fix bug when there are two multi vector TI in a prompt (#5356)

## What type of PR is this? (check all applicable)

- [ ] Refactor
- [ ] Feature
- [X] Bug Fix
- [ ] Optimization
- [ ] Documentation Update
- [ ] Community Node Submission


## Have you discussed this change with the InvokeAI team?
- [ ] Yes
- [X] No, because: it's a simple fix

      
## Have you updated all relevant documentation?
- [X] Yes
- [ ] No


## Description
if there are two multi vector TI in a prompt eg `<ti-1> <ti-2>` with
ti-1 has vector size 16 and ti-2 has vector size 8 then the second one
uses the first ti_embedding.shape[0] and you get errors like eg
"<ti-2-!pad-8> is not found" because ti-2 only has vector size 8 but the
code is taking the wrong ti_embedding.shape[0]

## Related Tickets & Documents

<!--
For pull requests that relate or close an issue, please include them
below. 

For example having the text: "closes #1234" would connect the current
pull
request to issue 1234.  And when we merge the pull request, Github will
automatically close the issue.
-->

- Related Issue #
- Closes #

## QA Instructions, Screenshots, Recordings

<!-- 
Please provide steps on how to test changes, any hardware or 
software specifications as well as any other pertinent information. 
-->

## Merge Plan

<!--
A merge plan describes how this PR should be handled after it is
approved.

Example merge plans:
- "This PR can be merged when approved"
- "This must be squash-merged when approved"
- "DO NOT MERGE - I will rebase and tidy commits before merging"
- "#dev-chat on discord needs to be advised of this change when it is
merged"

A merge plan is particularly important for large PRs or PRs that touch
the
database in any way.
-->

## Added/updated tests?

- [ ] Yes
- [ ] No : _please replace this line with details on why tests
      have not been included_

## [optional] Are there any post deployment tasks we need to perform?
This commit is contained in:
Millun Atluri 2023-12-28 15:59:27 +11:00 committed by GitHub
commit 78fe9b642d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -215,7 +215,9 @@ class ModelPatcher:
text_encoder.resize_token_embeddings(init_tokens_count + new_tokens_added, pad_to_multiple_of)
model_embeddings = text_encoder.get_input_embeddings()
for ti_name, _ in ti_list:
for ti_name, ti in ti_list:
ti_embedding = _get_ti_embedding(text_encoder.get_input_embeddings(), ti)
ti_tokens = []
for i in range(ti_embedding.shape[0]):
embedding = ti_embedding[i]