Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improve the logic for selecting SDXL pooled embeds when handling multi-region prompts.

Browse Source
This commit is contained in:
Ryan Dick 2024-02-28 22:14:41 -05:00
parent e7f7ae660d
commit bdf3691ad0
1 changed files with 10 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
invokeai/app/invocations/latent.py
View File

@ -410,17 +410,19 @@ class DenoiseLatentsInvocation(BaseInvocation):
extra_conditioning = text_embedding_info.extra_conditioning extra_conditioning = text_embedding_info.extra_conditioning
if is_sdxl: if is_sdxl:
# HACK(ryand): We just use the the first SDXLConditioningInfo's pooled_embeds and add_time_ids. This is # We choose a random SDXLConditioningInfo's pooled_embeds and add_time_ids here, with a preference for
# fundamentally an interface issue, as the SDXL Compel nodes are not designed to be used in the way that # prompts without a mask. We prefer prompts without a mask, because they are more likely to contain
# we use them for regional prompting. Ideally, the DenoiseLatents invocation should accept a single # global prompt information. In an ideal case, there should be exactly one global prompt without a
# mask, but we don't enforce this.
# HACK(ryand): The fact that we have to choose a single pooled_embedding and add_time_ids here is a
# fundamental interface issue. The SDXL Compel nodes are not designed to be used in the way that we use
# them for regional prompting. Ideally, the DenoiseLatents invocation should accept a single
# pooled_embeds tensor and a list of standard text embeds with region masks. This change would be a # pooled_embeds tensor and a list of standard text embeds with region masks. This change would be a
# pretty major breaking change to a popular node, so for now we use this hack. # pretty major breaking change to a popular node, so for now we use this hack.
# if pooled_embedding is None or mask is None:
# An improvement could be to use the pooled embeds from the prompt with the largest region, as this is
# most likely to be a global prompt.
if pooled_embedding is None:
pooled_embedding = text_embedding_info.pooled_embeds pooled_embedding = text_embedding_info.pooled_embeds
if add_time_ids is None: if add_time_ids is None or mask is None:
add_time_ids = text_embedding_info.add_time_ids add_time_ids = text_embedding_info.add_time_ids
text_embedding.append(text_embedding_info.embeds) text_embedding.append(text_embedding_info.embeds)