Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add TODO comment about peformance bottleneck in LoRA loading code.

Browse Source
This commit is contained in:
Ryan Dick 2024-04-04 11:33:27 -04:00
parent 6e4de001f1
commit dcf11a01ce
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
invokeai/backend/lora.py
View File

@ -544,6 +544,10 @@ class LoRAModelRaw(RawModel): # (torch.nn.Module):
for layer_key, values in state_dict.items():
layer = layer_cls(layer_key, values)
# TODO(ryand): This .to() call causes an implicit CUDA sync point in a tight loop. This is very slow (even
# slower than loading the weights from disk). We should ideally only be copying the weights once - right
# before they are used. Or, if we want to do this here, then setting non_blocking = True would probably
# help.
layer.to(device=device, dtype=dtype)
model.layers[layer_key] = layer
return model