Sergey Borisov
2c2ec8f0bc
Comments, a bit refactor
2024-07-17 04:20:31 +03:00
Sergey Borisov
79e35bd0d3
Minor fixes
2024-07-17 03:48:37 +03:00
Sergey Borisov
137202b77c
Remove patch_unet logic for now
2024-07-17 03:40:27 +03:00
Sergey Borisov
03e22c257b
Convert conditioning_mode to enum
2024-07-17 03:37:11 +03:00
Sergey Borisov
ae6d4fbc78
Move out _concat_conditionings_for_batch submethods
2024-07-17 03:31:26 +03:00
Sergey Borisov
cd1bc1595a
Rename sequential as private variable
2024-07-17 03:24:11 +03:00
Sergey Borisov
b7c6c63005
Added some comments
2024-07-16 22:52:44 +03:00
Sergey Borisov
cec345cb5c
Change attention processor apply logic
2024-07-16 20:03:29 +03:00
Sergey Borisov
608cbe3f5c
Separate inputs in denoise context
2024-07-16 19:30:29 +03:00
Sergey Borisov
9f088d1bf5
Multiple small fixes
2024-07-16 00:51:25 +03:00
Sergey Borisov
fd8d1c12d4
Remove 'del' operator overload
2024-07-16 00:43:32 +03:00
Sergey Borisov
d623bd429b
Fix condtionings logic
2024-07-16 00:31:56 +03:00
Sergey Borisov
499e4d4fde
Add preview extension to check logic
2024-07-13 00:45:04 +03:00
Sergey Borisov
e961dd1dec
Remove remains of priority logic
2024-07-13 00:44:21 +03:00
Sergey Borisov
7e00526999
Remove overrides logic for now
2024-07-13 00:28:56 +03:00
Sergey Borisov
3a9dda9177
Renames
2024-07-12 22:44:00 +03:00
Sergey Borisov
bd8ae5d896
Simplify guidance modes
2024-07-12 22:01:37 +03:00
Sergey Borisov
87e96e1be2
Rename modifiers to callbacks, convert order to int, a bit unify injection points
2024-07-12 22:01:05 +03:00
Sergey Borisov
0bc60378d3
A bit rework conditioning convert to unet kwargs
2024-07-12 20:43:32 +03:00
Sergey Borisov
9cc852cf7f
Base code from draft PR
2024-07-12 20:31:26 +03:00
Ryan Dick
35f8781ea2
Fix static type errors with SCHEDULER_NAME_VALUES. And, avoid bi-directional cross-directory imports, which contribute to circular import issues.
2024-07-05 07:38:35 -07:00
Ryan Dick
1d449097cc
Apply ruff rule to disallow all relative imports.
2024-07-04 09:35:37 -04:00
Ryan Dick
3752509066
Expose the VAE tile_size on the VAE encode and decode invocations.
2024-07-02 09:07:03 -04:00
Ryan Dick
79640ba14e
Add context manager for overriding VAE tiling params.
2024-07-02 09:07:03 -04:00
Ryan Dick
9a3b8c6fcb
Fix handling of init_timestep in StableDiffusionGeneratorPipeline and improve its documentation.
2024-06-26 12:51:51 -04:00
Ryan Dick
bd74b84cc5
Revert "Remove the redundant init_timestep parameter that was being passed around. It is simply the first element of the timesteps array."
...
This reverts commit fa40061eca
2024-06-26 12:51:51 -04:00
Ryan Dick
cd9dfefe3c
Fix inpainting mask shape assertions.
2024-06-25 11:31:52 -07:00
Ryan Dick
e1af78c702
Make the tile_overlap input to MultiDiffusion *strictly* control the amount of overlap rather than being a lower bound.
2024-06-25 11:31:52 -07:00
Ryan Dick
c5588e1ff7
Add TODO comment explaining why some schedulers do not interact well with MultiDiffusion.
2024-06-25 11:31:52 -07:00
Ryan Dick
07ac292680
Consolidate _region_step() function - the separation wasn't really adding any value.
2024-06-25 11:31:52 -07:00
Ryan Dick
7c032ea604
(minor) Fix some documentation typos.
2024-06-25 11:31:52 -07:00
Ryan Dick
fa40061eca
Remove the redundant init_timestep parameter that was being passed around. It is simply the first element of the timesteps array.
2024-06-25 11:31:52 -07:00
Ryan Dick
25067e4f0d
Delete rough notes.
2024-06-25 11:31:52 -07:00
Ryan Dick
fb0aaa3e6d
Fix advanced scheduler behaviour in MultiDiffusionPipeline.
2024-06-25 11:31:52 -07:00
Ryan Dick
c22526b9d0
Fix handling of stateful schedulers in MultiDiffusionPipeline.
2024-06-25 11:31:52 -07:00
Ryan Dick
c881882f73
Connect TiledMultiDiffusionDenoiseLatents to the MultiDiffusionPipeline backend.
2024-06-25 11:31:52 -07:00
Ryan Dick
36473fc52a
Remove regional conditioning logic from MultiDiffusionPipeline - it is not yet supported.
2024-06-25 11:31:52 -07:00
Ryan Dick
b9964ecc4a
Initial (untested) implementation of MultiDiffusionPipeline.
2024-06-25 11:31:52 -07:00
Ryan Dick
051af802fe
Remove inpainting support from MultiDiffusionPipeline.
2024-06-25 11:31:52 -07:00
Ryan Dick
3ff2e558d9
Remove IP-Adapter and T2I-Adapter support from MultiDiffusionPipeline.
2024-06-25 11:31:52 -07:00
Ryan Dick
fc187c9253
Document plan for the rest of the MultiDiffusion implementation.
2024-06-25 11:31:52 -07:00
Ryan Dick
605f460c7d
Add detailed docstring to latents_from_embeddings().
2024-06-25 11:31:52 -07:00
Ryan Dick
60d1e686d8
Copy StableDiffusionGeneratorPipeline as a starting point for a new MultiDiffusionPipeline.
2024-06-25 11:31:52 -07:00
Ryan Dick
22704dd542
Simplify handling of inpainting models. Improve the in-code documentation around inpainting.
2024-06-25 11:31:52 -07:00
Ryan Dick
875673c9ba
Minor tidying of latents_from_embeddings(...).
2024-06-25 11:31:52 -07:00
Ryan Dick
f604575862
Consolidate latents_from_embeddings(...) and generate_latents_from_embeddings(...) into a single function.
2024-06-25 11:31:52 -07:00
Ryan Dick
60ac937698
Improve clarity of comments regarded when 'noise' and 'latents' are expected to be set.
2024-06-25 11:31:52 -07:00
Ryan Dick
1e41949a02
Fix static check errors on imports in diffusers_pipeline.py.
2024-06-25 11:31:52 -07:00
Ryan Dick
5f0e330ed2
Remove a condition for handling inpainting models that never resolves to True. The same logic is already applied earlier by AddsMaskLatents.
2024-06-25 11:31:52 -07:00
Ryan Dick
9dd779b414
Add clarifying comment to explain why noise might be None in latents_from_embedding().
2024-06-25 11:31:52 -07:00
Ryan Dick
fa183025ac
Remove unused are_like_tensors() function.
2024-06-25 11:31:52 -07:00
Ryan Dick
d3c85aa91a
Remove unused StableDiffusionGeneratorPipeline.use_ip_adapter member.
2024-06-25 11:31:52 -07:00
Ryan Dick
82619602a5
Remove unused StableDiffusionGeneratorPipeline.control_model.
2024-06-25 11:31:52 -07:00
Ryan Dick
196f3b721d
Stricter typing for the is_gradient_mask: bool.
2024-06-25 11:31:52 -07:00
Ryan Dick
244c28859d
Fix typing of control_data to reflect that it can be None.
2024-06-25 11:31:52 -07:00
Ryan Dick
40ae174c41
Fix typing of timesteps and init_timestep.
2024-06-25 11:31:52 -07:00
Ryan Dick
afaebdf151
Fix typing to reflect that the callback arg to latents_from_embeddings is never None.
2024-06-25 11:31:52 -07:00
Ryan Dick
d661517d94
Move seed above optional params.
2024-06-25 11:31:52 -07:00
Ryan Dick
82a69a54ac
Simplify handling of AddsMaskGuidance, and fix some related type errors.
2024-06-25 11:31:52 -07:00
Ryan Dick
ffc28176fe
Remove unused num_inference_steps.
2024-06-25 11:31:52 -07:00
Ryan Dick
d08e405017
Fix ControlNetModel type hint import source.
2024-06-25 11:31:52 -07:00
blessedcoolant
da61396b1c
cleanup: seamless unused older code cleanup
2024-05-13 08:11:08 +10:00
blessedcoolant
6c9fb617dc
fix: fix seamless
2024-05-13 08:11:08 +10:00
blessedcoolant
39ab4dd83e
Merge branch 'main' into pr/6086
2024-05-01 00:37:06 +05:30
blessedcoolant
6bab040d24
Merge branch 'main' into ip-adapter-style-comp
2024-04-16 21:14:06 +05:30
blessedcoolant
f46bbaf8c4
fix: make ip-adapter weights not be optional
2024-04-16 21:12:45 +05:30
blessedcoolant
d27907cc6d
fix: entire reshaping block needs to be skipped
2024-04-16 04:29:53 +05:30
blessedcoolant
7ee3fef2db
cleanup: better var names for the ip adapter weight collection block
2024-04-16 04:23:50 +05:30
blessedcoolant
a148c4322c
fix: IP Adapter weights being incorrectly applied
...
They were being overwritten rather than being appended
2024-04-16 04:10:41 +05:30
blessedcoolant
5f6c6abf9c
chore: change IPAdapterAttentionWeights to a dataclass
2024-04-15 23:38:55 +05:30
Lincoln Stein
e93f4d632d
[util] Add generic torch device class ( #6174 )
...
* introduce new abstraction layer for GPU devices
* add unit test for device abstraction
* fix ruff
* convert TorchDeviceSelect into a stateless class
* move logic to select context-specific execution device into context API
* add mock hardware environments to pytest
* remove dangling mocker fixture
* fix unit test for running on non-CUDA systems
* remove unimplemented get_execution_device() call
* remove autocast precision
* Multiple changes:
1. Remove TorchDeviceSelect.get_execution_device(), as well as calls to
context.models.get_execution_device().
2. Rename TorchDeviceSelect to TorchDevice
3. Added back the legacy public API defined in `invocation_api`, including
choose_precision().
4. Added a config file migration script to accommodate removal of precision=autocast.
* add deprecation warnings to choose_torch_device() and choose_precision()
* fix test crash
* remove app_config argument from choose_torch_device() and choose_torch_dtype()
---------
Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-04-15 13:12:49 +00:00
blessedcoolant
8426f1e7b2
fix(experimental): Possible fix for conflict with regional embed length mismatch
...
Pushing this so people can test it out and see if this needs to be handled in a different way.
2024-04-14 12:19:19 +05:30
blessedcoolant
9cb0f63c44
refactor: fix a bunch of type issues in custom_attention
2024-04-13 14:17:25 +05:30
blessedcoolant
d4393e4170
chore: linter fixes
2024-04-13 12:14:45 +05:30
blessedcoolant
6ea183f0d4
wip: Initial Implementation IP Adapter Style & Comp Modes
2024-04-13 11:09:45 +05:30
psychedelicious
7bc77ddb40
fix(nodes): doubly-noised latents
...
When using refiner with a mask (i.e. inpainting), we don't have noise provided as an input to the node.
This situation uniquely hits a code path that wasn't reviewed when gradient denoising was implemented.
That code path does two things wrong:
- It lerp'd the input latents. This was fixed in 5a1f4cb1ce
2024-04-11 07:21:50 -04:00
Ryan Dick
f9af32a6d1
Fix the padding behavior when max-pooling regional IP-Adapter masks to mirror the downscaling behavior of SD and SDXL. Prior to this change, denoising with input latent dimensions that were not evenly divisible by 8 would raise an exception.
2024-04-09 16:50:43 -04:00
Ryan Dick
fba40eb1bd
Fix the padding behavior when max-pooling regional prompt masks to mirror the downscaling behavior of SD and SDXL. Prior to this change, denoising with input latent dimensions that were not evenly divisible by 8 would raise an exception.
2024-04-09 16:50:43 -04:00
Ryan Dick
0c6dd32ece
(minor) Fix IP-Adapter conditional logic in CustomAttnProcessor2_0.
2024-04-09 15:06:51 -04:00
Ryan Dick
0bdbfd4d1d
Add support for IP-Adapter masks.
2024-04-09 15:06:51 -04:00
Ryan Dick
2e27ed5f3d
Pass IP-Adapter scales through the cross_attn_kwargs pathway, since they are the same for all attention layers. This change also helps to prepare for adding IP-Adapter region masks.
2024-04-09 15:06:51 -04:00
Ryan Dick
4a828818da
Remove support for Prompt-to-Prompt cross-attention control (aka .swap()). This feature is not widely used. It does not work with SDXL and is incompatible with IP-Adapter and regional prompting. The implementation is also intertwined with both text embedding and the UNet attention layers, resulting in a high maintenance burden. For all of these reasons, we have decided to drop support.
2024-04-09 10:57:02 -04:00
Ryan Dick
182810337c
Add utility to_standard_float_mask(...) to convert various mask formats to a standardized format.
2024-04-09 08:12:12 -04:00
Ryan Dick
75ef473748
Pull the upstream changes from diffusers' AttnProcessor2_0 into CustomAttnProcessor2_0. This fixes a bug in CustomAttnProcessor2_0 that was being triggered when peft was not installed. The bug was present in a block of code that was previously copied from diffusers. The bug seems to have been introduced during diffusers' migration to PEFT for their LoRA handling. The upstream bug was fixed in 531e719163.
2024-04-09 08:12:12 -04:00
Ryan Dick
a78df8123f
Update the diffusion logic to use the new regional prompting feature.
2024-04-09 08:12:12 -04:00
Ryan Dick
7ca677578e
Create a UNetAttentionPatcher for patching UNet models with CustomAttnProcessor2_0 modules.
2024-04-09 08:12:12 -04:00
Ryan Dick
31c456c1e6
Update CustomAttention to support both IP-Adapters and regional prompting.
2024-04-09 08:12:12 -04:00
Ryan Dick
2ce79b61f5
Initialize a RegionalPromptAttnProcessor2_0 class by copying AttnProcessor2_0 from diffusers.
2024-04-09 08:12:12 -04:00
Ryan Dick
109e3f0e7f
Add RegionalPromptData class for managing prompt region masks.
2024-04-09 08:12:12 -04:00
Ryan Dick
d1e45585d0
Add TextConditioningRegions to the TextConditioningData data structure.
2024-04-09 08:12:12 -04:00
Ryan Dick
aba023e0c5
Improve documentation of conditioning_data.py.
2024-04-09 08:12:12 -04:00
Ryan Dick
e354c29b52
Rename ConditioningData -> TextConditioningData.
2024-04-09 08:12:12 -04:00
Ryan Dick
a7f363e654
Split ip_adapter_conditioning out from ConditioningData.
2024-04-09 08:12:12 -04:00
Ryan Dick
9b2162e564
Remove scheduler_args from ConditioningData structure.
2024-04-09 08:12:12 -04:00
blessedcoolant
fd1f240853
fix: SDXL Refiner not working properly with Inpainting
2024-04-09 14:13:10 +10:00
psychedelicious
51e515b925
tidy: use lowercase for tcd scheduler identifier
2024-04-03 17:03:02 +11:00
psychedelicious
8c509295f9
chore: ruff
2024-04-03 17:02:45 +11:00
l0stl0rd
1a83936cdd
Merge branch 'invoke-ai:main' into main
2024-03-29 11:14:28 +01:00
psychedelicious
2526ef52c5
fix(nodes): workaround seamless multi gpu error #6010
...
The seamless logic errors when a second GPU is selected. I don't understand why, but a workaround is to skip the model patching when there there are no seamless axes specified.
This is also just a good practice regardless - don't patch the model unless we need to. Probably a negligible perf impact.
Closes #6010
2024-03-29 08:56:38 +11:00
l0stl0rd
80e311a069
Update schedulers.py
2024-03-28 22:52:15 +01:00
