Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,844 Commits 274 Branches 239 Tags 397 MiB
cd9dfefe3c
Commit Graph

1605 Commits

Author SHA1 Message Date
Lincoln Stein
21a60af881
when unlocking models, offload_unlocked_models should prune to vram limit only (#6450) 
Co-authored-by: Lincoln Stein <lstein@gmail.com>
 2024-05-29 03:01:21 +00:00
Ryan Dick
829b9ad66b Add a callout about the hackiness of dropping tokens in the TextualInversionManager. 2024-05-28 05:11:54 -07:00
Ryan Dick
3aa1c8d3a8 Update TextualInversionManager for compatibility with the latest transformers release. See https://github.com/invoke-ai/InvokeAI/issues/6445. 2024-05-28 05:11:54 -07:00
Ryan Dick
994c61b67a Add docs to TextualInversionManager and improve types. No changes to functionality. 2024-05-28 05:11:54 -07:00
Lincoln Stein
34e1eb19f9 merge with main and resolve conflicts 2024-05-27 22:20:34 -04:00
Lincoln Stein
532f82cb97
Optimize RAM to VRAM transfer (#6312) 
* avoid copying model back from cuda to cpu

* handle models that don't have state dicts

* add assertions that models need a `device()` method

* do not rely on torch.nn.Module having the device() method

* apply all patches after model is on the execution device

* fix model patching in latents too

* log patched tokenizer

* closes #6375

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
 2024-05-24 17:06:09 +00:00
Lincoln Stein
987ee704a1
Merge branch 'main' into lstein/feat/simple-mm2-api 2024-05-17 22:54:03 -04:00
Lincoln Stein
f29c406fed refactor model_install to work with refactored download queue 2024-05-13 22:49:15 -04:00
psychedelicious
93da75209c feat(nodes): use new blur_if_nsfw method 2024-05-14 07:23:38 +10:00
psychedelicious
9c819f0fd8 fix(nodes): fix nsfw checker model download 2024-05-14 07:23:38 +10:00
Lincoln Stein
0bf14c2830 add multifile_download() method to download service 2024-05-12 20:14:00 -06:00
blessedcoolant
da61396b1c cleanup: seamless unused older code cleanup 2024-05-13 08:11:08 +10:00
blessedcoolant
6c9fb617dc fix: fix seamless 2024-05-13 08:11:08 +10:00
Lincoln Stein
b48d4a049d bad implementation of diffusers folder download 2024-05-08 21:21:01 -07:00
Lincoln Stein
e9a20051bd refactor DWOpenPose and add type hints 2024-05-03 18:08:53 -04:00
Lincoln Stein
3b64e7a1fd
Merge branch 'main' into lstein/feat/simple-mm2-api 2024-05-02 21:20:35 -04:00
Lincoln Stein
49c84cd423
Merge branch 'main' into lstein/feat/simple-mm2-api 2024-04-30 18:13:42 -04:00
blessedcoolant
39ab4dd83e Merge branch 'main' into pr/6086 2024-05-01 00:37:06 +05:30
psychedelicious
2d7b8c2a1b fix(backend): do not round image dims to 64 in controlnet processor resize 
Rounding the dims results in control images that are subtly different than the input. We round to the nearest 8px later, there's no need to round now.
 2024-04-30 08:10:59 -04:00
psychedelicious
1fe90c357c feat(backend): lift managed model loading out of depthanything class 2024-04-29 08:56:00 +10:00
psychedelicious
fcb071f30c feat(backend): lift managed model loading out of lama class 2024-04-29 08:12:51 +10:00
Lincoln Stein
57c831442e fix safe_filename() on windows 2024-04-28 14:42:40 -04:00
Lincoln Stein
7c39929758 support VRAM caching of dict models that lack to() 2024-04-28 13:41:06 -04:00
Lincoln Stein
a26667d3ca make download and convert cache keys safe for filename length 2024-04-28 12:24:36 -04:00
Lincoln Stein
bb04f496e0 Merge branch 'main' into lstein/feat/simple-mm2-api 2024-04-28 11:33:26 -04:00
Lincoln Stein
70903ef057 refactor load_ckpt_from_url() 2024-04-28 11:33:23 -04:00
psychedelicious
241a1fdb57 feat(mm): support sdxl ckpt inpainting models 
There are only a couple SDXL inpainting models, and my tests indicate they are not as good as SD1.5 inpainting, but at least we support them now.

- Add the config file. This matches what is used in A1111. The only difference from the non-inpainting SDXL config is the number of in-channels.
- Update the legacy config maps to use this config file.
 2024-04-28 12:57:27 +10:00
Lincoln Stein
d72f272f16 Address change requests in first round of PR reviews. 
Pending:

- Move model install calls into model manager and create passthrus in invocation_context.
- Consider splitting load_model_from_url() into a call to get the path and a call to load the path.
 2024-04-24 23:53:30 -04:00
psychedelicious
6b0bf59682 feat(backend): update nms util to make blur/thresholding optional 2024-04-25 13:20:09 +10:00
blessedcoolant
260e24733f fix: update SDXL IP Adpater starter model to be ViT-H 2024-04-24 00:08:21 -04:00
blessedcoolant
6b394554e2 fix: update ip adapter starter models path 2024-04-24 08:48:25 +05:30
psychedelicious
a461537087 chore: ruff 2024-04-23 07:32:53 -04:00
psychedelicious
0aa5aadfe8 fix(mm): move variant to MainConfigBase 
shoulda been here all along
 2024-04-23 07:32:53 -04:00
Lincoln Stein
2b9f06dc4c
Re-enable app shutdown actions (#6244) 
* closes #6242

* only override sigINT during slow model scanning

* fix ruff formatting

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
 2024-04-19 06:45:42 -04:00
Lincoln Stein
34cdfc61ab
Merge branch 'main' into lstein/feat/simple-mm2-api 2024-04-17 17:18:13 -04:00
blessedcoolant
6bab040d24 Merge branch 'main' into ip-adapter-style-comp 2024-04-16 21:14:06 +05:30
blessedcoolant
f46bbaf8c4 fix: make ip-adapter weights not be optional 2024-04-16 21:12:45 +05:30
blessedcoolant
d27907cc6d fix: entire reshaping block needs to be skipped 2024-04-16 04:29:53 +05:30
blessedcoolant
7ee3fef2db cleanup: better var names for the ip adapter weight collection block 2024-04-16 04:23:50 +05:30
blessedcoolant
a148c4322c fix: IP Adapter weights being incorrectly applied 
They were being overwritten rather than being appended
 2024-04-16 04:10:41 +05:30
blessedcoolant
5f6c6abf9c chore: change IPAdapterAttentionWeights to a dataclass 2024-04-15 23:38:55 +05:30
Lincoln Stein
470a39935c fix merge conflicts with main 2024-04-15 09:24:57 -04:00
Lincoln Stein
e93f4d632d
[util] Add generic torch device class (#6174) 
* introduce new abstraction layer for GPU devices

* add unit test for device abstraction

* fix ruff

* convert TorchDeviceSelect into a stateless class

* move logic to select context-specific execution device into context API

* add mock hardware environments to pytest

* remove dangling mocker fixture

* fix unit test for running on non-CUDA systems

* remove unimplemented get_execution_device() call

* remove autocast precision

* Multiple changes:

1. Remove TorchDeviceSelect.get_execution_device(), as well as calls to
   context.models.get_execution_device().
2. Rename TorchDeviceSelect to TorchDevice
3. Added back the legacy public API defined in `invocation_api`, including
   choose_precision().
4. Added a config file migration script to accommodate removal of precision=autocast.

* add deprecation warnings to choose_torch_device() and choose_precision()

* fix test crash

* remove app_config argument from choose_torch_device() and choose_torch_dtype()

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
 2024-04-15 13:12:49 +00:00
Lincoln Stein
41b909cbe3 port dw_openpose, depth_anything, and lama processors to new model download scheme 2024-04-14 15:57:03 -04:00
blessedcoolant
8426f1e7b2 fix(experimental): Possible fix for conflict with regional embed length mismatch 
Pushing this so people can test it out and see if this needs to be handled in a different way.
 2024-04-14 12:19:19 +05:30
blessedcoolant
9cb0f63c44 refactor: fix a bunch of type issues in custom_attention 2024-04-13 14:17:25 +05:30
blessedcoolant
d4393e4170 chore: linter fixes 2024-04-13 12:14:45 +05:30
blessedcoolant
6ea183f0d4 wip: Initial Implementation IP Adapter Style & Comp Modes 2024-04-13 11:09:45 +05:30
Lincoln Stein
3a26c7bb9e fix merge conflicts 2024-04-12 00:58:11 -04:00
Lincoln Stein
df5ebdbc4f add invocation_context.load_ckpt_from_url() method 2024-04-12 00:55:21 -04:00
Lincoln Stein
651c0b39b1 clear cache on all exceptions 2024-04-12 07:19:16 +10:00
Lincoln Stein
46d23cd868 catch RunTimeError during model to() call rather than OutOfMemoryError 2024-04-12 07:19:16 +10:00
Lincoln Stein
579082ac10 [mm] clear the cache entry for a model that got an OOM during loading 2024-04-12 07:19:16 +10:00
psychedelicious
7bc77ddb40 fix(nodes): doubly-noised latents 
When using refiner with a mask (i.e. inpainting), we don't have noise provided as an input to the node.

This situation uniquely hits a code path that wasn't reviewed when gradient denoising was implemented.

That code path does two things wrong:
- It lerp'd the input latents. This was fixed in 5a1f4cb1ce.
- It added noise to the latents an extra time. This is fixed in this change.

We don't need to add noise in `latents_from_embeddings` because we do it just a lines later in `AddsMaskGuidance`.

- Remove the extraneous call to `add_noise`
- Make `seed` a required arg. We never call the function without seed anyways. If we refactor this in the future, it will be clearer that we need to look at how seed is handled.
- Move the call to create the noise to a deeper conditional, just before we call `AddsMaskGuidance`. The created noise tensor is now only used in that function, no need to create it every time.

Note: Whether or not having both noise and latents as inputs on the node is correct is a separate conversation. This change just fixes the issue with the current setup.
 2024-04-11 07:21:50 -04:00
Ryan Dick
f9af32a6d1 Fix the padding behavior when max-pooling regional IP-Adapter masks to mirror the downscaling behavior of SD and SDXL. Prior to this change, denoising with input latent dimensions that were not evenly divisible by 8 would raise an exception. 2024-04-09 16:50:43 -04:00
Ryan Dick
fba40eb1bd Fix the padding behavior when max-pooling regional prompt masks to mirror the downscaling behavior of SD and SDXL. Prior to this change, denoising with input latent dimensions that were not evenly divisible by 8 would raise an exception. 2024-04-09 16:50:43 -04:00
Ryan Dick
0c6dd32ece (minor) Fix IP-Adapter conditional logic in CustomAttnProcessor2_0. 2024-04-09 15:06:51 -04:00
Ryan Dick
0bdbfd4d1d Add support for IP-Adapter masks. 2024-04-09 15:06:51 -04:00
Ryan Dick
2e27ed5f3d Pass IP-Adapter scales through the cross_attn_kwargs pathway, since they are the same for all attention layers. This change also helps to prepare for adding IP-Adapter region masks. 2024-04-09 15:06:51 -04:00
Ryan Dick
4a828818da Remove support for Prompt-to-Prompt cross-attention control (aka .swap()). This feature is not widely used. It does not work with SDXL and is incompatible with IP-Adapter and regional prompting. The implementation is also intertwined with both text embedding and the UNet attention layers, resulting in a high maintenance burden. For all of these reasons, we have decided to drop support. 2024-04-09 10:57:02 -04:00
Ryan Dick
182810337c Add utility to_standard_float_mask(...) to convert various mask formats to a standardized format. 2024-04-09 08:12:12 -04:00
Ryan Dick
75ef473748 Pull the upstream changes from diffusers' AttnProcessor2_0 into CustomAttnProcessor2_0. This fixes a bug in CustomAttnProcessor2_0 that was being triggered when peft was not installed. The bug was present in a block of code that was previously copied from diffusers. The bug seems to have been introduced during diffusers' migration to PEFT for their LoRA handling. The upstream bug was fixed in 531e719163. 2024-04-09 08:12:12 -04:00
Ryan Dick
a78df8123f Update the diffusion logic to use the new regional prompting feature. 2024-04-09 08:12:12 -04:00
Ryan Dick
7ca677578e Create a UNetAttentionPatcher for patching UNet models with CustomAttnProcessor2_0 modules. 2024-04-09 08:12:12 -04:00
Ryan Dick
31c456c1e6 Update CustomAttention to support both IP-Adapters and regional prompting. 2024-04-09 08:12:12 -04:00
Ryan Dick
2ce79b61f5 Initialize a RegionalPromptAttnProcessor2_0 class by copying AttnProcessor2_0 from diffusers. 2024-04-09 08:12:12 -04:00
Ryan Dick
109e3f0e7f Add RegionalPromptData class for managing prompt region masks. 2024-04-09 08:12:12 -04:00
Ryan Dick
d1e45585d0 Add TextConditioningRegions to the TextConditioningData data structure. 2024-04-09 08:12:12 -04:00
Ryan Dick
aba023e0c5 Improve documentation of conditioning_data.py. 2024-04-09 08:12:12 -04:00
Ryan Dick
e354c29b52 Rename ConditioningData -> TextConditioningData. 2024-04-09 08:12:12 -04:00
Ryan Dick
a7f363e654 Split ip_adapter_conditioning out from ConditioningData. 2024-04-09 08:12:12 -04:00
Ryan Dick
9b2162e564 Remove scheduler_args from ConditioningData structure. 2024-04-09 08:12:12 -04:00
blessedcoolant
fd1f240853 fix: SDXL Refiner not working properly with Inpainting 2024-04-09 14:13:10 +10:00
psychedelicious
9ab6655491 feat(backend): clean up choose_precision 
- Allow user-defined precision on MPS.
- Use more explicit logic to handle all possible cases.
- Add comments.
- Remove the app_config args (they were effectively unused, just get the config using the singleton getter util)
 2024-04-07 09:41:05 -04:00
psychedelicious
4068e817d6 fix(mm): typing issues in model cache 2024-04-06 14:35:36 +11:00
psychedelicious
a09d705e4c fix(mm): remove vram check 
This check prematurely reports insufficient VRAM on Windows. See #6106 for details.
 2024-04-06 14:35:36 +11:00
Lincoln Stein
4571986c63 fix misplaced lock call 2024-04-05 14:32:18 +11:00
Lincoln Stein
812f10730f
adjust free vram calculation for models that will be removed by lazy offloading (#6150) 
Co-authored-by: Lincoln Stein <lstein@gmail.com>
 2024-04-04 22:51:12 -04:00
psychedelicious
f0b1bb0327 feat(nodes): redo tile infill 
The previous algorithm errored if the image wasn't divisible by the tile size. I've reimplemented it from scratch to mitigate this issue.

The new algorithm is simpler. We create a pool of tiles, then use them to create an image composed completely of tiles. If there is any awkwardly sized space on the edge of the image, the tiles are cropped to fit.

Finally, paste the original image over the tile image.

I've added a jupyter notebook to do a smoke test of infilling methods, and 10 test images.

The other infill algorithms can be easily tested with the notebook on the same images, though I didn't set that up yet.

Tested and confirmed this gives results just as good as the earlier infill, though of course they aren't the same due to the change in the algorithm.
 2024-04-05 08:49:13 +11:00
blessedcoolant
3c195d74a5 fix: bypass edge pixels which cannot transform to tile size 
Still need to fix this somehow
 2024-04-05 08:49:13 +11:00
blessedcoolant
32a6b758cd wip: Initial Infill Methods Refactor 2024-04-05 08:49:13 +11:00
brandonrising
51ca59c088 Update probe to always use cpu for loading models 2024-04-04 07:34:43 +11:00
psychedelicious
85f53f94f8 feat(mm): include needed vs free in OOM 
Gives us a bit more visibility into these errors, which seem to be popping up more frequently with the new MM.
 2024-04-04 06:26:15 +11:00
blessedcoolant
be574cb764 fix: incorrect suffix check in ip adapter checkpoint file 2024-04-03 22:38:28 +05:30
blessedcoolant
5f01de1993 chore: ruff and lint fixes 2024-04-03 20:41:51 +05:30
blessedcoolant
e574815413 chore: clean up merge conflicts 2024-04-03 20:28:00 +05:30
blessedcoolant
fb293dcd84 Merge branch 'checkpoint-ip-adapter' of https://github.com/blessedcoolant/InvokeAI into checkpoint-ip-adapter 2024-04-03 20:23:07 +05:30
blessedcoolant
414851f2f0 fix: raise and present the runtime error from the exception 2024-04-03 20:21:50 +05:30
blessedcoolant
2dcbb7223b fix: use Path for ip_adapter_ckpt_path instead of str 2024-04-03 20:21:03 +05:30
blessedcoolant
14a9f74b17 cleanup: use load_file of safetensors directly for loading ip adapters 2024-04-03 12:40:13 +05:30
blessedcoolant
1372ef15b3 fix: Fail when unexpected keys are found in IP Adapter models 2024-04-03 12:40:11 +05:30
blessedcoolant
be1212de9a fix: Raise a better error when incorrect CLIP Vision model is used 2024-04-03 12:40:10 +05:30
blessedcoolant
a14ce0edab chore: rename IPAdapterDiffusersConfig to IPAdapterInvokeAIConfig 2024-04-03 12:40:10 +05:30
blessedcoolant
4a0dfc3b2d ui: improve the clip vision model picker layout 2024-04-03 12:40:08 +05:30
blessedcoolant
936b99bd3c chore: improve types in ip_adapter backend file 2024-04-03 12:40:02 +05:30
blessedcoolant
79f7b61dfe fix: cleanup across various ip adapter files 2024-04-03 12:39:52 +05:30
blessedcoolant
b1c8266e22 feat: add base model recognition for ip adapter safetensor files 2024-04-03 12:39:52 +05:30
blessedcoolant
67afb1763e wip: Initial implementation of safetensor support for IP Adapter 2024-04-03 12:39:52 +05:30
psychedelicious
0259114d9c
Merge branch 'main' into main 2024-04-03 17:03:19 +11:00
psychedelicious
51e515b925 tidy: use lowercase for tcd scheduler identifier 2024-04-03 17:03:02 +11:00