Ryan Dick
77090070bd
Check the size of a model on disk and make room for it in the cache before loading it.
2024-08-29 19:08:18 +00:00
Ryan Dick
6ba9b1b6b0
Tidy up GIG -> GB and remove unused GIG constant.
2024-08-29 19:08:18 +00:00
Ryan Dick
c578b8df1e
Improve ModelCache docs.
2024-08-29 19:08:18 +00:00
Ryan Dick
cad9a41433
Remove unused MOdelCache.exists(...) function.
2024-08-29 19:08:18 +00:00
Ryan Dick
5fefb3b0f4
Remove unused param from ModelCache.
2024-08-29 19:08:18 +00:00
Ryan Dick
5284a870b0
Remove unused constructor params from ModelCache.
2024-08-29 19:08:18 +00:00
Ryan Dick
e064377c05
Remove default model cache sizes from model_cache_default.py. These defaults were misleading, because the config defaults take precedence over them.
2024-08-29 19:08:18 +00:00
Brandon Rising
65bb46bcca
Rename params for flux and flux vae, add comments explaining use of the config_path in model config
2024-08-26 20:17:50 -04:00
Ryan Dick
bbf934d980
Remove outdated TODO.
2024-08-26 20:17:50 -04:00
Ryan Dick
635d2f480d
ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
70c278c810
Remove dependency on flux config files
2024-08-26 20:17:50 -04:00
Ryan Dick
83f82c5ddf
Switch the CLIP-L start model to use our hosted version - which is much smaller.
2024-08-26 20:17:50 -04:00
Brandon Rising
101de8c25d
Update t5 encoder formats to accurately reflect the quantization strategy and data type
2024-08-26 20:17:50 -04:00
Ryan Dick
75d8ac378c
Update the T5 8-bit quantized starter model to use the BnB LLM.int8() variant.
2024-08-26 20:17:50 -04:00
Brandon Rising
1047584b3e
Only import bnb quantize file if bitsandbytes is installed
2024-08-26 20:17:50 -04:00
Ryan Dick
a0bf20bcee
Run FLUX VAE decoding in the user's preferred dtype rather than float32. Tested, and seems to work well at float16.
2024-08-26 20:17:50 -04:00
Ryan Dick
1c1f2c6664
Add comment about incorrect T5 Tokenizer size calculation.
2024-08-26 20:17:50 -04:00
Brandon Rising
c27d59baf7
Run ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
72398350b4
More flux loader cleanup
2024-08-26 20:17:50 -04:00
Brandon Rising
df9445c351
Various styling and exception type updates
2024-08-26 20:17:50 -04:00
Brandon Rising
87b7a2e39b
Switch inheritance class of flux model loaders
2024-08-26 20:17:50 -04:00
Brandon Rising
57168d719b
Fix styling/lint
2024-08-26 20:17:50 -04:00
Brandon Rising
dee6d2c98e
Fix support for 8b quantized t5 encoders, update exception messages in flux loaders
2024-08-26 20:17:50 -04:00
Ryan Dick
0c5e11f521
Fix FLUX output image clamping. And a few other minor fixes to make inference work with the full bfloat16 FLUX transformer model.
2024-08-26 20:17:50 -04:00
Brandon Rising
a63f842a13
Select dev/schnell based on state dict, use correct max seq len based on dev/schnell, and shift in inference, separate vae flux params into separate config
2024-08-26 20:17:50 -04:00
Brandon Rising
4bd7fda694
Install sub directories with folders correctly, ensure consistent dtype of tensors in flux pipeline and vae
2024-08-26 20:17:50 -04:00
Brandon Rising
81f0886d6f
Working inference node with quantized bnb nf4 checkpoint
2024-08-26 20:17:50 -04:00
Brandon Rising
1bd90e0fd4
Run ruff, setup initial text to image node
2024-08-26 20:17:50 -04:00
Brandon Rising
436f18ff55
Add backend functions and classes for Flux implementation, Update the way flux encoders/tokenizers are loaded for prompt encoding, Update way flux vae is loaded
2024-08-26 20:17:50 -04:00
Brandon Rising
9ed53af520
Run Ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
56fda669fd
Manage quantization of models within the loader
2024-08-26 20:17:50 -04:00
blessedcoolant
4f8a4b0f22
Merge branch 'main' into depth_anything_v2
2024-08-03 00:38:57 +05:30
Ryan Dick
b9dc3460ba
Rename SegmentAnythingModel -> SegmentAnythingPipeline.
2024-08-01 09:57:47 -04:00
Ryan Dick
fca119773b
Split invokeai/backend/image_util/segment_anything/ dir into grounding_dino/ and segment_anything/
2024-07-31 12:28:47 -04:00
Ryan Dick
9f448fecb7
Move invokeai/backend/grounded_sam -> invokeai/backend/image_util/grounded_sam
2024-07-31 10:00:30 -04:00
blessedcoolant
18f89ed5ed
fix: Make DepthAnything work with Invoke's Model Management
2024-07-31 03:57:54 +05:30
Ryan Dick
ff6398f7d8
Add a GroundedSamInvocation for image segmentation from a text prompt (Grounding DINO + Segment Anything Model).
2024-07-30 11:12:26 -04:00
psychedelicious
74cef38bcf
fix(backend): add refiner to single-file load_classes
...
Fixes single-file refiner loading.
2024-07-26 05:08:01 +10:00
Lincoln Stein
97a7f51721
don't use cpu state_dict for model unpatching when executing on cpu ( #6631 )
...
Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-07-18 15:34:01 -04:00
Ryan Dick
81991e072b
Merge branch 'main' into ryan/spandrel-upscale
2024-07-16 15:14:08 -04:00
psychedelicious
38343917f8
fix(backend): revert non-blocking device transfer
...
In #6490 we enabled non-blocking torch device transfers throughout the model manager's memory management code. When using this torch feature, torch attempts to wait until the tensor transfer has completed before allowing any access to the tensor. Theoretically, that should make this a safe feature to use.
This provides a small performance improvement but causes race conditions in some situations. Specific platforms/systems are affected, and complicated data dependencies can make this unsafe.
- Intermittent black images on MPS devices - reported on discord and #6545 , fixed with special handling in #6549 .
- Intermittent OOMs and black images on a P4000 GPU on Windows - reported in #6613 , fixed in this commit.
On my system, I haven't experience any issues with generation, but targeted testing of non-blocking ops did expose a race condition when moving tensors from CUDA to CPU.
One workaround is to use torch streams with manual sync points. Our application logic is complicated enough that this would be a lot of work and feels ripe for edge cases and missed spots.
Much safer is to fully revert non-locking - which is what this change does.
2024-07-16 08:59:42 +10:00
Ryan Dick
7b5d4935b4
Merge branch 'main' into ryan/spandrel-upscale
2024-07-09 13:47:11 -04:00
Ryan Dick
af63c538ed
Demote error log to warning to models treated as having size 0.
2024-07-09 08:35:43 -04:00
Ryan Dick
1d449097cc
Apply ruff rule to disallow all relative imports.
2024-07-04 09:35:37 -04:00
Ryan Dick
9da5925287
Add ruff rule to disallow relative parent imports.
2024-07-04 09:35:37 -04:00
Ryan Dick
414750a45d
Update calc_model_size_by_data(...) to handle all expected model types, and to log an error if an unexpected model type is received.
2024-07-04 09:08:25 -04:00
Ryan Dick
a405f14ea2
Fix SpandrelImageToImageModel size calculation for the model cache.
2024-07-03 16:38:16 -04:00
Ryan Dick
2a1514272f
Set the dtype correctly for SpandrelImageToImageModels when they are loaded.
2024-07-03 16:28:21 -04:00
Ryan Dick
59ce9cf41c
WIP - Begin to integrate SpandreImageToImageModel type into the model manager.
2024-07-03 16:28:21 -04:00
Ryan Dick
e4813f800a
Update calc_model_size_by_data(...) to handle all expected model types, and to log an error if an unexpected model type is received.
2024-07-02 21:51:45 -04:00
