Brandon Rising
|
1047584b3e
|
Only import bnb quantize file if bitsandbytes is installed
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
a0bf20bcee
|
Run FLUX VAE decoding in the user's preferred dtype rather than float32. Tested, and seems to work well at float16.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
14ab339b33
|
Move prepare_latent_image_patches(...) to sampling.py with all of the related FLUX inference code.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
1c1f2c6664
|
Add comment about incorrect T5 Tokenizer size calculation.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
185f2a395f
|
Make FLUX get_noise(...) consistent across devices/dtypes.
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
94aba5892a
|
Attribute black-forest-labs/flux for much of the flux code
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
34451e5f27
|
added FLUX dev to starter models
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
c27d59baf7
|
Run ruff
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
b66f19d4d1
|
Add docs to the quantization scripts.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
4105a78b83
|
Update load_flux_model_bnb_llm_int8.py to work with a single-file FLUX transformer checkpoint.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
19a68afb3a
|
Fix bug in InvokeInt8Params that was causing it to use double the necessary VRAM.
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
e210c96485
|
add FLUX schnell starter models and submodels as dependenices or adhoc download options
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
5f567f41f4
|
add case for clip embed models in probe
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
cfac7c8189
|
Move requantize.py to the quatnization/ dir.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
1787de6836
|
Add docs to the requantize(...) function explaining why it was copied from optimum-quanto.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
ac96f187bd
|
Remove duplicate log_time(...) function.
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
72398350b4
|
More flux loader cleanup
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
df9445c351
|
Various styling and exception type updates
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
87b7a2e39b
|
Switch inheritance class of flux model loaders
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
57168d719b
|
Fix styling/lint
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
dee6d2c98e
|
Fix support for 8b quantized t5 encoders, update exception messages in flux loaders
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
e49105ece5
|
Add tqdm progress bar to FLUX denoising.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
0c5e11f521
|
Fix FLUX output image clamping. And a few other minor fixes to make inference work with the full bfloat16 FLUX transformer model.
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
a63f842a13
|
Select dev/schnell based on state dict, use correct max seq len based on dev/schnell, and shift in inference, separate vae flux params into separate config
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
4bd7fda694
|
Install sub directories with folders correctly, ensure consistent dtype of tensors in flux pipeline and vae
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
81f0886d6f
|
Working inference node with quantized bnb nf4 checkpoint
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
723f3ab0a9
|
Add nf4 bnb quantized format
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
1bd90e0fd4
|
Run ruff, setup initial text to image node
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
436f18ff55
|
Add backend functions and classes for Flux implementation, Update the way flux encoders/tokenizers are loaded for prompt encoding, Update way flux vae is loaded
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
2d9042fb93
|
Run Ruff
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
9ed53af520
|
Run Ruff
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
56fda669fd
|
Manage quantization of models within the loader
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
5f59a828f9
|
Setup flux model loading in the UI
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
1fa6bddc89
|
WIP on moving from diffusers to FLUX
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
d3a5ca5247
|
More improvements for LLM.int8() - not fully tested.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
f01f56a98e
|
LLM.int8() quantization is working, but still some rough edges to solve.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
99b0f79784
|
Clean up NF4 implementation.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
e1eb104345
|
NF4 inference working
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
5c2f95ef50
|
NF4 loading working... I think.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
b63df9bab9
|
wip
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
a52c899c6d
|
Split a FluxTextEncoderInvocation out from the FluxTextToImageInvocation. This has the advantage that we benfit from automatic caching when the prompt isn't changed.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
eeabb7ebe5
|
Make quantized loading fast for both T5XXL and FLUX transformer.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
8b1cef978c
|
Make quantized loading fast.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
152da482cd
|
WIP - experimentation
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
0b77511271
|
Update HF download logic to work for black-forest-labs/FLUX.1-schnell.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
c99cd989c1
|
Update imports for compatibility with bumped diffusers version.
|
2024-08-26 20:17:50 -04:00 |
|
Sergey Borisov
|
17ff8196cb
|
Remove tmp code
|
2024-08-07 22:06:05 -04:00 |
|
Sergey Borisov
|
68f993998a
|
Add support for norm layer
|
2024-08-07 22:06:05 -04:00 |
|
Sergey Borisov
|
7da6120b39
|
Fix LoKR refactor bug
|
2024-08-07 22:06:05 -04:00 |
|
blessedcoolant
|
4f8a4b0f22
|
Merge branch 'main' into depth_anything_v2
|
2024-08-03 00:38:57 +05:30 |
|