Brandon Rising
|
2d185fb766
|
Run ruff
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
2ba9b02932
|
Fix type error in tsc
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
849da67cc7
|
Remove no longer used code in the flux denoise function
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
3ea6c9666e
|
Remove in progress images until we're able to make the valuable
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
cf633e4ef2
|
Only install starter models if not already installed
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
bbf934d980
|
Remove outdated TODO.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
620f733110
|
ruff format
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
635d2f480d
|
ruff
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
70c278c810
|
Remove dependency on flux config files
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
56b9906e2e
|
Setup scaffolding for in progress images and add ability to cancel the flux node
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
a808ce81fd
|
Replace swish() with torch.nn.functional.silu(h). They are functionally equivalent, but in my test VAE deconding was ~8% faster after the change.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
83f82c5ddf
|
Switch the CLIP-L start model to use our hosted version - which is much smaller.
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
101de8c25d
|
Update t5 encoder formats to accurately reflect the quantization strategy and data type
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
3339a4baf0
|
Downgrade revert torch version after removing optimum-qanto, and other minor version-related fixes.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
dff4a88baa
|
Move quantization scripts to a scripts/ subdir.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
a21f6c4964
|
Update docs for T5 quantization script.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
97562504b7
|
Remove all references to optimum-quanto and downgrade diffusers.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
75d8ac378c
|
Update the T5 8-bit quantized starter model to use the BnB LLM.int8() variant.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
b9dd354e2b
|
Fixes to the T5XXL quantization script.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
33c2fbd201
|
Add script for quantizing a T5 model.
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
5063be92bf
|
Switch flux to using its own conditioning field
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
1047584b3e
|
Only import bnb quantize file if bitsandbytes is installed
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
6764dcfdaa
|
Load and unload clip/t5 encoders and run inference separately in text encoding
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
a0bf20bcee
|
Run FLUX VAE decoding in the user's preferred dtype rather than float32. Tested, and seems to work well at float16.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
14ab339b33
|
Move prepare_latent_image_patches(...) to sampling.py with all of the related FLUX inference code.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
25c91efbb6
|
Rename field positive_prompt -> prompt.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
1c1f2c6664
|
Add comment about incorrect T5 Tokenizer size calculation.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
d7c22b3bf7
|
Tidy is_schnell detection logic.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
185f2a395f
|
Make FLUX get_noise(...) consistent across devices/dtypes.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
0c5649491e
|
Mark FLUX nodes as prototypes.
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
94aba5892a
|
Attribute black-forest-labs/flux for much of the flux code
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
34451e5f27
|
added FLUX dev to starter models
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
1f9bdd1a9a
|
Undo changes to the v2 dir of frontend types
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
c27d59baf7
|
Run ruff
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
f130ddec7c
|
Remove automatic install of models during flux model loader, remove no longer used import function on context
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
a0a259eef1
|
Fix max_seq_len field description.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
b66f19d4d1
|
Add docs to the quantization scripts.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
4105a78b83
|
Update load_flux_model_bnb_llm_int8.py to work with a single-file FLUX transformer checkpoint.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
19a68afb3a
|
Fix bug in InvokeInt8Params that was causing it to use double the necessary VRAM.
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
fd68a2475b
|
add better workflow name
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
28ff7ba830
|
add better workflow description
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
5d0b248fdb
|
fix(worker) fix T5 type
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
01a4e0f6ef
|
update default workflow
|
2024-08-26 20:17:50 -04:00 |
|
Mary Hipp
|
91e0731506
|
fix schema
|
2024-08-26 20:17:50 -04:00 |
|
Mary Hipp
|
d1f904d41f
|
tsc and lint fix
|
2024-08-26 20:17:50 -04:00 |
|
Mary Hipp
|
269388c9f4
|
feat(ui): create new field for t5 encoder models in nodes
|
2024-08-26 20:17:50 -04:00 |
|
Mary Hipp
|
b8486379ce
|
fix(ui): pass base/type when installing models, add flux formats to MM badges
|
2024-08-26 20:17:50 -04:00 |
|
Mary Hipp
|
400eb94d3b
|
fix(ui): only exclude flux main models from linear UI dropdown, not model manager list
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
e210c96485
|
add FLUX schnell starter models and submodels as dependenices or adhoc download options
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
5f567f41f4
|
add case for clip embed models in probe
|
2024-08-26 20:17:50 -04:00 |
|
maryhipp
|
5fed573a29
|
update flux_model_loader node to take a T5 encoder from node field instead of hardcoded list, assume all models have been downloaded
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
cfac7c8189
|
Move requantize.py to the quatnization/ dir.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
1787de6836
|
Add docs to the requantize(...) function explaining why it was copied from optimum-quanto.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
ac96f187bd
|
Remove duplicate log_time(...) function.
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
72398350b4
|
More flux loader cleanup
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
df9445c351
|
Various styling and exception type updates
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
87b7a2e39b
|
Switch inheritance class of flux model loaders
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
f7e46622a1
|
Update doc string for import_local_model and remove access_token since it's only usable for local file paths
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
71f18353a9
|
Address minor review comments.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
4228de707b
|
Rename t5Encoder -> t5_encoder.
|
2024-08-26 20:17:50 -04:00 |
|
Mary Hipp
|
b6a05629ef
|
add default workflow for flux t2i
|
2024-08-26 20:17:50 -04:00 |
|
Mary Hipp
|
fbaa820643
|
exclude flux models from main model dropdown
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
db2a2d5e38
|
Some cleanup of the tags and description of flux nodes
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
8ba6e6b1f8
|
Add t5 encoders and clip embeds to the model manager
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
57168d719b
|
Fix styling/lint
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
dee6d2c98e
|
Fix support for 8b quantized t5 encoders, update exception messages in flux loaders
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
e49105ece5
|
Add tqdm progress bar to FLUX denoising.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
0c5e11f521
|
Fix FLUX output image clamping. And a few other minor fixes to make inference work with the full bfloat16 FLUX transformer model.
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
a63f842a13
|
Select dev/schnell based on state dict, use correct max seq len based on dev/schnell, and shift in inference, separate vae flux params into separate config
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
4bd7fda694
|
Install sub directories with folders correctly, ensure consistent dtype of tensors in flux pipeline and vae
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
81f0886d6f
|
Working inference node with quantized bnb nf4 checkpoint
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
2eb87f3306
|
Remove unused param on _run_vae_decoding in flux text to image
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
723f3ab0a9
|
Add nf4 bnb quantized format
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
1bd90e0fd4
|
Run ruff, setup initial text to image node
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
436f18ff55
|
Add backend functions and classes for Flux implementation, Update the way flux encoders/tokenizers are loaded for prompt encoding, Update way flux vae is loaded
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
cde9696214
|
Some UI cleanup, regenerate schema
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
2d9042fb93
|
Run Ruff
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
9ed53af520
|
Run Ruff
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
56fda669fd
|
Manage quantization of models within the loader
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
1d8545a76c
|
Remove changes to v1 workflow
|
2024-08-26 20:17:50 -04:00 |
|
Brandon Rising
|
5f59a828f9
|
Setup flux model loading in the UI
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
1fa6bddc89
|
WIP on moving from diffusers to FLUX
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
d3a5ca5247
|
More improvements for LLM.int8() - not fully tested.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
f01f56a98e
|
LLM.int8() quantization is working, but still some rough edges to solve.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
99b0f79784
|
Clean up NF4 implementation.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
e1eb104345
|
NF4 inference working
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
5c2f95ef50
|
NF4 loading working... I think.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
b63df9bab9
|
wip
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
a52c899c6d
|
Split a FluxTextEncoderInvocation out from the FluxTextToImageInvocation. This has the advantage that we benfit from automatic caching when the prompt isn't changed.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
eeabb7ebe5
|
Make quantized loading fast for both T5XXL and FLUX transformer.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
8b1cef978c
|
Make quantized loading fast.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
152da482cd
|
WIP - experimentation
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
3cf0365a35
|
Make float16 inference work with FLUX on 24GB GPU.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
5870742bb9
|
Add support for 8-bit quantizatino of the FLUX T5XXL text encoder.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
01d8c62c57
|
Make 8-bit quantization save/reload work for the FLUX transformer. Reload is still very slow with the current optimum.quanto implementation.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
55a242b2d6
|
Minor improvements to FLUX workflow.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
45263b339f
|
Got FLUX schnell working with 8-bit quantization. Still lots of rough edges to clean up.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
3319491861
|
Use the FluxPipeline.encode_prompt() api rather than trying to run the two text encoders separately.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
b39031ea53
|
First draft of FluxTextToImageInvocation.
|
2024-08-26 20:17:50 -04:00 |
|
Ryan Dick
|
0b77511271
|
Update HF download logic to work for black-forest-labs/FLUX.1-schnell.
|
2024-08-26 20:17:50 -04:00 |
|