Lincoln Stein
0719a46372
add support for SDXL textual inversion/embeddings
2023-12-01 01:28:28 -05:00
Ryan Dick
bfdef120d1
Re-organize merge_tiles_with_linear_blending(...) to merge rows horizontally first and then vertically. This change achieves slightly more natural blending on the corners where 4 tiles overlap.
2023-11-30 07:53:27 -08:00
Ryan Dick
76b888de17
Add unit tests for merge_tiles_with_linear_blending(...).
2023-11-30 07:53:27 -08:00
Ryan Dick
65a16be299
Add unit tests for calc_tiles_with_overlap(...) and fix a bug in its implementation.
2023-11-30 07:53:27 -08:00
Ryan Dick
29eade4880
Add nodes for tile splitting and merging. The main motivation for these nodes is for use in tiled upscaling workflows.
2023-11-30 07:53:27 -08:00
Damian Stewart
0beb08686c
Add CFG Rescale option for supporting zero-terminal SNR models ( #4335 )
...
* add support for CFG rescale
* fix typo
* move rescale position and tweak docs
* move input position
* implement suggestions from github and discord
* cleanup unused code
* add back dropped FieldDescription
* fix(ui): revert unrelated UI changes
* chore(nodes): bump denoise_latents version 1.4.0 -> 1.5.0
* feat(nodes): add cfg_rescale_multiplier to metadata node
* feat(ui): add cfg rescale multiplier to linear UI
- add param to state
- update graph builders
- add UI under advanced
- add metadata handling & recall
- regen types
* chore: black
* fix(backend): make `StableDiffusionGeneratorPipeline._rescale_cfg()` staticmethod
This doesn't need access to class.
* feat(backend): add docstring for `_rescale_cfg()` method
* feat(ui): update cfg rescale mult translation string
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-30 20:55:20 +11:00
Ryan Dick
693c6cf5e4
Add support for IPAdapterFull models. The changes are based on this upstream PR: https://github.com/tencent-ailab/IP-Adapter/pull/139 .
2023-11-29 15:07:21 -08:00
Lincoln Stein
ecd3dcd5df
Merge branch 'main' into refactor/model-manager-3
2023-11-27 22:15:51 -05:00
Lincoln Stein
ae82df0fda
fix a bunch of type mismatches in the logging module
2023-11-28 09:38:35 +11:00
Steven Frank
e509d719ee
Fix attempt to deserialize on CUDA on Mac
...
Without specifying "cpu", attempts to use non-existent CUDA to deserialize embeddings on macOS, resulting in a warning / failure to load.
2023-11-28 09:24:57 +11:00
Lincoln Stein
a79e814c8d
Merge branch 'main' into refactor/model-manager-3
2023-11-27 16:06:42 -05:00
psychedelicious
7653d21cf5
feat(backend): rename realesrgan class & upscale method
2023-11-28 07:58:22 +11:00
psychedelicious
46a2d83b84
feat(backend): organise realesrgan code, add license
...
- Moved util to own folder
- BSD3 License for RealESRGAN repo added
2023-11-28 07:58:22 +11:00
psychedelicious
2192210910
feat(nodes): remove dependency on realesrgan
...
We used the `RealESRGANer` utility class from the repo. It handled model loading and tiled upscaling logic.
Unfortunately, it hasn't been updated in over a year, had no types, and annoyingly printed to console.
I've adapted the class, cleaning it up a bit and removing the bits that are not relevant for us.
Upscaling functionality is identical.
2023-11-28 07:58:22 +11:00
Lincoln Stein
8ef596eac7
further changes for ruff
2023-11-26 17:13:31 -05:00
Lincoln Stein
8695ad6f59
all features implemented, docs updated, ready for review
2023-11-26 13:18:21 -05:00
Lincoln Stein
8aefe2cefe
import_model and list_install_jobs router APIs written
2023-11-25 21:45:59 -05:00
Lincoln Stein
ec510d34b5
fix model probing for controlnet checkpoint legacy config files
2023-11-25 15:53:22 -05:00
Lincoln Stein
19baea1883
all backend features in place; config scanning is failing on controlnet
2023-11-24 19:37:46 -05:00
Lincoln Stein
80bc9be3ab
make install_path and register_path work; refactor model probing
2023-11-23 23:15:32 -05:00
Lincoln Stein
8c7a7bc897
Merge branch 'main' into refactor/model-manager-3
2023-11-22 22:29:23 -05:00
Lincoln Stein
4aab728590
move name/description logic into model_probe.py
2023-11-22 22:29:02 -05:00
Lincoln Stein
9cf060115d
Merge branch 'main' into refactor/model-manager-3
2023-11-22 22:28:31 -05:00
Lincoln Stein
98a4930a52
add probe support for LCM main models
2023-11-22 14:58:27 -05:00
psychedelicious
1a596a5684
fix(backend): fix unintentional change to import orders
...
- Ignore I001 (isort rules) for this file
- Ignore F401 (unused imports) for this file
2023-11-21 20:22:27 +11:00
psychedelicious
da443973cb
chore: ruff
2023-11-21 20:22:27 +11:00
Lincoln Stein
9ea3126118
start implementation of installer
2023-11-20 23:02:30 -05:00
Lincoln Stein
acc0a29dca
fixed ruff formatting issues
2023-11-13 18:15:17 -05:00
Lincoln Stein
38c1436f02
resolve conflicts; blackify
2023-11-13 18:12:45 -05:00
Lincoln Stein
efbdb75568
implement psychedelicious recommendations as of 13 November
2023-11-13 17:05:01 -05:00
psychedelicious
bc64cde6f9
chore: ruff lint
2023-11-14 07:57:07 +11:00
psychedelicious
4465f97cdf
Merge branch 'main' into refactor/model-manager-2
2023-11-14 07:51:57 +11:00
psychedelicious
04d8f2dfea
fix(backend): fix controlnet zip len
...
Do not use `strict=True` when scaling controlnet conditioning.
When using `guess_mode` (e.g. `more_control` or `more_prompt`), `down_block_res_samples` and `scales` are zipped.
These two objects are of different lengths, so using zip's strict mode raises an error.
In testing, `len(scales) === len(down_block_res_samples) + 1`.
It appears this behaviour is intentional, as the final "extra" item in `scales` is used immediately afterwards.
2023-11-13 15:45:03 +11:00
Lincoln Stein
8c71ff37ae
Update config.py
...
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-12 19:03:39 -05:00
Lincoln Stein
8afe517204
add note about discriminated union and Body() issue; blackified
2023-11-12 16:50:05 -05:00
Lincoln Stein
7ea2a135f1
remove dangling import
2023-11-11 12:24:58 -05:00
Lincoln Stein
af2264b6eb
implement workaround for FastAPI and discriminated unions in Body parameter
2023-11-11 12:22:38 -05:00
Lincoln Stein
2b36565e9e
awkward workaround for double-Annotated in model_record route
2023-11-10 21:32:44 -05:00
Lincoln Stein
cb8cdefd59
Merge branch 'main' into refactor/model-manager-2
2023-11-10 19:24:19 -05:00
Lincoln Stein
f1c846ba5c
blackify
2023-11-10 19:14:29 -05:00
Lincoln Stein
3a6ba236f5
replace _class_map in ModelConfigFactory with a nested discriminated union
2023-11-10 19:14:15 -05:00
psychedelicious
6494e8e551
chore: ruff format
2023-11-11 10:55:40 +11:00
psychedelicious
513fceac82
chore: ruff check - fix pycodestyle
2023-11-11 10:55:33 +11:00
psychedelicious
99a8ebe3a0
chore: ruff check - fix flake8-bugbear
2023-11-11 10:55:28 +11:00
psychedelicious
3a136420d5
chore: ruff check - fix flake8-comprensions
2023-11-11 10:55:23 +11:00
Lincoln Stein
0544917161
multiple small fixes suggested in reviews from psychedelicious and ryan
2023-11-10 18:25:37 -05:00
Lincoln Stein
f1c195afb7
Merge branch 'main' into refactor/model-manager-2
2023-11-10 17:54:28 -05:00
Wubbbi
6001d3d71d
Change pad_to_multiple_of to be 8 for all cases. Add comment about it's temporary status
2023-11-10 17:51:59 -05:00
Wubbbi
8831d1ee41
Update Documentation
2023-11-10 17:51:59 -05:00
Wubbbi
a0be83e370
Update Transformers to 4.34 and fix pad_to_multiple_of
2023-11-10 17:51:59 -05:00
Lincoln Stein
8702a63197
add support for downloading and installing LCM lora diffusers models
2023-11-10 17:51:30 -05:00
blessedcoolant
356b5a41a9
wip: Add LCMScheduler
2023-11-10 06:54:36 -08:00
psychedelicious
2570497d83
fix(installer): fix import of ValidationError
...
It was being imported from a deprecated module
2023-11-10 06:11:15 +11:00
psychedelicious
6aa87f973e
fix(nodes): create app/shared/ module to prevent circular imports
...
We have a number of shared classes, objects, and functions that are used in multiple places. This causes circular import issues.
This commit creates a new `app/shared/` module to hold these shared classes, objects, and functions.
Initially, only `FreeUConfig` and `FieldDescriptions` are moved here. This resolves a circular import issue with custom nodes.
Other shared classes, objects, and functions will be moved here in future commits.
2023-11-09 16:41:55 +11:00
Lincoln Stein
927f8a66e6
Merge branch 'main' into refactor/model-manager-2
2023-11-08 16:46:08 -05:00
Lincoln Stein
6b173cc66f
multiple small stylistic changes requested by reviewers
2023-11-08 16:45:26 -05:00
Lincoln Stein
ce22c0fbaa
sync pydantic and sql field names; merge routes
2023-11-06 18:08:57 -05:00
Kent Keirsey
ff8a8a1963
Merge branch 'main' into feat/nodes/freeu
2023-11-06 09:04:54 -08:00
Kent Keirsey
67f2616d5a
Merge branch 'main' into revert-4923-revert-4914-feat/mix-cnet-t2iadapter
2023-11-06 07:34:51 -08:00
Kent Keirsey
e66d0f7372
Merge branch 'main' into feat/nodes/freeu
2023-11-06 05:39:58 -08:00
Lincoln Stein
edeea5237b
add sql-based model config store and api
2023-11-04 23:03:26 -04:00
Ryan Dick
aa02ebf8f5
Fix model cache gc.collect() condition.
2023-11-04 08:52:10 -04:00
Ryan Dick
fb3d0c4b12
Fix bug in model cache reference count checking.
2023-11-03 13:50:40 -07:00
Ryan Dick
8488ab0134
Reduce frequency that we call gc.collect() in the model cache.
2023-11-03 13:50:40 -07:00
Ryan Dick
875231ed3d
Add reminder to clean up our model cache clearing logic.
2023-11-03 13:50:40 -07:00
Ryan Dick
43b300498f
Remove explicit gc.collect() after transferring models from device to CPU. I'm not sure why this was there in the first place, but it was taking a significant amount of time (up to ~1sec in my tests).
2023-11-03 13:50:40 -07:00
Ryan Dick
e391f3c9a8
Skip torch.nn.Embedding.reset_parameters(...) when loading a text encoder model.
2023-11-02 19:41:33 -07:00
Ryan Dick
6e7a3f0546
(minor) Fix static checks and typo.
2023-11-02 19:20:37 -07:00
Ryan Dick
4a683cc669
Add a app config parameter to control the ModelCache logging behavior.
2023-11-02 19:20:37 -07:00
Ryan Dick
3781e56e57
Add log_memory_usage param to ModelCache.
2023-11-02 19:20:37 -07:00
Ryan Dick
8ff49109a8
Update get_pretty_snapshot_diff(...) to handle None-snapshots.
2023-11-02 19:20:37 -07:00
Ryan Dick
bac2a757e8
Replace deepcopy with a pickle roundtrip in apply_ti(...) to improve speed.
2023-11-02 19:05:24 -07:00
Ryan Dick
fa7f6a6a10
Further tidying of LoRA patching. Revert some changes that didn't end up being important under the constraint that calculations are done on the same device as the model.
2023-11-02 10:03:17 -07:00
Ryan Dick
61b17c475a
Add TODO note about improving _resolve_lora_key(...).
2023-11-02 10:03:17 -07:00
Ryan Dick
379d68f595
Patch LoRA on device when model is already on device.
2023-11-02 10:03:17 -07:00
Ryan Dick
545c811bf1
Remove device and dtype members from LoRAModelRaw, they can too easily get out-of-sync with the underlying layer states.
2023-11-02 10:03:17 -07:00
Ryan Dick
2ba5b44ec4
Remove unused _lora_forward_hook(...).
2023-11-02 10:03:17 -07:00
Ryan Dick
7f4ce518b7
auto-format lora.py
2023-11-02 10:03:17 -07:00
Lincoln Stein
5c9046580f
Merge branch 'main' into bugfix/textual-inversion
2023-10-24 14:17:43 -04:00
Lincoln Stein
6cbc69f3b7
support conversion of controlnets from safetensors to diffusers
2023-10-23 22:06:10 -04:00
Lincoln Stein
3546c41f4a
close #4975
2023-10-23 18:48:14 -04:00
d8ahazard
fdf02c33d0
Catch generic model errors
...
Prevent the app from dying on invalid models.
2023-10-19 07:28:33 +11:00
Ryan Dick
a078efc0f2
Merge branch 'main' into ryan/multi-image-ip
2023-10-18 08:59:12 -04:00
psychedelicious
252c9a5f5a
fix(backend): fix nsfw/watermarker util types
2023-10-18 09:08:13 +11:00
Kent Keirsey
b7555ddae8
Revert "Revert "chore: lint""
...
This reverts commit 38e7eb8878
2023-10-17 11:59:19 -04:00
Kent Keirsey
8afc47018b
Revert "Revert "Cleaning up (removing diagnostic prints)""
...
This reverts commit 6e697b7b6f
2023-10-17 11:59:19 -04:00
Kent Keirsey
a97ec88e06
Revert "Revert "Changes to _apply_standard_conditioning_sequentially() and _apply_cross_attention_controlled_conditioning() to reflect changes to T2I-Adapter implementation to allow usage of T2I-Adapter and ControlNet at the same time.""
...
This reverts commit c04fb451ee
2023-10-17 11:59:19 -04:00
Kent Keirsey
282d36b640
Revert "Revert "Fixing some var and arg names.""
...
This reverts commit 58a0709c1e
2023-10-17 11:59:19 -04:00
psychedelicious
58a0709c1e
Revert "Fixing some var and arg names."
...
This reverts commit f11ba81a8d
2023-10-17 11:59:11 -04:00
psychedelicious
c04fb451ee
Revert "Changes to _apply_standard_conditioning_sequentially() and _apply_cross_attention_controlled_conditioning() to reflect changes to T2I-Adapter implementation to allow usage of T2I-Adapter and ControlNet at the same time."
...
This reverts commit 378689a519
2023-10-17 11:59:11 -04:00
psychedelicious
6e697b7b6f
Revert "Cleaning up (removing diagnostic prints)"
...
This reverts commit 06f8a3276d
2023-10-17 11:59:11 -04:00
psychedelicious
38e7eb8878
Revert "chore: lint"
...
This reverts commit fff29d663d
2023-10-17 11:59:11 -04:00
psychedelicious
fff29d663d
chore: lint
2023-10-17 19:42:06 +11:00
user1
06f8a3276d
Cleaning up (removing diagnostic prints)
2023-10-17 19:42:06 +11:00
user1
378689a519
Changes to _apply_standard_conditioning_sequentially() and _apply_cross_attention_controlled_conditioning() to reflect changes to T2I-Adapter implementation to allow usage of T2I-Adapter and ControlNet at the same time.
...
Also, the PREVIOUS commit (@8d3885d, which was already pushed to github repo) was wrongly commented, but too late to fix without a force push or other mucking that I'm reluctant to do. That commit is actually the one that has all the changes to diffusers_pipeline.py to use additional arg down_intrablock_additional_residuals (introduced in diffusers PR https://github.com/huggingface/diffusers/pull/5362 ) to detangle T2I-Adapter from ControlNet inputs to main UNet.
2023-10-17 19:42:06 +11:00
user1
f11ba81a8d
Fixing some var and arg names.
2023-10-17 19:42:06 +11:00
Millun Atluri
001bba1719
Merge branch 'main' into feat/nodes/freeu
2023-10-17 15:58:00 +11:00
psychedelicious
c238a7f18b
feat(api): chore: pydantic & fastapi upgrade
...
Upgrade pydantic and fastapi to latest.
- pydantic~=2.4.2
- fastapi~=103.2
- fastapi-events~=0.9.1
**Big Changes**
There are a number of logic changes needed to support pydantic v2. Most changes are very simple, like using the new methods to serialized and deserialize models, but there are a few more complex changes.
**Invocations**
The biggest change relates to invocation creation, instantiation and validation.
Because pydantic v2 moves all validation logic into the rust pydantic-core, we may no longer directly stick our fingers into the validation pie.
Previously, we (ab)used models and fields to allow invocation fields to be optional at instantiation, but required when `invoke()` is called. We directly manipulated the fields and invocation models when calling `invoke()`.
With pydantic v2, this is much more involved. Changes to the python wrapper do not propagate down to the rust validation logic - you have to rebuild the model. This causes problem with concurrent access to the invocation classes and is not a free operation.
This logic has been totally refactored and we do not need to change the model any more. The details are in `baseinvocation.py`, in the `InputField` function and `BaseInvocation.invoke_internal()` method.
In the end, this implementation is cleaner.
**Invocation Fields**
In pydantic v2, you can no longer directly add or remove fields from a model.
Previously, we did this to add the `type` field to invocations.
**Invocation Decorators**
With pydantic v2, we instead use the imperative `create_model()` API to create a new model with the additional field. This is done in `baseinvocation.py` in the `invocation()` wrapper.
A similar technique is used for `invocation_output()`.
**Minor Changes**
There are a number of minor changes around the pydantic v2 models API.
**Protected `model_` Namespace**
All models' pydantic-provided methods and attributes are prefixed with `model_` and this is considered a protected namespace. This causes some conflict, because "model" means something to us, and we have a ton of pydantic models with attributes starting with "model_".
Forunately, there are no direct conflicts. However, in any pydantic model where we define an attribute or method that starts with "model_", we must tell set the protected namespaces to an empty tuple.
```py
class IPAdapterModelField(BaseModel):
model_name: str = Field(description="Name of the IP-Adapter model")
base_model: BaseModelType = Field(description="Base model")
model_config = ConfigDict(protected_namespaces=())
```
**Model Serialization**
Pydantic models no longer have `Model.dict()` or `Model.json()`.
Instead, we use `Model.model_dump()` or `Model.model_dump_json()`.
**Model Deserialization**
Pydantic models no longer have `Model.parse_obj()` or `Model.parse_raw()`, and there are no `parse_raw_as()` or `parse_obj_as()` functions.
Instead, you need to create a `TypeAdapter` object to parse python objects or JSON into a model.
```py
adapter_graph = TypeAdapter(Graph)
deserialized_graph_from_json = adapter_graph.validate_json(graph_json)
deserialized_graph_from_dict = adapter_graph.validate_python(graph_dict)
```
**Field Customisation**
Pydantic `Field`s no longer accept arbitrary args.
Now, you must put all additional arbitrary args in a `json_schema_extra` arg on the field.
**Schema Customisation**
FastAPI and pydantic schema generation now follows the OpenAPI version 3.1 spec.
This necessitates two changes:
- Our schema customization logic has been revised
- Schema parsing to build node templates has been revised
The specific aren't important, but this does present additional surface area for bugs.
**Performance Improvements**
Pydantic v2 is a full rewrite with a rust backend. This offers a substantial performance improvement (pydantic claims 5x to 50x depending on the task). We'll notice this the most during serialization and deserialization of sessions/graphs, which happens very very often - a couple times per node.
I haven't done any benchmarks, but anecdotally, graph execution is much faster. Also, very larges graphs - like with massive iterators - are much, much faster.
2023-10-17 14:59:25 +11:00
Ryan Dick
3079c75a60
(minor) Make it more clear that shape annotations are just comments and not commented lines of code.
2023-10-16 08:35:32 -04:00
psychedelicious
53b6f0dc73
Merge branch 'main' into ryan/multi-image-ip
2023-10-16 17:16:10 +11:00
blessedcoolant
2c6772f92f
Merge branch 'main' into bugfix/trim-whitespace-from-urls
2023-10-15 17:41:41 +05:30
blessedcoolant
c9d95e5758
Merge branch 'main' into bugfix/ignore-dot-directories-on-model-scan
2023-10-15 17:23:02 +05:30
psychedelicious
48626c40fd
fix(backend): handle systems with glibc < 2.33
...
`mallinfo2` is not available on `glibc` < 2.33.
On these systems, we successfully load the library but get an `AttributeError` on attempting to access `mallinfo2`.
I'm not sure if the old `mallinfo` will work, and not sure how to install it safely to test, so for now we just handle the `AttributeError`.
This means the enhanced memory snapshot logic will be skipped for these systems, which isn't a big deal.
2023-10-15 07:56:55 +11:00
Ryan Dick
8464450a53
Add support for multi-image IP-Adapter.
2023-10-14 12:50:33 -04:00
Lincoln Stein
a1001b6d10
Merge branch 'main' into bugfix/ignore-dot-directories-on-model-scan
2023-10-14 10:37:55 -04:00
Lincoln Stein
15cabc4968
Possibly closes #4815
2023-10-12 23:37:05 -04:00
Lincoln Stein
29c3f49182
enable the ram cache slider in invokeai-configure
2023-10-12 23:04:16 -04:00
Lincoln Stein
21d5969942
strip leading and trailing quotes as well as whitespace
2023-10-12 22:35:02 -04:00
Lincoln Stein
52274087f3
close #4536
2023-10-12 21:24:07 -04:00
psychedelicious
9646157ad5
fix: fix test imports
2023-10-12 12:15:06 -04:00
Ryan Dick
40f9e49b5e
Demote model cache logs from warning to debug based on the conversation here: https://discord.com/channels/1020123559063990373/1049495067846524939/1161647290189090816
2023-10-11 12:02:46 -04:00
psychedelicious
15b33ad501
feat(nodes): add freeu support
...
Add support for FreeU. See:
- https://huggingface.co/docs/diffusers/main/en/using-diffusers/freeu
- https://github.com/ChenyangSi/FreeU
Implementation:
- `ModelPatcher.apply_freeu()` handles the enabling freeu (which is very simple with diffusers).
- `FreeUConfig` model added to hold the hyperparameters.
- `freeu_config` added as optional sub-field on `UNetField`.
- `FreeUInvocation` added, works like LoRA - chain it to add the FreeU config to the UNet
- No support for model-dependent presets, this will be a future workflow editor enhancement
Closes #4845
2023-10-11 13:49:28 +11:00
Ryan Dick
61242bf86a
Fix bug in skip_torch_weight_init() where the original behavior of torch.nn.Conv*d modules wasn't being restored correctly.
2023-10-10 10:05:50 -04:00
Ryan Dick
58b56e9b1e
Add a skip_torch_weight_init() context manager to improve model load times (from disk).
2023-10-09 14:12:56 -04:00
Lincoln Stein
fe0cf2c160
remove hardcoded subfolder name from model downloader
2023-10-08 17:45:39 -04:00
Ryan Dick
971ccfb081
Refactor multi-IP-Adapter to clean up the interface around changing scales.
2023-10-06 20:43:43 -04:00
Ryan Dick
43a3c3c7ea
Fix typo in setting IP-Adapter scales.
2023-10-06 20:43:43 -04:00
Ryan Dick
4df1cdb34d
Tidy _prepare_attention_processors(...) logic.
2023-10-06 20:43:43 -04:00
Ryan Dick
d8d0c9af09
Fix handling of scales with multiple IP-Adapters.
2023-10-06 20:43:43 -04:00
Ryan Dick
26b91a538a
Fixes to get IP-Adapter tests working with new multi-IP-Adapter support.
2023-10-06 20:43:43 -04:00
Ryan Dick
7ca456d674
Update IP-Adapter model to enable running multiple IP-Adapters at once. (Not tested yet.)
2023-10-06 20:43:43 -04:00
Ryan Dick
78828b6b9c
WIP - Accept a list of IPAdapterFields in DenoiseLatents.
2023-10-06 20:43:43 -04:00
Ryan Dick
4f97bd4418
Merge branch 'main' into ryan/model-tests
2023-10-06 19:47:28 -04:00
Ryan Dick
e0e001758a
Remove @slow decorator in favor of @pytest.mark.slow.
2023-10-06 18:26:06 -04:00
Ryan Dick
096d195d6e
Merge branch 'main' into ryan/model-cache-logging-only
2023-10-06 09:52:45 -04:00
Ryan Dick
1c8b1fbc53
POC of a test that depends on models.
2023-10-05 15:35:58 -04:00
Ryan Dick
78377469db
Add support for T2I-Adapter in node workflows ( #4612 )
...
* Bump diffusers to 0.21.2.
* Add T2IAdapterInvocation boilerplate.
* Add T2I-Adapter model to model-management.
* (minor) Tidy prepare_control_image(...).
* Add logic to run the T2I-Adapter models at the start of the DenoiseLatentsInvocation.
* Add logic for applying T2I-Adapter weights and accumulating.
* Add T2IAdapter to MODEL_CLASSES map.
* yarn typegen
* Add model probes for T2I-Adapter models.
* Add all of the frontend boilerplate required to use T2I-Adapter in the nodes editor.
* Add T2IAdapterModel.convert_if_required(...).
* Fix errors in T2I-Adapter input image sizing logic.
* Fix bug with handling of multiple T2I-Adapters.
* black / flake8
* Fix typo
* yarn build
* Add num_channels param to prepare_control_image(...).
* Link to upstream diffusers bugfix PR that currently requires a workaround.
* feat: Add Color Map Preprocessor
Needed for the color T2I Adapter
* feat: Add Color Map Preprocessor to Linear UI
* Revert "feat: Add Color Map Preprocessor"
This reverts commit a1119a00bfbd8a9b82d8
2023-10-05 16:29:16 +11:00
Ryan Dick
fbe6452c45
Add support for IPAdapterPlusXL based on 6219530507.
2023-10-04 22:35:17 -04:00
Ryan Dick
2479a59e5e
Re-enable garbage collection in model cache MemorySnapshots.
2023-10-03 15:18:47 -04:00
Ryan Dick
7d0ac2c36d
(minor) clean up typos.
2023-10-03 15:00:03 -04:00
Ryan Dick
519b892f0c
Add unit test for Struct_mallinfo2.__str__()
2023-10-03 14:25:34 -04:00
Ryan Dick
763dcacfd3
Add unit test for get_pretty_snapshot_diff(...).
2023-10-03 14:25:34 -04:00
Ryan Dick
22a84930f6
Disable garbage collection in ModelCache calls to MemorySnapshot in order minimize snapshot overhead.
2023-10-03 14:25:34 -04:00
Ryan Dick
d64e17e043
Add README with info about glib memory fragmentation caused by the model cache.
2023-10-03 14:25:34 -04:00
Ryan Dick
ba54277011
Catch a more specific exception in environments that do not have a libc shared library.
2023-10-03 14:25:34 -04:00
Ryan Dick
5915a4a51c
Minor fixes.
2023-10-03 14:25:34 -04:00
Ryan Dick
4580ba0d87
Remove logic to update model cache size estimates dynamically.
2023-10-03 14:25:34 -04:00
Ryan Dick
b9fd2e9e76
Improve get_pretty_snapshot_diff(...) message formatting.
2023-10-03 14:25:34 -04:00
Ryan Dick
75b65597af
Add malloc info to MemorySnapshot.
2023-10-03 14:25:34 -04:00
Ryan Dick
2a3c0ab5d2
Move MemorySnapshot to its own file.
2023-10-03 14:25:34 -04:00
Ryan Dick
7d61373b82
Add LibcUtil class.
2023-10-03 14:25:34 -04:00
Ryan Dick
7d65555a5a
Fix type error in torch device comparison.
2023-10-03 14:25:34 -04:00
Ryan Dick
123f2b2dbc
Update cache model size estimates based on changes in VRAM when moving models to/from CUDA.
2023-10-03 14:25:34 -04:00
Ryan Dick
1e4e42556e
Update model cache device comparison to treat 'cuda' and 'cuda:0' as the same device type.
2023-10-03 14:25:34 -04:00
Ryan Dick
1f6699ac43
Consolidate all model.to(...) calls in the model cache to use a utility function with better logging.
2023-10-03 14:25:34 -04:00
Ryan Dick
ace8665411
Add warning log if moving a model from cuda to cpu causes unexpected change in VRAM usage.
2023-10-03 14:25:34 -04:00
Ryan Dick
7fa5bae8fd
Add warning log if moving model from RAM to VRAM causes an unexpected change in VRAM usage.
2023-10-03 14:25:34 -04:00
Ryan Dick
f9faca7c91
Add warning log if model mis-reports its required cache memory before load from disk.
2023-10-03 14:25:34 -04:00
Ryan Dick
594fd3ba6d
Add debug logging of changes in RAM and VRAM for all model cache operations.
2023-10-03 14:25:34 -04:00
Ryan Dick
44d68f5ed5
Auto-format model_cache.py.
2023-10-03 14:25:34 -04:00
psychedelicious
045d3f6139
chore: flake8
2023-09-28 13:49:31 +10:00
Lincoln Stein
f6bbcd0589
remove dangling debug statement
2023-09-27 22:26:26 -04:00
Lincoln Stein
ada22a799e
remove dangling debug statement
2023-09-27 22:26:06 -04:00
Lincoln Stein
034af2d9f8
enable downloading from subfolders for repo_ids
2023-09-27 22:11:56 -04:00
Ryan Dick
0fc14afcf0
Merge branch 'main' into bug/ip-adapter-calc-size
2023-09-27 09:42:51 -04:00
Lincoln Stein
3fd27b1aa9
run correct version of black
2023-09-26 08:03:34 -04:00
Lincoln Stein
d59e534cad
use heuristic to select RAM cache size during headless install; blackified
2023-09-26 08:03:34 -04:00
Ryan Dick
399ebe443e
Fix IP-Adapter calculation of memory footprint.
2023-09-25 18:28:10 -04:00
Lincoln Stein
43ebd68313
Merge branch 'main' into install/install-ip-adapters
2023-09-24 23:19:25 -04:00
Lincoln Stein
912087e4dc
blackify
2023-09-24 19:00:38 -04:00
Lincoln Stein
593fb95213
ip_adapter_sd15 & its encoder will now be installed by default during headless install
2023-09-24 19:00:21 -04:00
Lincoln Stein
a1d9e6b871
Merge branch 'main' into bugfix/probe_ip_adapter
2023-09-24 15:39:43 -04:00
Lincoln Stein
e34e6d6e80
enable v_prediction for sd-1 models
2023-09-24 12:22:29 -04:00
Lincoln Stein
86cb53342a
fix probing for ip_adapter folders
2023-09-23 22:32:03 -04:00
Lincoln Stein
25a71a1791
Merge branch 'main' into refactor/rename-get-logger
2023-09-23 14:49:07 -07:00
Ryan Dick
bf62553150
(minor) Update documentation to reflect that a bug was fixed in InvokeAI/ip_adapter_sdxl_vit_h by e178288fb6
2023-09-20 20:18:33 -04:00
Lincoln Stein
0960518088
add techjedi's database maintenance script
2023-09-20 17:46:49 -04:00
Kevin Turner
b1b5f70ea6
Merge branch 'main' into feat/taesd
2023-09-20 12:54:17 -07:00
Kevin Turner
e487bcd0f7
feat(model management): guess whether a VAE is for SDXL based on its name
2023-09-20 12:07:12 -07:00
Kevin Turner
e0f8274f49
feat(model management): guess whether a VAE is for SDXL based on its name
2023-09-20 12:06:55 -07:00
Lincoln Stein
0420874f56
reimplement the old invokeai-metadata command
2023-09-20 13:49:29 -04:00
Kevin Turner
f222b871e9
Merge remote-tracking branch 'origin/main' into feat/taesd
...
# Conflicts:
# invokeai/backend/model_management/model_probe.py
2023-09-20 10:46:55 -07:00
Ryan Dick
92b49e45bb
Address flake8 error.
2023-09-18 16:33:16 -04:00
Ryan Dick
b05b8ef677
Switch to using torch 2.0 attention for IP-Adapter (more memory-efficient).
2023-09-18 16:30:53 -04:00
blessedcoolant
e0dddbd38e
chore: fix isort issues
2023-09-17 12:13:03 +12:00
blessedcoolant
b7773c9962
chore: black & lint fixes
2023-09-17 12:00:21 +12:00
user1
c48e648cbb
Added per-step setting of IP-Adapter weights (for param easing, etc.)
2023-09-16 12:36:16 -07:00
user1
7ee13879e3
Added check in IP-Adapter to avoid begin/end step percent handling if use of IP-Adapter is already turned off due to potential clash with other cross attention control.
2023-09-16 09:29:50 -07:00
user1
ced297ed21
Initial implementation of IP-Adapter "begin_step_percent" and "end_step_percent" for controlling on which steps IP-Adapter is applied in the denoising loop.
2023-09-16 08:24:12 -07:00
Ryan Dick
343df03a92
isort
2023-09-15 13:18:00 -04:00
Ryan Dick
b57acb7353
Merge branch 'main' into feat/ip-adapter
2023-09-15 13:15:25 -04:00
Kent Keirsey
afe9756667
Merge branch 'main' into feat/taesd
2023-09-15 12:19:19 -04:00
Sergey Borisov
ff3150a818
Update lora hotfix to new diffusers version(scale argument added)
2023-09-15 12:19:01 -04:00
Ryan Dick
16664da5b6
black
2023-09-14 23:49:02 -04:00
Ryan Dick
c104807201
Update list of supported IP-Adapters.
2023-09-14 23:43:19 -04:00
Ryan Dick
990ce9a1da
Lookup IP-Adapter linked image encoder from disk instead of storing in model config metadata.
2023-09-14 23:06:57 -04:00
Ryan Dick
c2f074dc2f
Fix python static checks.
2023-09-14 16:48:47 -04:00
Ryan Dick
fca60862e2
Add README.md describing IP-Adapter model formats.
2023-09-14 16:02:07 -04:00
Ryan Dick
94c186bb4c
Fix bug in IPAdapter.to(...).
2023-09-14 15:45:25 -04:00
Ryan Dick
a22c8cb3a1
Improve robustness of check for IPAdapter vs IPAdapterPlus.
2023-09-14 15:25:41 -04:00
Ryan Dick
781e8521d5
Eliminate the need for IPAdapter.initialize().
2023-09-14 15:02:59 -04:00
Ryan Dick
d114d0ba95
Remove need for the image_encoder param in IPAdapter.initialize().
2023-09-14 14:14:35 -04:00
Ryan Dick
388554448a
Add CLIP Vision model to IP-Adapter info and use this to infer which model to use.
2023-09-14 11:57:53 -04:00
Ryan Dick
d5160648d0
Add support for downloading IP-Adapter models from HF.
2023-09-14 11:18:43 -04:00
Ryan Dick
6d0ea42a94
Get CLIPVision model download from HF working.
2023-09-14 09:54:10 -04:00
Ryan Dick
2c1100509f
Add BaseModelType.Any to be used by CLIPVisionModel.
2023-09-14 08:19:55 -04:00
Ryan Dick
c34b359c36
(minor) Remove duplicate TODO.
2023-09-13 21:25:20 -04:00
Ryan Dick
77d135967f
Update IPAdapterModel to respect requested torch_dtype.
2023-09-13 21:06:42 -04:00
Ryan Dick
ebf26687cb
(minor) Remove unnecessary TODO.
2023-09-13 21:03:42 -04:00
Ryan
2f5e923008
Removed duplicate import in model_cache.py
2023-09-13 19:33:43 -04:00
Ryan
b7296000e4
made MPS calls conditional on MPS actually being the chosen device with backend available
2023-09-13 19:33:43 -04:00
Ryan
fab055995e
Add empty_cache() for MPS hardware.
2023-09-13 19:33:43 -04:00
Ryan Dick
1c8991a3df
Use CLIPVisionModel under model management for IP-Adapter.
2023-09-13 19:10:02 -04:00
Ryan Dick
3d52656176
Add CLIPVisionModel to model management.
2023-09-13 17:14:20 -04:00
Kevin Turner
090db1ab3a
Merge remote-tracking branch 'origin/main' into feat/taesd
2023-09-13 09:17:53 -07:00
Ryan Dick
3ee9a21647
Initial (barely) working version of IP-Adapter model management.
2023-09-13 08:27:24 -04:00
Ryan Dick
0d823901ef
Add IPAdapter to model_management __init__.py
2023-09-13 08:27:24 -04:00
Ryan Dick
7ee55489bb
Improve model search warning messages.
2023-09-13 08:27:24 -04:00
Ryan Dick
163ece9aee
Initial skeleton for IPAdapter model management.
2023-09-13 08:27:24 -04:00
Martin Kristiansen
e467ca7f1b
Apply black, isort, flake8
2023-09-12 13:01:58 -04:00
Martin Kristiansen
e88d7c242f
isort wip 3
2023-09-12 13:01:58 -04:00
Martin Kristiansen
caea6d11c6
isort wip 2
2023-09-12 13:01:58 -04:00
Millun Atluri
88db094cf2
Merge branch 'main' into feat/taesd
2023-09-11 22:11:25 +10:00
Ryan Dick
50a0691514
flake8
2023-09-08 18:05:31 -04:00
Ryan Dick
a255624984
black
2023-09-08 17:55:23 -04:00
Ryan Dick
2630fe3608
Remove unused ip_adapter/utils.py file.
2023-09-08 16:25:34 -04:00
Ryan Dick
6ca6cf713c
Tidy IPAdapter. Add types, improve field/method naming.
2023-09-08 16:00:58 -04:00
Ryan Dick
3f7d5b4e0f
Remove redundant IPAdapterXL class.
2023-09-08 15:46:10 -04:00
Ryan Dick
91596d9527
Re-factor IPAdapter to patch UNet in a context manager.
2023-09-08 15:39:22 -04:00
Ryan Dick
d669f0855d
Comment unused IPAdapter generate(...) methods.
2023-09-08 13:12:42 -04:00
Ryan Dick
b2d5b53b5f
Pass IP-Adapter conditioning via cross_attention_kwargs instead of concatenating to the text embedding. This avoids interference with other features that manipulate the text embedding (e.g. long prompts).
2023-09-08 11:47:36 -04:00
Ryan Dick
ddc148b70b
Move ConditioningData and its field classes to their own file. This will allow new conditioning types to be added more cleanly without introducing circular dependencies.
2023-09-08 11:00:11 -04:00
Ryan Dick
c2d43f007b
Specify the image_embedding_len in the IPAttnProcessor rather than the text embedding length. This enables the IPAttnProcessor to handle text embeddings of varying lengths.
2023-09-07 18:20:21 -04:00
Ryan Dick
7703bf2ca1
Delete IP-Adapter copies of AttnProcessor and AttnProcessor2_0, which were unmodified from diffusers.
2023-09-07 15:00:13 -04:00
Millun Atluri
b5e1ba34b3
Merge branch 'main' into refactor/rename-get-logger
2023-09-07 23:19:59 +10:00
Ryan Dick
23fdf0156f
Clean up IP-Adapter in diffusers_pipeline.py - WIP
2023-09-06 20:42:20 -04:00
Ryan Dick
d776e0a0a9
Split ControlField and IpAdapterField.
2023-09-06 17:03:37 -04:00
Lincoln Stein
c54c1f603b
Merge branch 'main' into bugfix/set-vram-on-macs
2023-09-05 07:09:39 -04:00
blessedcoolant
f44496a579
Merge branch 'main' into feat/ip-adapter
2023-09-05 15:22:15 +12:00
psychedelicious
95ecb1a0c1
fix(ip_adapter): add None to types
2023-09-05 12:30:00 +10:00
blessedcoolant
a9fafad5b5
chore: sync, lint & update
2023-09-05 14:17:23 +12:00
Lincoln Stein
d2e327add9
install models/core/misc/lama/lama.pt
2023-09-04 21:54:40 -04:00
psychedelicious
58aa159a50
fix(backend): fix remaining instances of getLogger()
2023-09-05 10:43:30 +10:00
psychedelicious
d8f7c19030
Merge branch 'main' into refactor/rename-get-logger
2023-09-05 10:37:53 +10:00
blessedcoolant
3ac68cde66
chore: flake8 cleanup
2023-09-05 12:07:12 +12:00
blessedcoolant
65a76a086b
cleanup: Some basic cleanup
2023-09-05 11:54:28 +12:00
blessedcoolant
07381e5a26
cleanup: merge conflicts
2023-09-05 11:37:12 +12:00
blessedcoolant
6bb378a101
Merge branch 'main' into feat/ip-adapter
2023-09-05 11:35:19 +12:00
blessedcoolant
917c2c480e
Merge branch 'main' into lama-infill
2023-09-03 23:16:34 +12:00
blessedcoolant
f1f7778e73
Merge branch 'main' into chore/clean-up-unused-files
2023-09-03 02:59:31 +12:00
Lincoln Stein
c965d3eb6b
Merge branch 'main' into bugfix/set-vram-on-macs
2023-09-02 10:08:13 -04:00
blessedcoolant
ec09e21fc2
Merge branch 'main' into lama-infill
2023-09-02 23:02:38 +12:00
Kevin Turner
7df67d077a
Merge branch 'main' into feat/taesd
2023-09-01 22:18:40 -07:00
Lincoln Stein
6f6d920686
[Feature] Support the XL inpainting model ( #4431 )
...
* add StableDiffusionXLInpaintPipeline to probe list
* add StableDiffusionXLInpaintPipeline to probe list
* Blackified (?)
---------
Authored-by: Lincoln Stein <lstein@gmail.com>
Mucked about with to get it merged by: Kent Keirsey <31807370+hipsterusername@users.noreply.github.com>
2023-09-01 22:58:14 -04:00
psychedelicious
4b78deba92
Merge branch 'main' into bugfix/set-vram-on-macs
2023-09-02 11:33:20 +10:00
psychedelicious
b761807219
Merge branch 'main' into feat/ip-adapter
2023-09-02 11:31:08 +10:00
psychedelicious
9cb04f6f80
chore: remove unused files
2023-09-02 11:12:19 +10:00
blessedcoolant
48484e9fc8
Merge branch 'main' into lama-infill
2023-09-02 11:08:31 +12:00
blessedcoolant
a12fbc7406
chore: black fix
2023-09-02 10:51:53 +12:00
Sergey Borisov
5151798a16
Cleanup memory after model run
2023-09-01 20:50:39 +03:00
