Commit Graph

1796 Commits

Author SHA1 Message Date
Brandon Rising
8ba4b2a150 Run ruff 2024-03-08 15:36:14 -05:00
Brandon Rising
df12e12e09 Run ruff 2024-03-08 15:36:14 -05:00
Brandon Rising
e52274ecac Experiment with using absolute paths within model management 2024-03-08 15:36:14 -05:00
psychedelicious
a10dccdd43 fix(mm): fix bug in control adapter probe default settings
Wasn't checking for matches correctly.
2024-03-08 12:44:58 -05:00
psychedelicious
50bb9a6b41 fix(mm): remove default settings from IP adapter config 2024-03-08 12:44:58 -05:00
psychedelicious
13bb3c5e15 feat(mm): add control adapter default settings while probing 2024-03-08 12:44:58 -05:00
psychedelicious
80c2a4b925 feat(mm): add AnyDefaultSettings union 2024-03-08 12:44:58 -05:00
psychedelicious
8ce485b036 feat(mm): add default settings for control adapters
Only includes `preprocessor` at this time.
2024-03-08 12:44:58 -05:00
psychedelicious
6fc3e86061 tidy(mm): only main models get the main default settings 2024-03-08 12:44:58 -05:00
Brandon Rising
b6065d6328 Run ruff with newest version of ruff 2024-03-08 13:59:59 +11:00
Brandon Rising
04229f4a21 Run ruff 2024-03-08 13:59:59 +11:00
Brandon Rising
952d97741e Remove civit ai from tests and documentation 2024-03-08 13:59:59 +11:00
Brandon Rising
d1f859a446 Remove civit AI model install resources 2024-03-08 13:59:59 +11:00
psychedelicious
bbcbcd9b63 fix(mm): only loras and main models get trigger_phrases 2024-03-08 12:26:35 +11:00
Jennifer Player
4af5a09a68 cleanup 2024-03-06 21:57:41 -05:00
Jennifer Player
aa88fadc30 use webp images 2024-03-06 21:57:41 -05:00
Jennifer Player
8411029d93 get model image url from model config, added thumbnail formatting for images 2024-03-06 21:57:41 -05:00
Jennifer Player
8a68355926 got model images displaying, still need to clean up types and unused code 2024-03-06 21:57:41 -05:00
Jennifer Player
2f6964bfa5 fetching model image, still not working 2024-03-06 21:57:41 -05:00
psychedelicious
132790eebe tidy(nodes): use canonical capitalizations 2024-03-07 10:56:59 +11:00
psychedelicious
afd9ae7712 tidy(mm): remove convenience methods from high level model manager service
These were added as a hold-me-over for the nodes API changes, no longer needed. A followup commit will fix the nodes API to not rely on these.
2024-03-07 10:56:59 +11:00
Brandon Rising
46f32c5e3c Remove references to the no longer existing invokeai.app.services.model_metadata package 2024-03-05 19:58:25 -05:00
psychedelicious
e866d90ab2 tidy(mm): remove unused method on probe 2024-03-05 23:50:19 +11:00
psychedelicious
e8797787cf fix(mm): fix incorrect calls to update_model 2024-03-05 23:50:19 +11:00
psychedelicious
7c9128b253 tidy(mm): use canonical capitalization for all model-related enums, classes
For example, "Lora" -> "LoRA", "Vae" -> "VAE".
2024-03-05 23:50:19 +11:00
psychedelicious
3030a34b88 fix(mm): make type and format required in openapi schema for model config 2024-03-05 23:50:19 +11:00
psychedelicious
a8b6635050 fix(mm): make key required in openapi schema for model config 2024-03-05 23:50:19 +11:00
psychedelicious
5551cf8ac4 feat(mm): revise update_model to use ModelRecordChanges 2024-03-05 23:50:19 +11:00
psychedelicious
37b969d339 tidy(mm): add default_settings to model config 2024-03-05 23:50:19 +11:00
psychedelicious
c953e61294 tidy(mm): "trigger_words" -> "trigger_phrases" 2024-03-05 23:50:19 +11:00
psychedelicious
93dd3c848e tidy(mm): remove unused code in select_hf_files.py 2024-03-05 23:50:19 +11:00
psychedelicious
3391c19926 chore: ruff 2024-03-05 23:50:19 +11:00
psychedelicious
0f60b1ced4 fix(mm): use .value for model config discriminators
There is a breaking change in python 3.11 related to how enums with `str` as a mixin are formatted. This appears to have not caused any grief for us until now.

Re-jigger the discriminator setup to use `.value` so everything works on both python 3.10 and 3.11.
2024-03-05 23:50:19 +11:00
psychedelicious
44c40d7d1a refactor(mm): remove unused metadata logic, fix tests
- Metadata is merged with the config. We can simplify the MM substantially and remove the handling for metadata.
- Per discussion, we don't have an ETA for frontend implementation of tags, and with the realization that the tags from CivitAI are largely useless, there's no reason to keep tags in the MM right now. When we are ready to implement tags on the frontend, we can refer back to the implementation here and use it if it supports the design.
- Fix all tests.
2024-03-05 23:50:19 +11:00
psychedelicious
c3aa985c93 refactor(mm): get metadata working 2024-03-05 23:50:19 +11:00
psychedelicious
7cb0da1f66 refactor(mm): wip schema changes 2024-03-05 23:50:19 +11:00
psychedelicious
3534366146 fix(mm): fix extraneous downloaded files in diffusers
Sometimes, diffusers model components (tokenizer, unet, etc.) have multiple weights files in the same directory.

In this situation, we assume the files are different versions of the same weights. For example, we may have multiple
formats (`.bin`, `.safetensors`) with different precisions. When downloading model files, we want to select only
the best of these files for the requested format and precision/variant.

The previous logic assumed that each model weights file would have the same base filename, but this assumption was
not always true. The logic is revised score each file and choose the best scoring file, resulting in only a single
file being downloaded for each submodel/subdirectory.
2024-03-05 23:50:19 +11:00
psychedelicious
f2b5f8753f tidy(mm): remove json_schema_extra from config - not needed 2024-03-05 23:50:19 +11:00
psychedelicious
94e1e64296 chore: ruff 2024-03-05 23:50:19 +11:00
psychedelicious
2411bf53c0 tidy(mm): better descriptions for model configs 2024-03-05 23:50:19 +11:00
psychedelicious
9378e47a06 feat(mm): add source_type to model configs 2024-03-05 23:50:19 +11:00
psychedelicious
4471ea8ad1 refactor(mm): simplify model metadata schemas 2024-03-05 23:50:19 +11:00
psychedelicious
61b737bb9f tidy(mm): remove update method from ModelConfigBase
It's only used in the soon-to-be-removed model merge logic
2024-03-05 23:50:19 +11:00
psychedelicious
a8cd3dfc99 refactor(mm): add models table (schema WIP), rename "original_hash" -> "hash" 2024-03-05 23:50:19 +11:00
psychedelicious
0cce582f2f tidy(mm): remove current_hash 2024-03-05 23:50:19 +11:00
psychedelicious
bd4fd9693d tidy(mm): rename ckpt "last_modified" -> "converted_at"
Clarify what this timestamp means
2024-03-05 23:50:19 +11:00
psychedelicious
9b40c28144 tidy(mm): rename ckpy "config" -> "config_path" 2024-03-05 23:50:19 +11:00
psychedelicious
16a5d718bf fix(mm): add config field to ckpt vaes 2024-03-05 23:50:19 +11:00
psychedelicious
76cbc745e1 refactor(mm): add CheckpointConfigBase for all ckpt models 2024-03-05 23:50:19 +11:00
psychedelicious
0a614943f6 fix(mm): fix broken get_model_discriminator_value 2024-03-05 23:50:19 +11:00
psychedelicious
e426096d32 fix(mm): misc typing fixes for model loaders 2024-03-05 23:50:19 +11:00
psychedelicious
c561cd751f fix(mm): use correct import path for ConfigMixin, ModelMixin 2024-03-05 23:50:19 +11:00
psychedelicious
af9298f0ef tidy(mm): tidy class names in config.py 2024-03-05 23:50:19 +11:00
psychedelicious
5b74117836 fix(mm): use generic for model loader registry
This preserves the typing for classes using the decorator
2024-03-05 23:50:19 +11:00
psychedelicious
38474c9797 fix(mm): use correct import path for ModelMixin 2024-03-05 23:50:19 +11:00
psychedelicious
b880a31039 refactor(mm): remove ztsnr_training field on _MainConfig
This is used to determine the CFG Rescale Multiplier setting. We'll handle this in the UI as a default setting.
2024-03-05 23:50:19 +11:00
psychedelicious
dd31bc4586 refactor(mm): remove vae field on _MainConfig
We will handle default VAE selection in the UI.
2024-03-05 23:50:19 +11:00
psychedelicious
316573df2d feat(mm): use callable discriminator for AnyModelConfig union 2024-03-05 23:50:19 +11:00
Mary Hipp Rogers
8b34f5298c
Default model settings (#5850)
* UI in MM to create trigger phrases

* add scheduler and vaePrecision to config

* UI for configuring default settings for models'

* hook MM default model settings up to API

* add button to set default settings in parameters

* pull out trigger phrases

* back-end for default settings

* lint

* remove log;
gi

* ruff

* ruff format

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2024-03-04 09:39:03 -05:00
psychedelicious
554d175792 feat(mm): improved model hash class
- Use memory view for hashlib algorithms (closer to python 3.11's filehash API in hashlib)
- Remove `sha1_fast` (realized it doesn't even hash the whole file, it just does the first block)
- Add support for custom file filters
- Update docstrings
- Update tests
2024-03-03 14:32:14 +11:00
psychedelicious
86982f3059 feat(mm): make ModelHash instantiatable, taking an algorithm as arg 2024-03-03 14:32:14 +11:00
psychedelicious
ec8ed530a7 feat(mm): modularize ModelHash to facilitate testing 2024-03-03 14:32:14 +11:00
psychedelicious
982076d7d7 feat(mm): add hashing algos to ModelHash
- Some algos are slow, so it is now just called ModelHash
- Added all hashlib algos, plus BLAKE3 and the fast (but incorrect) SHA1 algo
2024-03-03 14:32:14 +11:00
psychedelicious
2e4672f931 feat(mm): make hash.py a script for testing 2024-03-03 14:32:14 +11:00
psychedelicious
908e915a71 feat(mm): use blake3 for hashing 2024-03-03 14:32:14 +11:00
Lincoln Stein
a72056e0df make model key assignment deterministic
- When installing, model keys are now calculated from the model contents.
- .safetensors, .ckpt and other single file models are hashed with sha1
- The contents of diffusers directories are hashed using imohash (faster)

fixup yaml->sql db migration script to assign deterministic key

- this commit also detects and assigns the correct image encoder for
  ip adapter models.
2024-03-03 14:32:14 +11:00
Ryan Dick
cc45007dc4 Remove unused code for attention map saving. 2024-03-02 08:25:41 -05:00
Ryan Dick
73bec56c59 Delete unused functions from shared_invokeai_diffusion.py. 2024-03-02 08:25:41 -05:00
Ryan Dick
6935830f99 Remove unused constructor declared with typo in name: __int__. 2024-03-01 15:12:03 -05:00
Ryan Dick
7651eeea8d Merge sequential conditioning and cac conditioning logic to eliminate a bunch of duplication. 2024-03-01 15:12:03 -05:00
Ryan Dick
204e7d383b Remove outdated comments related to T2I-Adapters and ControlNets. 2024-03-01 15:12:03 -05:00
Ryan Dick
9bc4e7a593 Remove use of **kwargs in do_unet_step(...), where full parameter list is known and supported. 2024-03-01 15:12:03 -05:00
Ryan Dick
ad96857e0f Fix avoid storing extra conditioning info in two places. 2024-03-01 15:12:03 -05:00
Lincoln Stein
0832e1818e Fix problem of all installed models being assigned "<NOKEY>"
- Also fix redundant scanning of models directory at startup.
2024-02-29 22:31:05 -05:00
psychedelicious
dd9daf8efb chore: ruff 2024-03-01 10:42:33 +11:00
psychedelicious
0305e90287 chore: ruff 2024-03-01 10:42:33 +11:00
blessedcoolant
ae34bcfbc0 fix: Assertion issue with SDXL Compel 2024-03-01 10:42:33 +11:00
Brandon Rising
e7afae0159 Switch absolute path to as_posix in _walk_directory 2024-03-01 10:42:33 +11:00
Brandon Rising
f16e64084b Ruff checks 2024-03-01 10:42:33 +11:00
Brandon Rising
8992d89817 Fix directory called on _walk_directory 2024-03-01 10:42:33 +11:00
Brandon Rising
0fc2f90824 Switch ModelSearch from os.walk to os.scandir 2024-03-01 10:42:33 +11:00
Lincoln Stein
94e3857110 handle change to Civitai metadata schema for commercial usage 2024-03-01 10:42:33 +11:00
psychedelicious
7d2e840590 tidy: remove some traces of ONNX 2024-03-01 10:42:33 +11:00
psychedelicious
10d36b4045 tidy(mm): remove ONNX from AnyModelConfig 2024-03-01 10:42:33 +11:00
psychedelicious
c80c0f0fb9 fix(mm): fix ModelCacheBase method name 2024-03-01 10:42:33 +11:00
psychedelicious
37d66488c5 chore: ruff 2024-03-01 10:42:33 +11:00
Lincoln Stein
371e3cc260 recover gracefuly from GPU out of memory errors (next version) 2024-03-01 10:42:33 +11:00
Lincoln Stein
d22738723d clear out VRAM when an OOM occurs 2024-03-01 10:42:33 +11:00
Brandon Rising
c778ab8db4 Allow passing in key on register 2024-03-01 10:42:33 +11:00
psychedelicious
239ecfaf79 fix(nodes): make fields on ModelConfigBase required
The setup of `ModelConfigBase` means autogenerated types have critical fields flagged as nullable (like `key` and `base`). Need to manually flag them as required.
2024-03-01 10:42:33 +11:00
Lincoln Stein
cc41e8912c several small model install enhancements
- Support extended HF repoid syntax in TUI. This allows
  installation of subfolders and safetensors files, as in
  `XpucT/Deliberate::Deliberate_v5.safetensors`

- Add `error` and `error_traceback` properties to the install
  job objects.

- Rename the `heuristic_import` route to `heuristic_install`.

- Fix the example `config` input in the `heuristic_install` route.
2024-03-01 10:42:33 +11:00
Lincoln Stein
5bb3aeaccd remove startup dependency on legacy models.yaml file 2024-03-01 10:42:33 +11:00
dunkeroni
06cc57d82a feat(nodes): added gradient mask node 2024-03-01 10:42:33 +11:00
Lincoln Stein
af2117dc0c remove errant def that was crashing invokeai-configure 2024-03-01 10:42:33 +11:00
psychedelicious
5a3195f757 final tidying before marking PR as ready for review
- Replace AnyModelLoader with ModelLoaderRegistry
- Fix type check errors in multiple files
- Remove apparently unneeded `get_model_config_enum()` method from model manager
- Remove last vestiges of old model manager
- Updated tests and documentation

resolve conflict with seamless.py
2024-03-01 10:42:33 +11:00
Lincoln Stein
5d612ec095 Tidy names and locations of modules
- Rename old "model_management" directory to "model_management_OLD" in order to catch
  dangling references to original model manager.
- Caught and fixed most dangling references (still checking)
- Rename lora, textual_inversion and model_patcher modules
- Introduce a RawModel base class to simplfy the Union returned by the
  model loaders.
- Tidy up the model manager 2-related tests. Add useful fixtures, and
  a finalizer to the queue and installer fixtures that will stop the
  services and release threads.
2024-03-01 10:42:33 +11:00
Lincoln Stein
996eb96b4e Fix issues identified during PR review by RyanjDick and brandonrising
- ModelMetadataStoreService is now injected into ModelRecordStoreService
  (these two services are really joined at the hip, and should someday be merged)
- ModelRecordStoreService is now injected into ModelManagerService
- Reduced timeout value for the various installer and download wait*() methods
- Introduced a Mock modelmanager for testing
- Removed bare print() statement with _logger in the install helper backend.
- Removed unused code from model loader init file
- Made `locker` a private variable in the `LoadedModel` object.
- Fixed up model merge frontend (will be deprecated anyway!)
2024-03-01 10:42:33 +11:00
Brandon Rising
88d6de4101 Raise InvalidModelConfigException when unable to detect load class in ModelLoader 2024-03-01 10:42:33 +11:00
Brandon Rising
4c6e34b216 Update _get_hf_load_class to support clipvision models 2024-03-01 10:42:33 +11:00
Brandon Rising
262cbaacdd References to context.services.model_manager.store.get_model can only accept keys, remove invalid assertion 2024-03-01 10:42:33 +11:00
Brandon Rising
35e8a33dfd Remove references to model_records service, change submodel property on ModelInfo to submodel_type to support new params in model manager 2024-03-01 10:42:33 +11:00
Lincoln Stein
b0835db47d improve swagger documentation 2024-03-01 10:42:33 +11:00
Lincoln Stein
3e330d7d9d fix a number of typechecking errors 2024-03-01 10:42:33 +11:00
Lincoln Stein
ff6e94f828 add route for model conversion from safetensors to diffusers
- Begin to add SwaggerUI documentation for AnyModelConfig and other
  discriminated Unions.
2024-03-01 10:42:33 +11:00
Lincoln Stein
a2cc4047f9 add a JIT download_and_cache() call to the model installer 2024-03-01 10:42:33 +11:00
Lincoln Stein
4027e845d4 add back the heuristic_import() method and extend repo_ids to arbitrary file paths 2024-03-01 10:42:33 +11:00
Lincoln Stein
a23dedd2ee make model manager v2 ready for PR review
- Replace legacy model manager service with the v2 manager.

- Update invocations to use new load interface.

- Fixed many but not all type checking errors in the invocations. Most
  were unrelated to model manager

- Updated routes. All the new routes live under the route tag
  `model_manager_v2`. To avoid confusion with the old routes,
  they have the URL prefix `/api/v2/models`. The old routes
  have been de-registered.

- Added a pytest for the loader.

- Updated documentation in contributing/MODEL_MANAGER.md
2024-03-01 10:42:33 +11:00
Lincoln Stein
7956602b19 consolidate model manager parts into a single class 2024-03-01 10:42:33 +11:00
Lincoln Stein
8db01ab1b3 probe for required encoder for IPAdapters and add to config 2024-03-01 10:42:33 +11:00
Lincoln Stein
db340bc253 fix invokeai_configure script to work with new mm; rename CLIs 2024-03-01 10:42:33 +11:00
Lincoln Stein
78ef946e01 BREAKING CHANGES: invocations now require model key, not base/type/name
- Implement new model loader and modify invocations and embeddings

- Finish implementation loaders for all models currently supported by
  InvokeAI.

- Move lora, textual_inversion, and model patching support into
  backend/embeddings.

- Restore support for model cache statistics collection (a little ugly,
  needs work).

- Fixed up invocations that load and patch models.

- Move seamless and silencewarnings utils into better location
2024-03-01 10:42:33 +11:00
Lincoln Stein
5745ce9c7d Multiple refinements on loaders:
- Cache stat collection enabled.
- Implemented ONNX loading.
- Add ability to specify the repo version variant in installer CLI.
- If caller asks for a repo version that doesn't exist, will fall back
  to empty version rather than raising an error.
2024-03-01 10:42:33 +11:00
Lincoln Stein
0d3addc69b added textual inversion and lora loaders 2024-03-01 10:42:33 +11:00
Lincoln Stein
67eb715093 loaders for main, controlnet, ip-adapter, clipvision and t2i 2024-03-01 10:42:33 +11:00
Lincoln Stein
8ba5360269 model loading and conversion implemented for vaes 2024-03-01 10:42:33 +11:00
Lincoln Stein
b8e875bb73 add ram cache module and support files 2024-03-01 10:42:33 +11:00
Lincoln Stein
010c4eae65 add concept of repo variant 2024-03-01 10:42:33 +11:00
psychedelicious
0f8af643d1 chore(backend): rename ModelInfo -> LoadedModelInfo
We have two different classes named `ModelInfo` which might need to be used by API consumers. We need to export both but have to deal with this naming collision.

The `ModelInfo` I've renamed here is the one that is returned when a model is loaded. It's the object least likely to be used by API consumers.
2024-03-01 10:42:33 +11:00
psychedelicious
e5d8921cf2 feat(nodes): extract LATENT_SCALE_FACTOR to constants.py 2024-03-01 10:42:33 +11:00
psychedelicious
05fb485d33 feat(nodes): move ConditioningFieldData to conditioning_data.py 2024-03-01 10:42:33 +11:00
blessedcoolant
ff9bd040cc possible fix: Seamless not working with Custom VAE's 2024-02-14 16:13:11 -05:00
Kent Keirsey
17d5f7bebd Critical Space Removal 2024-02-14 16:13:11 -05:00
Kent Keirsey
30dae0f5aa adding back skipped layer 2024-02-14 16:13:11 -05:00
psychedelicious
3339ad4df8 feat(nodes): seamless.py minor cleanup 2024-02-13 13:34:48 +11:00
Kent Keirsey
c3b2a8cb27 Quick Seamless Fixes 2024-02-13 13:34:48 +11:00
blessedcoolant
e82c21b5ba chore: rename DWPose to DW Openpose 2024-02-12 11:12:45 -05:00
blessedcoolant
f8e566d62a cleanup: unused util functions 2024-02-12 11:12:45 -05:00
blessedcoolant
f588b95c7f cleanup: remove unused code from the DWPose implementation 2024-02-12 11:12:45 -05:00
blessedcoolant
7d80261d47 chore: Add code attribution for the DWPoseDetector 2024-02-12 11:12:45 -05:00
blessedcoolant
67cbfeb33d feat: Add output image resizing for DWPose 2024-02-12 11:12:45 -05:00
blessedcoolant
675c73c94f fix: ruff lint errors 2024-02-12 11:12:45 -05:00
blessedcoolant
0a27b0379f feat: Initial implementation of DWPoseDetector 2024-02-12 11:12:45 -05:00
psychedelicious
c5f069a255 feat(backend): remove dependency on basicsr
`basicsr` has a hard dependency on torchvision <= 0.16 and is unmaintained. Extract the code we need from it and remove the dep.

Closes #5108
2024-02-11 08:34:54 +11:00
Lincoln Stein
8e500283b6
Fix broken import in checkpoint_convert (#5635)
* Fix broken import in checkpoint_convert

* simplify the fix

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-02-04 12:56:51 +00:00
Lincoln Stein
f2777f5096
Port the command-line tools to use model_manager2 (#5546)
* Port the command-line tools to use model_manager2

1.Reimplement the following:

  - invokeai-model-install
  - invokeai-merge
  - invokeai-ti

  To avoid breaking the original modeal manager, the udpated tools
  have been renamed invokeai-model-install2 and invokeai-merge2. The
  textual inversion training script should continue to work with
  existing installations. The "starter" models now live in
  `invokeai/configs/INITIAL_MODELS2.yaml`.

  When the full model manager 2 is in place and working, I'll rename
  these files and commands.

2. Add the `merge` route to the web API. This will merge two or three models,
   resulting a new one.

   - Note that because the model installer selectively installs the `fp16` variant
     of models (rather than both 16- and 32-bit versions as previous),
     the diffusers merge script will choke on any huggingface diffuserse models
     that were downloaded with the new installer. Previously-downloaded models
     should continue to merge correctly. I have a PR
     upstream https://github.com/huggingface/diffusers/pull/6670 to fix
     this.

3. (more important!)
  During implementation of the CLI tools, found and fixed a number of small
  runtime bugs in the model_manager2 implementation:

  - During model database migration, if a registered models file was
    not found on disk, the migration would be aborted. Now the
    offending model is skipped with a log warning.

  - Caught and fixed a condition in which the installer would download the
    entire diffusers repo when the user provided a single `.safetensors`
    file URL.

  - Caught and fixed a condition in which the installer would raise an
    exception and stop the app when a request for an unknown model's metadata
    was passed to Civitai. Now an error is logged and the installer continues.

  - Replaced the LoWRA starter LoRA with FlatColor. The former has been removed
    from Civitai.

* fix ruff issue

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-02-02 17:18:47 +00:00
Brandon
72db2ee352
Merge branch 'main' into sdxl-convert-safetensors 2024-02-02 10:10:49 -05:00
Wubbbi
1057314508 Fix ruff? 2024-02-01 20:40:28 -05:00
Wubbbi
73a077956b Why did my IDE change the comment? 2024-02-01 20:40:28 -05:00
Wubbbi
5e1e50bd47 Fix hopefully last import 2024-02-01 20:40:28 -05:00
Wubbbi
413fe566b8 Fix imports 2024-02-01 20:40:28 -05:00
Wubbbi
c9b5f06c42 Update diffusers + hotfix 2024-02-01 20:40:28 -05:00
Brandon
06bcc07f65
Merge branch 'main' into sdxl-convert-safetensors 2024-01-31 17:00:19 -05:00
Brandon Rising
a0996b1c0a Fix ruff styling 2024-01-31 07:16:14 -06:00
Brandon Rising
522ff4a042 civit -> civitai 2024-01-31 07:16:14 -06:00
Brandon Rising
a769f93be0 Remove unnecessary change 2024-01-31 07:16:14 -06:00
Brandon Rising
5d773dc94c Remove debug line 2024-01-31 07:16:14 -06:00
Brandon Rising
088e3420e6 Allow passing of civit api key via config 2024-01-31 07:16:14 -06:00
Brandon Rising
14efc95707 Allow passing of a civit api key 2024-01-31 07:16:14 -06:00
Brandon Rising
a04981b418 This seems to work now 2024-01-30 21:32:08 -05:00
blessedcoolant
35184dbd86 fix: incorrect local file path 2024-01-24 03:37:16 +05:30
blessedcoolant
92fb09c4df fix: Move the models to any folder to avoid boot warnings 2024-01-24 03:35:37 +05:30
blessedcoolant
7cb49e65bd feat: Add Resolution to DepthAnything 2024-01-23 14:13:50 -06:00
blessedcoolant
6a2eb1d2e4 fix: Change the path of the annotator folder to annotators
Just making this change in case there are other models added to the folder in the future
2024-01-23 14:13:50 -06:00
blessedcoolant
c859eb865e fix: lint & other minor issues 2024-01-23 14:13:50 -06:00
blessedcoolant
8f5e2cbcc7 feat: Add Depth Anything PreProcessor 2024-01-23 14:13:50 -06:00
Neil Wang
aa25ea62a5
fix(backend) installed models being redownloaded (#5526)
* fix

* fix ruff errors

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-01-18 16:53:53 -05:00
Lincoln Stein
bd31b5606c Retain suffix (.safetensors, .bin) when renaming a checkpoint file or LoRA
- closes #5518
2024-01-16 17:27:50 -05:00
Lincoln Stein
4536e4a8b6
Model Manager Refactor: Install remote models and store their tags and other metadata (#5361)
* add basic functionality for model metadata fetching from hf and civitai

* add storage

* start unit tests

* add unit tests and documentation

* add missing dependency for pytests

* remove redundant fetch; add modified/published dates; updated docs

* add code to select diffusers files based on the variant type

* implement Civitai installs

* make huggingface parallel downloading work

* add unit tests for model installation manager

- Fixed race condition on selection of download destination path
- Add fixtures common to several model_manager_2 unit tests
- Added dummy model files for testing diffusers and safetensors downloading/probing
- Refactored code for selecting proper variant from list of huggingface repo files
- Regrouped ordering of methods in model_install_default.py

* improve Civitai model downloading

- Provide a better error message when Civitai requires an access token (doesn't give a 403 forbidden, but redirects
  to the HTML of an authorization page -- arrgh)
- Handle case of Civitai providing a primary download link plus additional links for VAEs, config files, etc

* add routes for retrieving metadata and tags

* code tidying and documentation

* fix ruff errors

* add file needed to maintain test root diretory in repo for unit tests

* fix self->cls in classmethod

* add pydantic plugin for mypy

* use TestSession instead of requests.Session to prevent any internet activity

improve logging

fix error message formatting

fix logging again

fix forward vs reverse slash issue in Windows install tests

* Several fixes of problems detected during PR review:

- Implement cancel_model_install_job and get_model_install_job routes
  to allow for better control of model download and install.
- Fix thread deadlock that occurred after cancelling an install.
- Remove unneeded pytest_plugins section from tests/conftest.py
- Remove unused _in_terminal_state() from model_install_default.
- Remove outdated documentation from several spots.
- Add workaround for Civitai API results which don't return correct
  URL for the default model.

* fix docs and tests to match get_job_by_source() rather than get_job()

* Update invokeai/backend/model_manager/metadata/fetch/huggingface.py

Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>

* Call CivitaiMetadata.model_validate_json() directly

Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>

* Second round of revisions suggested by @ryanjdick:

- Fix type mismatch in `list_all_metadata()` route.
- Do not have a default value for the model install job id
- Remove static class variable declarations from non Pydantic classes
- Change `id` field to `model_id` for the sqlite3 `model_tags` table.
- Changed AFTER DELETE triggers to ON DELETE CASCADE for the metadata and tags tables.
- Made the `id` field of the `model_metadata` table into a primary key to achieve uniqueness.

* Code cleanup suggested in PR review:

- Narrowed the declaration of the `parts` attribute of the download progress event
- Removed auto-conversion of str to Url in Url-containing sources
- Fixed handling of `InvalidModelConfigException`
- Made unknown sources raise `NotImplementedError` rather than `Exception`
- Improved status reporting on cached HuggingFace access tokens

* Multiple fixes:

- `job.total_size` returns a valid size for locally installed models
- new route `list_models` returns a paged summary of model, name,
  description, tags and other essential info
- fix a few type errors

* consolidated all invokeai root pytest fixtures into a single location

* Update invokeai/backend/model_manager/metadata/metadata_store.py

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>

* Small tweaks in response to review comments:

- Remove flake8 configuration from pyproject.toml
- Use `id` rather than `modelId` for huggingface `ModelInfo` object
- Use `last_modified` rather than `LastModified` for huggingface `ModelInfo` object
- Add `sha256` field to file metadata downloaded from huggingface
- Add `Invoker` argument to the model installer `start()` and `stop()` routines
  (but made it optional in order to facilitate use of the service outside the API)
- Removed redundant `PRAGMA foreign_keys` from metadata store initialization code.

* Additional tweaks and minor bug fixes

- Fix calculation of aggregate diffusers model size to only count the
  size of files, not files + directories (which gives different unit test
  results on different filesystems).
- Refactor _get_metadata() and _get_download_urls() to have distinct code paths
  for Civitai, HuggingFace and URL sources.
- Forward the `inplace` flag from the source to the job and added unit test for this.
- Attach cached model metadata to the job rather than to the model install service.

* fix unit test that was breaking on windows due to CR/LF changing size of test json files

* fix ruff formatting

* a few last minor fixes before merging:

- Turn job `error` and `error_type` into properties derived from the exception.
- Add TODO comment about the reason for handling temporary directory destruction
  manually rather than using tempfile.tmpdir().

* add unit tests for reporting HTTP download errors

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2024-01-14 19:54:53 +00:00
Millun Atluri
74e644c4ba
Allow bfloat16 to be configurable in invoke.yaml (#5469)
* feat: allow bfloat16 to be configurable in invoke.yaml

* fix: `torch_dtype()` util

- Use `choose_precision` to get the precision string
- Do not reference deprecated `config.full_precision` flat (why does this still exist?), if a user had this enabled it would override their actual precision setting and potentially cause a lot of confusion.

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2024-01-12 18:40:37 +00:00
Lincoln Stein
7269c9f02e
Enable correct probing of LoRA latent-consistency/lcm-lora-sdxl (#5449)
- Closes #5435

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-01-08 17:18:26 -05:00
Lincoln Stein
ffa05a0bb3 Only replace vae when it is the broken SDXL 1.0 version 2024-01-06 14:06:47 -05:00
Lincoln Stein
a20e17330b blackify 2024-01-06 14:06:47 -05:00
Lincoln Stein
4e83644433 if sdxl-vae-fp16-fix model is available then bake it in when converting ckpts 2024-01-06 14:06:47 -05:00
Ryan Dick
f7f697849c Skip weight initialization when resizing text encoder token embeddings to accomodate new TI embeddings. This saves time. 2024-01-05 15:16:00 -05:00
Lincoln Stein
6460dcc7e0 use torch.bfloat16 on cuda systems 2024-01-04 23:25:52 -05:00
Jonathan
83a9e26cd8
Respect torch-sdp in config.yaml (#5353)
If the user specifies `torch-sdp` as the attention type in `config.yaml`, we can go ahead and use it (if available) rather than always throwing an exception.
2023-12-28 05:46:28 +00:00
Millun Atluri
53b835945f Updated with ruff formatting 2023-12-28 11:05:19 +11:00
woweenie
e38d0e39b7
fix bug when there are two multi vector TI in a prompt 2023-12-27 22:14:14 +01:00
Lincoln Stein
4957a360ff close #5209 2023-12-21 23:02:57 -05:00
gogurtenjoyer
2d11d97dad
remove MacOS Sonoma check in devices.py (#5312)
* remove MacOS Sonoma check in devices.py

As of pytorch 2.1.0, float16 works with our MPS fixes on Sonoma, so the check is no longer needed.

* remove unused platform import
2023-12-22 00:42:47 +00:00
Lincoln Stein
fa3f1b6e41
[Feat] reimport model config records after schema migration (#5281)
* add code to repopulate model config records after schema update

* reformat for ruff

* migrate model records using db cursor rather than the ModelRecordConfigService

* ruff fixes

* tweak exception reporting

* fix: build frontend in  pypi-release workflow

This was missing, resulting in the 3.5.0rc1 having no frontend.

* fix: use node 18, set working directory

- Node 20 has  a problem with `pnpm`; set it to Node 18
- Set the working directory for the frontend commands

* Don't copy extraneous paths into installer .zip

* feat(installer): delete frontend build after creating installer

This prevents an empty `dist/` from breaking the app on startup.

* feat: add python dist as release artifact, as input to enable publish to pypi

- The release workflow never runs automatically. It must be manually kicked off.
- The release workflow has an input. When running it from the GH actions UI, you will see a "Publish build on PyPi" prompt. If this value is "true", the workflow will upload the build to PyPi, releasing it. If this is anything else (e.g. "false", the default), the workflow will build but not upload to PyPi.
- The `dist/` folder (where the python package is built) is uploaded as a workflow artifact as a zip file. This can be downloaded and inspected. This allows "dry" runs of the workflow.
- The workflow job and some steps have been renamed to clarify what they do

* translationBot(ui): update translation files

Updated by "Cleanup translation files" hook in Weblate.

Co-authored-by: Hosted Weblate <hosted@weblate.org>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/
Translation: InvokeAI/Web UI

* freeze yaml migration logic at upgrade to 3.5

* moved migration code to migration_3

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
Co-authored-by: Hosted Weblate <hosted@weblate.org>
2023-12-19 17:01:47 -05:00
Ryan Dick
cb698ff1fb Update model_probe to work with diffuser-format SD TI embeddings. 2023-12-18 09:51:16 -05:00
skunkworxdark
96a717c4ba In CalculateImageTilesEvenSplitInvocation to have overlap_fraction becomes just overlap. This is now in pixels rather than as a fraction of the tile size.
Update calc_tiles_even_split() with the same change. Ensuring Overlap is within allowed size

Update even_split tests
2023-12-17 15:10:50 +00:00
Lincoln Stein
212dbaf9a2 fix comment 2023-12-15 00:25:27 -05:00
Lincoln Stein
ac3cf48d7f make probe recognize lora format at https://civitai.com/models/224641 2023-12-15 00:25:27 -05:00
Ryan Dick
5127e9df2d Fix error caused by bump to diffusers 0.24. We pass kwargs now instead of positional args so that we are more robust to future changes. 2023-12-13 09:17:30 -05:00
psychedelicious
cc18d86f29
Merge branch 'main' into feat/db/migrations 2023-12-13 11:24:55 +11:00
skunkworxdark
9620f9336c updated comment 2023-12-12 10:24:00 -05:00
skunkworxdark
a64ced7b29 remove unneeded if else 2023-12-12 10:24:00 -05:00
skunkworxdark
dd7deff1a3 fix for calc_tiles_min_overlap when tile size is bigger than image size 2023-12-12 10:24:00 -05:00
psychedelicious
2cdda1fda2 Merge remote-tracking branch 'origin/main' into feat/db/migrations 2023-12-12 17:22:52 +11:00
Lincoln Stein
15a3e8076f
Merge branch 'main' into feat/enhance-model-db-migrate-script 2023-12-11 23:10:04 -05:00
Lincoln Stein
2fbe3a3104 fix ruff error 2023-12-11 23:04:18 -05:00
Lincoln Stein
b0cfa58526 allow the model record migrate script to update existing model records 2023-12-11 22:47:19 -05:00
psychedelicious
417db71471 feat(db): decouple SqliteDatabase from config object
- Simplify init args to path (None means use memory), logger, and verbose
- Add docstrings to SqliteDatabase (it had almost none)
- Update all usages of the class
2023-12-12 10:30:37 +11:00
skunkworxdark
c84526fae5 Fixed Tests that where using round_to_8 and removed redundant tests 2023-12-11 17:05:45 +00:00
skunkworxdark
f762940335 Merge branch 'main' into tiled-upscaling-graph 2023-12-11 16:57:36 +00:00
skunkworxdark
fefb78795f - Even_spilt overlap renamed to overlap_fraction
- min_overlap removed * restrictions and round_to_8
- min_overlap handles tile size > image size by clipping the num tiles to 1.
- Updated assert test on min_overlap.
2023-12-11 16:55:27 +00:00
psychedelicious
0710ec30cf feat(db): incorporate feedback 2023-12-11 16:14:25 +11:00
psychedelicious
a2dc780188 feat: add script to migrate image workflows 2023-12-11 16:14:25 +11:00
Lincoln Stein
36043bf38b fixed docstring in probe module 2023-12-10 21:33:54 -05:00
Lincoln Stein
de2879f602 port new code for detecting sdxl-based embeddings 2023-12-10 15:48:02 -05:00
Lincoln Stein
d7f7fbc8c2 Merge branch 'main' into refactor/model-manager-3 2023-12-10 12:55:28 -05:00
Lincoln Stein
e2567a7e31 Merge branch 'refactor/model-manager-3' of github.com:invoke-ai/InvokeAI into refactor/model-manager-3 2023-12-10 12:55:24 -05:00
Lincoln Stein
2f3457c02a rename installer __del__() to stop(). Improve probe error messages 2023-12-10 12:55:01 -05:00
Lincoln Stein
aab6369ffe
Update invokeai/backend/model_manager/search.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
2023-12-10 12:24:50 -05:00
skunkworxdark
abdd840fb9 Merge branch 'main' into tiled-upscaling-graph 2023-12-09 22:03:18 +00:00
skunkworxdark
e656768eb2 more fixes from code review 2023-12-09 21:56:31 +00:00
skunkworxdark
494c2a9b05 Updates based on code review by @RyanJDick 2023-12-09 18:38:07 +00:00
psychedelicious
c42d692ea6
feat: workflow library (#5148)
* chore: bump pydantic to 2.5.2

This release fixes pydantic/pydantic#8175 and allows us to use `JsonValue`

* fix(ui): exclude public/en.json from prettier config

* fix(workflow_records): fix SQLite workflow insertion to ignore duplicates

* feat(backend): update workflows handling

Update workflows handling for Workflow Library.

**Updated Workflow Storage**

"Embedded Workflows" are workflows associated with images, and are now only stored in the image files. "Library Workflows" are not associated with images, and are stored only in DB.

This works out nicely. We have always saved workflows to files, but recently began saving them to the DB in addition to in image files. When that happened, we stopped reading workflows from files, so all the workflows that only existed in images were inaccessible. With this change, access to those workflows is restored, and no workflows are lost.

**Updated Workflow Handling in Nodes**

Prior to this change, workflows were embedded in images by passing the whole workflow JSON to a special workflow field on a node. In the node's `invoke()` function, the node was able to access this workflow and save it with the image. This (inaccurately) models workflows as a property of an image and is rather awkward technically.

A workflow is now a property of a batch/session queue item. It is available in the InvocationContext and therefore available to all nodes during `invoke()`.

**Database Migrations**

Added a `SQLiteMigrator` class to handle database migrations. Migrations were needed to accomodate the DB-related changes in this PR. See the code for details.

The `images`, `workflows` and `session_queue` tables required migrations for this PR, and are using the new migrator. Other tables/services are still creating tables themselves. A followup PR will adapt them to use the migrator.

**Other/Support Changes**

- Add a `has_workflow` column to `images` table to indicate that the image has an embedded workflow.
- Add handling for retrieving the workflow from an image in python. The image file must be fetched, the workflow extracted, and then sent to client, avoiding needing the browser to parse the image file. With the `has_workflow` column, the UI knows if there is a workflow to be fetched, and only fetches when the user requests to load the workflow.
- Add route to get the workflow from an image
- Add CRUD service/routes for the library workflows
- `workflow_images` table and services removed (no longer needed now that embedded workflows are not in the DB)

* feat(ui): updated workflow handling (WIP)

Clientside updates for the backend workflow changes.

Includes roughed-out workflow library UI.

* feat: revert SQLiteMigrator class

Will pursue this in a separate PR.

* feat(nodes): do not overwrite custom node module names

Use a different, simpler method to detect if a node is custom.

* feat(nodes): restore WithWorkflow as no-op class

This class is deprecated and no longer needed. Set its workflow attr value to None (meaning it is now a no-op), and issue a warning when an invocation subclasses it.

* fix(nodes): fix get_workflow from queue item dict func

* feat(backend): add WorkflowRecordListItemDTO

This is the id, name, description, created at and updated at workflow columns/attrs. Used to display lists of workflowsl

* chore(ui): typegen

* feat(ui): add workflow loading, deleting to workflow library UI

* feat(ui): workflow library pagination button styles

* wip

* feat: workflow library WIP

- Save to library
- Duplicate
- Filter/sort
- UI/queries

* feat: workflow library - system graphs - wip

* feat(backend): sync system workflows to db

* fix: merge conflicts

* feat: simplify default workflows

- Rename "system" -> "default"
- Simplify syncing logic
- Update UI to match

* feat(workflows): update default workflows

- Update TextToImage_SD15
- Add TextToImage_SDXL
- Add README

* feat(ui): refine workflow list UI

* fix(workflow_records): typo

* fix(tests): fix tests

* feat(ui): clean up workflow library hooks

* fix(db): fix mis-ordered db cleanup step

It was happening before pruning queue items - should happen afterwards, else you have to restart the app again to free disk space made available by the pruning.

* feat(ui): tweak reset workflow editor translations

* feat(ui): split out workflow redux state

The `nodes` slice is a rather complicated slice. Removing `workflow` makes it a bit more reasonable.

Also helps to flatten state out a bit.

* docs: update default workflows README

* fix: tidy up unused files, unrelated changes

* fix(backend): revert unrelated service organisational changes

* feat(backend): workflow_records.get_many arg "filter_text" -> "query"

* feat(ui): use custom hook in current image buttons

Already in use elsewhere, forgot to use it here.

* fix(ui): remove commented out property

* fix(ui): fix workflow loading

- Different handling for loading from library vs external
- Fix bug where only nodes and edges loaded

* fix(ui): fix save/save-as workflow naming

* fix(ui): fix circular dependency

* fix(db): fix bug with releasing without lock in db.clean()

* fix(db): remove extraneous lock

* chore: bump ruff

* fix(workflow_records): default `category` to `WorkflowCategory.User`

This allows old workflows to validate when reading them from the db or image files.

* hide workflow library buttons if feature is disabled

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-12-09 09:48:38 +11:00
skunkworxdark
d3ad356c6a Ruff Formatting
Fix pyTest issues
2023-12-08 18:31:33 +00:00
skunkworxdark
fed2bdafeb Added Defaults to calc_tiles_min_overlap for overlap and round
Added tests for min_overlap and even_split tile gen
2023-12-08 18:16:13 +00:00
skunkworxdark
cd15d8b7a9 ruff formatting
reformatted due to ruff errors
2023-12-06 08:10:22 +00:00
skunkworxdark
674d9796d0 First check-in of new tile nodes
- calc_tiles_even_split
- calc_tiles_min_overlap
- merge_tiles_with_seam_blending
Update MergeTilesToImageInvocation with seam blending
2023-12-05 21:03:16 +00:00
Lincoln Stein
3bfaee9c57
Merge branch 'main' into refactor/model-manager-3 2023-12-04 22:51:45 -05:00
Lincoln Stein
3c7d1fcd32 clean up get_logger() call 2023-12-04 22:41:59 -05:00
Lincoln Stein
2b583ffcdf implement review suggestions from @RyanjDick 2023-12-04 21:12:10 -05:00
Lincoln Stein
2d2ef5d72c ensure that setting loglevel on one logger doesn't change others 2023-12-02 11:48:51 -05:00
Lincoln Stein
e46ac45741 port probing changes from main model_probe.py to refactored probe.py 2023-12-01 09:19:24 -05:00
Lincoln Stein
75089b7a9d merge in changes from main 2023-12-01 09:18:07 -05:00
Lincoln Stein
778fd55f0d Merge branch 'main' into refactor/model-manager-3 2023-12-01 09:15:18 -05:00
Lincoln Stein
5a3f1f2b22 fix ruff github format errors 2023-12-01 01:59:26 -05:00
Lincoln Stein
f95ce1870c fix ruff format check 2023-12-01 01:46:12 -05:00
Lincoln Stein
0719a46372 add support for SDXL textual inversion/embeddings 2023-12-01 01:28:28 -05:00
Ryan Dick
bfdef120d1 Re-organize merge_tiles_with_linear_blending(...) to merge rows horizontally first and then vertically. This change achieves slightly more natural blending on the corners where 4 tiles overlap. 2023-11-30 07:53:27 -08:00
Ryan Dick
76b888de17 Add unit tests for merge_tiles_with_linear_blending(...). 2023-11-30 07:53:27 -08:00
Ryan Dick
65a16be299 Add unit tests for calc_tiles_with_overlap(...) and fix a bug in its implementation. 2023-11-30 07:53:27 -08:00
Ryan Dick
29eade4880 Add nodes for tile splitting and merging. The main motivation for these nodes is for use in tiled upscaling workflows. 2023-11-30 07:53:27 -08:00
Damian Stewart
0beb08686c
Add CFG Rescale option for supporting zero-terminal SNR models (#4335)
* add support for CFG rescale

* fix typo

* move rescale position and tweak docs

* move input position

* implement suggestions from github and discord

* cleanup unused code

* add back dropped FieldDescription

* fix(ui): revert unrelated UI changes

* chore(nodes): bump denoise_latents version 1.4.0 -> 1.5.0

* feat(nodes): add cfg_rescale_multiplier to metadata node

* feat(ui): add cfg rescale multiplier to linear UI

- add param to state
- update graph builders
- add UI under advanced
- add metadata handling & recall
- regen types

* chore: black

* fix(backend): make `StableDiffusionGeneratorPipeline._rescale_cfg()` staticmethod

This doesn't need access to class.

* feat(backend): add docstring for `_rescale_cfg()` method

* feat(ui): update cfg rescale mult translation string

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-30 20:55:20 +11:00
Ryan Dick
693c6cf5e4 Add support for IPAdapterFull models. The changes are based on this upstream PR: https://github.com/tencent-ailab/IP-Adapter/pull/139 . 2023-11-29 15:07:21 -08:00
Lincoln Stein
ecd3dcd5df
Merge branch 'main' into refactor/model-manager-3 2023-11-27 22:15:51 -05:00
Lincoln Stein
ae82df0fda fix a bunch of type mismatches in the logging module 2023-11-28 09:38:35 +11:00
Steven Frank
e509d719ee Fix attempt to deserialize on CUDA on Mac
Without specifying "cpu", attempts to use non-existent CUDA to deserialize embeddings on macOS, resulting in a warning / failure to load.
2023-11-28 09:24:57 +11:00
Lincoln Stein
a79e814c8d
Merge branch 'main' into refactor/model-manager-3 2023-11-27 16:06:42 -05:00
psychedelicious
7653d21cf5 feat(backend): rename realesrgan class & upscale method 2023-11-28 07:58:22 +11:00
psychedelicious
46a2d83b84 feat(backend): organise realesrgan code, add license
- Moved util to own folder
- BSD3 License for RealESRGAN repo added
2023-11-28 07:58:22 +11:00
psychedelicious
2192210910 feat(nodes): remove dependency on realesrgan
We used the `RealESRGANer` utility class from the repo. It handled model loading and tiled upscaling logic.

Unfortunately, it hasn't been updated in over a year, had no types, and annoyingly printed to console.

I've adapted the class, cleaning it up a bit and removing the bits that are not relevant for us.

Upscaling functionality is identical.
2023-11-28 07:58:22 +11:00
Lincoln Stein
8ef596eac7 further changes for ruff 2023-11-26 17:13:31 -05:00
Lincoln Stein
8695ad6f59 all features implemented, docs updated, ready for review 2023-11-26 13:18:21 -05:00
Lincoln Stein
8aefe2cefe import_model and list_install_jobs router APIs written 2023-11-25 21:45:59 -05:00
Lincoln Stein
ec510d34b5 fix model probing for controlnet checkpoint legacy config files 2023-11-25 15:53:22 -05:00
Lincoln Stein
19baea1883 all backend features in place; config scanning is failing on controlnet 2023-11-24 19:37:46 -05:00
Lincoln Stein
80bc9be3ab make install_path and register_path work; refactor model probing 2023-11-23 23:15:32 -05:00
Lincoln Stein
8c7a7bc897 Merge branch 'main' into refactor/model-manager-3 2023-11-22 22:29:23 -05:00
Lincoln Stein
4aab728590 move name/description logic into model_probe.py 2023-11-22 22:29:02 -05:00
Lincoln Stein
9cf060115d Merge branch 'main' into refactor/model-manager-3 2023-11-22 22:28:31 -05:00
Lincoln Stein
98a4930a52 add probe support for LCM main models 2023-11-22 14:58:27 -05:00
psychedelicious
1a596a5684 fix(backend): fix unintentional change to import orders
- Ignore I001 (isort rules) for this file
- Ignore F401 (unused imports) for this file
2023-11-21 20:22:27 +11:00
psychedelicious
da443973cb chore: ruff 2023-11-21 20:22:27 +11:00
Lincoln Stein
9ea3126118 start implementation of installer 2023-11-20 23:02:30 -05:00
Lincoln Stein
acc0a29dca fixed ruff formatting issues 2023-11-13 18:15:17 -05:00
Lincoln Stein
38c1436f02 resolve conflicts; blackify 2023-11-13 18:12:45 -05:00
Lincoln Stein
efbdb75568 implement psychedelicious recommendations as of 13 November 2023-11-13 17:05:01 -05:00
psychedelicious
bc64cde6f9 chore: ruff lint 2023-11-14 07:57:07 +11:00
psychedelicious
4465f97cdf
Merge branch 'main' into refactor/model-manager-2 2023-11-14 07:51:57 +11:00
psychedelicious
04d8f2dfea fix(backend): fix controlnet zip len
Do not use `strict=True` when scaling controlnet conditioning.

When using `guess_mode` (e.g. `more_control` or `more_prompt`), `down_block_res_samples` and `scales` are zipped.

These two objects are of different lengths, so using zip's strict mode raises an error.

In testing, `len(scales) === len(down_block_res_samples) + 1`.

It appears this behaviour is intentional, as the final "extra" item in `scales` is used immediately afterwards.
2023-11-13 15:45:03 +11:00
Lincoln Stein
8c71ff37ae
Update config.py
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-11-12 19:03:39 -05:00
Lincoln Stein
8afe517204 add note about discriminated union and Body() issue; blackified 2023-11-12 16:50:05 -05:00
Lincoln Stein
7ea2a135f1 remove dangling import 2023-11-11 12:24:58 -05:00
Lincoln Stein
af2264b6eb implement workaround for FastAPI and discriminated unions in Body parameter 2023-11-11 12:22:38 -05:00
Lincoln Stein
2b36565e9e awkward workaround for double-Annotated in model_record route 2023-11-10 21:32:44 -05:00
Lincoln Stein
cb8cdefd59
Merge branch 'main' into refactor/model-manager-2 2023-11-10 19:24:19 -05:00
Lincoln Stein
f1c846ba5c blackify 2023-11-10 19:14:29 -05:00
Lincoln Stein
3a6ba236f5 replace _class_map in ModelConfigFactory with a nested discriminated union 2023-11-10 19:14:15 -05:00
psychedelicious
6494e8e551 chore: ruff format 2023-11-11 10:55:40 +11:00
psychedelicious
513fceac82 chore: ruff check - fix pycodestyle 2023-11-11 10:55:33 +11:00
psychedelicious
99a8ebe3a0 chore: ruff check - fix flake8-bugbear 2023-11-11 10:55:28 +11:00
psychedelicious
3a136420d5 chore: ruff check - fix flake8-comprensions 2023-11-11 10:55:23 +11:00
Lincoln Stein
0544917161 multiple small fixes suggested in reviews from psychedelicious and ryan 2023-11-10 18:25:37 -05:00
Lincoln Stein
f1c195afb7 Merge branch 'main' into refactor/model-manager-2 2023-11-10 17:54:28 -05:00
Wubbbi
6001d3d71d Change pad_to_multiple_of to be 8 for all cases. Add comment about it's temporary status 2023-11-10 17:51:59 -05:00
Wubbbi
8831d1ee41 Update Documentation 2023-11-10 17:51:59 -05:00
Wubbbi
a0be83e370 Update Transformers to 4.34 and fix pad_to_multiple_of 2023-11-10 17:51:59 -05:00
Lincoln Stein
8702a63197 add support for downloading and installing LCM lora diffusers models 2023-11-10 17:51:30 -05:00
blessedcoolant
356b5a41a9 wip: Add LCMScheduler 2023-11-10 06:54:36 -08:00
psychedelicious
2570497d83 fix(installer): fix import of ValidationError
It was being imported from a deprecated module
2023-11-10 06:11:15 +11:00
psychedelicious
6aa87f973e fix(nodes): create app/shared/ module to prevent circular imports
We have a number of shared classes, objects, and functions that are used in multiple places. This causes circular import issues.

This commit creates a new `app/shared/` module to hold these shared classes, objects, and functions.

Initially, only `FreeUConfig` and `FieldDescriptions` are moved here. This resolves a circular import issue with custom nodes.

Other shared classes, objects, and functions will be moved here in future commits.
2023-11-09 16:41:55 +11:00
Lincoln Stein
927f8a66e6 Merge branch 'main' into refactor/model-manager-2 2023-11-08 16:46:08 -05:00
Lincoln Stein
6b173cc66f multiple small stylistic changes requested by reviewers 2023-11-08 16:45:26 -05:00
Lincoln Stein
ce22c0fbaa sync pydantic and sql field names; merge routes 2023-11-06 18:08:57 -05:00
Kent Keirsey
ff8a8a1963
Merge branch 'main' into feat/nodes/freeu 2023-11-06 09:04:54 -08:00
Kent Keirsey
67f2616d5a
Merge branch 'main' into revert-4923-revert-4914-feat/mix-cnet-t2iadapter 2023-11-06 07:34:51 -08:00
Kent Keirsey
e66d0f7372
Merge branch 'main' into feat/nodes/freeu 2023-11-06 05:39:58 -08:00
Lincoln Stein
edeea5237b add sql-based model config store and api 2023-11-04 23:03:26 -04:00
Ryan Dick
aa02ebf8f5 Fix model cache gc.collect() condition. 2023-11-04 08:52:10 -04:00
Ryan Dick
fb3d0c4b12 Fix bug in model cache reference count checking. 2023-11-03 13:50:40 -07:00
Ryan Dick
8488ab0134 Reduce frequency that we call gc.collect() in the model cache. 2023-11-03 13:50:40 -07:00
Ryan Dick
875231ed3d Add reminder to clean up our model cache clearing logic. 2023-11-03 13:50:40 -07:00
Ryan Dick
43b300498f Remove explicit gc.collect() after transferring models from device to CPU. I'm not sure why this was there in the first place, but it was taking a significant amount of time (up to ~1sec in my tests). 2023-11-03 13:50:40 -07:00
Ryan Dick
e391f3c9a8 Skip torch.nn.Embedding.reset_parameters(...) when loading a text encoder model. 2023-11-02 19:41:33 -07:00
Ryan Dick
6e7a3f0546 (minor) Fix static checks and typo. 2023-11-02 19:20:37 -07:00
Ryan Dick
4a683cc669 Add a app config parameter to control the ModelCache logging behavior. 2023-11-02 19:20:37 -07:00
Ryan Dick
3781e56e57 Add log_memory_usage param to ModelCache. 2023-11-02 19:20:37 -07:00
Ryan Dick
8ff49109a8 Update get_pretty_snapshot_diff(...) to handle None-snapshots. 2023-11-02 19:20:37 -07:00
Ryan Dick
bac2a757e8 Replace deepcopy with a pickle roundtrip in apply_ti(...) to improve speed. 2023-11-02 19:05:24 -07:00
Ryan Dick
fa7f6a6a10 Further tidying of LoRA patching. Revert some changes that didn't end up being important under the constraint that calculations are done on the same device as the model. 2023-11-02 10:03:17 -07:00
Ryan Dick
61b17c475a Add TODO note about improving _resolve_lora_key(...). 2023-11-02 10:03:17 -07:00
Ryan Dick
379d68f595 Patch LoRA on device when model is already on device. 2023-11-02 10:03:17 -07:00
Ryan Dick
545c811bf1 Remove device and dtype members from LoRAModelRaw, they can too easily get out-of-sync with the underlying layer states. 2023-11-02 10:03:17 -07:00
Ryan Dick
2ba5b44ec4 Remove unused _lora_forward_hook(...). 2023-11-02 10:03:17 -07:00
Ryan Dick
7f4ce518b7 auto-format lora.py 2023-11-02 10:03:17 -07:00
Lincoln Stein
5c9046580f
Merge branch 'main' into bugfix/textual-inversion 2023-10-24 14:17:43 -04:00
Lincoln Stein
6cbc69f3b7 support conversion of controlnets from safetensors to diffusers 2023-10-23 22:06:10 -04:00
Lincoln Stein
3546c41f4a close #4975 2023-10-23 18:48:14 -04:00
d8ahazard
fdf02c33d0 Catch generic model errors
Prevent the app from dying on invalid models.
2023-10-19 07:28:33 +11:00
Ryan Dick
a078efc0f2 Merge branch 'main' into ryan/multi-image-ip 2023-10-18 08:59:12 -04:00
psychedelicious
252c9a5f5a fix(backend): fix nsfw/watermarker util types 2023-10-18 09:08:13 +11:00
Kent Keirsey
b7555ddae8 Revert "Revert "chore: lint""
This reverts commit 38e7eb8878.
2023-10-17 11:59:19 -04:00
Kent Keirsey
8afc47018b Revert "Revert "Cleaning up (removing diagnostic prints)""
This reverts commit 6e697b7b6f.
2023-10-17 11:59:19 -04:00
Kent Keirsey
a97ec88e06 Revert "Revert "Changes to _apply_standard_conditioning_sequentially() and _apply_cross_attention_controlled_conditioning() to reflect changes to T2I-Adapter implementation to allow usage of T2I-Adapter and ControlNet at the same time.""
This reverts commit c04fb451ee.
2023-10-17 11:59:19 -04:00
Kent Keirsey
282d36b640 Revert "Revert "Fixing some var and arg names.""
This reverts commit 58a0709c1e.
2023-10-17 11:59:19 -04:00
psychedelicious
58a0709c1e Revert "Fixing some var and arg names."
This reverts commit f11ba81a8d.
2023-10-17 11:59:11 -04:00
psychedelicious
c04fb451ee Revert "Changes to _apply_standard_conditioning_sequentially() and _apply_cross_attention_controlled_conditioning() to reflect changes to T2I-Adapter implementation to allow usage of T2I-Adapter and ControlNet at the same time."
This reverts commit 378689a519.
2023-10-17 11:59:11 -04:00
psychedelicious
6e697b7b6f Revert "Cleaning up (removing diagnostic prints)"
This reverts commit 06f8a3276d.
2023-10-17 11:59:11 -04:00
psychedelicious
38e7eb8878 Revert "chore: lint"
This reverts commit fff29d663d.
2023-10-17 11:59:11 -04:00
psychedelicious
fff29d663d chore: lint 2023-10-17 19:42:06 +11:00
user1
06f8a3276d Cleaning up (removing diagnostic prints) 2023-10-17 19:42:06 +11:00
user1
378689a519 Changes to _apply_standard_conditioning_sequentially() and _apply_cross_attention_controlled_conditioning() to reflect changes to T2I-Adapter implementation to allow usage of T2I-Adapter and ControlNet at the same time.
Also, the PREVIOUS commit (@8d3885d, which was already pushed to github repo) was wrongly commented, but too late to fix without a force push or other mucking that I'm reluctant to do. That commit is actually the one that has all the changes to diffusers_pipeline.py to use additional arg down_intrablock_additional_residuals (introduced in diffusers PR https://github.com/huggingface/diffusers/pull/5362) to detangle T2I-Adapter from ControlNet inputs to main UNet.
2023-10-17 19:42:06 +11:00
user1
f11ba81a8d Fixing some var and arg names. 2023-10-17 19:42:06 +11:00
Millun Atluri
001bba1719
Merge branch 'main' into feat/nodes/freeu 2023-10-17 15:58:00 +11:00
psychedelicious
c238a7f18b feat(api): chore: pydantic & fastapi upgrade
Upgrade pydantic and fastapi to latest.

- pydantic~=2.4.2
- fastapi~=103.2
- fastapi-events~=0.9.1

**Big Changes**

There are a number of logic changes needed to support pydantic v2. Most changes are very simple, like using the new methods to serialized and deserialize models, but there are a few more complex changes.

**Invocations**

The biggest change relates to invocation creation, instantiation and validation.

Because pydantic v2 moves all validation logic into the rust pydantic-core, we may no longer directly stick our fingers into the validation pie.

Previously, we (ab)used models and fields to allow invocation fields to be optional at instantiation, but required when `invoke()` is called. We directly manipulated the fields and invocation models when calling `invoke()`.

With pydantic v2, this is much more involved. Changes to the python wrapper do not propagate down to the rust validation logic - you have to rebuild the model. This causes problem with concurrent access to the invocation classes and is not a free operation.

This logic has been totally refactored and we do not need to change the model any more. The details are in `baseinvocation.py`, in the `InputField` function and `BaseInvocation.invoke_internal()` method.

In the end, this implementation is cleaner.

**Invocation Fields**

In pydantic v2, you can no longer directly add or remove fields from a model.

Previously, we did this to add the `type` field to invocations.

**Invocation Decorators**

With pydantic v2, we instead use the imperative `create_model()` API to create a new model with the additional field. This is done in `baseinvocation.py` in the `invocation()` wrapper.

A similar technique is used for `invocation_output()`.

**Minor Changes**

There are a number of minor changes around the pydantic v2 models API.

**Protected `model_` Namespace**

All models' pydantic-provided methods and attributes are prefixed with `model_` and this is considered a protected namespace. This causes some conflict, because "model" means something to us, and we have a ton of pydantic models with attributes starting with "model_".

Forunately, there are no direct conflicts. However, in any pydantic model where we define an attribute or method that starts with "model_", we must tell set the protected namespaces to an empty tuple.

```py
class IPAdapterModelField(BaseModel):
    model_name: str = Field(description="Name of the IP-Adapter model")
    base_model: BaseModelType = Field(description="Base model")

    model_config = ConfigDict(protected_namespaces=())
```

**Model Serialization**

Pydantic models no longer have `Model.dict()` or `Model.json()`.

Instead, we use `Model.model_dump()` or `Model.model_dump_json()`.

**Model Deserialization**

Pydantic models no longer have `Model.parse_obj()` or `Model.parse_raw()`, and there are no `parse_raw_as()` or `parse_obj_as()` functions.

Instead, you need to create a `TypeAdapter` object to parse python objects or JSON into a model.

```py
adapter_graph = TypeAdapter(Graph)
deserialized_graph_from_json = adapter_graph.validate_json(graph_json)
deserialized_graph_from_dict = adapter_graph.validate_python(graph_dict)
```

**Field Customisation**

Pydantic `Field`s no longer accept arbitrary args.

Now, you must put all additional arbitrary args in a `json_schema_extra` arg on the field.

**Schema Customisation**

FastAPI and pydantic schema generation now follows the OpenAPI version 3.1 spec.

This necessitates two changes:
- Our schema customization logic has been revised
- Schema parsing to build node templates has been revised

The specific aren't important, but this does present additional surface area for bugs.

**Performance Improvements**

Pydantic v2 is a full rewrite with a rust backend. This offers a substantial performance improvement (pydantic claims 5x to 50x depending on the task). We'll notice this the most during serialization and deserialization of sessions/graphs, which happens very very often - a couple times per node.

I haven't done any benchmarks, but anecdotally, graph execution is much faster. Also, very larges graphs - like with massive iterators - are much, much faster.
2023-10-17 14:59:25 +11:00
Ryan Dick
3079c75a60 (minor) Make it more clear that shape annotations are just comments and not commented lines of code. 2023-10-16 08:35:32 -04:00
psychedelicious
53b6f0dc73
Merge branch 'main' into ryan/multi-image-ip 2023-10-16 17:16:10 +11:00
blessedcoolant
2c6772f92f
Merge branch 'main' into bugfix/trim-whitespace-from-urls 2023-10-15 17:41:41 +05:30
blessedcoolant
c9d95e5758
Merge branch 'main' into bugfix/ignore-dot-directories-on-model-scan 2023-10-15 17:23:02 +05:30
psychedelicious
48626c40fd fix(backend): handle systems with glibc < 2.33
`mallinfo2` is not available on `glibc` < 2.33.

On these systems, we successfully load the library but get an `AttributeError` on attempting to access `mallinfo2`.

I'm not sure if the old `mallinfo` will work, and not sure how to install it safely to test, so for now we just handle the `AttributeError`.

This means the enhanced memory snapshot logic will be skipped for these systems, which isn't a big deal.
2023-10-15 07:56:55 +11:00
Ryan Dick
8464450a53 Add support for multi-image IP-Adapter. 2023-10-14 12:50:33 -04:00
Lincoln Stein
a1001b6d10
Merge branch 'main' into bugfix/ignore-dot-directories-on-model-scan 2023-10-14 10:37:55 -04:00
Lincoln Stein
15cabc4968 Possibly closes #4815 2023-10-12 23:37:05 -04:00
Lincoln Stein
29c3f49182 enable the ram cache slider in invokeai-configure 2023-10-12 23:04:16 -04:00
Lincoln Stein
21d5969942 strip leading and trailing quotes as well as whitespace 2023-10-12 22:35:02 -04:00
Lincoln Stein
52274087f3 close #4536 2023-10-12 21:24:07 -04:00
psychedelicious
9646157ad5 fix: fix test imports 2023-10-12 12:15:06 -04:00
Ryan Dick
40f9e49b5e Demote model cache logs from warning to debug based on the conversation here: https://discord.com/channels/1020123559063990373/1049495067846524939/1161647290189090816 2023-10-11 12:02:46 -04:00
psychedelicious
15b33ad501 feat(nodes): add freeu support
Add support for FreeU. See:
- https://huggingface.co/docs/diffusers/main/en/using-diffusers/freeu
- https://github.com/ChenyangSi/FreeU

Implementation:
- `ModelPatcher.apply_freeu()` handles the enabling freeu (which is very simple with diffusers).
- `FreeUConfig` model added to hold the hyperparameters.
- `freeu_config` added as optional sub-field on `UNetField`.
- `FreeUInvocation` added, works like LoRA - chain it to add the FreeU config to the UNet
- No support for model-dependent presets, this will be a future workflow editor enhancement

Closes #4845
2023-10-11 13:49:28 +11:00
Ryan Dick
61242bf86a Fix bug in skip_torch_weight_init() where the original behavior of torch.nn.Conv*d modules wasn't being restored correctly. 2023-10-10 10:05:50 -04:00
Ryan Dick
58b56e9b1e Add a skip_torch_weight_init() context manager to improve model load times (from disk). 2023-10-09 14:12:56 -04:00
Lincoln Stein
fe0cf2c160 remove hardcoded subfolder name from model downloader 2023-10-08 17:45:39 -04:00
Ryan Dick
971ccfb081 Refactor multi-IP-Adapter to clean up the interface around changing scales. 2023-10-06 20:43:43 -04:00
Ryan Dick
43a3c3c7ea Fix typo in setting IP-Adapter scales. 2023-10-06 20:43:43 -04:00
Ryan Dick
4df1cdb34d Tidy _prepare_attention_processors(...) logic. 2023-10-06 20:43:43 -04:00
Ryan Dick
d8d0c9af09 Fix handling of scales with multiple IP-Adapters. 2023-10-06 20:43:43 -04:00
Ryan Dick
26b91a538a Fixes to get IP-Adapter tests working with new multi-IP-Adapter support. 2023-10-06 20:43:43 -04:00
Ryan Dick
7ca456d674 Update IP-Adapter model to enable running multiple IP-Adapters at once. (Not tested yet.) 2023-10-06 20:43:43 -04:00
Ryan Dick
78828b6b9c WIP - Accept a list of IPAdapterFields in DenoiseLatents. 2023-10-06 20:43:43 -04:00
Ryan Dick
4f97bd4418
Merge branch 'main' into ryan/model-tests 2023-10-06 19:47:28 -04:00
Ryan Dick
e0e001758a Remove @slow decorator in favor of @pytest.mark.slow. 2023-10-06 18:26:06 -04:00
Ryan Dick
096d195d6e
Merge branch 'main' into ryan/model-cache-logging-only 2023-10-06 09:52:45 -04:00
Ryan Dick
1c8b1fbc53 POC of a test that depends on models. 2023-10-05 15:35:58 -04:00
Ryan Dick
78377469db
Add support for T2I-Adapter in node workflows (#4612)
* Bump diffusers to 0.21.2.

* Add T2IAdapterInvocation boilerplate.

* Add T2I-Adapter model to model-management.

* (minor) Tidy prepare_control_image(...).

* Add logic to run the T2I-Adapter models at the start of the DenoiseLatentsInvocation.

* Add logic for applying T2I-Adapter weights and accumulating.

* Add T2IAdapter to MODEL_CLASSES map.

* yarn typegen

* Add model probes for T2I-Adapter models.

* Add all of the frontend boilerplate required to use T2I-Adapter in the nodes editor.

* Add T2IAdapterModel.convert_if_required(...).

* Fix errors in T2I-Adapter input image sizing logic.

* Fix bug with handling of multiple T2I-Adapters.

* black / flake8

* Fix typo

* yarn build

* Add num_channels param to prepare_control_image(...).

* Link to upstream diffusers bugfix PR that currently requires a workaround.

* feat: Add Color Map Preprocessor

Needed for the color T2I Adapter

* feat: Add Color Map Preprocessor to Linear UI

* Revert "feat: Add Color Map Preprocessor"

This reverts commit a1119a00bf.

* Revert "feat: Add Color Map Preprocessor to Linear UI"

This reverts commit bd8a9b82d8.

* Fix T2I-Adapter field rendering in workflow editor.

* yarn build, yarn typegen

---------

Co-authored-by: blessedcoolant <54517381+blessedcoolant@users.noreply.github.com>
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-10-05 16:29:16 +11:00
Ryan Dick
fbe6452c45 Add support for IPAdapterPlusXL based on 6219530507. 2023-10-04 22:35:17 -04:00
Ryan Dick
2479a59e5e Re-enable garbage collection in model cache MemorySnapshots. 2023-10-03 15:18:47 -04:00
Ryan Dick
7d0ac2c36d (minor) clean up typos. 2023-10-03 15:00:03 -04:00
Ryan Dick
519b892f0c Add unit test for Struct_mallinfo2.__str__() 2023-10-03 14:25:34 -04:00
Ryan Dick
763dcacfd3 Add unit test for get_pretty_snapshot_diff(...). 2023-10-03 14:25:34 -04:00
Ryan Dick
22a84930f6 Disable garbage collection in ModelCache calls to MemorySnapshot in order minimize snapshot overhead. 2023-10-03 14:25:34 -04:00
Ryan Dick
d64e17e043 Add README with info about glib memory fragmentation caused by the model cache. 2023-10-03 14:25:34 -04:00
Ryan Dick
ba54277011 Catch a more specific exception in environments that do not have a libc shared library. 2023-10-03 14:25:34 -04:00
Ryan Dick
5915a4a51c Minor fixes. 2023-10-03 14:25:34 -04:00
Ryan Dick
4580ba0d87 Remove logic to update model cache size estimates dynamically. 2023-10-03 14:25:34 -04:00
Ryan Dick
b9fd2e9e76 Improve get_pretty_snapshot_diff(...) message formatting. 2023-10-03 14:25:34 -04:00
Ryan Dick
75b65597af Add malloc info to MemorySnapshot. 2023-10-03 14:25:34 -04:00
Ryan Dick
2a3c0ab5d2 Move MemorySnapshot to its own file. 2023-10-03 14:25:34 -04:00
Ryan Dick
7d61373b82 Add LibcUtil class. 2023-10-03 14:25:34 -04:00
Ryan Dick
7d65555a5a Fix type error in torch device comparison. 2023-10-03 14:25:34 -04:00
Ryan Dick
123f2b2dbc Update cache model size estimates based on changes in VRAM when moving models to/from CUDA. 2023-10-03 14:25:34 -04:00
Ryan Dick
1e4e42556e Update model cache device comparison to treat 'cuda' and 'cuda:0' as the same device type. 2023-10-03 14:25:34 -04:00
Ryan Dick
1f6699ac43 Consolidate all model.to(...) calls in the model cache to use a utility function with better logging. 2023-10-03 14:25:34 -04:00
Ryan Dick
ace8665411 Add warning log if moving a model from cuda to cpu causes unexpected change in VRAM usage. 2023-10-03 14:25:34 -04:00
Ryan Dick
7fa5bae8fd Add warning log if moving model from RAM to VRAM causes an unexpected change in VRAM usage. 2023-10-03 14:25:34 -04:00
Ryan Dick
f9faca7c91 Add warning log if model mis-reports its required cache memory before load from disk. 2023-10-03 14:25:34 -04:00
Ryan Dick
594fd3ba6d Add debug logging of changes in RAM and VRAM for all model cache operations. 2023-10-03 14:25:34 -04:00
Ryan Dick
44d68f5ed5 Auto-format model_cache.py. 2023-10-03 14:25:34 -04:00
psychedelicious
045d3f6139 chore: flake8 2023-09-28 13:49:31 +10:00
Lincoln Stein
f6bbcd0589 remove dangling debug statement 2023-09-27 22:26:26 -04:00
Lincoln Stein
ada22a799e remove dangling debug statement 2023-09-27 22:26:06 -04:00
Lincoln Stein
034af2d9f8 enable downloading from subfolders for repo_ids 2023-09-27 22:11:56 -04:00
Ryan Dick
0fc14afcf0
Merge branch 'main' into bug/ip-adapter-calc-size 2023-09-27 09:42:51 -04:00
Lincoln Stein
3fd27b1aa9 run correct version of black 2023-09-26 08:03:34 -04:00
Lincoln Stein
d59e534cad use heuristic to select RAM cache size during headless install; blackified 2023-09-26 08:03:34 -04:00
Ryan Dick
399ebe443e Fix IP-Adapter calculation of memory footprint. 2023-09-25 18:28:10 -04:00
Lincoln Stein
43ebd68313
Merge branch 'main' into install/install-ip-adapters 2023-09-24 23:19:25 -04:00
Lincoln Stein
912087e4dc blackify 2023-09-24 19:00:38 -04:00
Lincoln Stein
593fb95213 ip_adapter_sd15 & its encoder will now be installed by default during headless install 2023-09-24 19:00:21 -04:00
Lincoln Stein
a1d9e6b871
Merge branch 'main' into bugfix/probe_ip_adapter 2023-09-24 15:39:43 -04:00
Lincoln Stein
e34e6d6e80 enable v_prediction for sd-1 models 2023-09-24 12:22:29 -04:00
Lincoln Stein
86cb53342a fix probing for ip_adapter folders 2023-09-23 22:32:03 -04:00
Lincoln Stein
25a71a1791
Merge branch 'main' into refactor/rename-get-logger 2023-09-23 14:49:07 -07:00
Ryan Dick
bf62553150 (minor) Update documentation to reflect that a bug was fixed in InvokeAI/ip_adapter_sdxl_vit_h by e178288fb6 2023-09-20 20:18:33 -04:00
Lincoln Stein
0960518088 add techjedi's database maintenance script 2023-09-20 17:46:49 -04:00
Kevin Turner
b1b5f70ea6
Merge branch 'main' into feat/taesd 2023-09-20 12:54:17 -07:00
Kevin Turner
e487bcd0f7 feat(model management): guess whether a VAE is for SDXL based on its name 2023-09-20 12:07:12 -07:00
Kevin Turner
e0f8274f49 feat(model management): guess whether a VAE is for SDXL based on its name 2023-09-20 12:06:55 -07:00
Lincoln Stein
0420874f56 reimplement the old invokeai-metadata command 2023-09-20 13:49:29 -04:00
Kevin Turner
f222b871e9 Merge remote-tracking branch 'origin/main' into feat/taesd
# Conflicts:
#	invokeai/backend/model_management/model_probe.py
2023-09-20 10:46:55 -07:00
Ryan Dick
92b49e45bb Address flake8 error. 2023-09-18 16:33:16 -04:00
Ryan Dick
b05b8ef677 Switch to using torch 2.0 attention for IP-Adapter (more memory-efficient). 2023-09-18 16:30:53 -04:00
blessedcoolant
e0dddbd38e chore: fix isort issues 2023-09-17 12:13:03 +12:00
blessedcoolant
b7773c9962 chore: black & lint fixes 2023-09-17 12:00:21 +12:00
user1
c48e648cbb Added per-step setting of IP-Adapter weights (for param easing, etc.) 2023-09-16 12:36:16 -07:00
user1
7ee13879e3 Added check in IP-Adapter to avoid begin/end step percent handling if use of IP-Adapter is already turned off due to potential clash with other cross attention control. 2023-09-16 09:29:50 -07:00
user1
ced297ed21 Initial implementation of IP-Adapter "begin_step_percent" and "end_step_percent" for controlling on which steps IP-Adapter is applied in the denoising loop. 2023-09-16 08:24:12 -07:00
Ryan Dick
343df03a92 isort 2023-09-15 13:18:00 -04:00
Ryan Dick
b57acb7353 Merge branch 'main' into feat/ip-adapter 2023-09-15 13:15:25 -04:00
Kent Keirsey
afe9756667
Merge branch 'main' into feat/taesd 2023-09-15 12:19:19 -04:00
Sergey Borisov
ff3150a818 Update lora hotfix to new diffusers version(scale argument added) 2023-09-15 12:19:01 -04:00
Ryan Dick
16664da5b6 black 2023-09-14 23:49:02 -04:00
Ryan Dick
c104807201 Update list of supported IP-Adapters. 2023-09-14 23:43:19 -04:00
Ryan Dick
990ce9a1da Lookup IP-Adapter linked image encoder from disk instead of storing in model config metadata. 2023-09-14 23:06:57 -04:00
Ryan Dick
c2f074dc2f Fix python static checks. 2023-09-14 16:48:47 -04:00
Ryan Dick
fca60862e2 Add README.md describing IP-Adapter model formats. 2023-09-14 16:02:07 -04:00
Ryan Dick
94c186bb4c Fix bug in IPAdapter.to(...). 2023-09-14 15:45:25 -04:00
Ryan Dick
a22c8cb3a1 Improve robustness of check for IPAdapter vs IPAdapterPlus. 2023-09-14 15:25:41 -04:00
Ryan Dick
781e8521d5 Eliminate the need for IPAdapter.initialize(). 2023-09-14 15:02:59 -04:00
Ryan Dick
d114d0ba95 Remove need for the image_encoder param in IPAdapter.initialize(). 2023-09-14 14:14:35 -04:00
Ryan Dick
388554448a Add CLIP Vision model to IP-Adapter info and use this to infer which model to use. 2023-09-14 11:57:53 -04:00
Ryan Dick
d5160648d0 Add support for downloading IP-Adapter models from HF. 2023-09-14 11:18:43 -04:00
Ryan Dick
6d0ea42a94 Get CLIPVision model download from HF working. 2023-09-14 09:54:10 -04:00
Ryan Dick
2c1100509f Add BaseModelType.Any to be used by CLIPVisionModel. 2023-09-14 08:19:55 -04:00
Ryan Dick
c34b359c36 (minor) Remove duplicate TODO. 2023-09-13 21:25:20 -04:00
Ryan Dick
77d135967f Update IPAdapterModel to respect requested torch_dtype. 2023-09-13 21:06:42 -04:00
Ryan Dick
ebf26687cb (minor) Remove unnecessary TODO. 2023-09-13 21:03:42 -04:00
Ryan
2f5e923008 Removed duplicate import in model_cache.py 2023-09-13 19:33:43 -04:00
Ryan
b7296000e4 made MPS calls conditional on MPS actually being the chosen device with backend available 2023-09-13 19:33:43 -04:00
Ryan
fab055995e Add empty_cache() for MPS hardware. 2023-09-13 19:33:43 -04:00
Ryan Dick
1c8991a3df Use CLIPVisionModel under model management for IP-Adapter. 2023-09-13 19:10:02 -04:00
Ryan Dick
3d52656176 Add CLIPVisionModel to model management. 2023-09-13 17:14:20 -04:00
Kevin Turner
090db1ab3a Merge remote-tracking branch 'origin/main' into feat/taesd 2023-09-13 09:17:53 -07:00
Ryan Dick
3ee9a21647 Initial (barely) working version of IP-Adapter model management. 2023-09-13 08:27:24 -04:00
Ryan Dick
0d823901ef Add IPAdapter to model_management __init__.py 2023-09-13 08:27:24 -04:00
Ryan Dick
7ee55489bb Improve model search warning messages. 2023-09-13 08:27:24 -04:00
Ryan Dick
163ece9aee Initial skeleton for IPAdapter model management. 2023-09-13 08:27:24 -04:00
Martin Kristiansen
e467ca7f1b Apply black, isort, flake8 2023-09-12 13:01:58 -04:00
Martin Kristiansen
e88d7c242f isort wip 3 2023-09-12 13:01:58 -04:00
Martin Kristiansen
caea6d11c6 isort wip 2 2023-09-12 13:01:58 -04:00
Millun Atluri
88db094cf2
Merge branch 'main' into feat/taesd 2023-09-11 22:11:25 +10:00
Ryan Dick
50a0691514 flake8 2023-09-08 18:05:31 -04:00
Ryan Dick
a255624984 black 2023-09-08 17:55:23 -04:00
Ryan Dick
2630fe3608 Remove unused ip_adapter/utils.py file. 2023-09-08 16:25:34 -04:00
Ryan Dick
6ca6cf713c Tidy IPAdapter. Add types, improve field/method naming. 2023-09-08 16:00:58 -04:00
Ryan Dick
3f7d5b4e0f Remove redundant IPAdapterXL class. 2023-09-08 15:46:10 -04:00
Ryan Dick
91596d9527 Re-factor IPAdapter to patch UNet in a context manager. 2023-09-08 15:39:22 -04:00
Ryan Dick
d669f0855d Comment unused IPAdapter generate(...) methods. 2023-09-08 13:12:42 -04:00
Ryan Dick
b2d5b53b5f Pass IP-Adapter conditioning via cross_attention_kwargs instead of concatenating to the text embedding. This avoids interference with other features that manipulate the text embedding (e.g. long prompts). 2023-09-08 11:47:36 -04:00
Ryan Dick
ddc148b70b Move ConditioningData and its field classes to their own file. This will allow new conditioning types to be added more cleanly without introducing circular dependencies. 2023-09-08 11:00:11 -04:00
Ryan Dick
c2d43f007b Specify the image_embedding_len in the IPAttnProcessor rather than the text embedding length. This enables the IPAttnProcessor to handle text embeddings of varying lengths. 2023-09-07 18:20:21 -04:00
Ryan Dick
7703bf2ca1 Delete IP-Adapter copies of AttnProcessor and AttnProcessor2_0, which were unmodified from diffusers. 2023-09-07 15:00:13 -04:00
Millun Atluri
b5e1ba34b3
Merge branch 'main' into refactor/rename-get-logger 2023-09-07 23:19:59 +10:00
Ryan Dick
23fdf0156f Clean up IP-Adapter in diffusers_pipeline.py - WIP 2023-09-06 20:42:20 -04:00
Ryan Dick
d776e0a0a9 Split ControlField and IpAdapterField. 2023-09-06 17:03:37 -04:00
Lincoln Stein
c54c1f603b
Merge branch 'main' into bugfix/set-vram-on-macs 2023-09-05 07:09:39 -04:00
blessedcoolant
f44496a579 Merge branch 'main' into feat/ip-adapter 2023-09-05 15:22:15 +12:00
psychedelicious
95ecb1a0c1 fix(ip_adapter): add None to types 2023-09-05 12:30:00 +10:00
blessedcoolant
a9fafad5b5 chore: sync, lint & update 2023-09-05 14:17:23 +12:00
Lincoln Stein
d2e327add9 install models/core/misc/lama/lama.pt 2023-09-04 21:54:40 -04:00
psychedelicious
58aa159a50 fix(backend): fix remaining instances of getLogger() 2023-09-05 10:43:30 +10:00
psychedelicious
d8f7c19030
Merge branch 'main' into refactor/rename-get-logger 2023-09-05 10:37:53 +10:00
blessedcoolant
3ac68cde66 chore: flake8 cleanup 2023-09-05 12:07:12 +12:00
blessedcoolant
65a76a086b cleanup: Some basic cleanup 2023-09-05 11:54:28 +12:00
blessedcoolant
07381e5a26 cleanup: merge conflicts 2023-09-05 11:37:12 +12:00
blessedcoolant
6bb378a101 Merge branch 'main' into feat/ip-adapter 2023-09-05 11:35:19 +12:00
blessedcoolant
917c2c480e Merge branch 'main' into lama-infill 2023-09-03 23:16:34 +12:00
blessedcoolant
f1f7778e73
Merge branch 'main' into chore/clean-up-unused-files 2023-09-03 02:59:31 +12:00
Lincoln Stein
c965d3eb6b
Merge branch 'main' into bugfix/set-vram-on-macs 2023-09-02 10:08:13 -04:00
blessedcoolant
ec09e21fc2 Merge branch 'main' into lama-infill 2023-09-02 23:02:38 +12:00
Kevin Turner
7df67d077a
Merge branch 'main' into feat/taesd 2023-09-01 22:18:40 -07:00
Lincoln Stein
6f6d920686
[Feature] Support the XL inpainting model (#4431)
* add StableDiffusionXLInpaintPipeline to probe list

* add StableDiffusionXLInpaintPipeline to probe list

* Blackified (?)

---------

Authored-by: Lincoln Stein <lstein@gmail.com>
Mucked about with to get it merged by: Kent Keirsey <31807370+hipsterusername@users.noreply.github.com>
2023-09-01 22:58:14 -04:00
psychedelicious
4b78deba92
Merge branch 'main' into bugfix/set-vram-on-macs 2023-09-02 11:33:20 +10:00
psychedelicious
b761807219
Merge branch 'main' into feat/ip-adapter 2023-09-02 11:31:08 +10:00
psychedelicious
9cb04f6f80 chore: remove unused files 2023-09-02 11:12:19 +10:00
blessedcoolant
48484e9fc8 Merge branch 'main' into lama-infill 2023-09-02 11:08:31 +12:00
blessedcoolant
a12fbc7406 chore: black fix 2023-09-02 10:51:53 +12:00
Sergey Borisov
5151798a16 Cleanup memory after model run 2023-09-01 20:50:39 +03:00
blessedcoolant
1a9f552a75 experimental: Add CV2 Infill 2023-09-02 04:48:18 +12:00
user1
fb1b03960e Added IP-Adapter SDXL support. Added IP-Adapter "Plus" (more detail) model support. 2023-09-01 04:40:30 -07:00
user1
74bfb5e1f9 First commit of separate node for IP-Adapter.
And it own dataclasses for passing info.
2023-08-31 23:07:15 -07:00
Kevin Turner
bc1bce18b0 Merge branch 'main' into feat/taesd 2023-08-31 20:26:10 -07:00
Lincoln Stein
44b49c7f2d fixed true source of problem 2023-08-31 22:55:17 -04:00
Lincoln Stein
52a5f1f56f prevent from trying to set vram on macs 2023-08-31 22:50:53 -04:00
blessedcoolant
b94ec14853 chore: Black lint fix 2023-09-01 09:19:10 +12:00
blessedcoolant
54cda8ea42 chore: Change LaMA log statement to use InvokeAI Logger 2023-09-01 09:17:41 +12:00
Lincoln Stein
b567d65032 blackify and rerun frontend build 2023-08-31 10:35:17 -04:00
user1
5a9993772d Added ip_adapter_strength parameter to adjust weighting of IP-Adapter's added cross-attention layers 2023-08-30 17:28:30 -07:00
user1
9f86cfa471 Working POC of IP-Adapters. Not fully nodified yet. 2023-08-30 17:28:30 -07:00
user1
8c1390166f Modifying code from https://github.com/tencent-ailab/IP-Adapter. Also adding license notice at top. 2023-08-30 17:28:30 -07:00
user1
1ad98ce999 Core ip_adapter files from https://github.com/tencent-ailab/IP-Adapter
Copied into InvokeAI since IP-Adapter repo is not a package. Is there a better way to do this for non-packaged Python code while still keeping InvokeAI install easy?
2023-08-30 17:28:30 -07:00
Lincoln Stein
83163ddd9a fix migrate script to work when autoimport directories are None 2023-08-30 18:46:17 -04:00
Sergey Borisov
8562dbaaa8 Hotfix to make second order schedulers work with mask 2023-08-30 02:18:08 +03:00
blessedcoolant
cfee8d9804 chore: seamless print statement cleanup 2023-08-29 13:09:30 +12:00
blessedcoolant
4196c669a0 chore: black / flake lint errors 2023-08-29 12:57:26 +12:00
blessedcoolant
a1398dec91 Merge branch 'main' into pr/4112 2023-08-29 12:56:59 +12:00
blessedcoolant
605e13eac0 chore: black fix 2023-08-29 07:50:17 +12:00
Kent Keirsey
2a1d7342a7 Seamless Patch from Stalker 2023-08-28 15:48:05 -04:00
blessedcoolant
99475ab800 chore: pyflake lint fixes 2023-08-29 05:16:23 +12:00
Sergey Borisov
2bf747caf6 Blackify 2023-08-28 18:36:27 +03:00
StAlKeR7779
cd548f73fd
Merge branch 'main' into feat_compel_and 2023-08-28 18:31:41 +03:00
Sergey Borisov
bb085c5fba Move monkeypatch for diffusers/torch bug to hotfixes.py 2023-08-28 18:29:49 +03:00
Kent Keirsey
421f5b7d75 Seamless Updates 2023-08-28 08:43:08 -04:00
blessedcoolant
3ef36707a8 chore: Black lint 2023-08-28 23:10:00 +12:00
Kent Keirsey
b9731cb434
Merge branch 'main' into Seamless 2023-08-28 00:12:23 -04:00
Kent Keirsey
1f476692da Seamless fixes 2023-08-28 00:10:46 -04:00
Kent Keirsey
5fdd25501b updates per stalkers comments 2023-08-27 22:54:53 -04:00
Sergey Borisov
b65c9ad612 Add monkeypatch for xformers to align unaligned attention_mask 2023-08-28 04:50:58 +03:00
Millun Atluri
24132a7950
Merge branch 'main' into refactor/rename-get-logger 2023-08-28 11:38:37 +10:00
StAlKeR7779
3e6c49001c
Change antialias to True as input - image
Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
2023-08-28 02:54:39 +03:00
Kent Keirsey
ea40a7844a add VAE 2023-08-27 14:53:57 -04:00
Kent Keirsey
0d2e194213 Fixed dict error 2023-08-27 14:21:56 -04:00
Kent Keirsey
3de45af734 updates 2023-08-27 14:13:00 -04:00
Sergey Borisov
526c7e7737 Provide antialias argument as behaviour will be changed in future(deprecation warning) 2023-08-27 20:04:55 +03:00
blessedcoolant
e9633a3adb Merge branch 'main' into fix/inpaint_gen 2023-08-27 02:54:19 +12:00