Commit Graph

6031 Commits

Author SHA1 Message Date
psychedelicious
e8f4012b56 feat(ui): extract FetchingModelsLoader into reusable component 2024-03-20 15:05:25 +11:00
psychedelicious
bc12ca9220 chore(ui): typegen 2024-03-20 15:05:25 +11:00
psychedelicious
5ceaeb234d feat(mm): add starter models route
The models from INITIAL_MODELS.yaml have been recreated as a structured python object. This data is served on a new route. The model sources are compared against currently-installed models to determine if they are already installed or not.
2024-03-20 15:05:25 +11:00
psychedelicious
429f87c60b fix(mm): HFModelSource string format
The dunder `__str__` method for `HFModelSource` was appending a colon `:` to the end of the source strings.
2024-03-20 15:05:25 +11:00
psychedelicious
ee3096f616 feat(config): add flag to indicate if args were parsed
This flag acts as a proxy for the `get_config()` function to determine if the full application is running.

If it was, the config will set the root, do HF login, etc.

If not (e.g. it's called by an external script), all that stuff will be skipped.
2024-03-20 15:05:25 +11:00
psychedelicious
6af6673a4f feat: move all config-related initialization to app
HF login, legacy yaml confs, and default init file are all handled during app setup.

All directories are created as they are needed by the app.

No need to check for a valid root dir - we will make it if it doesn't exist.
2024-03-20 15:05:25 +11:00
psychedelicious
b173e4c08d tidy(config): type checker ignores + comment 2024-03-20 15:05:25 +11:00
psychedelicious
059f869737 tidy(config): remove ignore_missing_core_models CLI arg and setting
This is now a no-op, with all models being downloaded when they are first requested.
2024-03-20 15:05:25 +11:00
psychedelicious
813e679b77 feat: add hf_login util
This provides a simple way to provide a HF token. If HF reports no valid token, one is prompted for until a valid token is provided, or the user presses Ctrl + C to cancel.
2024-03-20 15:05:25 +11:00
psychedelicious
857e9c9b5f feat: add SuppressOutput util
This context manager suppresses/hides stdout.
2024-03-20 15:05:25 +11:00
psychedelicious
5c1aa02e7b fix(config): set default legacy_conf_dir to configs
It was `configs/stable-diffusion` before, which broke conversions.
2024-03-20 15:05:25 +11:00
psychedelicious
6e882d3fd6 feat(config): dynamic ram cache size
Use the util function to calculate ram cache size on startup. This way, the `ram` setting will always be optimized for a system, even if they add or remove RAM. In other words, the default value is now dynamic.
2024-03-20 15:05:25 +11:00
psychedelicious
fabef8b45b feat(mm): download upscaling & lama models as they are requested 2024-03-20 15:05:25 +11:00
psychedelicious
97f16b2b7e fix(ui): fix model install progress display 2024-03-20 15:05:25 +11:00
dunkeroni
609c2c0abf Fix: progress image preview for inpainting 2024-03-20 13:36:05 +11:00
dunkeroni
fe5fa7f8cc chore: make ruff 2024-03-20 13:36:05 +11:00
dunkeroni
8b30cbe81e chore: clean up old code comments 2024-03-20 13:36:05 +11:00
dunkeroni
2af9286345 fix: denoise mask incorectly applied after step 2024-03-20 13:36:05 +11:00
psychedelicious
29b04b7e83 chore: bump nodes versions
Bump all nodes in prep for v4.0.0.
2024-03-20 10:28:07 +11:00
Mary Hipp
39fa8874fc undo 2024-03-20 10:05:46 +11:00
Mary Hipp
4e245e9331 fix refiner metadata 2024-03-20 10:05:46 +11:00
Lincoln Stein
74a51571a0
Fix race condition causing hangs during model install unit tests (#5994)
* fix race condition causing hangs during model install unit tests

* remove extraneous sanity checks

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-03-19 20:54:49 +00:00
Lincoln Stein
c87497fd54
record model_variant in t2i and clip_vision configs (#5989)
- Move base of t2i and clip_vision config models to DiffusersBase, which contains
  a field to record the model variant (e.g. "fp16")
- This restore the ability to load fp16 t2i and clip_vision models
- Also add defensive coding to load the vanilla model when the fp16 model
  has been replaced (or more likely, user's preferences changed since installation)

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-03-19 20:14:12 +00:00
psychedelicious
3f61c51c3a fix(ui): model list refreshes after changes
When consolidating all the model queries I messed up the query tags. Fixed now, so that when a model is installed, removed, or changed, the list refreshes.
2024-03-20 06:25:57 +11:00
Васянатор
07c9c0b0ab translationBot(ui): update translation (Russian)
Currently translated at 99.5% (1091 of 1096 strings)

Co-authored-by: Васянатор <ilabulanov339@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/ru/
Translation: InvokeAI/Web UI
2024-03-19 22:34:53 +11:00
Gohsuke Shimada
2322d3cbbe translationBot(ui): update translation (Japanese)
Currently translated at 52.5% (576 of 1096 strings)

translationBot(ui): update translation (Japanese)

Currently translated at 52.0% (570 of 1096 strings)

Co-authored-by: Gohsuke Shimada <ghoskay@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/ja/
Translation: InvokeAI/Web UI
2024-03-19 22:34:53 +11:00
Riccardo Giovanetti
419ce02aae translationBot(ui): update translation (Italian)
Currently translated at 98.2% (1077 of 1096 strings)

translationBot(ui): update translation (Italian)

Currently translated at 98.2% (1077 of 1096 strings)

Co-authored-by: Riccardo Giovanetti <riccardo.giovanetti@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/it/
Translation: InvokeAI/Web UI
2024-03-19 22:34:53 +11:00
Hosted Weblate
629ccd059e translationBot(ui): update translation files
Updated by "Cleanup translation files" hook in Weblate.

translationBot(ui): update translation files

Updated by "Cleanup translation files" hook in Weblate.

Co-authored-by: Hosted Weblate <hosted@weblate.org>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/
Translation: InvokeAI/Web UI
2024-03-19 22:34:53 +11:00
Васянатор
2dfa51c2e5 translationBot(ui): update translation (Russian)
Currently translated at 99.0% (1518 of 1533 strings)

translationBot(ui): update translation (Russian)

Currently translated at 99.0% (1518 of 1533 strings)

Co-authored-by: Васянатор <ilabulanov339@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/ru/
Translation: InvokeAI/Web UI
2024-03-19 22:34:53 +11:00
Sufi2425
edde0fe174 translationBot(ui): update translation (Bulgarian)
Currently translated at 3.9% (61 of 1533 strings)

translationBot(ui): update translation (Bulgarian)

Currently translated at 1.8% (28 of 1533 strings)

translationBot(ui): added translation (Bulgarian)

Co-authored-by: Sufi2425 <sufi24251@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/bg/
Translation: InvokeAI/Web UI
2024-03-19 22:34:53 +11:00
Riccardo Giovanetti
3e46f7a010 translationBot(ui): update translation (Italian)
Currently translated at 97.8% (1510 of 1543 strings)

translationBot(ui): update translation (Italian)

Currently translated at 98.1% (1503 of 1532 strings)

translationBot(ui): update translation (Italian)

Currently translated at 98.1% (1503 of 1532 strings)

Co-authored-by: Riccardo Giovanetti <riccardo.giovanetti@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/it/
Translation: InvokeAI/Web UI
2024-03-19 22:34:53 +11:00
psychedelicious
faa555df20 chore(ui): lint 2024-03-19 22:11:48 +11:00
psychedelicious
7a3e19227f feat(ui): display created_by using valueOrNull 2024-03-19 22:11:48 +11:00
psychedelicious
e706afe8a6 feat(ui): add valueOrNull to useMetadataItem
In order to allow for null and undefined metadata values, this hook returned a symbol to indicate that parsing failed or was pending.

For values where the parsed value will never be null or undefined, it is useful get the value or null (instead of a symbol).
2024-03-19 22:11:48 +11:00
psychedelicious
acca197893 revert(ui): restore metadata parsers for created_by 2024-03-19 22:11:48 +11:00
Mary Hipp
aa2c404cab move created_by out of recall panel 2024-03-19 22:11:48 +11:00
Mary Hipp
300a4693ae prettier 2024-03-19 21:59:51 +11:00
maryhipp
820614e4d8 ruff 2024-03-19 21:59:51 +11:00
Mary Hipp
fe563f05fc tsc 2024-03-19 21:59:51 +11:00
Mary Hipp
d89e653588 fix(ui): remove image_resolution from colormap 2024-03-19 21:59:51 +11:00
maryhipp
4e9207a10b fix(worker): remove resolution from zoe as it seems to break it 2024-03-19 21:59:51 +11:00
Mary Hipp
8c6c33a315 'feat(ui): update processor constants to calculate default resolution based on current base model, add image_resolution to the processors that didn't have it in the UI as a configurable op
tion
2024-03-19 21:59:51 +11:00
maryhipp
ed0f9f7d66 feat(worker): add image_resolution as option for all cnet procesors 2024-03-19 21:59:51 +11:00
Mary Hipp
b25850a585 typegen 2024-03-19 21:59:51 +11:00
Brandon Rising
9d5b96c119 Pull format type setting out of model_type if statement 2024-03-19 01:16:11 -04:00
Brandon Rising
5daefccf77 Simplify logic for determining model type in probe 2024-03-19 01:16:11 -04:00
Brandon Rising
1f3c35ee90 Run Ruff 2024-03-19 01:16:11 -04:00
Brandon Rising
f78ed3a952 Add unit test 2024-03-19 01:16:11 -04:00
Brandon Rising
d38262a7ea Allow type field to be a string 2024-03-19 01:16:11 -04:00
Brandon Rising
5feb62d440 Allow users to specify model type and skip detection step of probe 2024-03-19 01:16:11 -04:00
Lincoln Stein
f8df293d2c Revert "fix(mm): provide ckpt config as stream to diffusers"
This reverts commit 9d045964d6.
2024-03-19 14:24:54 +11:00
psychedelicious
9d045964d6 fix(mm): provide ckpt config as stream to diffusers
Fixes converting ckpt main models
2024-03-19 09:24:28 +11:00
psychedelicious
9fa9ebe386 fix(config): set ignore_missing_core_models when provided as CLI arg 2024-03-19 09:24:28 +11:00
psychedelicious
1cb1b60b4c tidy: "check_root.py" -> "check_directories.py" 2024-03-19 09:24:28 +11:00
psychedelicious
1d4517d00d tidy: "validate_root" -> "validate_directories" 2024-03-19 09:24:28 +11:00
psychedelicious
982b513af3 tidy(config): move a few get_config calls to inside the functions where they are needed 2024-03-19 09:24:28 +11:00
Lincoln Stein
f1450c2c24 update textual inversion training to use root_path rather than root_dir 2024-03-19 09:24:28 +11:00
Lincoln Stein
5d16a40b95 fix invokeai-configure to use isolated argument-parsing pattern 2024-03-19 09:24:28 +11:00
Lincoln Stein
d871fca643 partially address --root CLI argument handling
- fix places where `get_config()` is being called at import time rather
  than at run time.

- add regression test for import time get_config() calling.
2024-03-19 09:24:28 +11:00
Lincoln Stein
8cd65755ef make invokeai-model-install use the --root argument correctly 2024-03-19 09:24:28 +11:00
psychedelicious
e76cc71e81 fix(config): edge cases in models.yaml migration
When running the configurator, the `legacy_models_conf_path` was stripped when saving the config file. Then the migration logic didn't fire correctly, and the custom models.yaml paths weren't migrated into the db.

- Rework the logic to migrate this path by adding it to the config object as a normal field that is not excluded from serialization.
- Rearrange the models.yaml migration logic to remove the legacy path after migrating, then write the config file. This way, the legacy path doesn't stick around.
- Move the schema version into the config object.
- Back up the config file before attempting migration.
- Add tests to cover this edge case
2024-03-19 09:24:28 +11:00
psychedelicious
4633242503 tidy(config): move config docstring builder to its script 2024-03-19 09:24:28 +11:00
psychedelicious
e8b030427d fix(config): do not discard conf_path, migrate custom models.yaml
Hold onto `conf_path` temporarily while migrating `invokeai.yaml` so that it gets migrated correctly as the model installer starts up. Stashed as `legacy_models_yaml_path` in the config, excluded from serialization.
2024-03-19 09:24:28 +11:00
psychedelicious
415a4baf78 docs: add note about pydantic-settings' yaml support 2024-03-19 09:24:28 +11:00
psychedelicious
e32c609fec fix(config): ignore empty environment variables (use default values instead) 2024-03-19 09:24:28 +11:00
psychedelicious
a281671e6c docs: update InvokeAIAppConfig doc generator
It now renders the valid values.
2024-03-19 09:24:28 +11:00
psychedelicious
5179587b5a feat(config): restore ignore_missing_core_models arg 2024-03-19 09:24:28 +11:00
psychedelicious
cb180909f7 fix(install): resolve config-related issues with configurator
- Do not use the singleton app config, this causes a lot of weirdness
- Update logic to use new config object
- Remove unused code
2024-03-19 09:24:28 +11:00
psychedelicious
ce9aeeece3 feat: single app entrypoint with CLI arg parsing
We have two problems with how argparse is being utilized:
- We parse CLI args as the `api_app.py` file is read. This causes a problem pytest, which has an incompatible set of CLI args. Some tests import the FastAPI app, which triggers the config to parse CLI args, which receives the pytest args and fails.
- We've repeatedly had problems when something that uses the config is imported before the CLI args are parsed. When this happens, the root dir may not be set correctly, so we attempt to operate on incorrect paths.

To resolve these issues, we need to lift CLI arg parsing outside of the application code, but still let the application access the CLI args. We can create a external app entrypoint to do this.

- `InvokeAIArgs` is a simple helper class that parses CLI args and stores the result.
- `run_app()` is the new entrypoint. It first parses CLI args, then runs `invoke_api` to start the app.

The `invokeai-web` project script and `invokeai-web.py` dev script now call `run_app()` instead of `invoke_api()`.

The first time `get_config()` is called to get the singleton config object, it retrieves the args from `InvokeAIArgs`, sets the root dir if provided, then merges settings in from `invokeai.yaml`.

CLI arg parsing is now safely insulated from application code, but still accessible. And we don't need to worry about import order having an impact on anything, because by the time the app is running, we have already parsed CLI args. Whew!
2024-03-19 09:24:28 +11:00
psychedelicious
d09f03ef25 fix(config): if no invokeai.yaml is found, create a default one
This fixes an issue with `test_images.py`, which tests the bulk images routers and imports the whole FastAPI app. This triggers the config logic which fails on the test runner, because it has no `invokeai.yaml`.

Also probably just good for graceful fallback.
2024-03-19 09:24:28 +11:00
psychedelicious
3f8e2bfd18 fix(config): migrate deprecated max_cache_size and max_vram_cache_size settings 2024-03-19 09:24:28 +11:00
psychedelicious
60492500db chore: ruff 2024-03-19 09:24:28 +11:00
psychedelicious
f69938c6a8 fix(config): revised config methods
- `write_file` requires an destination file path
- `read_config` -> `merge_from_file`, if no path is provided, reads from `self.init_file_path`
- update app, tests to use new methods
- fix configurator, was overwriting config file data unexpectedly
2024-03-19 09:24:28 +11:00
psychedelicious
5e39e46954 feat(config): more resiliant update_config method
Only set values that have changed.
2024-03-19 09:24:28 +11:00
psychedelicious
1079bf3ccf feat(config): fix bad compress_level setting
Tweak the name of it so that incoming configs with the old default value of 6 have the setting stripped out. The result is all configs will now have the new, much better default value of 1.
2024-03-19 09:24:28 +11:00
psychedelicious
53c8f36029 docs(config): clarify comment during config migration 2024-03-19 09:24:28 +11:00
psychedelicious
b9884a6166 feat(config): split out parse_args and read_config logic from get_config
Having this all in the `get_config` function makes testing hard. Move these two functions to their own methods, and call them on app startup explicitly.
2024-03-19 09:24:28 +11:00
psychedelicious
77b86e9ad5 fix(install): remove broken v2.3 -> v3 migration logic from configurator 2024-03-19 09:24:28 +11:00
psychedelicious
a6181b5759 fix(install): update configurator to use new config system 2024-03-19 09:24:28 +11:00
psychedelicious
b4b0af7c60 fix(install): do not use deprecated pydantic methods 2024-03-19 09:24:28 +11:00
psychedelicious
3d1f3818cb fix(config): use set_root to set root 2024-03-19 09:24:28 +11:00
psychedelicious
deffeb9655 fix(config): use get_config singleton, new paths 2024-03-19 09:24:28 +11:00
psychedelicious
b8c46fb15b fix(config): split check_invokeai_root into separate function to validate, use this in model_install to determine if need to run configurator 2024-03-19 09:24:28 +11:00
psychedelicious
9539ecce79 fix(config): use correct config in textual_inversion_training 2024-03-19 09:24:28 +11:00
psychedelicious
7716a4a8c7 fix(config): use correct config in install_helper 2024-03-19 09:24:28 +11:00
psychedelicious
dedce2d896 fix(config): remove unnecessary resolve on config path 2024-03-19 09:24:28 +11:00
psychedelicious
897fe497dc fix(config): use new get_config across the app, use correct settings 2024-03-19 09:24:28 +11:00
psychedelicious
7b1f9409bc fix(config): drop nonexistent config.use_cpu setting 2024-03-19 09:24:28 +11:00
psychedelicious
a72cea014c fix(config): drop usage of deprecated config.xformers, just use the existing utility function 2024-03-19 09:24:28 +11:00
psychedelicious
b4182b190f fix(config): use new config.patchmatch 2024-03-19 09:24:28 +11:00
psychedelicious
22ac204678 fix(config): fix invisible_watermark handling
This setting was hardcoded to True. Simplified logic around it to not have a conditional that does nothing.
2024-03-19 09:24:28 +11:00
psychedelicious
7ca447ded1 fix(config): use new config setup in api_app.py 2024-03-19 09:24:28 +11:00
psychedelicious
4df28f1de6 fix(config): use yaml module instead of omegaconf when migrating models.yaml
Also use new paths.
2024-03-19 09:24:28 +11:00
psychedelicious
ebd0cb6113 fix(config): remove reference to internet_available
Nothing ever set this. Only a debug print statement referenced it.
2024-03-19 09:24:28 +11:00
psychedelicious
fbe3afa5e1 fix(config): fix nsfw_checker handling
This setting was hardcoded to True. Rework logic around it to not conditionally check the setting.
2024-03-19 09:24:28 +11:00
psychedelicious
3fb116155b refactor(config): simplified config
- Remove OmegaConf. It functioned as an intermediary data format, between YAML/argparse and pydantic. It's not necessary - we can parse YAML or CLI args directly with pydantic.

- Remove dynamic CLI args. Only `root` is explicitly supported. This greatly simplifies config handling. Configuration is done by editing the YAML file. Frequently-used args can be added if there is a demand.

- A separate arg parser is created to handle the slimmed-down CLI args. It's run immediately in the `invokeai-web` script to handle `--version` and `--help`. It is also used inside the singleton config getter (see below).

- Remove categories from the config. Our settings model is mostly flat. Handling categories adds complexity for both us and users - we have to handle transforming a flat config to categorized config (and vice-versa), while users have to be careful with indentation in their YAML file.

- Add a `meta` key to the config file. Currently, this holds the config schema version only. It is not a part of the config object itself.

- Remove legacy settings that are no longer referenced, or were effectively no-op settings when referenced in code.

- Implement simple migration logic to for v3 configs. If migration is successful, the v3 config file is backed up to `invokeai.yaml.bak` and the new config written to `invokeai.yaml`.

- Previously, the singleton config was accessed by calling `InvokeAIAppConfig.get_config()`. This returned an instance of `InvokeAIAppConfig`, which _also_ has the `get_config` function. This created to a confusing situation where you weren't sure if you needed to call `get_config` or just use the config object. This method is replaced by a standalone `get_config` function which returns a singleton config object.

- Wrap CLI arg parsing (for `root`) and loading/migrating `invokeai.yaml` into the new `get_config()` function.

- Move `generate_config_docstrings` into standalone utility function.

- Make `root` a private attr (`_root`). This reduces the temptation to directly modify and or use this sensitive field and ensures it is neither serialized nor read from input data. Use `root_path` to access the resolved root path, or `set_root` to set the root to something.
2024-03-19 09:24:28 +11:00
Lincoln Stein
7387b0bdc9
install missing clip_vision encoders if required by an ip adapter (#5982)
Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-03-18 02:19:53 +00:00
Lincoln Stein
7ea9cac9a3
Add sdxl controlnet models (#5980)
* allow removal of models with legacy relative path addressing

* added five controlnet models for sdxl to INITIAL_MODELS

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-03-18 01:15:15 +00:00
Brandon Rising
ea5bc94b9c Resolve when instantiating _cached_model_paths 2024-03-18 11:17:23 +11:00
Brandon Rising
a1743647b7 Stop registering and moving models which have symlinks in the models dir 2024-03-18 11:17:23 +11:00
Lincoln Stein
71a1740740 Remove core safetensors->diffusers conversion models
- No longer install core conversion models. Use the HuggingFace cache to load
  them if and when needed.

- Call directly into the diffusers library to perform conversions with only shallow
   wrappers around them to massage arguments, etc.

- At root configuration time, do not create all the possible model subdirectories,
  but let them be created and populated at model install time.

- Remove checks for missing core conversion files, since they are no
  longer installed.
2024-03-17 19:13:18 -04:00
Lincoln Stein
a0420d1442 fix ruff error 2024-03-17 14:01:04 -04:00
Lincoln Stein
a17021ba0c allow removal of models with legacy relative path addressing 2024-03-17 09:58:16 -04:00
Mary Hipp
8c04eec210 fix initial main model logic 2024-03-15 10:22:16 -04:00
Brandon Rising
330e1354b4 Run typegen, update version to 4.0.0rc2 2024-03-14 17:01:36 -04:00
psychedelicious
21621eebf0 feat(ui): handle control adapter processed images
- Add helper functions to build metadata for control adapters, including the processed images
- Update parses to parse the new metadata
2024-03-14 12:34:03 -07:00
psychedelicious
c24f2046e7 chore(ui): typegen 2024-03-14 12:34:03 -07:00
psychedelicious
297408d67e feat(nodes): add control adapter processed images to metadata
In the client, a controlnet or t2i adapter has two images:
- The source control image: the image the user selected (required)
- The processed control image: the user's image after we've processed it (optional)

The processed image is optional because a user may provide a pre-processed image.

We only actually use one of these images when building the graph, and until this change, we only stored one of the in image metadata. This created a situation where only a processed image was stored in metadata - say, a canny edge map - and the user-selected image wasn't provided.

By adding the processed image to metadata, we can recall both the control image and optional processed image.

This commit is followed by a UI-facing changes to support the change.
2024-03-14 12:34:03 -07:00
psychedelicious
0131e7d928 fix(ui): recall control adapter metadata fields 2024-03-14 12:34:03 -07:00
psychedelicious
06ff105a1f fix(ui): reset loras/control adapters when using recall all or remix 2024-03-14 12:34:03 -07:00
psychedelicious
328dc99f3a fix(ui): log model load events
- Fix types
- Fix logging in listener
2024-03-14 18:29:55 +05:30
psychedelicious
ef55077e84 feat(events): add submodel_type to model load events
This was lost during MM2 migration
2024-03-14 18:29:55 +05:30
psychedelicious
ba3d8af161 fix(events): dump event payloads to serializable format 2024-03-14 18:29:55 +05:30
psychedelicious
19d66d5ec7 feat(ui): single getModelConfigs query
Single query, with simple wrapper hooks (type-safe). Updated everywhere in frontend.
2024-03-14 23:37:40 +11:00
psychedelicious
fed1f983db fix(nodes): depth anything processor (#5956)
We were passing a PIL image when we needed to pass the np image.

Closes #5956
2024-03-14 20:14:53 +11:00
psychedelicious
a386544a1d chore: ruff 2024-03-14 17:32:02 +11:00
Lincoln Stein
0851de9090 closes #5932 2024-03-14 17:32:02 +11:00
Lincoln Stein
1bd8e33f8c Work around missing core conversion model issue
- This adds additional logic to the safetensors->diffusers conversion script
  to check for and install missing core conversion models at runtime.

- Fixes #5934
2024-03-14 16:52:01 +11:00
psychedelicious
3fd824306c feat(mm): probe for main model default settings
Currently, this is just the width and height, derived from the model base.
2024-03-14 16:03:37 +11:00
psychedelicious
2584a950aa feat(ui): add w/h to default model settings 2024-03-14 16:03:37 +11:00
psychedelicious
1adaf63253 chore(ui): typegen 2024-03-14 16:03:37 +11:00
psychedelicious
b9f1a4bd65 feat(nodes): add w/h defaults for models 2024-03-14 16:03:37 +11:00
psychedelicious
731942dbed feat(nodes): add constraints & descriptions to default settings 2024-03-14 16:03:37 +11:00
psychedelicious
4117cea5bf tidy(mm): remove misplaced comment 2024-03-14 15:54:42 +11:00
psychedelicious
21617f3bc1 docs: update description for hashing_algorithm in config 2024-03-14 15:54:42 +11:00
psychedelicious
9fcd67b5c0 feat(mm): add algorithm prefix to hashes
For example:
- md5:a0cd925fc063f98dbf029eee315060c3
- sha1:9e362940e5603fdc60566ea100a288ba2fe48b8c
- blake3:ce3f0c5f3c05d119f4a5dcaf209b50d3149046a0d3a9adee9fed4c83cad6b4d0
2024-03-14 15:54:42 +11:00
psychedelicious
a4be935458 docs: update config docs 2024-03-14 15:54:42 +11:00
psychedelicious
eb6e6548ed feat(mm): faster hashing for spinning disk HDDs
BLAKE3 has poor performance on spinning disks when parallelized. See https://github.com/BLAKE3-team/BLAKE3/issues/31

- Replace `skip_model_hash` setting with `hashing_algorithm`. Any algorithm we support is accepted.
- Add `random` algorithm: hashes a UUID with BLAKE3 to create a random "hash". Equivalent to the previous skip functionality.
- Add `blake3_single` algorithm: hashes on a single thread using BLAKE3, fixes the aforementioned performance issue
- Update model probe to accept the algorithm to hash with as an optional arg, defaulting to `blake3`
- Update all calls of the probe to use the app's configured hashing algorithm
- Update an external script that probes models
- Update tests
- Move ModelHash into its own module to avoid circuclar import issues
2024-03-14 15:54:42 +11:00
Josh Corbett
8287fcf097 feat: ✏️ rename "Workflow Editor" tab label to "Workflows" 2024-03-14 12:22:23 +11:00
psychedelicious
dd475e28ed chore(ui): remove unused translation keys via script 2024-03-14 11:38:29 +11:00
psychedelicious
24e741e2d1 feat(ui): add script to clean translations
This script removes unused translations from the `en.json` source translation file:
- Parse `en.json` to build a list of all keys, e.g. `controlnet.depthAnything`
- Check every frontend file for every key
- If the key is not found, it is removed from the translation file
- Exact matches (e.g. `controlnet.depthAnything`) and stem matches (e.g. `depthAnything`) are ignored
2024-03-14 11:38:29 +11:00
psychedelicious
e0bf9ce5c6 tidy(ui): use normal quotes in translations 2024-03-14 11:38:29 +11:00
psychedelicious
c66e8b395e fix(ui): remove unused input on depth anything processor node 2024-03-14 10:53:57 +11:00
psychedelicious
4c417adc82 fix(ui): use revised metadata model types
We can also totally remove the fetch logic because we store the same model data in state now.
2024-03-14 10:53:57 +11:00
psychedelicious
437a413ca3 chore(ui): typegen 2024-03-14 10:53:57 +11:00
psychedelicious
4492bedd19 tidy(nodes): use ModelIdentifierField for model metadata
Until recently, this had a different shape than the ModelMetadataField. They are now the same, so we can re-use the ModelIdentifierField.
2024-03-14 10:53:57 +11:00
psychedelicious
db12ce95a8 fix(ui): invalid collect node error w/ control adapters
The graph builders used awaited functions within `Array.prototype.forEach` loops. This doesn't do what you'd think. This caused graphs to be enqueued before they were fully constructed.

 Changed to `for..of` loops to fix this.
2024-03-14 10:53:57 +11:00
psychedelicious
ee3a1a95ef fix(ui): control adapters require control images
There wasn't enough validation of control adapters during graph building. It would be possible for a graph to be built with empty collect node, causing an error. Addressed with an extra check.

This should never happen in practice, because the invoke button should be disabled if an invalid CA is active.
2024-03-14 10:53:57 +11:00
psychedelicious
4bb5aba70e feat(ui): only fetch TIs on first load, add comment 2024-03-14 07:38:09 +11:00
Mary Hipp
cd55c23713 initiate TI model query when socket connects so user doesnt have to wait when opening prompt trigger phrases 2024-03-14 07:38:09 +11:00
Mary Hipp
1d2743af1b remove log 2024-03-14 07:25:48 +11:00
Mary Hipp
99d2099ccd add key for controladapter CustomSelect too 2024-03-14 07:25:48 +11:00
Mary Hipp
b64a693f16 try adding a key to force rerender when items load 2024-03-14 07:25:48 +11:00
blessedcoolant
af660163ca chore: cleanup DepthAnything code 2024-03-13 20:35:52 +05:30
psychedelicious
c73f58e486 docs: move frontend docs to mkdocs 2024-03-13 22:43:20 +11:00
psychedelicious
614fece147 chore(ui): prettier 2024-03-13 21:02:29 +11:00
psychedelicious
8ef8082d65 feat(ui): style add model panel 2024-03-13 21:02:29 +11:00
psychedelicious
d93d4afbb7 feat(ui): style HF scan tab 2024-03-13 21:02:29 +11:00
psychedelicious
01207a2fa5 fix(mm): config.json to indicates diffusers model 2024-03-13 21:02:29 +11:00
Jennifer Player
d0800c4888 ui consistency, moved is_diffusers logic to backend, extended HuggingFaceMetadata, removed logic from service 2024-03-13 21:02:29 +11:00
Jennifer Player
2a300ecada updated add model copy, added search to hugging face results 2024-03-13 21:02:29 +11:00
Jennifer Player
90340a39c7 clean up python errors 2024-03-13 21:02:29 +11:00
Jennifer Player
ee77abb4fe updated simple install button to match other tabs 2024-03-13 21:02:29 +11:00
Jennifer Player
004bca5c42 updated endpoint types 2024-03-13 21:02:29 +11:00
Jennifer Player
5ad048a161 fixed error handling 2024-03-13 21:02:29 +11:00
Jennifer Player
6369ccd05e added placeholders, updated some copy 2024-03-13 21:02:29 +11:00
Jennifer Player
3a5314f1ca install model if diffusers or single file, cleaned up backend logic to not mess with existing model install 2024-03-13 21:02:29 +11:00
Jennifer Player
4c0896e436 removed log 2024-03-13 21:02:29 +11:00
Jennifer Player
f7cd3cf1f4 added hf models import tab and route for getting available hf models 2024-03-13 21:02:29 +11:00
Mary Hipp
d0d695c020 disable trigger phrase form if empty 2024-03-12 21:08:15 -04:00
Jennifer Player
2a648da557 updated model manager to display when import item is cancelled 2024-03-13 09:18:05 +11:00
Brandon Rising
8d2a4db902 Found another instance of expecting a mid_block on the decoder in a vae 2024-03-12 12:11:38 -04:00
Brandon Rising
7b393656de Update l2i invoke and seamless to support AutoencoderTiny, remove attention processors if no mid_block is detected 2024-03-12 12:00:24 -04:00
psychedelicious
43948e0758 feat(ui): add setting for always show image size badge 2024-03-12 18:52:23 +11:00
psychedelicious
cc03fcbcb6 style(ui): tweak image dimension badge overlay styles 2024-03-12 18:52:23 +11:00
Rohinish
d1e445fa49 fix(ui): changed to theme tokens 2024-03-12 18:52:23 +11:00
Rohinish
adba8489f2 fix(ui): made changes to avoid overlapping 2024-03-12 18:52:23 +11:00
Rohinish
d919022ba5 fix(ui): fixed requested changes and made the badge display on hover 2024-03-12 18:52:23 +11:00
Rohinish
e076898798 fix(ui): logic to remove badge for small image size 2024-03-12 18:52:23 +11:00
Rohinish
9f19b766a4 feat(ui): Add image size badge to gallery images 2024-03-12 18:52:23 +11:00
Brandon Rising
be951da99d {release} 4.0.0rc1 2024-03-12 10:05:03 +11:00
Ryan Dick
9ee2e7ff25 Do not override log_memory_usage when debug logs are enabled. The speed cost of log_memory_usage=True is large. It is common to want debug log without enabling log_memory_usage. 2024-03-12 09:48:50 +11:00
Brandon Rising
149ff758b9 Run ruff 2024-03-11 15:53:00 -04:00
Brandon Rising
65d415d5aa Remove redundant with_suffix call 2024-03-11 15:53:00 -04:00
Brandon Rising
c74c1927ec Gracefully error without deleting invokeai.yaml 2024-03-11 15:53:00 -04:00
Brandon Rising
c454ccc65c Run ruff 2024-03-11 15:53:00 -04:00
Brandon Rising
46fd3465ce Skip list logic if the list only contains primitives 2024-03-11 15:53:00 -04:00
Brandon Rising
97afa6e2a6 Allow lists of basemodel objects in omegaconf 2024-03-11 15:53:00 -04:00
psychedelicious
8c2ff794d5 fix(nodes): ip adapter uses valid ModelIdentifierField for image encoder model
- Add class method to `ModelIdentifierField` to construct the field from a model config
- Use this to construct a valid IP adapter model field
2024-03-10 17:28:58 -05:00
Ryan Dick
145bb45858 Remove dead code related to an old symmetry feature. 2024-03-10 00:13:18 -06:00
psychedelicious
9376b13435 fix(mm): models lose file extension when syncing
We were stripping the file extension from file models when  moving them in `_sync_model_path`. For example, `some_model.safetensors` would be moved to `some_model`, which of course breaks things.

Instead of using the model's name as the new path, use the model's path's last segment. This is the same behaviour for directories, but for files, it retains the file extension.
2024-03-10 13:36:09 +11:00
psychedelicious
eec82afd89 fix(mm): fix models.yaml backup filename
Was erroneously `models.bak`, now `models.yaml.bak`
2024-03-10 13:36:09 +11:00
psychedelicious
c47dbf7258 docs(mm): format docstrings for ModelSearch 2024-03-10 12:09:47 +11:00
psychedelicious
92b2e8186a tidy(mm): simplify types for ModelSearch
- Use `set` instead of `Set`
- Methods accept only `Path`s
2024-03-10 12:09:47 +11:00
psychedelicious
70a88c6b99 docs(mm): update docstrings for ModelSearch 2024-03-10 12:09:47 +11:00
psychedelicious
56e7c04475 tidy(mm): remove extraneous dependencies in model search
- `config` is unused
- `stats` is created on instantiation
- `logger` uses the app logger
2024-03-10 12:09:47 +11:00
psychedelicious
bd5b43c00d tidy(mm): ModelSearch cleanup
- No need for it to by a pydantic model. Just a class now.
- Remove ABC, it made it hard to understand what was going on as attributes were spread across the ABC and implementation. Also, there is no other implementation.
- Add tests
2024-03-10 12:09:47 +11:00
dunkeroni
631e789195 fix(canvas): create masked latents when None 2024-03-10 11:58:41 +11:00
psychedelicious
133c90e116 fix(ui): update all components and logic to use enriched ModelIdentifierField 2024-03-10 11:03:38 +11:00
psychedelicious
4433b78e59 chore(ui): typegen 2024-03-10 11:03:38 +11:00
psychedelicious
daeb766468 feat(api): add ModelIdentifierField to openapi schema
- Also add `ProgressImage`
2024-03-10 11:03:38 +11:00
psychedelicious
92b0d13d0e feat(nodes): "ModelField" -> "ModelIdentifierField", add hash/name/base/type 2024-03-10 11:03:38 +11:00
psychedelicious
5b51ebf1c4 docs: regenerate config docstrings 2024-03-10 10:38:52 +11:00
psychedelicious
59228643a9 docs: skip_model_hash -> model install category, use_memory_db -> development category 2024-03-10 10:38:52 +11:00
psychedelicious
b24657df11 docs: roll back adding examples to config docstrings
This isn't a valid docstring syntax and breaks the autogeneration
2024-03-10 10:38:52 +11:00
psychedelicious
d4686b7f64 fix(mm): yaml migration fixup
- If the metadata yaml has an invalid version, exist the app. If we don't, the app will crawl the models dir and add models to the db without having first parsed `models.yaml`. This should not happen often, as the vast majority of users are on v3.0.0 models.yaml files.
- Fix off-by-one error with models count (need to pop the `__metadata__` stanza
- After a successful migration, rename `models.yaml` to `models.yaml.bak` to prevent the migration logic from re-running on subsequent app startups.
2024-03-09 08:37:45 -06:00
psychedelicious
67163c2224 fix(mm): only move model files if necessary
The old logic to check if a model needed to be moved relied on the model path being a relative path. Paths are now absolute, causing this check to fail. We then assumed the paths were different and moved the model from its current location to, well, its current location.

Use more resilient method to check if a model should be moved.
2024-03-09 22:58:26 +11:00
Brandon Rising
f01e81d382 Run ruff 2024-03-08 18:46:17 -05:00
maryhipp
a50e0a4802 use correct key name from yaml 2024-03-08 18:46:17 -05:00
maryhipp
df0a5aa92a pass config_path to migration path, make sure it uses absolute path 2024-03-08 18:46:17 -05:00
Brandon Rising
0bd9a0a9ea Add ability to provide config examples in docs 2024-03-08 16:31:39 -05:00
Brandon Rising
4ae2cd242e Update to include remote_api_tokens in the config docs 2024-03-08 16:31:39 -05:00
psychedelicious
deb1d4eb14 docs: run script to update config class's docstring 2024-03-08 16:31:39 -05:00
psychedelicious
eba1fc1355 docs: autogenerated app config docs
mkdocs can autogenerate python class docs from its docstrings. Our config is a pydantic model.

It's tedious and error-prone to duplicate docstrings from the pydantic field descriptions to the class docstrings.

- Add helper function to generate a mkdocs-compatible docstring from the InvokeAIAppConfig class fields
2024-03-08 16:31:39 -05:00
psychedelicious
96702c395e feat(config): add deprecated category for config settings
It's not clear why these are still in the config class.
2024-03-08 16:31:39 -05:00
psychedelicious
3361aec065 docs(nodes): update config field descriptions 2024-03-08 16:31:39 -05:00
Brandon Rising
8ba4b2a150 Run ruff 2024-03-08 15:36:14 -05:00
Brandon Rising
df12e12e09 Run ruff 2024-03-08 15:36:14 -05:00
Brandon Rising
ee38fbe89c Remove check for models dir in model deletion, update tests to always assume the model path is an absolute path 2024-03-08 15:36:14 -05:00
Brandon Rising
6e2cef1db5 Remove instances making models relative to the model dir 2024-03-08 15:36:14 -05:00
Brandon Rising
b1f5ac4548 fix path 2024-03-08 15:36:14 -05:00
Brandon Rising
e52274ecac Experiment with using absolute paths within model management 2024-03-08 15:36:14 -05:00
maryhipp
66f0ff5b13 add ordering to search_by_attr that is used for model lists 2024-03-08 13:38:38 -06:00
Mary Hipp
cab5b64f0b only render convert button if ckpt model 2024-03-08 13:19:08 -06:00
blessedcoolant
a42812d78d ui(model_manager): Remember Scan Path 2024-03-08 14:05:57 -05:00
maryhipp
281222df3c remove old data migration from previous schema version 2024-03-08 13:10:27 -05:00
maryhipp
d5674150fa ruff 2024-03-08 13:02:04 -05:00
maryhipp
0cb2cf6644 wrap version check in try/except 2024-03-08 13:02:04 -05:00
maryhipp
da87266c9c remove log 2024-03-08 13:02:04 -05:00
maryhipp
35731a6f51 fix null description, add logging 2024-03-08 13:02:04 -05:00
Brandon Rising
a3dfa161a8 Run ruff 2024-03-08 13:02:04 -05:00
Brandon Rising
42d606f07c use register instead of heuristic import, get rid of typing warnings 2024-03-08 13:02:04 -05:00
maryhipp
9063b1ae61 on model manager start, look to see if yaml needs to be migrated and do it if so 2024-03-08 13:02:04 -05:00
Brandon Rising
6aae88bd88 Rerun typegen 2024-03-08 12:44:58 -05:00
psychedelicious
57c1954da7 feat(ui): use control adapter processor helper in metadata parser 2024-03-08 12:44:58 -05:00
psychedelicious
a10dccdd43 fix(mm): fix bug in control adapter probe default settings
Wasn't checking for matches correctly.
2024-03-08 12:44:58 -05:00
psychedelicious
a3570901f7 fix(ui): do not show default settings for refiner models 2024-03-08 12:44:58 -05:00
psychedelicious
fd457955bc feat(ui): update default settings for control adapters
- Split out main model defaults
- Add controlnet/t2i defaults (which includes only the preprocessor)
2024-03-08 12:44:58 -05:00
psychedelicious
1f69613f5d chore(ui): typegen 2024-03-08 12:44:58 -05:00
psychedelicious
7a87ebb3b2 fix(mm): add control adapter default settings to ModelRecordChanges schema
This is needed to update Control Adapter defaults.
2024-03-08 12:44:58 -05:00
psychedelicious
4ee4a801c6 feat(ui): update default settings for main models
Needed some massaging now that only main models get main model default settings.
2024-03-08 12:44:58 -05:00
psychedelicious
53b7f6be37 feat(ui): use default settings for control adapters for processor 2024-03-08 12:44:58 -05:00
psychedelicious
dbd7c94e7c chore(ui): typegen 2024-03-08 12:44:58 -05:00
psychedelicious
50bb9a6b41 fix(mm): remove default settings from IP adapter config 2024-03-08 12:44:58 -05:00
psychedelicious
13bb3c5e15 feat(mm): add control adapter default settings while probing 2024-03-08 12:44:58 -05:00
psychedelicious
80c2a4b925 feat(mm): add AnyDefaultSettings union 2024-03-08 12:44:58 -05:00
psychedelicious
8ce485b036 feat(mm): add default settings for control adapters
Only includes `preprocessor` at this time.
2024-03-08 12:44:58 -05:00
psychedelicious
6fc3e86061 tidy(mm): only main models get the main default settings 2024-03-08 12:44:58 -05:00
Brandon Rising
33ded359e6 Run typegen 2024-03-08 11:10:44 -05:00
psychedelicious
effbd8a1ba chore: ruff 2024-03-08 11:10:44 -05:00
psychedelicious
ddde355b09 fix(mm): add ui_type to model fields
Recently the schema for models was changed to a generic `ModelField`, and the UI was unable to derive the type of those fields. This didn't affect functionality, but it did break the styling of handles.

Add `ui_type` to the affected fields and update the UI to use the correct capitalizations.
2024-03-08 11:10:44 -05:00
psychedelicious
fe2c6f621a fix(ui): do not allow model add when no location is provided 2024-03-08 14:41:03 +11:00
psychedelicious
d0fcdbe8a3 tweak(ui): simplify layout of inplace install form elements 2024-03-08 14:41:03 +11:00
Mary Hipp
a28547b3dd make inplace optional, default to true 2024-03-08 14:41:03 +11:00
Mary Hipp
c7b2bdb846 allow inplace installs 2024-03-08 14:41:03 +11:00
psychedelicious
4a20377fef tidy(config): move version "setting" to new CLIArgs category
It's not actually a setting.
2024-03-08 13:59:59 +11:00
psychedelicious
ed803640f7 tidy(mm): move remote_api_tokens to new ModelInstall category 2024-03-08 13:59:59 +11:00
psychedelicious
576bb4a61d feat(mm): support generic API tokens via regex/token pairs in config
A list of regex and token pairs is accepted. As a file is downloaded by the model installer, the URL is tested against the provided regex/token pairs. The token for the first matching regex is used during download, added as a bearer token.
2024-03-08 13:59:59 +11:00
Brandon Rising
b6065d6328 Run ruff with newest version of ruff 2024-03-08 13:59:59 +11:00
Brandon Rising
04229f4a21 Run ruff 2024-03-08 13:59:59 +11:00