Commit Graph

1774 Commits

Author SHA1 Message Date
psychedelicious
eec82afd89 fix(mm): fix models.yaml backup filename
Was erroneously `models.bak`, now `models.yaml.bak`
2024-03-10 13:36:09 +11:00
psychedelicious
56e7c04475 tidy(mm): remove extraneous dependencies in model search
- `config` is unused
- `stats` is created on instantiation
- `logger` uses the app logger
2024-03-10 12:09:47 +11:00
dunkeroni
631e789195 fix(canvas): create masked latents when None 2024-03-10 11:58:41 +11:00
psychedelicious
daeb766468 feat(api): add ModelIdentifierField to openapi schema
- Also add `ProgressImage`
2024-03-10 11:03:38 +11:00
psychedelicious
92b0d13d0e feat(nodes): "ModelField" -> "ModelIdentifierField", add hash/name/base/type 2024-03-10 11:03:38 +11:00
psychedelicious
5b51ebf1c4 docs: regenerate config docstrings 2024-03-10 10:38:52 +11:00
psychedelicious
59228643a9 docs: skip_model_hash -> model install category, use_memory_db -> development category 2024-03-10 10:38:52 +11:00
psychedelicious
b24657df11 docs: roll back adding examples to config docstrings
This isn't a valid docstring syntax and breaks the autogeneration
2024-03-10 10:38:52 +11:00
psychedelicious
d4686b7f64 fix(mm): yaml migration fixup
- If the metadata yaml has an invalid version, exist the app. If we don't, the app will crawl the models dir and add models to the db without having first parsed `models.yaml`. This should not happen often, as the vast majority of users are on v3.0.0 models.yaml files.
- Fix off-by-one error with models count (need to pop the `__metadata__` stanza
- After a successful migration, rename `models.yaml` to `models.yaml.bak` to prevent the migration logic from re-running on subsequent app startups.
2024-03-09 08:37:45 -06:00
psychedelicious
67163c2224 fix(mm): only move model files if necessary
The old logic to check if a model needed to be moved relied on the model path being a relative path. Paths are now absolute, causing this check to fail. We then assumed the paths were different and moved the model from its current location to, well, its current location.

Use more resilient method to check if a model should be moved.
2024-03-09 22:58:26 +11:00
Brandon Rising
f01e81d382 Run ruff 2024-03-08 18:46:17 -05:00
maryhipp
a50e0a4802 use correct key name from yaml 2024-03-08 18:46:17 -05:00
maryhipp
df0a5aa92a pass config_path to migration path, make sure it uses absolute path 2024-03-08 18:46:17 -05:00
Brandon Rising
0bd9a0a9ea Add ability to provide config examples in docs 2024-03-08 16:31:39 -05:00
Brandon Rising
4ae2cd242e Update to include remote_api_tokens in the config docs 2024-03-08 16:31:39 -05:00
psychedelicious
deb1d4eb14 docs: run script to update config class's docstring 2024-03-08 16:31:39 -05:00
psychedelicious
eba1fc1355 docs: autogenerated app config docs
mkdocs can autogenerate python class docs from its docstrings. Our config is a pydantic model.

It's tedious and error-prone to duplicate docstrings from the pydantic field descriptions to the class docstrings.

- Add helper function to generate a mkdocs-compatible docstring from the InvokeAIAppConfig class fields
2024-03-08 16:31:39 -05:00
psychedelicious
96702c395e feat(config): add deprecated category for config settings
It's not clear why these are still in the config class.
2024-03-08 16:31:39 -05:00
psychedelicious
3361aec065 docs(nodes): update config field descriptions 2024-03-08 16:31:39 -05:00
Brandon Rising
8ba4b2a150 Run ruff 2024-03-08 15:36:14 -05:00
Brandon Rising
df12e12e09 Run ruff 2024-03-08 15:36:14 -05:00
Brandon Rising
ee38fbe89c Remove check for models dir in model deletion, update tests to always assume the model path is an absolute path 2024-03-08 15:36:14 -05:00
Brandon Rising
6e2cef1db5 Remove instances making models relative to the model dir 2024-03-08 15:36:14 -05:00
Brandon Rising
b1f5ac4548 fix path 2024-03-08 15:36:14 -05:00
Brandon Rising
e52274ecac Experiment with using absolute paths within model management 2024-03-08 15:36:14 -05:00
maryhipp
66f0ff5b13 add ordering to search_by_attr that is used for model lists 2024-03-08 13:38:38 -06:00
maryhipp
281222df3c remove old data migration from previous schema version 2024-03-08 13:10:27 -05:00
maryhipp
d5674150fa ruff 2024-03-08 13:02:04 -05:00
maryhipp
0cb2cf6644 wrap version check in try/except 2024-03-08 13:02:04 -05:00
maryhipp
da87266c9c remove log 2024-03-08 13:02:04 -05:00
maryhipp
35731a6f51 fix null description, add logging 2024-03-08 13:02:04 -05:00
Brandon Rising
a3dfa161a8 Run ruff 2024-03-08 13:02:04 -05:00
Brandon Rising
42d606f07c use register instead of heuristic import, get rid of typing warnings 2024-03-08 13:02:04 -05:00
maryhipp
9063b1ae61 on model manager start, look to see if yaml needs to be migrated and do it if so 2024-03-08 13:02:04 -05:00
psychedelicious
7a87ebb3b2 fix(mm): add control adapter default settings to ModelRecordChanges schema
This is needed to update Control Adapter defaults.
2024-03-08 12:44:58 -05:00
psychedelicious
6fc3e86061 tidy(mm): only main models get the main default settings 2024-03-08 12:44:58 -05:00
psychedelicious
effbd8a1ba chore: ruff 2024-03-08 11:10:44 -05:00
psychedelicious
ddde355b09 fix(mm): add ui_type to model fields
Recently the schema for models was changed to a generic `ModelField`, and the UI was unable to derive the type of those fields. This didn't affect functionality, but it did break the styling of handles.

Add `ui_type` to the affected fields and update the UI to use the correct capitalizations.
2024-03-08 11:10:44 -05:00
psychedelicious
4a20377fef tidy(config): move version "setting" to new CLIArgs category
It's not actually a setting.
2024-03-08 13:59:59 +11:00
psychedelicious
ed803640f7 tidy(mm): move remote_api_tokens to new ModelInstall category 2024-03-08 13:59:59 +11:00
psychedelicious
576bb4a61d feat(mm): support generic API tokens via regex/token pairs in config
A list of regex and token pairs is accepted. As a file is downloaded by the model installer, the URL is tested against the provided regex/token pairs. The token for the first matching regex is used during download, added as a bearer token.
2024-03-08 13:59:59 +11:00
Brandon Rising
b6065d6328 Run ruff with newest version of ruff 2024-03-08 13:59:59 +11:00
Brandon Rising
04229f4a21 Run ruff 2024-03-08 13:59:59 +11:00
Brandon Rising
73a190fb6e Add remote_repo_api_key config to be added as a token query param for all remote url model downloads 2024-03-08 13:59:59 +11:00
Brandon Rising
952d97741e Remove civit ai from tests and documentation 2024-03-08 13:59:59 +11:00
Brandon Rising
d1f859a446 Remove civit AI model install resources 2024-03-08 13:59:59 +11:00
psychedelicious
5118160282 docs(mm): update comment about model images 2024-03-08 12:26:35 +11:00
psychedelicious
a3a42d25d3 fix(mm): model images reload when changed
When we change a model image, its URL remains the same. The browser will aggressively cache the image. The easiest way to fix this is to append a random query parameter to the URL whenever we build a model config in the API.
2024-03-08 12:26:35 +11:00
psychedelicious
9b48029bc9 tidy(mm): ModelImages service 2024-03-06 21:57:41 -05:00
Jennifer Player
347f1fd0b7 fix tests 2024-03-06 21:57:41 -05:00
Jennifer Player
4af5a09a68 cleanup 2024-03-06 21:57:41 -05:00
Jennifer Player
8df02623f2 cleanup 2024-03-06 21:57:41 -05:00
Jennifer Player
aa88fadc30 use webp images 2024-03-06 21:57:41 -05:00
Jennifer Player
8411029d93 get model image url from model config, added thumbnail formatting for images 2024-03-06 21:57:41 -05:00
Jennifer Player
239b1e8cc7 moved upload image field and added delete image functionality 2024-03-06 21:57:41 -05:00
Jennifer Player
2f6964bfa5 fetching model image, still not working 2024-03-06 21:57:41 -05:00
Jennifer Player
c1cdfd132b moved model image to edit page, added model_images service 2024-03-06 21:57:41 -05:00
psychedelicious
132790eebe tidy(nodes): use canonical capitalizations 2024-03-07 10:56:59 +11:00
psychedelicious
528ac5dd25 refactor(nodes): model identifiers
- All models are identified by a key and optionally a submodel type via new model `ModelField`. Previously, a few model types had their own class, but not all of them. This inconsistency just added complexity without any benefit.
- Update all invocation to use the new format.
- In the node API, models are loaded by key or an instance of `ModelField` as a convenience.
- Add an enriched model schema for metadata. It includes key, hash, name, base and type.
2024-03-07 10:56:59 +11:00
psychedelicious
afd9ae7712 tidy(mm): remove convenience methods from high level model manager service
These were added as a hold-me-over for the nodes API changes, no longer needed. A followup commit will fix the nodes API to not rely on these.
2024-03-07 10:56:59 +11:00
psychedelicious
99c0662e3f fix(nodes): load config before doing anything else
This was preventing custom nodes from loading if a custom nodes dir was specified

Closes #5862
2024-03-07 10:36:27 +11:00
maryhipp
cdc0d0c182 add config_path to ModelRecordChanges 2024-03-07 10:29:29 +11:00
psychedelicious
b0615bdfd4 fix(nodes): correctly serialize outputs
In order for delete by match to work, we need the whole invocation output to be stringified.

For some reason, the serialization of the output was set to only include the `type` field. It should instead include the whole output.

I don't understand how this ever worked unless pydantic had different serialization behaviour in v1 (though it appears to have been the same).

Closes #5805
2024-03-06 08:14:12 -05:00
psychedelicious
bab20467fb fix(nodes): fix invocation cache clear method args 2024-03-06 08:14:12 -05:00
psychedelicious
e24624109e fix(nodes): fix invocation cache ABC typing 2024-03-06 08:14:12 -05:00
psychedelicious
bcf58cac59 feat(mm): add config to skip model hash
This is useful for when you are using a memory DB and do not want to wait for all models to be hashed on startup.
2024-03-05 23:50:19 +11:00
psychedelicious
e8797787cf fix(mm): fix incorrect calls to update_model 2024-03-05 23:50:19 +11:00
psychedelicious
0082ecb22b feat(mm): add path to ModelRecordChanges 2024-03-05 23:50:19 +11:00
psychedelicious
656839fcd1 fix(mm): fix typing on heuristic_import 2024-03-05 23:50:19 +11:00
psychedelicious
48119d9010 revert(mm): restore convert route 2024-03-05 23:50:19 +11:00
psychedelicious
7c9128b253 tidy(mm): use canonical capitalization for all model-related enums, classes
For example, "Lora" -> "LoRA", "Vae" -> "VAE".
2024-03-05 23:50:19 +11:00
psychedelicious
4f9bb00275 tidy(api): tidy mm routes
Rename MM routes to be consistent:
- "import" -> "install"
- "model_record" -> "model"

Comment several unused routes while I work (may end up removing them?):
- list model summary (we use the search route instead)
- add model record
- convert model
- merge models
2024-03-05 23:50:19 +11:00
psychedelicious
78895b3e80 fix(mm): add missing inplace parameter to model install abc 2024-03-05 23:50:19 +11:00
psychedelicious
58fa9c2fac fix(mm): do not allow extra fields on ModelRecordChanges 2024-03-05 23:50:19 +11:00
psychedelicious
5551cf8ac4 feat(mm): revise update_model to use ModelRecordChanges 2024-03-05 23:50:19 +11:00
psychedelicious
37b969d339 tidy(mm): add default_settings to model config 2024-03-05 23:50:19 +11:00
psychedelicious
c953e61294 tidy(mm): "trigger_words" -> "trigger_phrases" 2024-03-05 23:50:19 +11:00
psychedelicious
44c40d7d1a refactor(mm): remove unused metadata logic, fix tests
- Metadata is merged with the config. We can simplify the MM substantially and remove the handling for metadata.
- Per discussion, we don't have an ETA for frontend implementation of tags, and with the realization that the tags from CivitAI are largely useless, there's no reason to keep tags in the MM right now. When we are ready to implement tags on the frontend, we can refer back to the implementation here and use it if it supports the design.
- Fix all tests.
2024-03-05 23:50:19 +11:00
psychedelicious
c3aa985c93 refactor(mm): get metadata working 2024-03-05 23:50:19 +11:00
psychedelicious
7cb0da1f66 refactor(mm): wip schema changes 2024-03-05 23:50:19 +11:00
psychedelicious
f13f5984c0 fix(mm): update db schema & migration 2024-03-05 23:50:19 +11:00
psychedelicious
9378e47a06 feat(mm): add source_type to model configs 2024-03-05 23:50:19 +11:00
psychedelicious
2c835fd550 refactor(mm): WIP db schema 2024-03-05 23:50:19 +11:00
psychedelicious
a8cd3dfc99 refactor(mm): add models table (schema WIP), rename "original_hash" -> "hash" 2024-03-05 23:50:19 +11:00
psychedelicious
0cce582f2f tidy(mm): remove current_hash 2024-03-05 23:50:19 +11:00
psychedelicious
bd4fd9693d tidy(mm): rename ckpt "last_modified" -> "converted_at"
Clarify what this timestamp means
2024-03-05 23:50:19 +11:00
psychedelicious
9b40c28144 tidy(mm): rename ckpy "config" -> "config_path" 2024-03-05 23:50:19 +11:00
psychedelicious
76cbc745e1 refactor(mm): add CheckpointConfigBase for all ckpt models 2024-03-05 23:50:19 +11:00
psychedelicious
b880a31039 refactor(mm): remove ztsnr_training field on _MainConfig
This is used to determine the CFG Rescale Multiplier setting. We'll handle this in the UI as a default setting.
2024-03-05 23:50:19 +11:00
psychedelicious
dd31bc4586 refactor(mm): remove vae field on _MainConfig
We will handle default VAE selection in the UI.
2024-03-05 23:50:19 +11:00
Mary Hipp Rogers
8b34f5298c
Default model settings (#5850)
* UI in MM to create trigger phrases

* add scheduler and vaePrecision to config

* UI for configuring default settings for models'

* hook MM default model settings up to API

* add button to set default settings in parameters

* pull out trigger phrases

* back-end for default settings

* lint

* remove log;
gi

* ruff

* ruff format

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2024-03-04 09:39:03 -05:00
Brandon Rising
893bcd16fc Next: Allow in place local installs of models 2024-03-04 23:11:41 +11:00
Ryan Dick
f6028a4c61 Log a stack trace for invocation errors. 2024-03-04 23:01:56 +11:00
dunkeroni
735857479d fix(canvas): use corrected mask for pasteback 2024-03-03 12:58:47 -05:00
psychedelicious
ae99428883 fix(mm): use UUIDv4 for key
This changes the functionality of this PR to only use the updated hashing for model hashes with a UUID for the key.
2024-03-03 14:32:14 +11:00
psychedelicious
86982f3059 feat(mm): make ModelHash instantiatable, taking an algorithm as arg 2024-03-03 14:32:14 +11:00
psychedelicious
982076d7d7 feat(mm): add hashing algos to ModelHash
- Some algos are slow, so it is now just called ModelHash
- Added all hashlib algos, plus BLAKE3 and the fast (but incorrect) SHA1 algo
2024-03-03 14:32:14 +11:00
Lincoln Stein
a72056e0df make model key assignment deterministic
- When installing, model keys are now calculated from the model contents.
- .safetensors, .ckpt and other single file models are hashed with sha1
- The contents of diffusers directories are hashed using imohash (faster)

fixup yaml->sql db migration script to assign deterministic key

- this commit also detects and assigns the correct image encoder for
  ip adapter models.
2024-03-03 14:32:14 +11:00
Ryan Dick
cc45007dc4 Remove unused code for attention map saving. 2024-03-02 08:25:41 -05:00
Ryan Dick
ad96857e0f Fix avoid storing extra conditioning info in two places. 2024-03-01 15:12:03 -05:00