Commit Graph

158 Commits

Author SHA1 Message Date
5feb62d440 Allow users to specify model type and skip detection step of probe 2024-03-19 01:16:11 -04:00
f8df293d2c Revert "fix(mm): provide ckpt config as stream to diffusers"
This reverts commit 9d045964d6.
2024-03-19 14:24:54 +11:00
9d045964d6 fix(mm): provide ckpt config as stream to diffusers
Fixes converting ckpt main models
2024-03-19 09:24:28 +11:00
897fe497dc fix(config): use new get_config across the app, use correct settings 2024-03-19 09:24:28 +11:00
71a1740740 Remove core safetensors->diffusers conversion models
- No longer install core conversion models. Use the HuggingFace cache to load
  them if and when needed.

- Call directly into the diffusers library to perform conversions with only shallow
   wrappers around them to massage arguments, etc.

- At root configuration time, do not create all the possible model subdirectories,
  but let them be created and populated at model install time.

- Remove checks for missing core conversion files, since they are no
  longer installed.
2024-03-17 19:13:18 -04:00
1bd8e33f8c Work around missing core conversion model issue
- This adds additional logic to the safetensors->diffusers conversion script
  to check for and install missing core conversion models at runtime.

- Fixes #5934
2024-03-14 16:52:01 +11:00
3fd824306c feat(mm): probe for main model default settings
Currently, this is just the width and height, derived from the model base.
2024-03-14 16:03:37 +11:00
b9f1a4bd65 feat(nodes): add w/h defaults for models 2024-03-14 16:03:37 +11:00
731942dbed feat(nodes): add constraints & descriptions to default settings 2024-03-14 16:03:37 +11:00
eb6e6548ed feat(mm): faster hashing for spinning disk HDDs
BLAKE3 has poor performance on spinning disks when parallelized. See https://github.com/BLAKE3-team/BLAKE3/issues/31

- Replace `skip_model_hash` setting with `hashing_algorithm`. Any algorithm we support is accepted.
- Add `random` algorithm: hashes a UUID with BLAKE3 to create a random "hash". Equivalent to the previous skip functionality.
- Add `blake3_single` algorithm: hashes on a single thread using BLAKE3, fixes the aforementioned performance issue
- Update model probe to accept the algorithm to hash with as an optional arg, defaulting to `blake3`
- Update all calls of the probe to use the app's configured hashing algorithm
- Update an external script that probes models
- Update tests
- Move ModelHash into its own module to avoid circuclar import issues
2024-03-14 15:54:42 +11:00
01207a2fa5 fix(mm): config.json to indicates diffusers model 2024-03-13 21:02:29 +11:00
d0800c4888 ui consistency, moved is_diffusers logic to backend, extended HuggingFaceMetadata, removed logic from service 2024-03-13 21:02:29 +11:00
3a5314f1ca install model if diffusers or single file, cleaned up backend logic to not mess with existing model install 2024-03-13 21:02:29 +11:00
f7cd3cf1f4 added hf models import tab and route for getting available hf models 2024-03-13 21:02:29 +11:00
9ee2e7ff25 Do not override log_memory_usage when debug logs are enabled. The speed cost of log_memory_usage=True is large. It is common to want debug log without enabling log_memory_usage. 2024-03-12 09:48:50 +11:00
c47dbf7258 docs(mm): format docstrings for ModelSearch 2024-03-10 12:09:47 +11:00
92b2e8186a tidy(mm): simplify types for ModelSearch
- Use `set` instead of `Set`
- Methods accept only `Path`s
2024-03-10 12:09:47 +11:00
70a88c6b99 docs(mm): update docstrings for ModelSearch 2024-03-10 12:09:47 +11:00
56e7c04475 tidy(mm): remove extraneous dependencies in model search
- `config` is unused
- `stats` is created on instantiation
- `logger` uses the app logger
2024-03-10 12:09:47 +11:00
bd5b43c00d tidy(mm): ModelSearch cleanup
- No need for it to by a pydantic model. Just a class now.
- Remove ABC, it made it hard to understand what was going on as attributes were spread across the ABC and implementation. Also, there is no other implementation.
- Add tests
2024-03-10 12:09:47 +11:00
e52274ecac Experiment with using absolute paths within model management 2024-03-08 15:36:14 -05:00
a10dccdd43 fix(mm): fix bug in control adapter probe default settings
Wasn't checking for matches correctly.
2024-03-08 12:44:58 -05:00
50bb9a6b41 fix(mm): remove default settings from IP adapter config 2024-03-08 12:44:58 -05:00
13bb3c5e15 feat(mm): add control adapter default settings while probing 2024-03-08 12:44:58 -05:00
80c2a4b925 feat(mm): add AnyDefaultSettings union 2024-03-08 12:44:58 -05:00
8ce485b036 feat(mm): add default settings for control adapters
Only includes `preprocessor` at this time.
2024-03-08 12:44:58 -05:00
6fc3e86061 tidy(mm): only main models get the main default settings 2024-03-08 12:44:58 -05:00
952d97741e Remove civit ai from tests and documentation 2024-03-08 13:59:59 +11:00
d1f859a446 Remove civit AI model install resources 2024-03-08 13:59:59 +11:00
bbcbcd9b63 fix(mm): only loras and main models get trigger_phrases 2024-03-08 12:26:35 +11:00
4af5a09a68 cleanup 2024-03-06 21:57:41 -05:00
aa88fadc30 use webp images 2024-03-06 21:57:41 -05:00
8411029d93 get model image url from model config, added thumbnail formatting for images 2024-03-06 21:57:41 -05:00
8a68355926 got model images displaying, still need to clean up types and unused code 2024-03-06 21:57:41 -05:00
2f6964bfa5 fetching model image, still not working 2024-03-06 21:57:41 -05:00
132790eebe tidy(nodes): use canonical capitalizations 2024-03-07 10:56:59 +11:00
e866d90ab2 tidy(mm): remove unused method on probe 2024-03-05 23:50:19 +11:00
e8797787cf fix(mm): fix incorrect calls to update_model 2024-03-05 23:50:19 +11:00
7c9128b253 tidy(mm): use canonical capitalization for all model-related enums, classes
For example, "Lora" -> "LoRA", "Vae" -> "VAE".
2024-03-05 23:50:19 +11:00
3030a34b88 fix(mm): make type and format required in openapi schema for model config 2024-03-05 23:50:19 +11:00
a8b6635050 fix(mm): make key required in openapi schema for model config 2024-03-05 23:50:19 +11:00
5551cf8ac4 feat(mm): revise update_model to use ModelRecordChanges 2024-03-05 23:50:19 +11:00
37b969d339 tidy(mm): add default_settings to model config 2024-03-05 23:50:19 +11:00
c953e61294 tidy(mm): "trigger_words" -> "trigger_phrases" 2024-03-05 23:50:19 +11:00
93dd3c848e tidy(mm): remove unused code in select_hf_files.py 2024-03-05 23:50:19 +11:00
3391c19926 chore: ruff 2024-03-05 23:50:19 +11:00
0f60b1ced4 fix(mm): use .value for model config discriminators
There is a breaking change in python 3.11 related to how enums with `str` as a mixin are formatted. This appears to have not caused any grief for us until now.

Re-jigger the discriminator setup to use `.value` so everything works on both python 3.10 and 3.11.
2024-03-05 23:50:19 +11:00
44c40d7d1a refactor(mm): remove unused metadata logic, fix tests
- Metadata is merged with the config. We can simplify the MM substantially and remove the handling for metadata.
- Per discussion, we don't have an ETA for frontend implementation of tags, and with the realization that the tags from CivitAI are largely useless, there's no reason to keep tags in the MM right now. When we are ready to implement tags on the frontend, we can refer back to the implementation here and use it if it supports the design.
- Fix all tests.
2024-03-05 23:50:19 +11:00
c3aa985c93 refactor(mm): get metadata working 2024-03-05 23:50:19 +11:00
7cb0da1f66 refactor(mm): wip schema changes 2024-03-05 23:50:19 +11:00