Commit Graph

123 Commits

Author SHA1 Message Date
psychedelicious
132790eebe tidy(nodes): use canonical capitalizations 2024-03-07 10:56:59 +11:00
psychedelicious
e866d90ab2 tidy(mm): remove unused method on probe 2024-03-05 23:50:19 +11:00
psychedelicious
e8797787cf fix(mm): fix incorrect calls to update_model 2024-03-05 23:50:19 +11:00
psychedelicious
7c9128b253 tidy(mm): use canonical capitalization for all model-related enums, classes
For example, "Lora" -> "LoRA", "Vae" -> "VAE".
2024-03-05 23:50:19 +11:00
psychedelicious
3030a34b88 fix(mm): make type and format required in openapi schema for model config 2024-03-05 23:50:19 +11:00
psychedelicious
a8b6635050 fix(mm): make key required in openapi schema for model config 2024-03-05 23:50:19 +11:00
psychedelicious
5551cf8ac4 feat(mm): revise update_model to use ModelRecordChanges 2024-03-05 23:50:19 +11:00
psychedelicious
37b969d339 tidy(mm): add default_settings to model config 2024-03-05 23:50:19 +11:00
psychedelicious
c953e61294 tidy(mm): "trigger_words" -> "trigger_phrases" 2024-03-05 23:50:19 +11:00
psychedelicious
93dd3c848e tidy(mm): remove unused code in select_hf_files.py 2024-03-05 23:50:19 +11:00
psychedelicious
3391c19926 chore: ruff 2024-03-05 23:50:19 +11:00
psychedelicious
0f60b1ced4 fix(mm): use .value for model config discriminators
There is a breaking change in python 3.11 related to how enums with `str` as a mixin are formatted. This appears to have not caused any grief for us until now.

Re-jigger the discriminator setup to use `.value` so everything works on both python 3.10 and 3.11.
2024-03-05 23:50:19 +11:00
psychedelicious
44c40d7d1a refactor(mm): remove unused metadata logic, fix tests
- Metadata is merged with the config. We can simplify the MM substantially and remove the handling for metadata.
- Per discussion, we don't have an ETA for frontend implementation of tags, and with the realization that the tags from CivitAI are largely useless, there's no reason to keep tags in the MM right now. When we are ready to implement tags on the frontend, we can refer back to the implementation here and use it if it supports the design.
- Fix all tests.
2024-03-05 23:50:19 +11:00
psychedelicious
c3aa985c93 refactor(mm): get metadata working 2024-03-05 23:50:19 +11:00
psychedelicious
7cb0da1f66 refactor(mm): wip schema changes 2024-03-05 23:50:19 +11:00
psychedelicious
3534366146 fix(mm): fix extraneous downloaded files in diffusers
Sometimes, diffusers model components (tokenizer, unet, etc.) have multiple weights files in the same directory.

In this situation, we assume the files are different versions of the same weights. For example, we may have multiple
formats (`.bin`, `.safetensors`) with different precisions. When downloading model files, we want to select only
the best of these files for the requested format and precision/variant.

The previous logic assumed that each model weights file would have the same base filename, but this assumption was
not always true. The logic is revised score each file and choose the best scoring file, resulting in only a single
file being downloaded for each submodel/subdirectory.
2024-03-05 23:50:19 +11:00
psychedelicious
f2b5f8753f tidy(mm): remove json_schema_extra from config - not needed 2024-03-05 23:50:19 +11:00
psychedelicious
94e1e64296 chore: ruff 2024-03-05 23:50:19 +11:00
psychedelicious
2411bf53c0 tidy(mm): better descriptions for model configs 2024-03-05 23:50:19 +11:00
psychedelicious
9378e47a06 feat(mm): add source_type to model configs 2024-03-05 23:50:19 +11:00
psychedelicious
4471ea8ad1 refactor(mm): simplify model metadata schemas 2024-03-05 23:50:19 +11:00
psychedelicious
61b737bb9f tidy(mm): remove update method from ModelConfigBase
It's only used in the soon-to-be-removed model merge logic
2024-03-05 23:50:19 +11:00
psychedelicious
a8cd3dfc99 refactor(mm): add models table (schema WIP), rename "original_hash" -> "hash" 2024-03-05 23:50:19 +11:00
psychedelicious
0cce582f2f tidy(mm): remove current_hash 2024-03-05 23:50:19 +11:00
psychedelicious
bd4fd9693d tidy(mm): rename ckpt "last_modified" -> "converted_at"
Clarify what this timestamp means
2024-03-05 23:50:19 +11:00
psychedelicious
9b40c28144 tidy(mm): rename ckpy "config" -> "config_path" 2024-03-05 23:50:19 +11:00
psychedelicious
16a5d718bf fix(mm): add config field to ckpt vaes 2024-03-05 23:50:19 +11:00
psychedelicious
76cbc745e1 refactor(mm): add CheckpointConfigBase for all ckpt models 2024-03-05 23:50:19 +11:00
psychedelicious
0a614943f6 fix(mm): fix broken get_model_discriminator_value 2024-03-05 23:50:19 +11:00
psychedelicious
e426096d32 fix(mm): misc typing fixes for model loaders 2024-03-05 23:50:19 +11:00
psychedelicious
c561cd751f fix(mm): use correct import path for ConfigMixin, ModelMixin 2024-03-05 23:50:19 +11:00
psychedelicious
af9298f0ef tidy(mm): tidy class names in config.py 2024-03-05 23:50:19 +11:00
psychedelicious
5b74117836 fix(mm): use generic for model loader registry
This preserves the typing for classes using the decorator
2024-03-05 23:50:19 +11:00
psychedelicious
38474c9797 fix(mm): use correct import path for ModelMixin 2024-03-05 23:50:19 +11:00
psychedelicious
b880a31039 refactor(mm): remove ztsnr_training field on _MainConfig
This is used to determine the CFG Rescale Multiplier setting. We'll handle this in the UI as a default setting.
2024-03-05 23:50:19 +11:00
psychedelicious
dd31bc4586 refactor(mm): remove vae field on _MainConfig
We will handle default VAE selection in the UI.
2024-03-05 23:50:19 +11:00
psychedelicious
316573df2d feat(mm): use callable discriminator for AnyModelConfig union 2024-03-05 23:50:19 +11:00
Mary Hipp Rogers
8b34f5298c
Default model settings (#5850)
* UI in MM to create trigger phrases

* add scheduler and vaePrecision to config

* UI for configuring default settings for models'

* hook MM default model settings up to API

* add button to set default settings in parameters

* pull out trigger phrases

* back-end for default settings

* lint

* remove log;
gi

* ruff

* ruff format

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2024-03-04 09:39:03 -05:00
psychedelicious
554d175792 feat(mm): improved model hash class
- Use memory view for hashlib algorithms (closer to python 3.11's filehash API in hashlib)
- Remove `sha1_fast` (realized it doesn't even hash the whole file, it just does the first block)
- Add support for custom file filters
- Update docstrings
- Update tests
2024-03-03 14:32:14 +11:00
psychedelicious
86982f3059 feat(mm): make ModelHash instantiatable, taking an algorithm as arg 2024-03-03 14:32:14 +11:00
psychedelicious
ec8ed530a7 feat(mm): modularize ModelHash to facilitate testing 2024-03-03 14:32:14 +11:00
psychedelicious
982076d7d7 feat(mm): add hashing algos to ModelHash
- Some algos are slow, so it is now just called ModelHash
- Added all hashlib algos, plus BLAKE3 and the fast (but incorrect) SHA1 algo
2024-03-03 14:32:14 +11:00
psychedelicious
2e4672f931 feat(mm): make hash.py a script for testing 2024-03-03 14:32:14 +11:00
psychedelicious
908e915a71 feat(mm): use blake3 for hashing 2024-03-03 14:32:14 +11:00
Lincoln Stein
a72056e0df make model key assignment deterministic
- When installing, model keys are now calculated from the model contents.
- .safetensors, .ckpt and other single file models are hashed with sha1
- The contents of diffusers directories are hashed using imohash (faster)

fixup yaml->sql db migration script to assign deterministic key

- this commit also detects and assigns the correct image encoder for
  ip adapter models.
2024-03-03 14:32:14 +11:00
Lincoln Stein
0832e1818e Fix problem of all installed models being assigned "<NOKEY>"
- Also fix redundant scanning of models directory at startup.
2024-02-29 22:31:05 -05:00
psychedelicious
dd9daf8efb chore: ruff 2024-03-01 10:42:33 +11:00
psychedelicious
0305e90287 chore: ruff 2024-03-01 10:42:33 +11:00
Brandon Rising
e7afae0159 Switch absolute path to as_posix in _walk_directory 2024-03-01 10:42:33 +11:00
Brandon Rising
f16e64084b Ruff checks 2024-03-01 10:42:33 +11:00