Commit Graph

118 Commits

Author SHA1 Message Date
Lincoln Stein
0fe92cd406
[MM bugfix] Put model install errors on the event bus (#6578)
* fix access token lookup

* fix bug preventing model install error events from being reported

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-07-03 22:44:34 -04:00
psychedelicious
41f46b846b chore: ruff 2024-06-28 10:36:05 +10:00
psychedelicious
6dfc406c52 tests: update test_bulk_download.py after addition of archived field 2024-06-28 10:36:05 +10:00
psychedelicious
e26125b734 tests: fix test_model_install.py 2024-06-17 10:57:11 +10:00
Lincoln Stein
dc134935c8 replace load_and_cache_model() with load_remote_model() and load_local_odel() 2024-06-07 14:12:16 +10:00
Lincoln Stein
9f9379682e ruff fixes 2024-06-07 13:54:41 +10:00
Lincoln Stein
f81b8bc9f6 add support for generic loading of diffusers directories 2024-06-07 13:54:30 +10:00
Lincoln Stein
ead1748c54 issue a download progress event when install download starts 2024-05-28 19:30:42 -04:00
Lincoln Stein
34e1eb19f9 merge with main and resolve conflicts 2024-05-27 22:20:34 -04:00
psychedelicious
5b390bb11c tests: clean up tests after events changes 2024-05-27 09:06:02 +10:00
psychedelicious
4756920282 tests: move fixtures import to conftest.py 2024-05-27 09:06:02 +10:00
psychedelicious
a876675448 tests: update tests to use new events 2024-05-27 09:06:02 +10:00
psychedelicious
9bd78823a3 refactor(events): use pydantic schemas for events
Our events handling and implementation has a couple pain points:
- Adding or removing data from event payloads requires changes wherever the events are dispatched from.
- We have no type safety for events and need to rely on string matching and dict access when interacting with events.
- Frontend types for socket events must be manually typed. This has caused several bugs.

`fastapi-events` has a neat feature where you can create a pydantic model as an event payload, give it an `__event_name__` attr, and then dispatch the model directly.

This allows us to eliminate a layer of indirection and some unpleasant complexity:
- Event handler callbacks get type hints for their event payloads, and can use `isinstance` on them if needed.
- Event payload construction is now the responsibility of the event itself (a pydantic model), not the service. Every event model has a `build` class method, encapsulating this logic. The build methods are provided as few args as possible. For example, `InvocationStartedEvent.build()` gets the invocation instance and queue item, and can choose the data it wants to include in the event payload.
- Frontend event types may be autogenerated from the OpenAPI schema. We use the payload registry feature of `fastapi-events` to collect all payload models into one place, making it trivial to keep our schema and frontend types in sync.

This commit moves the backend over to this improved event handling setup.
2024-05-27 09:06:02 +10:00
Lincoln Stein
e77c7e40b7 fix ruff error 2024-05-17 22:53:45 -04:00
Lincoln Stein
8aebc29b91 fix test to run on 32bit cpu 2024-05-17 22:48:54 -04:00
Lincoln Stein
d968c6f379 refactor multifile download code 2024-05-17 22:29:19 -04:00
Lincoln Stein
2dae5eb7ad more refactoring; HF subfolders not working 2024-05-16 22:26:18 -04:00
Lincoln Stein
911a24479b add tests for model install file size reporting 2024-05-16 07:18:33 -04:00
Lincoln Stein
f29c406fed refactor model_install to work with refactored download queue 2024-05-13 22:49:15 -04:00
Lincoln Stein
287c679f7b clean up type checking for single file and multifile download job callbacks 2024-05-13 18:31:40 -04:00
Lincoln Stein
0bf14c2830 add multifile_download() method to download service 2024-05-12 20:14:00 -06:00
Lincoln Stein
b48d4a049d bad implementation of diffusers folder download 2024-05-08 21:21:01 -07:00
Lincoln Stein
f211c95dbc move access token regex matching into download queue 2024-05-05 21:00:31 -04:00
Lincoln Stein
a26667d3ca make download and convert cache keys safe for filename length 2024-04-28 12:24:36 -04:00
Lincoln Stein
bb04f496e0 Merge branch 'main' into lstein/feat/simple-mm2-api 2024-04-28 11:33:26 -04:00
Lincoln Stein
70903ef057 refactor load_ckpt_from_url() 2024-04-28 11:33:23 -04:00
Lincoln Stein
d72f272f16 Address change requests in first round of PR reviews.
Pending:

- Move model install calls into model manager and create passthrus in invocation_context.
- Consider splitting load_model_from_url() into a call to get the path and a call to load the path.
2024-04-24 23:53:30 -04:00
psychedelicious
6b0bf59682 feat(backend): update nms util to make blur/thresholding optional 2024-04-25 13:20:09 +10:00
Lincoln Stein
3ddd7ced49 change names of convert and download caches and add migration script 2024-04-14 15:57:33 -04:00
Lincoln Stein
df5ebdbc4f add invocation_context.load_ckpt_from_url() method 2024-04-12 00:55:21 -04:00
psychedelicious
216b34ac44 tests: update model rename test 2024-04-04 07:17:38 +11:00
psychedelicious
31e9cf1f06 tests: update model install tests for change to paths 2024-03-29 10:56:06 -04:00
psychedelicious
eb15a7516f chore: ruff 2024-03-27 08:34:00 +11:00
psychedelicious
06e51b582f test(mm): test arbitrary error handling during model install 2024-03-27 08:34:00 +11:00
psychedelicious
d63d26924b tests: default settings union evaluation 2024-03-25 07:40:52 -04:00
Lincoln Stein
eb558d72d8
Fix minor bugs involving model manager handling of model paths (#6024)
* Fix minor bugs involving model manager handling of model paths

- Leave models found in the `autoimport` directory there. Do not move them
  into the `models` hierarchy.
- If model name, type or base is updated and model is in the `models` directory,
  update its path as appropriate.
- On startup during model scanning, if a model's path is a symbolic link, then resolve
  to an absolute path before deciding it is a new model that must be hashed and
  registered. (This prevents needless hashing at startup time).

* fix issue with dropped suffix

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-03-22 01:14:45 +00:00
Lincoln Stein
2038064a34 add timeouts to the download tests 2024-03-21 16:41:20 +11:00
Lincoln Stein
689cb9d31d after stopping install and download services, wait for thread exit 2024-03-21 16:41:20 +11:00
psychedelicious
6c558279dd feat(config): add CLI arg to specify config file
This allows users to create simple "profiles" via separate `invokeai.yaml` files.

- Remove `InvokeAIAppConfig.set_root()`, it's extraneous
- Remove `InvokeAIAppConfig.merge_from_file()`, it's extraneous
- Add `--config` to the app arg parser, add `InvokeAIAppConfig._config_file`, and consume in the config singleton getter
- `InvokeAIAppConfig.init_file_path` -> `InvokeAIAppConfig.config_file_path`
2024-03-20 15:05:25 +11:00
Lincoln Stein
74a51571a0
Fix race condition causing hangs during model install unit tests (#5994)
* fix race condition causing hangs during model install unit tests

* remove extraneous sanity checks

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
2024-03-19 20:54:49 +00:00
Brandon Rising
94257e35f5 Comment out test failing to run due to issue in model install service 2024-03-19 01:16:11 -04:00
Brandon Rising
a3ed6e694c Use wait_for_job instead of wait_for_installs 2024-03-19 01:16:11 -04:00
Brandon Rising
20d9d10798 Wrap in try except for InvalidModelConfigException 2024-03-19 01:16:11 -04:00
Brandon Rising
77a70a8a9c Skip hashing in test_heuristic_import_with_type 2024-03-19 01:16:11 -04:00
Brandon Rising
06abea8db0 Increase timeout for test_heuristic_import_with_type, fix Url import 2024-03-19 01:16:11 -04:00
Brandon Rising
a28f0932e6 Run ruff 2024-03-19 01:16:11 -04:00
Brandon Rising
6968a068bb Fix test to run on windows vms 2024-03-19 01:16:11 -04:00
Brandon Rising
5daefccf77 Simplify logic for determining model type in probe 2024-03-19 01:16:11 -04:00
Brandon Rising
1f3c35ee90 Run Ruff 2024-03-19 01:16:11 -04:00
Brandon Rising
f78ed3a952 Add unit test 2024-03-19 01:16:11 -04:00