Commit Graph

104 Commits

Author SHA1 Message Date
psychedelicious
160267c71a feat(nodes): refactor image types
- Remove `ImageType` entirely, it is confusing
- Create `ResourceOrigin`, may be `internal` or `external`
- Revamp `ImageCategory`, may be `general`, `mask`, `control`, `user`, `other`. Expect to add more as time goes on
- Update images `list` route to accept `include_categories` OR `exclude_categories` query parameters to afford finer-grained querying. All services are updated to accomodate this change.

The new setup should account for our types of images, including the combinations we couldn't really handle until now:
- Canvas init and masks
- Canvas when saved-to-gallery or merged
2023-05-28 20:19:56 -04:00
psychedelicious
fd47e70c92 feat(nodes): use higher precision timestamps in db 2023-05-28 20:19:56 -04:00
psychedelicious
9317b42e5f feat(nodes, ui): wip image types 2023-05-28 20:19:56 -04:00
psychedelicious
ee0225f4ba fix(nodes): handle intermediates during images.get_many() 2023-05-28 20:19:56 -04:00
psychedelicious
33a0af4637 feat(nodes): add nameservice
Currenly only used to make names for images, but when latents, conditioning, etc are managed in DB, will do the same for them.

Intended to eventually support custom naming schemes.
2023-05-28 20:19:56 -04:00
psychedelicious
020f3ccf07 fix(nodes): controlnet input accepts list or single controlnet 2023-05-26 21:44:00 -04:00
psychedelicious
d2c8a53c55 feat(nodes): change intermediates handling
- `ImageType` is now restricted to `results` and `uploads`.
- Add a reserved `meta` field to nodes to hold the `is_intermediate` boolean. We can extend it in the future to support other node `meta`.
- Add a `is_intermediate` column to the `images` table to hold this. (When `latents`, `conditioning` etc are added to the DB, they will also have this column.)
- All nodes default to `*not* intermediate`. Nodes must explicitly be marked `intermediate` for their outputs to be `intermediate`.
- When building a graph, you can set `node.meta.is_intermediate=True` and it will be handled as an intermediate.
- Add a new `update()` method to the `ImageService`, and a route to call it. Updates have a strict model, currently only `session_id` and `image_category` may be updated.
- Add a new `update()` method to the `ImageRecordStorageService` to update the image record using the model.
2023-05-25 22:17:14 -04:00
psychedelicious
3829ffbe66 fix(tests): add --use_memory_db flag; use it in tests 2023-05-25 12:12:31 +10:00
psychedelicious
3000436121 chore(nodes): remove unused imports 2023-05-25 12:12:31 +10:00
psychedelicious
37cdd91f5d fix(nodes): use forward declarations for InvocationServices
Also use `TYPE_CHECKING` to get IDE hints.
2023-05-25 12:12:31 +10:00
psychedelicious
ff6b345d45 fix(nodes): rebase fixes 2023-05-24 11:30:47 -04:00
psychedelicious
d2c223de8f feat(nodes): move fully* to new images service
* except i haven't rebuilt inpaint in latents
2023-05-24 11:30:47 -04:00
psychedelicious
23d9d58c08 fix(nodes): fix bugs with serving images
When returning a `FileResponse`, we must provide a valid path, else an exception is raised outside the route handler.

Add the `validate_path` method back to the service so we can validate paths before returning the file.

I don't like this but apparently this is just how `starlette` and `fastapi` works with `FileResponse`.
2023-05-24 11:30:47 -04:00
psychedelicious
035425ef24 feat(nodes): address feedback
- Address database feedback:
  - Remove all the extraneous tables. Only an `images` table now:
  - `image_type` and `image_category` are unrestricted strings. When creating images, the provided values are checked to ensure they are a valid type and category.
  - Add `updated_at` and `deleted_at` columns. `deleted_at` is currently unused.
  - Use SQLite's built-in timestamp features to populate these. Add a trigger to update `updated_at` when the row is updated. Currently no way to update a row.
  - Rename the `id` column in `images` to `image_name`
- Rename `ImageCategory.IMAGE` to `ImageCategory.GENERAL`
- Move all exceptions outside their base classes to make them more portable.
- Add `width` and `height` columns to the database. These store the actual dimensions of the image file, whereas the metadata's `width` and `height` refer to the respective generation parameters and are nullable.
- Make `deserialize_image_record` take a `dict` instead of `sqlite3.Row`
- Improve comments throughout
- Tidy up unused code/files and some minor organisation
2023-05-24 11:30:47 -04:00
psychedelicious
021e5a2aa3 feat(nodes): improve metadata service comments 2023-05-24 11:30:47 -04:00
psychedelicious
c31ff364ab fix(nodes): tidy images service 2023-05-24 11:30:47 -04:00
psychedelicious
5a7e611e0a fix(nodes): fix image url 2023-05-24 11:30:47 -04:00
psychedelicious
5de3c41d19 feat(nodes): add metadata handling 2023-05-24 11:30:47 -04:00
psychedelicious
b9375186a5 feat(nodes): consolidate image routers 2023-05-24 11:30:47 -04:00
psychedelicious
11bd932cba feat(nodes): revert invocation_complete url hack 2023-05-24 11:30:47 -04:00
psychedelicious
60d25f105f fix(nodes): restore metadata traverser 2023-05-24 11:30:47 -04:00
psychedelicious
52c9e6ec91 feat(nodes): organise/tidy 2023-05-24 11:30:47 -04:00
psychedelicious
c0f132e41a hack(nodes): hack to get image urls in the invocation complete event 2023-05-24 11:30:47 -04:00
psychedelicious
cc1160a43a feat(nodes): streamline urlservice 2023-05-24 11:30:47 -04:00
psychedelicious
5bf9891553 feat(nodes): it works 2023-05-24 11:30:47 -04:00
psychedelicious
22c34c343a feat(nodes): fix types for InvocationServices 2023-05-24 11:30:47 -04:00
psychedelicious
f7804f6126 feat(nodes): add logger to images service 2023-05-24 11:30:47 -04:00
psychedelicious
d14b02e93f feat(logger): fix logger type issues 2023-05-24 11:30:47 -04:00
psychedelicious
1b75d899ae feat(nodes): wip image storage implementation 2023-05-24 11:30:47 -04:00
psychedelicious
d4aa79acd7 fix(nodes): use save instead of set
`set` is a python builtin
2023-05-24 11:30:47 -04:00
psychedelicious
33d199c007 feat(nodes): image records router 2023-05-24 11:30:47 -04:00
psychedelicious
9c89d3452c feat(nodes): add high-level images service
feat(nodes): add ResultsServiceABC & SqliteResultsService

**Doesn't actually work bc of circular imports. Can't even test it.**

- add a base class for ResultsService and SQLite implementation
- use `graph_execution_manager` `on_changed` callback to keep `results` table in sync

fix(nodes): fix results service bugs

chore(ui): regen api

fix(ui): fix type guards

feat(nodes): add `result_type` to results table, fix types

fix(nodes): do not shadow `list` builtin

feat(nodes): add results router

It doesn't work due to circular imports still

fix(nodes): Result class should use outputs classes, not fields

feat(ui): crude results router

fix(ui): send to canvas in currentimagebuttons not working

feat(nodes): add core metadata builder

feat(nodes): add design doc

feat(nodes): wip latents db stuff

feat(nodes): images_db_service and resources router

feat(nodes): wip images db & router

feat(nodes): update image related names

feat(nodes): update urlservice

feat(nodes): add high-level images service
2023-05-24 11:30:47 -04:00
Lincoln Stein
7ea995149e fixes to env parsing, textual inversion & help text
- Make environment variable settings case InSenSiTive:
  INVOKEAI_MAX_LOADED_MODELS and InvokeAI_Max_Loaded_Models
  environment variables will both set `max_loaded_models`

- Updated realesrgan to use new config system.

- Updated textual_inversion_training to use new config system.

- Discovered a race condition when InvokeAIAppConfig is created
  at module load time, which makes it impossible to customize
  or replace the help message produced with --help on the command
  line. To fix this, moved all instances of get_invokeai_config()
  from module load time to object initialization time. Makes code
  cleaner, too.

- Added `--from_file` argument to `invokeai-node-cli` and changed
  github action to match. CI tests will hopefully work now.
2023-05-18 10:48:23 -04:00
Eugene
20ca9e1fc1 config: move 'CORS' settings to 'Web Server' in the docstring to match the actual category 2023-05-17 19:45:51 -04:00
Eugene
9e4e386c9b web and formatting fixes
- remove non-existent import InvokeAIWebConfig
- fix workflow file formatting
- clean up whitespace
2023-05-17 19:12:03 -04:00
Lincoln Stein
ffaadb9d05 reorder options in help text 2023-05-17 15:22:58 -04:00
Lincoln Stein
b7c5a39685 make invokeai.yaml more hierarchical; fix list configuration bug 2023-05-17 12:19:19 -04:00
Lincoln Stein
eadfd239a8 update config script to work with new config system 2023-05-17 00:18:19 -04:00
Lincoln Stein
8d75e50435 partial port of invokeai-configure 2023-05-16 01:50:01 -04:00
Lincoln Stein
1103ab2844 merge with main 2023-05-13 21:35:19 -04:00
Eugene Brodsky
63db3fc22f reduce queue check interval to 0.5s 2023-05-12 17:54:26 -04:00
Eugene
ad0bb3f61a fix: queue error should not crash InvocationProcessor
1. if retrieving an item from the queue raises an exception, the
   InvocationProcessor thread crashes, but the API continues running in
   a non-functional state. This fixes the issue
2. when there are no items in the queue, sleep 1 second before checking
   again.
3. Also ensures the thread isn't crashed if an exception is raised from
   invoker, and emits the error event

Intentionally using base Exceptions because for now we don't know which
specific exception to expect.

Fixes (sort of)? #3222
2023-05-12 17:54:26 -04:00
psychedelicious
e5b7dd63e9 fix(nodes): temporarily disable librarygraphs
- Do not retrieve graph from DB until we resolve the issue of changing node schemas causing application to fail to start up due to invalid graphs
2023-05-12 22:33:49 +10:00
psychedelicious
f488b1a7f2 fix(nodes): fix usage of Optional 2023-05-11 11:55:51 +10:00
psychedelicious
20f6a597ab fix(nodes): add MetadataColorField 2023-05-11 11:55:51 +10:00
psychedelicious
206e6b1730 feat(nodes): wip inpaint node 2023-05-11 11:55:51 +10:00
psychedelicious
5e09dd380d Revert "feat(nodes): free gpu mem after invocation"
This reverts commit 99cb33f477306d5dcc455efe04053ce41b8d85bd.
2023-05-11 11:55:51 +10:00
psychedelicious
a75148cb16 feat(nodes): free gpu mem after invocation 2023-05-11 11:55:51 +10:00
psychedelicious
7dfa135b2c fix(nodes): fix #3306
Check if the cache has the object before deleting it.
2023-05-10 15:29:10 +10:00
Lincoln Stein
afd2e32092
Merge branch 'main' into lstein/global-configuration 2023-05-06 21:20:25 -04:00