Commit Graph

195 Commits

Author SHA1 Message Date
Lincoln Stein
2aefa921fe fix "unknown model type" error when rebasing a model with API
- Add command-line model probing script for dev use
- Minor documentation tweak
2023-07-25 08:36:57 -04:00
Alexandre Macabies
07a90c0198 Fix incorrect use of a singleton list.
This was found through pylance type errors. Go types!
2023-07-23 15:28:05 +02:00
psychedelicious
594bf6fef1 fix(api,ui): fix canvas saved images have extra transparent regions
- add `crop_visible` param to upload image & set to true only for canvas saves
2023-07-21 07:26:12 -04:00
psychedelicious
896b77cf56 feat(api,db): allow creating an image with a board_id 2023-07-21 07:26:12 -04:00
Lincoln Stein
9370572169 prettify startup messages 2023-07-20 22:45:35 -04:00
Lincoln Stein
603989dc0d added get_log_level and set_log_level operations to the app route 2023-07-20 08:33:01 -04:00
blessedcoolant
0795d8764f
Merge branch 'main' into fix/post-model-sync 2023-07-20 20:16:14 +12:00
psychedelicious
187cf906fa
ui: enhance intermediates clear, enhance board auto-add (#3851)
* feat(ui): enhance clear intermediates feature

- retrieve the # of intermediates using a new query (just uses list images endpoint w/ limit of 0)
- display the count in the UI
- add types for clearIntermediates mutation
- minor styling and verbiage changes

* feat(ui): remove unused settings option for guides

* feat(ui): use solid badge variant

consistent with the rest of the usage of badges

* feat(ui): update board ctx menu, add board auto-add

- add context menu to system boards - only open is select board. did this so that you dont think its broken when you click it
- add auto-add board. you can right click a user board to enable it for auto-add, or use the gallery settings popover to select it. the invoke button has a tooltip on a short delay to remind you that you have auto-add enabled
- made useBoardName hook, provide it a board id and it gets your the board name
- removed `boardIdToAdTo` state & logic, updated workflows to auto-switch and auto-add on image generation

* fix(ui): clear controlnet when clearing intermediates

* feat: Make Add Board icon a button

* feat(db, api): clear intermediates now clears all of them

* feat(ui): make reset webui text subtext style

* feat(ui): board name change submits on blur

---------

Co-authored-by: blessedcoolant <54517381+blessedcoolant@users.noreply.github.com>
2023-07-20 17:44:22 +12:00
Lincoln Stein
d76bf4444c
Update invokeai/app/api/routers/models.py
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-07-19 22:46:49 -04:00
Lincoln Stein
170bbd7da3 change GET to POST method for model synchronization route 2023-07-19 22:16:56 -04:00
psychedelicious
0724eb9e0a
feat(ui): another go at gallery (#3791)
* feat(ui): migrate listImages to RTK query using createEntityAdapter

- see comments in `endpoints/images.ts` for explanation of the caching
- so far, only manually updating `all` images when new image is generated. no other manual cache updates are implemented, but will be needed.
- fixed some weirdness with loading state components (like the spinners in gallery)
- added `useThumbnailFallback` for `IAIDndImage`, this displays the tiny webp thumbnail while the full-size images load
- comment out some old thunk related stuff in gallerySlice, which is no longer needed

* feat(ui): add manual cache updates for board changes (wip)

- update RTK Query caches when adding/removing single image to/from board
- work more on migrating all image-related operations to RTK Query

* update AddImagesToBoardContext so that it works when user uses context menu + modal

* handle case where no image is selected

* get assets working for main list and boards - dnd only

* feat(ui): migrate image uploads to RTK Query

- minor refactor of `ImageUploader` and `useImageUploadButton` hooks, simplify some logic
- style filesystem upload overlay to match existing UI
- replace all old `imageUploaded` thunks with `uploadImage` RTK Query calls, update associated logic including canvas related uploads
- simplify `PostUploadAction`s that only need to display user input

* feat(ui): remove `receivedPageOfImages` thunks

* feat(ui): remove `receivedImageUrls` thunk

* feat(ui): finish removing all images thunks

stuff now broken:
- image usage
- delete board images
- on first load, no image selected

* feat(ui): simplify `updateImage` cache manipulation

- we don't actually ever change categories, so we can remove a lot of logic

* feat(ui): simplify canvas autosave

- instead of using a network request to set the canvas generation as not intermediate, we can just do that in the graph

* feat(ui): simplify & handle edge cases in cache updates

* feat(db, api): support `board_id='none'` for `get_many` images queries

This allows us to get all images that are not on a board.

* chore(ui): regen types

* feat(ui): add `All Assets`, `No Board` boards

Restructure boards:
- `all images` is all images
- `all assets` is all assets
- `no board` is all images/assets without a board set
- user boards may have images and assets

Update caching logic
- much simpler without every board having sub-views of images and assets
- update drag and drop operations for all possible interactions

* chore(ui): regen types

* feat(ui): move download to top of context menu

* feat(ui): improve drop overlay styles

* fix(ui): fix image not selected on first load

- listen for first load of all images board, then select the first image

* feat(ui): refactor board deletion

api changes:
- add route to list all image names for a board. this is required to handle board + image deletion. we need to know every image in the board to determine the image usage across the app. this is fetched only when the delete board and images modal is opened so it's as efficient as it can be.
- update the delete board route to respond with a list of deleted `board_images` and `images`, as image names. this is needed to perform accurate clientside state & cache updates after deleting.

db changes:
- remove unused `board_images` service method to get paginated images dtos for a board. this is now done thru the list images endpoint & images service. needs a small logic change on `images.delete_images_on_board`

ui changes:
- simplify the delete board modal - no context, just minor prop drilling. this is feasible for boards only because the components that need to trigger and manipulate the modal are very close together in the tree
- add cache updates for `deleteBoard` & `deleteBoardAndImages` mutations
- the only thing we cannot do directly is on `deleteBoardAndImages`, update the `No Board` board. we'd need to insert image dtos that we may not have loaded. instead, i am just invalidating the tags for that `listImages` cache. so when you `deleteBoardAndImages`, the `No Board` will re-fetch the initial image limit. i think this is more efficient than e.g. fetching all image dtos to insert then inserting them.
- handle image usage for `deleteBoardAndImages`
- update all (i think/hope) the little bits and pieces in the UI to accomodate these changes

* fix(ui): fix board selection logic

* feat(ui): add delete board modal loading state

* fix(ui): use thumbnails for board cover images

* fix(ui): fix race condition with board selection

when selecting a board that doesn't have any images loaded, we need to wait until the images haveloaded before selecting the first image.

this logic is debounced to ~1000ms.

* feat(ui): name 'No Board' correctly, change icon

* fix(ui): do not cache listAllImageNames query

if we cache it, we can end up with stale image usage during deletion.

we could of course manually update the cache as we are doing elsewhere. but because this is a relatively infrequent network request, i'd like to trade increased cache mgmt complexity here for increased resource usage.

* feat(ui): reduce drag preview opacity, remove border

* fix(ui): fix incorrect queryArg used in `deleteImage` and `updateImage` cache updates

* fix(ui): fix doubled open in new tab

* fix(ui): fix new generations not getting added to 'No Board'

* fix(ui): fix board id not changing on new image when autosave enabled

* fix(ui): context menu when selection is 0

need to revise how context menu is triggered later, when we approach multi select

* fix(ui): fix deleting does not update counts for all images and all assets

* fix(ui): fix all assets board name in boards list collapse button

* fix(ui): ensure we never go under 0 for total board count

* fix(ui): fix text overflow on board names

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-07-19 12:06:38 -04:00
Mary Hipp Rogers
9f00e055ac
Maryhipp/clear intermediates (#3820)
* new route to clear intermediates

* UI to clear intermediates from settings modal

* cleanup

* PR feedback

---------

Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-07-19 10:55:29 -04:00
blessedcoolant
b08ad28daa fix: typo in logger statement (import_model) 2023-07-18 15:17:52 +12:00
Lincoln Stein
1353bf98b3 add specific exception for model probe failures 2023-07-17 23:08:39 -04:00
Lincoln Stein
af1c1ab51f importing an unrecognized model now gives "Unsupported Media Type" error 2023-07-17 22:33:05 -04:00
blessedcoolant
13da881953 Merge branch 'main' into sdxl-support 2023-07-18 13:34:07 +12:00
Lincoln Stein
08854b6d68 keep model path consistent with model manager key in model update api 2023-07-17 10:00:28 -04:00
Kent Keirsey
675a92401c
Merge branch 'main' into lstein/default-model-install 2023-07-16 19:32:59 -04:00
Lincoln Stein
6fbb5ce780 add renaming capabilities to model update API route 2023-07-16 14:17:05 -04:00
psychedelicious
48a031dbaf fix(nodes): fix typing of configuration service 2023-07-16 10:52:18 +10:00
Lincoln Stein
ccbfa5d862 resolve conflicts 2023-07-15 19:47:50 -04:00
psychedelicious
c7b547ea3e feat(nodes): remove references to restoration services
- remove restoration services
- remove the restore faces nodes
- update tests
2023-07-16 01:12:39 +10:00
Lincoln Stein
2faa7cee37 add rename_model route 2023-07-14 23:03:18 -04:00
Lincoln Stein
8600aad12b multiple enhancements to model manager REACT API
1. add a /sync route for synchronizing the in-memory model lists to
   models.yaml, the models directory, and the autoimport directories.

2. add optional destination_directories to convert_model and merge_model
   operations.

3. add /ckpt_confs route for retrieving known legacy checkpoint configuration
   files.

4. add /search route for finding all models in a directory located in the server
   filesystem
2023-07-14 13:45:16 -04:00
Lincoln Stein
ad076b1174 add model directory search route 2023-07-14 11:14:33 -04:00
Mary Hipp Rogers
545e2f557f
Merge branch 'main' into feat/api/image-max-age 2023-07-14 08:21:44 -04:00
psychedelicious
4d25d702a1 feat(api): app/config route returns available infill methods 2023-07-13 23:26:52 +10:00
maryhipp
fba25792f9 add new route for app config 2023-07-13 23:26:52 +10:00
psychedelicious
50bef87da7 feat(db,nodes,api): refactor metadata
Metadata for the Linear UI is now sneakily provided via a `MetadataAccumulator` node, which the client populates / hooks up while building the graph.

Additionally, we provide the unexpanded graph with the metadata API response.

Both of these are embedded into the PNGs.

- Remove `metadata` from `ImageDTO`
- Split up the `images/` routes to accomodate this; metadata is only retrieved per-image
- `images/{image_name}` now gets the DTO
- `images/{image_name}/metadata` gets the new metadata
- `images/{image_name}/full` gets the full-sized image file
- Remove old metadata service
- Add `MetadataAccumulator` node, `CoreMetadataField`, hook up to `LatentsToImage` node
- Add `get_raw()` method to `ItemStorage`, retrieves the row from DB as a string, no pydantic parsing
- Update `images`related services to handle storing and retrieving the new metadata
- Add `get_metadata_graph_from_raw_session` which extracts the `graph` from `session` without needing to hydrate the session in pydantic, in preparation for providing it as metadata; also removes all references to the `MetadataAccumulator` node
2023-07-13 15:40:05 +10:00
Lincoln Stein
23d8a2777e add ability to filter list_models on list of base models 2023-07-10 21:59:32 -04:00
Lincoln Stein
8e42502dfd partial implementation of SDXL model loader 2023-07-10 20:18:30 -04:00
psychedelicious
c5539b442c feat(api): set max-age for images
Image files are immutable and we expect deletion to result in no further requests for a given image, so we can set the max-age to something thicc.

Resolves #3426
2023-07-09 22:42:05 +10:00
Lincoln Stein
bd9786564c merge with main 2023-07-08 10:11:25 -04:00
blessedcoolant
3333805821 feat: Add App Version to UI 2023-07-08 21:31:17 +12:00
Lincoln Stein
8e8f9cce0f print version when --version provided at command line 2023-07-07 20:47:29 -04:00
Lincoln Stein
e9352227f3 add merge api 2023-07-06 15:12:34 -04:00
Lincoln Stein
90c66aab3d merge with upstream 2023-07-06 13:17:02 -04:00
Lincoln Stein
3e925fbf34 model merging API ready for testing 2023-07-06 13:15:15 -04:00
psychedelicious
c21245f590 fix(api): make list models params querys, make path /, remove defaults
The list models route should just be the base route path, and should use query parameters as opposed to path parameters (which cannot be optional)

Removed defaults for update model route - for the purposes of the API, we should always be explicit with this
2023-07-06 15:34:50 +10:00
Lincoln Stein
f7daa6e71d all methods now return OPENAPI_MODEL_CONFIGS; convert uses PUT 2023-07-05 23:13:01 -04:00
Lincoln Stein
5b6dd47b9f add API for model convert 2023-07-05 15:13:21 -04:00
Lincoln Stein
5027d0a603 accept @psychedelicious suggestions above 2023-07-05 14:50:57 -04:00
Lincoln Stein
6112197edf convert implemented; need router 2023-07-05 09:05:05 -04:00
psychedelicious
56d4ea3252 fix(api): improve mm routes 2023-07-05 20:08:47 +10:00
Lincoln Stein
5d099f4a49 update_model working 2023-07-04 17:26:57 -04:00
Lincoln Stein
752b4d50cf model_delete method now working 2023-07-04 10:40:32 -04:00
Lincoln Stein
c1c49d9a76 import model returns 404 for invalid path, 409 for duplicate model 2023-07-04 10:08:10 -04:00
Lincoln Stein
96bf92ead4 add the import model router 2023-07-04 14:35:47 +10:00
maryhipp
45935caf1d fix query 2023-06-29 21:14:53 +10:00
Mary Hipp
73f2092ec5 (api) add option to board delete route and logic to services 2023-06-29 21:14:53 +10:00
Lincoln Stein
466ec3ab5e add router API support for model manager heuristic_import()` 2023-06-23 16:35:39 -04:00
psychedelicious
b937b7da01 feat(models): update model manager service & route to return list of models 2023-06-22 17:34:12 +10:00
Sergey Borisov
01d17601b8 Generate config names for openapi 2023-06-22 16:41:19 +10:00
psychedelicious
285195bf72 feat(api): add get_board route 2023-06-22 16:25:49 +10:00
psychedelicious
67a75f6895 feat(api, db): support board_id filter on images service get_many() 2023-06-22 16:25:49 +10:00
psychedelicious
be3bdae847 fix: resolve rebase conflicts 2023-06-22 16:25:49 +10:00
psychedelicious
661a94b3de feat(db): add get_all() method for boards
This is needed to show the full list of boards in the update boards modal.
2023-06-22 16:25:49 +10:00
psychedelicious
d604d986f9 feat(db, api): update get_board_for_image & service dependencies
- previously was `get_boards_for_image`, returning a list of `BoardDTO`, now returns a single `board_id`
2023-06-22 16:25:49 +10:00
psychedelicious
dd1b3c9f35 fix(api): update API models to use BoardDTOs 2023-06-22 16:25:49 +10:00
maryhipp
748016bdab routes working 2023-06-22 16:25:49 +10:00
psychedelicious
72e9ced889 feat(nodes): add boards and board_images services 2023-06-22 16:25:49 +10:00
maryhipp
3833304f57 [WIP] board list endpoint w cover photos 2023-06-22 16:25:49 +10:00
maryhipp
499a174832 some more 2023-06-22 16:25:49 +10:00
maryhipp
6ca5ad9075 filter images by board_id 2023-06-22 16:25:49 +10:00
maryhipp
207602f425 remove unused 2023-06-22 16:25:49 +10:00
maryhipp
a1671519d5 board CRUD 2023-06-22 16:25:49 +10:00
Sergey Borisov
5f2d07917d Fix lora import, fix sd2 config, fix list models api 2023-06-15 21:30:15 +03:00
StAlKeR7779
d0ee3558d1
Merge branch 'main' into lstein/new-model-manager 2023-06-14 17:29:01 +03:00
psychedelicious
a1773197e9 feat(nodes): remove image_origin from most places
- remove `image_origin` from most places where we interact with images
- consolidate image file storage into a single `images/` dir

Images have an `image_origin` attribute but it is not actually used when retrieving images, nor will it ever be. It is still used when creating images and helps to differentiate between internally generated images and uploads.

It was included in eg API routes and image service methods as a holdover from the previous app implementation where images were not managed in a database. Now that we have images in a db, we can do away with this and simplify basically everything that touches images.

The one potentially controversial change is to no longer separate internal and external images on disk. If we retain this separation, we have to keep `image_origin` around in a number of spots and it getting image paths on disk painful.

So, I am have gotten rid of this organisation. Images are now all stored in `images`, regardless of their origin. As we improve the image management features, this change will hopefully become transparent.
2023-06-14 23:08:27 +10:00
Sergey Borisov
26090011c4 Fix conflict resolve, add model configs to type annotation 2023-06-14 00:26:37 +03:00
Sergey Borisov
738ba40f51 Fixes 2023-06-11 06:12:21 +03:00
Lincoln Stein
6652f3405b merge with main 2023-06-08 21:08:43 -04:00
Lincoln Stein
2a6d11e645 create databases directory on startup 2023-06-08 07:17:54 -04:00
Lincoln Stein
31e97ead2a move invokeai.db to ~/invokeai/databases
- The invokeai.db database file has now been moved into
  `INVOKEAIROOT/databases`. Using plural here for possible
  future with more than one database file.

- Removed a few dangling debug messages that appeared during
  testing.

- Rebuilt frontend to test web.
2023-06-03 20:25:34 -04:00
Lincoln Stein
98773b20ac merge with main 2023-06-01 18:09:49 -04:00
Lincoln Stein
dc54cbb1fc
Merge branch 'main' into release/make-web-dist-startable 2023-05-29 14:16:10 -04:00
psychedelicious
f31e62afad feat(nodes): make list images route use offset pagination
Because we dynamically insert images into the DB and UI's images state, `page`/`per_page` pagination makes loading the images awkward.

Using `offset`/`limit` pagination lets us query for images with an offset equal to the number of images already loaded (which match the query parameters).

The result is that we always get the correct next page of images when loading more.
2023-05-28 20:19:56 -04:00
psychedelicious
160267c71a feat(nodes): refactor image types
- Remove `ImageType` entirely, it is confusing
- Create `ResourceOrigin`, may be `internal` or `external`
- Revamp `ImageCategory`, may be `general`, `mask`, `control`, `user`, `other`. Expect to add more as time goes on
- Update images `list` route to accept `include_categories` OR `exclude_categories` query parameters to afford finer-grained querying. All services are updated to accomodate this change.

The new setup should account for our types of images, including the combinations we couldn't really handle until now:
- Canvas init and masks
- Canvas when saved-to-gallery or merged
2023-05-28 20:19:56 -04:00
psychedelicious
9317b42e5f feat(nodes, ui): wip image types 2023-05-28 20:19:56 -04:00
psychedelicious
3ea5e78322 fix(nodes): fix list images route param descriptions 2023-05-28 20:19:56 -04:00
psychedelicious
ee0225f4ba fix(nodes): handle intermediates during images.get_many() 2023-05-28 20:19:56 -04:00
psychedelicious
33a0af4637 feat(nodes): add nameservice
Currenly only used to make names for images, but when latents, conditioning, etc are managed in DB, will do the same for them.

Intended to eventually support custom naming schemes.
2023-05-28 20:19:56 -04:00
Lincoln Stein
497a885c85
Merge branch 'main' into release/make-web-dist-startable 2023-05-25 22:49:18 -04:00
Lincoln Stein
5f8f51436a merge with main; fix conflicts 2023-05-25 22:40:45 -04:00
psychedelicious
d2c8a53c55 feat(nodes): change intermediates handling
- `ImageType` is now restricted to `results` and `uploads`.
- Add a reserved `meta` field to nodes to hold the `is_intermediate` boolean. We can extend it in the future to support other node `meta`.
- Add a `is_intermediate` column to the `images` table to hold this. (When `latents`, `conditioning` etc are added to the DB, they will also have this column.)
- All nodes default to `*not* intermediate`. Nodes must explicitly be marked `intermediate` for their outputs to be `intermediate`.
- When building a graph, you can set `node.meta.is_intermediate=True` and it will be handled as an intermediate.
- Add a new `update()` method to the `ImageService`, and a route to call it. Updates have a strict model, currently only `session_id` and `image_category` may be updated.
- Add a new `update()` method to the `ImageRecordStorageService` to update the image record using the model.
2023-05-25 22:17:14 -04:00
Lincoln Stein
9110838fe4
Merge branch 'main' into release/make-web-dist-startable 2023-05-25 19:06:09 -04:00
psychedelicious
ff6b345d45 fix(nodes): rebase fixes 2023-05-24 11:30:47 -04:00
psychedelicious
d2c223de8f feat(nodes): move fully* to new images service
* except i haven't rebuilt inpaint in latents
2023-05-24 11:30:47 -04:00
psychedelicious
23d9d58c08 fix(nodes): fix bugs with serving images
When returning a `FileResponse`, we must provide a valid path, else an exception is raised outside the route handler.

Add the `validate_path` method back to the service so we can validate paths before returning the file.

I don't like this but apparently this is just how `starlette` and `fastapi` works with `FileResponse`.
2023-05-24 11:30:47 -04:00
psychedelicious
035425ef24 feat(nodes): address feedback
- Address database feedback:
  - Remove all the extraneous tables. Only an `images` table now:
  - `image_type` and `image_category` are unrestricted strings. When creating images, the provided values are checked to ensure they are a valid type and category.
  - Add `updated_at` and `deleted_at` columns. `deleted_at` is currently unused.
  - Use SQLite's built-in timestamp features to populate these. Add a trigger to update `updated_at` when the row is updated. Currently no way to update a row.
  - Rename the `id` column in `images` to `image_name`
- Rename `ImageCategory.IMAGE` to `ImageCategory.GENERAL`
- Move all exceptions outside their base classes to make them more portable.
- Add `width` and `height` columns to the database. These store the actual dimensions of the image file, whereas the metadata's `width` and `height` refer to the respective generation parameters and are nullable.
- Make `deserialize_image_record` take a `dict` instead of `sqlite3.Row`
- Improve comments throughout
- Tidy up unused code/files and some minor organisation
2023-05-24 11:30:47 -04:00
psychedelicious
f310a39381 feat(nodes): finalize image routes 2023-05-24 11:30:47 -04:00
psychedelicious
5de3c41d19 feat(nodes): add metadata handling 2023-05-24 11:30:47 -04:00
psychedelicious
b9375186a5 feat(nodes): consolidate image routers 2023-05-24 11:30:47 -04:00
psychedelicious
52c9e6ec91 feat(nodes): organise/tidy 2023-05-24 11:30:47 -04:00
psychedelicious
5bf9891553 feat(nodes): it works 2023-05-24 11:30:47 -04:00
psychedelicious
f7804f6126 feat(nodes): add logger to images service 2023-05-24 11:30:47 -04:00
psychedelicious
d14b02e93f feat(logger): fix logger type issues 2023-05-24 11:30:47 -04:00
psychedelicious
1b75d899ae feat(nodes): wip image storage implementation 2023-05-24 11:30:47 -04:00
psychedelicious
33d199c007 feat(nodes): image records router 2023-05-24 11:30:47 -04:00
psychedelicious
9c89d3452c feat(nodes): add high-level images service
feat(nodes): add ResultsServiceABC & SqliteResultsService

**Doesn't actually work bc of circular imports. Can't even test it.**

- add a base class for ResultsService and SQLite implementation
- use `graph_execution_manager` `on_changed` callback to keep `results` table in sync

fix(nodes): fix results service bugs

chore(ui): regen api

fix(ui): fix type guards

feat(nodes): add `result_type` to results table, fix types

fix(nodes): do not shadow `list` builtin

feat(nodes): add results router

It doesn't work due to circular imports still

fix(nodes): Result class should use outputs classes, not fields

feat(ui): crude results router

fix(ui): send to canvas in currentimagebuttons not working

feat(nodes): add core metadata builder

feat(nodes): add design doc

feat(nodes): wip latents db stuff

feat(nodes): images_db_service and resources router

feat(nodes): wip images db & router

feat(nodes): update image related names

feat(nodes): update urlservice

feat(nodes): add high-level images service
2023-05-24 11:30:47 -04:00