Commit Graph

318 Commits

Author SHA1 Message Date
Lincoln Stein
951900a86a
Merge branch 'main' into lstein/config-management-fixes 2023-05-26 08:56:41 -04:00
psychedelicious
33e13820fc feat(nodes): remove meta node field; use individual is_intermediate field instead
as suggested by @Kyle0654
2023-05-25 22:17:14 -04:00
psychedelicious
291e9cf14b fix(nodes): add is_intermediate to all image-outputting nodes 2023-05-25 22:17:14 -04:00
psychedelicious
d2c8a53c55 feat(nodes): change intermediates handling
- `ImageType` is now restricted to `results` and `uploads`.
- Add a reserved `meta` field to nodes to hold the `is_intermediate` boolean. We can extend it in the future to support other node `meta`.
- Add a `is_intermediate` column to the `images` table to hold this. (When `latents`, `conditioning` etc are added to the DB, they will also have this column.)
- All nodes default to `*not* intermediate`. Nodes must explicitly be marked `intermediate` for their outputs to be `intermediate`.
- When building a graph, you can set `node.meta.is_intermediate=True` and it will be handled as an intermediate.
- Add a new `update()` method to the `ImageService`, and a route to call it. Updates have a strict model, currently only `session_id` and `image_category` may be updated.
- Add a new `update()` method to the `ImageRecordStorageService` to update the image record using the model.
2023-05-25 22:17:14 -04:00
Lincoln Stein
2f25dd7d0d
Merge branch 'main' into lstein/config-management-fixes 2023-05-25 21:10:12 -04:00
Lincoln Stein
e56965ad76 documentation tweaks; fixed initialization in a couple more places 2023-05-25 21:10:00 -04:00
Lincoln Stein
2273b3a8c8 fix potential race condition in config system 2023-05-25 20:41:26 -04:00
Kent Keirsey
05fb0ac2b2 Update latent.py 2023-05-26 10:27:33 +10:00
Kent Keirsey
d4acd49ee3 Update generate.py 2023-05-26 10:27:33 +10:00
Lincoln Stein
34f567abd4
Merge branch 'main' into lstein/logging-improvements 2023-05-25 08:48:47 -04:00
Lincoln Stein
b87f3043ae add logging configuration 2023-05-24 23:57:15 -04:00
psychedelicious
3829ffbe66 fix(tests): add --use_memory_db flag; use it in tests 2023-05-25 12:12:31 +10:00
psychedelicious
ad619ae880 fix(tests): log db_location 2023-05-25 12:12:31 +10:00
psychedelicious
d22ebe08be fix(tests): log db_location 2023-05-25 12:12:31 +10:00
psychedelicious
ee0c6ad86e fix(cli): fix invocation services for cli 2023-05-25 12:12:31 +10:00
psychedelicious
3000436121 chore(nodes): remove unused imports 2023-05-25 12:12:31 +10:00
psychedelicious
37cdd91f5d fix(nodes): use forward declarations for InvocationServices
Also use `TYPE_CHECKING` to get IDE hints.
2023-05-25 12:12:31 +10:00
psychedelicious
ff6b345d45 fix(nodes): rebase fixes 2023-05-24 11:30:47 -04:00
psychedelicious
1fb307abf4 feat(nodes): restore canvas functionality (non-latents) 2023-05-24 11:30:47 -04:00
psychedelicious
ad39680feb feat(nodes): wip inpainting nodes prep 2023-05-24 11:30:47 -04:00
psychedelicious
1e0ae8404c feat(nodes): comment out seamless
this will be a model config feature when model manager is ready
2023-05-24 11:30:47 -04:00
psychedelicious
460d555a3d feat(nodes): add image mul, channel, convert nodes
also make img node names consistent
2023-05-24 11:30:47 -04:00
psychedelicious
66ad04fcfc feat(nodes): add mask image category 2023-05-24 11:30:47 -04:00
psychedelicious
d2c223de8f feat(nodes): move fully* to new images service
* except i haven't rebuilt inpaint in latents
2023-05-24 11:30:47 -04:00
psychedelicious
dd16f788ed fix(nodes): fix RangeOfSizeInvocation off-by-one error 2023-05-24 11:30:47 -04:00
psychedelicious
b25c1af018 feat(nodes): add RangeOfSizeInvocation
The `RangeInvocation` is a simple wrapper around `range()`, but you must provide `stop > start`.

`RangeOfSizeInvocation` replaces the `stop` parameter with `size`, so that you can just provide the `start` and `step` and get a range of `size` length.
2023-05-24 11:30:47 -04:00
psychedelicious
8f393b64b8 feat(nodes): add seed validator
If `seed>SEED_MAX`, we can still continue if we parse the seed as `seed % SEED_MAX`.
2023-05-24 11:30:47 -04:00
psychedelicious
55b3193629 fix(nodes): add RangeInvocation validator
`stop` must be greater than `start`.
2023-05-24 11:30:47 -04:00
psychedelicious
23d9d58c08 fix(nodes): fix bugs with serving images
When returning a `FileResponse`, we must provide a valid path, else an exception is raised outside the route handler.

Add the `validate_path` method back to the service so we can validate paths before returning the file.

I don't like this but apparently this is just how `starlette` and `fastapi` works with `FileResponse`.
2023-05-24 11:30:47 -04:00
psychedelicious
035425ef24 feat(nodes): address feedback
- Address database feedback:
  - Remove all the extraneous tables. Only an `images` table now:
  - `image_type` and `image_category` are unrestricted strings. When creating images, the provided values are checked to ensure they are a valid type and category.
  - Add `updated_at` and `deleted_at` columns. `deleted_at` is currently unused.
  - Use SQLite's built-in timestamp features to populate these. Add a trigger to update `updated_at` when the row is updated. Currently no way to update a row.
  - Rename the `id` column in `images` to `image_name`
- Rename `ImageCategory.IMAGE` to `ImageCategory.GENERAL`
- Move all exceptions outside their base classes to make them more portable.
- Add `width` and `height` columns to the database. These store the actual dimensions of the image file, whereas the metadata's `width` and `height` refer to the respective generation parameters and are nullable.
- Make `deserialize_image_record` take a `dict` instead of `sqlite3.Row`
- Improve comments throughout
- Tidy up unused code/files and some minor organisation
2023-05-24 11:30:47 -04:00
psychedelicious
021e5a2aa3 feat(nodes): improve metadata service comments 2023-05-24 11:30:47 -04:00
psychedelicious
c31ff364ab fix(nodes): tidy images service 2023-05-24 11:30:47 -04:00
psychedelicious
f310a39381 feat(nodes): finalize image routes 2023-05-24 11:30:47 -04:00
psychedelicious
5a7e611e0a fix(nodes): fix image url 2023-05-24 11:30:47 -04:00
psychedelicious
5de3c41d19 feat(nodes): add metadata handling 2023-05-24 11:30:47 -04:00
psychedelicious
b9375186a5 feat(nodes): consolidate image routers 2023-05-24 11:30:47 -04:00
psychedelicious
11bd932cba feat(nodes): revert invocation_complete url hack 2023-05-24 11:30:47 -04:00
psychedelicious
60d25f105f fix(nodes): restore metadata traverser 2023-05-24 11:30:47 -04:00
psychedelicious
734b653a5f fix(nodes): add base images router 2023-05-24 11:30:47 -04:00
psychedelicious
52c9e6ec91 feat(nodes): organise/tidy 2023-05-24 11:30:47 -04:00
psychedelicious
c0f132e41a hack(nodes): hack to get image urls in the invocation complete event 2023-05-24 11:30:47 -04:00
psychedelicious
cc1160a43a feat(nodes): streamline urlservice 2023-05-24 11:30:47 -04:00
psychedelicious
adde8450bc fix(nodes): remove bad import 2023-05-24 11:30:47 -04:00
psychedelicious
5bf9891553 feat(nodes): it works 2023-05-24 11:30:47 -04:00
psychedelicious
22c34c343a feat(nodes): fix types for InvocationServices 2023-05-24 11:30:47 -04:00
psychedelicious
f7804f6126 feat(nodes): add logger to images service 2023-05-24 11:30:47 -04:00
psychedelicious
d14b02e93f feat(logger): fix logger type issues 2023-05-24 11:30:47 -04:00
psychedelicious
1b75d899ae feat(nodes): wip image storage implementation 2023-05-24 11:30:47 -04:00
psychedelicious
d4aa79acd7 fix(nodes): use save instead of set
`set` is a python builtin
2023-05-24 11:30:47 -04:00
psychedelicious
33d199c007 feat(nodes): image records router 2023-05-24 11:30:47 -04:00
psychedelicious
9c89d3452c feat(nodes): add high-level images service
feat(nodes): add ResultsServiceABC & SqliteResultsService

**Doesn't actually work bc of circular imports. Can't even test it.**

- add a base class for ResultsService and SQLite implementation
- use `graph_execution_manager` `on_changed` callback to keep `results` table in sync

fix(nodes): fix results service bugs

chore(ui): regen api

fix(ui): fix type guards

feat(nodes): add `result_type` to results table, fix types

fix(nodes): do not shadow `list` builtin

feat(nodes): add results router

It doesn't work due to circular imports still

fix(nodes): Result class should use outputs classes, not fields

feat(ui): crude results router

fix(ui): send to canvas in currentimagebuttons not working

feat(nodes): add core metadata builder

feat(nodes): add design doc

feat(nodes): wip latents db stuff

feat(nodes): images_db_service and resources router

feat(nodes): wip images db & router

feat(nodes): update image related names

feat(nodes): update urlservice

feat(nodes): add high-level images service
2023-05-24 11:30:47 -04:00
Lincoln Stein
7ea995149e fixes to env parsing, textual inversion & help text
- Make environment variable settings case InSenSiTive:
  INVOKEAI_MAX_LOADED_MODELS and InvokeAI_Max_Loaded_Models
  environment variables will both set `max_loaded_models`

- Updated realesrgan to use new config system.

- Updated textual_inversion_training to use new config system.

- Discovered a race condition when InvokeAIAppConfig is created
  at module load time, which makes it impossible to customize
  or replace the help message produced with --help on the command
  line. To fix this, moved all instances of get_invokeai_config()
  from module load time to object initialization time. Makes code
  cleaner, too.

- Added `--from_file` argument to `invokeai-node-cli` and changed
  github action to match. CI tests will hopefully work now.
2023-05-18 10:48:23 -04:00
Eugene
20ca9e1fc1 config: move 'CORS' settings to 'Web Server' in the docstring to match the actual category 2023-05-17 19:45:51 -04:00
Eugene
8a8b09a953 api_app: rename web_config to app_config for consistency 2023-05-17 19:42:13 -04:00
Eugene
9e4e386c9b web and formatting fixes
- remove non-existent import InvokeAIWebConfig
- fix workflow file formatting
- clean up whitespace
2023-05-17 19:12:03 -04:00
Lincoln Stein
eca1e449a8 Merge branch 'lstein/global-configuration' of github.com:invoke-ai/InvokeAI into lstein/global-configuration 2023-05-17 15:23:21 -04:00
Lincoln Stein
ffaadb9d05 reorder options in help text 2023-05-17 15:22:58 -04:00
Lincoln Stein
8adff96e29
Merge branch 'main' into lstein/global-configuration 2023-05-17 14:37:09 -04:00
Lincoln Stein
7593dc19d6 complete several steps needed to make 3.0 installable
- invokeai-configure updated to work with new config system
- migrate invokeai.init to invokeai.yaml during configure
- replace legacy invokeai with invokeai-node-cli
- add ability to run an invocation directly from invokeai-node-cli command line
- update CI tests to work with new invokeai syntax
2023-05-17 14:13:27 -04:00
Lincoln Stein
b7c5a39685 make invokeai.yaml more hierarchical; fix list configuration bug 2023-05-17 12:19:19 -04:00
Lincoln Stein
eadfd239a8 update config script to work with new config system 2023-05-17 00:18:19 -04:00
Lincoln Stein
8d75e50435 partial port of invokeai-configure 2023-05-16 01:50:01 -04:00
psychedelicious
1d9c115225 feat(nodes): add low and high to RandomIntInvocation 2023-05-16 13:50:52 +10:00
Eugene Brodsky
2623941d91
Merge branch 'main' into lstein/bugfix/compel 2023-05-13 22:23:59 -04:00
Lincoln Stein
1103ab2844 merge with main 2023-05-13 21:35:19 -04:00
Lincoln Stein
11b2076b46 implement change to web_config suggested by ebr 2023-05-13 21:33:19 -04:00
Eugene Brodsky
63db3fc22f reduce queue check interval to 0.5s 2023-05-12 17:54:26 -04:00
Eugene
ad0bb3f61a fix: queue error should not crash InvocationProcessor
1. if retrieving an item from the queue raises an exception, the
   InvocationProcessor thread crashes, but the API continues running in
   a non-functional state. This fixes the issue
2. when there are no items in the queue, sleep 1 second before checking
   again.
3. Also ensures the thread isn't crashed if an exception is raised from
   invoker, and emits the error event

Intentionally using base Exceptions because for now we don't know which
specific exception to expect.

Fixes (sort of)? #3222
2023-05-12 17:54:26 -04:00
psychedelicious
e5b7dd63e9 fix(nodes): temporarily disable librarygraphs
- Do not retrieve graph from DB until we resolve the issue of changing node schemas causing application to fail to start up due to invalid graphs
2023-05-12 22:33:49 +10:00
Eugene Brodsky
af060188bd
Merge branch 'main' into lstein/bugfix/compel 2023-05-12 08:22:18 -04:00
blessedcoolant
9c1e65f3a3
Merge branch 'main' into feat/nodes/add-randomintinvocation 2023-05-12 15:56:41 +12:00
psychedelicious
ebec200ba6
Remove unused import 2023-05-12 13:56:02 +10:00
blessedcoolant
0acb8ed85d
Merge branch 'main' into feat/nodes/add-w-h-latentsoutput 2023-05-12 15:23:29 +12:00
blessedcoolant
8c1c9cd702
Merge branch 'main' into feat/nodes/add-randomintinvocation 2023-05-12 15:21:49 +12:00
blessedcoolant
af95cef7f9
Merge branch 'main' into fix/nodes/fix-imageoutput-optionals 2023-05-12 15:08:19 +12:00
Sergey Borisov
27dc07d95a Set zero eta by default(fix ddim scheduler error) 2023-05-11 18:49:27 +03:00
blessedcoolant
f7dc171c4f Rename default schedulers across the app 2023-05-12 03:44:20 +12:00
blessedcoolant
8a836247c8 Add DPMPP Single, Euler Karras and DPMPP2 Multi Karras Schedulers 2023-05-12 02:23:33 +12:00
blessedcoolant
9a383e456d Codesplit SCHEDULER_MAP for reusage 2023-05-12 00:40:03 +12:00
blessedcoolant
c1e7460d39
Merge branch 'main' into unipc-sched 2023-05-12 00:11:09 +12:00
blessedcoolant
3ffff023b2 Add missing key to scheduler_map
It was breaking coz the sampler was not being reset. So needs a key on each. Will simplify this later.
2023-05-12 00:08:50 +12:00
psychedelicious
6cf308004a fix(nodes): remove Optionals on ImageOutputs 2023-05-11 20:54:57 +10:00
blessedcoolant
d1029138d2 Default to DDIM if scheduler is missing 2023-05-11 22:54:35 +12:00
blessedcoolant
06b5800d28 Add UniPC Scheduler 2023-05-11 22:43:18 +12:00
psychedelicious
483f2ccb56 feat(nodes): add RandomIntInvocation
just outputs a single random int
2023-05-11 20:33:32 +10:00
psychedelicious
93ced0bec6 feat(nodes): add w/h to latents outputs
This reduces the number of nodes needed when working with latents (ie fewer plain integer value nodes)

Also correct a few mistakes in the fields
2023-05-11 20:32:55 +10:00
psychedelicious
4333852c37 fix(nodes): fix missing context arg in LatentsToLatents 2023-05-11 19:28:42 +10:00
Eugene Brodsky
3baa230077
Merge branch 'main' into lstein/bugfix/compel 2023-05-11 00:50:45 -04:00
psychedelicious
f488b1a7f2 fix(nodes): fix usage of Optional 2023-05-11 11:55:51 +10:00
psychedelicious
34f3a0f0e3 feat(nodes): improve default model choosing output 2023-05-11 11:55:51 +10:00
psychedelicious
d0bac1675e fix(nodes): fix ImageOutput Config 2023-05-11 11:55:51 +10:00
psychedelicious
4e56c962f4 fix(nodes): fix infill docstrings 2023-05-11 11:55:51 +10:00
psychedelicious
4ef0e43759 fix(nodes): remove dataURL invocation 2023-05-11 11:55:51 +10:00
psychedelicious
a7786d5ff2 fix(nodes): restore seamless to TextToLatents 2023-05-11 11:55:51 +10:00
psychedelicious
6c1de975d9 feat(nodes): add infill nodes 2023-05-11 11:55:51 +10:00
psychedelicious
a1079e455a feat(nodes): cleanup unused params, seed generation 2023-05-11 11:55:51 +10:00
psychedelicious
da4eacdffe feat(nodes): add InfillInvocation 2023-05-11 11:55:51 +10:00
psychedelicious
6102e560ba feat(nodes): add LatentsToImage node (VAE encode) 2023-05-11 11:55:51 +10:00
psychedelicious
20f6a597ab fix(nodes): add MetadataColorField 2023-05-11 11:55:51 +10:00
psychedelicious
206e6b1730 feat(nodes): wip inpaint node 2023-05-11 11:55:51 +10:00
psychedelicious
357cee2849 fix(nodes): fix cfg scale min value 2023-05-11 11:55:51 +10:00
psychedelicious
0b49997bb6 feat(nodes): allow uploaded images to be any ImageType (eg intermediates) 2023-05-11 11:55:51 +10:00
psychedelicious
5e09dd380d Revert "feat(nodes): free gpu mem after invocation"
This reverts commit 99cb33f477306d5dcc455efe04053ce41b8d85bd.
2023-05-11 11:55:51 +10:00
psychedelicious
a75148cb16 feat(nodes): free gpu mem after invocation 2023-05-11 11:55:51 +10:00
psychedelicious
e0b9b5cc6c feat(nodes): add dataURL to image node 2023-05-11 11:55:51 +10:00
Lincoln Stein
aca4770481 fixed compel.py as requested 2023-05-10 21:40:44 -04:00
psychedelicious
ee24ad7b13 fix(nodes): fix broken docs routes 2023-05-10 08:28:17 -04:00
psychedelicious
f8e90ba3f0 feat(nodes): add ui build static route 2023-05-10 08:28:17 -04:00
psychedelicious
7dfa135b2c fix(nodes): fix #3306
Check if the cache has the object before deleting it.
2023-05-10 15:29:10 +10:00
Lincoln Stein
42d938fda5 remove debugging statement 2023-05-06 23:54:11 -04:00
Lincoln Stein
25ce47c44f remove reference to globals in compel.py 2023-05-06 22:49:35 -04:00
Lincoln Stein
afd2e32092
Merge branch 'main' into lstein/global-configuration 2023-05-06 21:20:25 -04:00
StAlKeR7779
a80fe05e23 Rename compel node 2023-05-05 21:30:16 +03:00
StAlKeR7779
58d7833c5c Review changes 2023-05-05 21:09:29 +03:00
StAlKeR7779
5012f61599 Separate conditionings back to positive and negative 2023-05-05 15:47:51 +03:00
blessedcoolant
85c33823c3
Merge branch 'main' into feat/compel_node 2023-05-05 14:41:45 +12:00
psychedelicious
e04ada1319
Merge branch 'main' into patch-1 2023-05-05 10:38:45 +10:00
StAlKeR7779
81ec476f3a Revert seed field addition 2023-05-04 21:50:40 +03:00
StAlKeR7779
1e6adf0a06 Fix default graph and test 2023-05-04 21:14:31 +03:00
StAlKeR7779
7d221e2518 Combine conditioning to one field(better fits for multiple type conditioning like perp-neg) 2023-05-04 20:14:22 +03:00
Lincoln Stein
742ed19d66 add missing config module 2023-05-04 01:20:30 -04:00
Lincoln Stein
e4196bbe5b adjust non-app modules to use new config system 2023-05-04 00:43:51 -04:00
Lincoln Stein
15ffb53e59 remove globals, args, generate and the legacy CLI 2023-05-03 23:36:51 -04:00
Lincoln Stein
90054ddf0d use InvokeAISettings for app-wide configuration 2023-05-03 22:30:30 -04:00
StAlKeR7779
56d3cbead0
Merge branch 'main' into feat/compel_node 2023-05-04 00:28:33 +03:00
Lincoln Stein
4687ad4ed6
Merge branch 'main' into enhance/invokeai-logs 2023-05-03 13:36:06 -04:00
psychedelicious
f9f40adcdc fix(nodes): fix t2i graph
Removed width and height edges.
2023-05-02 13:11:28 +10:00
psychedelicious
3264d30b44 feat(nodes): allow multiples of 8 for dimensions 2023-05-02 12:01:52 +10:00
Eugene
d39de0ad38 fix(nodes): fix duplicate Invoker start/stop events 2023-05-01 18:24:37 -04:00
Eugene
d14a7d756e nodes-api: enforce single thread for the processor
On hyperthreaded CPUs we get two threads operating on the queue by
default on each core. This cases two threads to process queue items.
This results in pytorch errors and sometimes generates garbage.

Locking this to single thread makes sense because we are bound by the
number of GPUs in the system, not by CPU cores. And to parallelize
across GPUs we should just start multiple processors (and use async
instead of threading)

Fixes #3289
2023-05-01 18:24:37 -04:00
Lincoln Stein
b050c1bb8f use logger in ApiDependencies 2023-05-01 16:27:44 -04:00
psychedelicious
b49d76ebee feat(nodes): fix image to image fit param
it was ignored previously.
2023-05-01 17:28:22 +10:00
Lincoln Stein
974841926d logger is a interchangeable service 2023-04-29 10:48:50 -04:00
Lincoln Stein
8db20e0d95 rename log to logger throughout 2023-04-29 09:43:40 -04:00
Lincoln Stein
6b79e2b407 Merge branch 'main' into enhance/invokeai-logs
- resolve conflicts
- remove unused code identified by pyflakes
2023-04-28 10:09:46 -04:00
psychedelicious
4a924c9b54 feat(nodes): hardcode resize latents downsampling 2023-04-28 09:52:09 +10:00
psychedelicious
0453d60c64 fix(nodes): fix slatents and rlatents bugs 2023-04-28 09:52:09 +10:00
psychedelicious
c4f4f8b1b8 fix(nodes): remove unused width and height from t2l 2023-04-28 09:52:09 +10:00
psychedelicious
3e80eaa342 feat(nodes): add resize and scale latents nodes
- this resize/scale latents is what is needed for hires fix
- also remove unused `seed` from t2l
2023-04-28 09:52:09 +10:00
Andy Luhrs
0f95f7cea3
Fix inpaint node
Seems like this is the only change needed for the existing inpaint node to work.
2023-04-27 11:03:07 -07:00
StAlKeR7779
0b0068ab86
Merge branch 'main' into feat/compel_node 2023-04-27 14:53:10 +03:00
psychedelicious
a824f47bc6 fix(nodes): use absolute path when deleting 2023-04-27 14:10:44 +10:00
psychedelicious
0cc739afc8 feat(nodes): use send2trash to delete images, fix thumbnail_path 2023-04-27 14:10:44 +10:00
maryhipp
82c4dd8b86 fix(api): return same URL on location header 2023-04-26 06:29:30 +10:00
psychedelicious
711d09a107 feat(nodes): add get_uri method to image storage
- gets the external URI of an image
2023-04-26 06:29:30 +10:00
psychedelicious
74013b6611 fix(nodes): address feedback 2023-04-26 06:29:30 +10:00
psychedelicious
790f399986 feat(nodes): tidy images routes 2023-04-26 06:29:30 +10:00
psychedelicious
73cdd36594 feat(nodes): raise HTTPExceptions instead of returning Reponses 2023-04-26 06:29:30 +10:00
psychedelicious
50ac3eb28d feat(nodes): add delete_image & delete_images routes 2023-04-26 06:29:30 +10:00
StAlKeR7779
89f1909e4b Update default graph 2023-04-25 13:11:50 +03:00