Commit Graph

657 Commits

Author SHA1 Message Date
Brandon Rising
35d5ef9118 Emit step completions 2023-07-18 12:35:07 -04:00
Sergey Borisov
3240f98f4e Rename clip1 to clip 2023-07-18 18:58:17 +03:00
blessedcoolant
7fde1f93ea fix: Missing context on string param node 2023-07-19 02:49:09 +12:00
blessedcoolant
3f1d5000c0 Merge branch 'main' into nodes-stuff 2023-07-19 02:37:50 +12:00
blessedcoolant
0c18c5d603 feat: Add titles and tags to all Nodes 2023-07-19 02:26:45 +12:00
blessedcoolant
7d49c727a0 feat: Add String Param & types to other params 2023-07-19 02:26:33 +12:00
StAlKeR7779
889b77d3d6
Merge branch 'main' into save_vram 2023-07-18 16:55:48 +03:00
Sergey Borisov
fbbc4b3f69 Fixes 2023-07-18 16:51:16 +03:00
Sergey Borisov
bc11296a5e Disable lazy offloading on disabled vram cache, move resulted tensors to cpu(to not stack vram tensors in cache), fix - text encoder not freed(detach) 2023-07-18 16:20:25 +03:00
psychedelicious
c19d48abd0 fix(nodes): fix inpaint cond logic for new compel version
thanks @StAlKeR7779
2023-07-18 22:39:34 +10:00
psychedelicious
42c440c73f
Merge branch 'main' into feat/ui/upscale 2023-07-18 22:08:02 +10:00
Lincoln Stein
9c3c393b84 merge with main 2023-07-18 07:00:55 -04:00
psychedelicious
56098f370c feat(nodes): add RealESRGAN_x2plus.pth, update upscale nodes
- add `RealESRGAN_x2plus.pth` model to installer
- add `RealESRGAN_x2plus.pth` to `realesrgan` node
- rename `RealESRGAN` to `ESRGAN` in nodes
- make `scale_factor` optional in `img_scale` node
2023-07-18 14:55:18 +10:00
Lincoln Stein
ef31837167 fix caption on sdxl raw prompt 2023-07-17 23:49:23 -04:00
blessedcoolant
13da881953 Merge branch 'main' into sdxl-support 2023-07-18 13:34:07 +12:00
Sergey Borisov
ada9b06e48 Implement compel prompt nodes for sdxl 2023-07-18 01:49:45 +03:00
Lincoln Stein
6ae10798b0
Merge branch 'main' into feat/model-events 2023-07-17 17:15:12 -04:00
Brandon Rising
bcce70fca6 Testing different session opts, added timings for testing 2023-07-17 16:27:33 -04:00
skunkworxdark
f767bf2330 use FileNotFoundError insted of "File path not found" 2023-07-17 05:49:09 -04:00
skunkworxdark
b1008af696 apply changes as suggested @psychedelicious in PR comments.
- filename -> file_path
- pre and post prompt changed to optional
- clearer pre and post prompt descriptions
- handle pre and post prompt passed as None
- max_prompts defaults to 1 isted of 0 to avoid accidentally processing large prompt files with it set to 0 when adding a new node.
2023-07-17 05:49:09 -04:00
skunkworxdark
956011066d Added class PromptsFromFileInvocation to prompt.py. A new PromptFromFile Custom node that reads prompts from a file one line per prompt and outputs them as a prompt collection. With inputs for filename, pre_prompt, post_prompt, start line number, and max_prompts 2023-07-17 05:49:09 -04:00
blessedcoolant
e039771d07 fix: Incorrect type on SDXL Model Loader 2023-07-17 21:47:41 +12:00
psychedelicious
3e2a948007
Merge branch 'main' into feat/model-events 2023-07-17 17:36:20 +10:00
psychedelicious
ba12849685 fix(nodes): fix some model load events not emitting
Missed adding the `context` arg to them initially
2023-07-17 17:16:55 +10:00
Lincoln Stein
cad3f96831 add model input to refiner 2023-07-16 12:38:04 -04:00
Lincoln Stein
6534288b75 refiner only has clip2 not clip 2023-07-16 12:36:38 -04:00
Lincoln Stein
0a2964d8c0 add differentiated sdxl and sdxl_refiner model loaders 2023-07-16 12:17:56 -04:00
Brandon Rising
91112167b1 Fix syntax err 2023-07-15 23:56:48 -04:00
Lincoln Stein
5206ddf9b2 truncate long prompts to avoid a crash with controlnet 2023-07-15 23:49:25 -04:00
Sergey Borisov
fe78a08e37 Fix sd1/2 models conditionings 2023-07-16 06:24:24 +03:00
Sergey Borisov
c9c2229917 Separate prompt to sdxl and sdxl-refiner, add denoising start-end fields, add l2l node(supports both sdxl and sdxl-refiner), add fp32 to vae encode 2023-07-16 06:00:37 +03:00
psychedelicious
5d59dd4b97 feat(nodes): use correctly-typed configuration service in upscale node 2023-07-16 10:54:52 +10:00
Lincoln Stein
ccbfa5d862 resolve conflicts 2023-07-15 19:47:50 -04:00
psychedelicious
7b6159f8d6 feat(nodes): emit model loading events
- remove dependency on having access to a `node` during emits, would need a bit of additional args passed through the system and I don't think its necessary at this point. this also allowed us to drop an extraneous fetching/parsing of the session from db.
- provide the invocation context to all `get_model()` calls, so the events are able to be emitted
- test all model loading events in the app and confirm socket events are received
2023-07-16 02:12:01 +10:00
psychedelicious
c7b547ea3e feat(nodes): remove references to restoration services
- remove restoration services
- remove the restore faces nodes
- update tests
2023-07-16 01:12:39 +10:00
psychedelicious
74ca87ac9e feat(nodes): add realesrgan node 2023-07-16 01:06:50 +10:00
psychedelicious
d270f21c85 feat(nodes): valid controlnet weights are -1 to 2 2023-07-15 19:56:44 +10:00
psychedelicious
ae72f372be fix(nodes): do not use hardcoded controlnet model 2023-07-15 19:56:44 +10:00
psychedelicious
29b2e59e65 fix(nodes): fix ref to ctx mgr service, missing import 2023-07-15 19:56:44 +10:00
psychedelicious
82fa39b531 feat(nodes): add controlnet nodes type hint 2023-07-15 19:56:44 +10:00
psychedelicious
788dcbde70 fix(nodes): add missing import 2023-07-15 19:56:44 +10:00
Sergey Borisov
6ab9a5e108 Draft 2023-07-15 19:56:44 +10:00
Sergey Borisov
7093e5d033 Pad conditionings using zeros and encoder_attention_mask 2023-07-15 00:52:54 +03:00
Brandon Rising
bd7b59910d Testing onnx in new ui updates 2023-07-14 14:24:15 -04:00
Jonathan
9960d7ca2a
Allow ImageResizeInvocation w/h to take inputs from other nodes (#3765) 2023-07-15 05:34:13 +12:00
psychedelicious
23c1a6b9d5 fix(nodes): make ResizeLatents w/h optional
now you can connect to them in node editor
2023-07-14 21:42:42 +10:00
Sergey Borisov
9fb0b0959f Make sde and ancestral schedulers reproducible 2023-07-14 05:25:09 +03:00
Sergey Borisov
d8f88c09ea Fix pink output on a lot of samplers 2023-07-14 05:00:33 +03:00
Brandon Rising
524888bf3b Merge branch 'main' into feat/onnx 2023-07-13 14:23:57 -04:00
blessedcoolant
430b9c291f fix: Loras not working correctly with Inpainting 2023-07-13 22:59:38 +12:00
psychedelicious
50bef87da7 feat(db,nodes,api): refactor metadata
Metadata for the Linear UI is now sneakily provided via a `MetadataAccumulator` node, which the client populates / hooks up while building the graph.

Additionally, we provide the unexpanded graph with the metadata API response.

Both of these are embedded into the PNGs.

- Remove `metadata` from `ImageDTO`
- Split up the `images/` routes to accomodate this; metadata is only retrieved per-image
- `images/{image_name}` now gets the DTO
- `images/{image_name}/metadata` gets the new metadata
- `images/{image_name}/full` gets the full-sized image file
- Remove old metadata service
- Add `MetadataAccumulator` node, `CoreMetadataField`, hook up to `LatentsToImage` node
- Add `get_raw()` method to `ItemStorage`, retrieves the row from DB as a string, no pydantic parsing
- Update `images`related services to handle storing and retrieving the new metadata
- Add `get_metadata_graph_from_raw_session` which extracts the `graph` from `session` without needing to hydrate the session in pydantic, in preparation for providing it as metadata; also removes all references to the `MetadataAccumulator` node
2023-07-13 15:40:05 +10:00
Lincoln Stein
75c5ce46bc merged SDXLModelLoader into ModelLoader invocation 2023-07-11 16:33:08 -04:00
Sergey Borisov
358ced6bab SDXL Prompt and t2l nodes draft, add fp32 to vae decode 2023-07-11 18:19:36 +03:00
Lincoln Stein
8e42502dfd partial implementation of SDXL model loader 2023-07-10 20:18:30 -04:00
Lincoln Stein
10d3bccf32
Mac MPS FP16 fixes (#3641)
This PR is to allow FP16 precision to work on Macs with MPS. In
addition, it centralizes the torch fixes/workarounds required for MPS
into a new backend utility `mps_fixes.py`. This is conditionally
imported in `api_app.py`/`cli_app.py`.

Many MANY thanks to @StAlKeR7779 for patiently working to debug and fix
these issues.
2023-07-07 17:43:23 -04:00
Eugene Brodsky
97b2ec58e2
Merge branch 'main' into release/invokeai-3-0-alpha 2023-07-07 14:18:12 -04:00
Sergey Borisov
a9e77675a8 Move clip skip to separate node 2023-07-06 17:39:49 +03:00
Sergey Borisov
04b57c408f Add clip skip option to prompt node 2023-07-06 16:09:40 +03:00
Lincoln Stein
83d3a043da merge latest changes from main 2023-07-05 19:15:53 -04:00
Lincoln Stein
021e1eca8e
Merge branch 'main' into mps-fp16-fixes 2023-07-05 13:19:52 -04:00
Sergey Borisov
0ac9dca926 Fix loading diffusers ti 2023-07-05 19:46:00 +03:00
Lincoln Stein
9edf78dd2e merge with main 2023-07-05 09:12:54 -04:00
gogurtenjoyer
ba7345deb4
Merge branch 'main' into mps-fp16-fixes 2023-07-05 07:38:41 -04:00
blessedcoolant
639d88afd6 revert: inference_mode to no_grad 2023-07-05 16:39:15 +12:00
blessedcoolant
c0501ed5c2 fix: Slow loading of Loras
Co-Authored-By: StAlKeR7779 <7768370+StAlKeR7779@users.noreply.github.com>
2023-07-05 12:47:34 +10:00
psychedelicious
08d428a5e7 feat(nodes): add lora field, update lora loader 2023-07-05 12:47:34 +10:00
gogurtenjoyer
233869b56a Mac MPS FP16 fixes
This PR is to allow FP16 precision to work on Macs with MPS. In addition, it centralizes the torch fixes/workarounds
required for MPS into a new backend utility file `mps_fixes.py`. This is conditionally imported in `api_app.py`/`cli_app.py`.

Many MANY thanks to StAlKeR7779 for patiently working to debug and fix these issues.
2023-07-04 18:10:53 -04:00
blessedcoolant
7e18814dd0 Add standard names for Model Loader Nodes 2023-07-04 14:35:06 +10:00
Lincoln Stein
a8a2209560 VAE loader is loading proper VAE. Unclear if it is changing the image 2023-07-04 14:35:06 +10:00
Lincoln Stein
fa8a5838d3 add vae lodaer 2023-07-04 14:35:06 +10:00
blessedcoolant
6c62f41f2e chore: Change PipelineModels to MainModels 2023-07-04 14:33:56 +10:00
Lincoln Stein
4d2c7806fc quash memory leak when compel invocation called 2023-07-03 14:12:35 -04:00
Lincoln Stein
3937428563 Merge branch 'release/invokeai-3-0-alpha' of github.com:invoke-ai/InvokeAI into release/invokeai-3-0-alpha 2023-07-03 14:11:28 -04:00
Lincoln Stein
ed86d0b708 Union[foo, None]=>Optional[foo] 2023-07-03 12:17:45 -04:00
psychedelicious
fa169b5517 feat(nodes): add ImageCollection node in prep for batch processing 2023-07-04 00:18:27 +10:00
Lincoln Stein
aae60b6142 quash memory leak when compel invocation called 2023-07-03 10:08:10 -04:00
Lincoln Stein
b79740d61d back out torch.no_grad() 2023-07-02 23:03:24 -04:00
Lincoln Stein
fa8ccd2a94 add no_grad() to compel node invoke() method 2023-07-02 18:20:16 -04:00
StAlKeR7779
ac46b129bf
Merge branch 'main' into feat/lora_model_patch 2023-06-28 22:43:58 +03:00
Lincoln Stein
72209d0cc3
Merge branch 'main' into lstein/installer-for-new-model-layout 2023-06-28 14:49:37 -04:00
blessedcoolant
75614bbba3
Merge branch 'main' into feat/nodes/cpu-noise 2023-06-28 18:22:08 +12:00
blessedcoolant
32883adf6e
Merge branch 'main' into feat/controlnet_extras 2023-06-28 17:36:21 +12:00
Kent Keirsey
fc322aa9f7 Update controlnet-aux to 0.0.6 and add LeReS 2023-06-27 23:45:47 -04:00
psychedelicious
73f63853ba fix(nodes): use context for logger in param_easing 2023-06-27 23:30:10 -04:00
psychedelicious
2e14528e4c feat(nodes): default to CPU noise 2023-06-27 13:57:31 +10:00
user1
10d2d85c83 Started to add ControlNet resize_crop and resize_fill options, but commented out, not ready to deploy yet. 2023-06-26 12:03:05 -07:00
user1
af566adf56 For MediapipeFace ControlNet preprocessor, if input image is RGBA format then convert to RGB (otherwise MediapipeFace image processing throws an error) 2023-06-26 04:29:43 -07:00
user1
873c18bc4b Added TileResampler ControlNet preprocessor node.
Also fixes to SegmentAnything ControlNet preprocessor node.
2023-06-26 04:27:26 -07:00
Eugene Brodsky
7b97639961
Merge branch 'main' into lstein/installer-for-new-model-layout 2023-06-26 01:24:30 -04:00
Sergey Borisov
5cebf67ee4 Apply lora by patching lora instead of hooks 2023-06-26 03:57:33 +03:00
user1
10e8389fa4 Commenting out LeReS ControlNet image preprocessor until release of controlnet_aux v0.0.6 (supported on controlnet_aux current main, but not on latest release v0.0.5) 2023-06-25 14:25:14 -07:00
user1
414a04774c Added LeReS ControlNet image preprocessor. 2023-06-25 14:19:55 -07:00
user1
b872e7a5e0 Simplifying ControlNet SAM preprocessor segmentation color mapping. 2023-06-25 12:54:48 -07:00
user1
de4064bdac Fixed problem with with non-reproducible results from ControlNet SegmentAnything preprocessor. Cause was controlnet_aux randomization of segmentation coloring, which seems to lead to some randomization of resulting images using ControlNet seg model. Switched to using deterministic ADE20K color palette instead, which solved the problem. 2023-06-25 12:38:17 -07:00
user1
10c3753d7f Added SAM preprocessor 2023-06-25 11:16:39 -07:00
user1
c5faffc18b Merge branch 'main' of github.com:invoke-ai/InvokeAI into feat/controlnet-control-modes
Only "real" conflicts were in:
     invokeai/frontend/web/src/features/controlNet/components/ControlNet.tsx
     invokeai/frontend/web/src/features/controlNet/store/controlNetSlice.ts
2023-06-24 17:05:57 -07:00
Lincoln Stein
d5f742620f Merge branch 'main' into lstein/installer-for-new-model-layout 2023-06-24 11:58:06 -04:00
Lincoln Stein
ba1371a88f rename ModelType.Pipeline to ModelType.Main 2023-06-24 11:45:49 -04:00
psychedelicious
bab3a9504e fix(nodes): fix LatentsToImage not using is_intermediate when creating images
Appears this was removed during a merge conflict resolution.
2023-06-24 17:57:39 +10:00
blessedcoolant
bb85608890 Merge branch 'main' into feat/onnx 2023-06-23 05:18:41 +12:00
Sergey Borisov
6c7668aaca Update onnx model structure, change code according 2023-06-22 20:03:17 +03:00
psychedelicious
1bc170727b tidy(nodes): rename sd_model_loader to pipeline_model_loader
this is more accurate bc it can do eg kandinsky also
2023-06-22 17:47:58 +10:00
psychedelicious
42a59aa147 feat(nodes): add sd_model_loader node
Loads any pipeline model.

Also introduced is `PipelineModelField`, which includes a model name and base model.
2023-06-22 17:36:05 +10:00
Sergey Borisov
7759b3f75a Small refactor 2023-06-21 04:24:25 +03:00
Sergey Borisov
4d337f6abc ONNX Model/runtime first implementation 2023-06-21 02:12:21 +03:00
blessedcoolant
4f5693040e
Merge branch 'main' into fix/inpaint_new_manager 2023-06-19 22:55:00 +12:00
Sergey Borisov
a01998d095 Remove more old logic 2023-06-19 15:57:28 +10:00
Sergey Borisov
7b35162b9e Remove old logic except for inpaint, add support for lora and ti to inpaint node 2023-06-19 15:57:28 +10:00
Sergey Borisov
c26e1a9271 Rewrite inpaint node to new model manager, remove TextToImage and ImageToImage nodes 2023-06-19 15:57:28 +10:00
Sergey Borisov
9b32407744 Provide generator to all schedulers step function to make both ancestral and sde schedulers reproducible 2023-06-19 00:34:01 +03:00
Sergey Borisov
f3d9797ebe Add dpmpp_sde and dpmpp_2m_sde schedulers(with karras) 2023-06-18 23:38:15 +03:00
Lincoln Stein
82c2498043
Merge branch 'main' into lstein/new-model-manager 2023-06-14 08:41:40 -07:00
blessedcoolant
6b8e88ad7f Merge branch 'main' into feat/controlnet-control-modes 2023-06-15 03:18:41 +12:00
psychedelicious
34ebee67b7 fix(nodes): fix revert conflict 2023-06-15 01:05:16 +10:00
psychedelicious
b51e9a6bdb Revert "feat(nodes): add warning socket event"
This reverts commit cefdd9d634e515239bd85666c872a0d64bb9d772.
2023-06-15 01:05:16 +10:00
psychedelicious
abee37eab3 feat(nodes): add warning socket event 2023-06-15 01:05:16 +10:00
psychedelicious
42e48b2bef feat(nodes): add dynamic prompt node 2023-06-15 01:05:16 +10:00
StAlKeR7779
d0ee3558d1
Merge branch 'main' into lstein/new-model-manager 2023-06-14 17:29:01 +03:00
psychedelicious
a1773197e9 feat(nodes): remove image_origin from most places
- remove `image_origin` from most places where we interact with images
- consolidate image file storage into a single `images/` dir

Images have an `image_origin` attribute but it is not actually used when retrieving images, nor will it ever be. It is still used when creating images and helps to differentiate between internally generated images and uploads.

It was included in eg API routes and image service methods as a holdover from the previous app implementation where images were not managed in a database. Now that we have images in a db, we can do away with this and simplify basically everything that touches images.

The one potentially controversial change is to no longer separate internal and external images on disk. If we retain this separation, we have to keep `image_origin` around in a number of spots and it getting image paths on disk painful.

So, I am have gotten rid of this organisation. Images are now all stored in `images`, regardless of their origin. As we improve the image management features, this change will hopefully become transparent.
2023-06-14 23:08:27 +10:00
user1
5cd0e90816 Renamed ControlNet control_mode option "even_more_control" to "unbalanced" 2023-06-13 22:30:17 -07:00
user1
de3e6cdb02 Switched over to ControlNet control_mode with 4 options: balanced, more_prompt, more_control, even_more_control. Based on True/False combinations of internal booleans cfg_injection and soft_injection 2023-06-13 21:08:34 -07:00
Sergey Borisov
26090011c4 Fix conflict resolve, add model configs to type annotation 2023-06-14 00:26:37 +03:00
StAlKeR7779
c9ae26a176
Merge branch 'main' into lstein/new-model-manager 2023-06-13 23:37:52 +03:00
Sergey Borisov
e7db6d8120 Fix ckpt and vae conversion, migrate script, remove sd2-base 2023-06-13 18:05:12 +03:00
user1
8495764d45 Moving from ControlNet guess_mode to separate booleans for cfg_injection and soft_injection for testing control modes 2023-06-13 00:41:36 -07:00
user1
8b7fac75ed First pass at ControlNet "guess mode" implementation. 2023-06-13 00:41:36 -07:00
blessedcoolant
2a814d886b
Merge branch 'main' into diffusers-upgrade 2023-06-13 05:29:15 +12:00
Sergey Borisov
36eb1bd893 Fixes 2023-06-12 16:14:09 +03:00
Gregg Helt
c647056287
Feat/easy param (#3504)
* Testing change to LatentsToText to allow setting different cfg_scale values per diffusion step.

* Adding first attempt at float param easing node, using Penner easing functions.

* Core implementation of ControlNet and MultiControlNet.

* Added support for ControlNet and MultiControlNet to legacy non-nodal Txt2Img in backend/generator. Although backend/generator will likely disappear by v3.x, right now they are very useful for testing core ControlNet and MultiControlNet functionality while node codebase is rapidly evolving.

* Added example of using ControlNet with legacy Txt2Img generator

* Resolving rebase conflict

* Added first controlnet preprocessor node for canny edge detection.

* Initial port of controlnet node support from generator-based TextToImageInvocation node to latent-based TextToLatentsInvocation node

* Switching to ControlField for output from controlnet nodes.

* Resolving conflicts in rebase to origin/main

* Refactored ControlNet nodes so they subclass from PreprocessedControlInvocation, and only need to override run_processor(image) (instead of reimplementing invoke())

* changes to base class for controlnet nodes

* Added HED, LineArt, and OpenPose ControlNet nodes

* Added an additional "raw_processed_image" output port to controlnets, mainly so could route ImageField to a ShowImage node

* Added more preprocessor nodes for:
      MidasDepth
      ZoeDepth
      MLSD
      NormalBae
      Pidi
      LineartAnime
      ContentShuffle
Removed pil_output options, ControlNet preprocessors should always output as PIL. Removed diagnostics and other general cleanup.

* Prep for splitting pre-processor and controlnet nodes

* Refactored controlnet nodes: split out controlnet stuff into separate node, stripped controlnet stuff form image processing/analysis nodes.

* Added resizing of controlnet image based on noise latent. Fixes a tensor mismatch issue.

* More rebase repair.

* Added support for using multiple control nets. Unfortunately this breaks direct usage of Control node output port  ==> TextToLatent control input port -- passing through a Collect node is now required. Working on fixing this...

* Fixed use of ControlNet control_weight parameter

* Fixed lint-ish formatting error

* Core implementation of ControlNet and MultiControlNet.

* Added first controlnet preprocessor node for canny edge detection.

* Initial port of controlnet node support from generator-based TextToImageInvocation node to latent-based TextToLatentsInvocation node

* Switching to ControlField for output from controlnet nodes.

* Refactored controlnet node to output ControlField that bundles control info.

* changes to base class for controlnet nodes

* Added more preprocessor nodes for:
      MidasDepth
      ZoeDepth
      MLSD
      NormalBae
      Pidi
      LineartAnime
      ContentShuffle
Removed pil_output options, ControlNet preprocessors should always output as PIL. Removed diagnostics and other general cleanup.

* Prep for splitting pre-processor and controlnet nodes

* Refactored controlnet nodes: split out controlnet stuff into separate node, stripped controlnet stuff form image processing/analysis nodes.

* Added resizing of controlnet image based on noise latent. Fixes a tensor mismatch issue.

* Cleaning up TextToLatent arg testing

* Cleaning up mistakes after rebase.

* Removed last bits of dtype and and device hardwiring from controlnet section

* Refactored ControNet support to consolidate multiple parameters into data struct. Also redid how multiple controlnets are handled.

* Added support for specifying which step iteration to start using
each ControlNet, and which step to end using each controlnet (specified as fraction of total steps)

* Cleaning up prior to submitting ControlNet PR. Mostly turning off diagnostic printing. Also fixed error when there is no controlnet input.

* Added dependency on controlnet-aux v0.0.3

* Commented out ZoeDetector. Will re-instate once there's a controlnet-aux release that supports it.

* Switched CotrolNet node modelname input from free text to default list of popular ControlNet model names.

* Fix to work with current stable release of controlnet_aux (v0.0.3). Turned of pre-processor params that were added post v0.0.3. Also change defaults for shuffle.

* Refactored most of controlnet code into its own method to declutter TextToLatents.invoke(), and make upcoming integration with LatentsToLatents easier.

* Cleaning up after ControlNet refactor in TextToLatentsInvocation

* Extended node-based ControlNet support to LatentsToLatentsInvocation.

* chore(ui): regen api client

* fix(ui): add value to conditioning field

* fix(ui): add control field type

* fix(ui): fix node ui type hints

* fix(nodes): controlnet input accepts list or single controlnet

* Moved to controlnet_aux v0.0.4, reinstated Zoe controlnet preprocessor. Also in pyproject.toml  had to specify downgrade of timm to 0.6.13 _after_ controlnet-aux installs timm >= 0.9.2, because timm >0.6.13 breaks Zoe preprocessor.

* Core implementation of ControlNet and MultiControlNet.

* Added first controlnet preprocessor node for canny edge detection.

* Switching to ControlField for output from controlnet nodes.

* Resolving conflicts in rebase to origin/main

* Refactored ControlNet nodes so they subclass from PreprocessedControlInvocation, and only need to override run_processor(image) (instead of reimplementing invoke())

* changes to base class for controlnet nodes

* Added HED, LineArt, and OpenPose ControlNet nodes

* Added more preprocessor nodes for:
      MidasDepth
      ZoeDepth
      MLSD
      NormalBae
      Pidi
      LineartAnime
      ContentShuffle
Removed pil_output options, ControlNet preprocessors should always output as PIL. Removed diagnostics and other general cleanup.

* Prep for splitting pre-processor and controlnet nodes

* Refactored controlnet nodes: split out controlnet stuff into separate node, stripped controlnet stuff form image processing/analysis nodes.

* Added resizing of controlnet image based on noise latent. Fixes a tensor mismatch issue.

* Added support for using multiple control nets. Unfortunately this breaks direct usage of Control node output port  ==> TextToLatent control input port -- passing through a Collect node is now required. Working on fixing this...

* Fixed use of ControlNet control_weight parameter

* Core implementation of ControlNet and MultiControlNet.

* Added first controlnet preprocessor node for canny edge detection.

* Initial port of controlnet node support from generator-based TextToImageInvocation node to latent-based TextToLatentsInvocation node

* Switching to ControlField for output from controlnet nodes.

* Refactored controlnet node to output ControlField that bundles control info.

* changes to base class for controlnet nodes

* Added more preprocessor nodes for:
      MidasDepth
      ZoeDepth
      MLSD
      NormalBae
      Pidi
      LineartAnime
      ContentShuffle
Removed pil_output options, ControlNet preprocessors should always output as PIL. Removed diagnostics and other general cleanup.

* Prep for splitting pre-processor and controlnet nodes

* Refactored controlnet nodes: split out controlnet stuff into separate node, stripped controlnet stuff form image processing/analysis nodes.

* Added resizing of controlnet image based on noise latent. Fixes a tensor mismatch issue.

* Cleaning up TextToLatent arg testing

* Cleaning up mistakes after rebase.

* Removed last bits of dtype and and device hardwiring from controlnet section

* Refactored ControNet support to consolidate multiple parameters into data struct. Also redid how multiple controlnets are handled.

* Added support for specifying which step iteration to start using
each ControlNet, and which step to end using each controlnet (specified as fraction of total steps)

* Cleaning up prior to submitting ControlNet PR. Mostly turning off diagnostic printing. Also fixed error when there is no controlnet input.

* Commented out ZoeDetector. Will re-instate once there's a controlnet-aux release that supports it.

* Switched CotrolNet node modelname input from free text to default list of popular ControlNet model names.

* Fix to work with current stable release of controlnet_aux (v0.0.3). Turned of pre-processor params that were added post v0.0.3. Also change defaults for shuffle.

* Refactored most of controlnet code into its own method to declutter TextToLatents.invoke(), and make upcoming integration with LatentsToLatents easier.

* Cleaning up after ControlNet refactor in TextToLatentsInvocation

* Extended node-based ControlNet support to LatentsToLatentsInvocation.

* chore(ui): regen api client

* fix(ui): fix node ui type hints

* fix(nodes): controlnet input accepts list or single controlnet

* Added Mediapipe image processor for use as ControlNet preprocessor.
Also hacked in ability to specify HF subfolder when loading ControlNet models from string.

* Fixed bug where MediapipFaceProcessorInvocation was ignoring max_faces and min_confidence params.

* Added nodes for float params: ParamFloatInvocation and FloatCollectionOutput. Also added FloatOutput.

* Added mediapipe install requirement. Should be able to remove once controlnet_aux package adds mediapipe to its requirements.

* Added float to FIELD_TYPE_MAP ins constants.ts

* Progress toward improvement in fieldTemplateBuilder.ts  getFieldType()

* Fixed controlnet preprocessors and controlnet handling in TextToLatents to work with revised Image services.

* Cleaning up from merge, re-adding cfg_scale to FIELD_TYPE_MAP

* Making sure cfg_scale of type list[float] can be used in image metadata, to support param easing for cfg_scale

* Fixed math for per-step param easing.

* Added option to show plot of param value at each step

* Just cleaning up after adding param easing plot option, removing vestigial code.

* Modified control_weight ControlNet param to be polistmorphic --
can now be either a single float weight applied for all steps, or a list of floats of size total_steps, that specifies weight for each step.

* Added more informative error message when _validat_edge() throws an error.

* Just improving parm easing bar chart title to include easing type.

* Added requirement for easing-functions package

* Taking out some diagnostic prints.

* Added option to use both easing function and mirror of easing function together.

* Fixed recently introduced problem (when pulled in main), triggered by num_steps in StepParamEasingInvocation not having a default value -- just added default.

---------

Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com>
2023-06-11 16:27:44 +10:00
Sergey Borisov
738ba40f51 Fixes 2023-06-11 06:12:21 +03:00
Sergey Borisov
2c056ead42 New models structure draft 2023-06-10 03:14:10 +03:00
blessedcoolant
7bce455d16
Merge branch 'main' into diffusers-upgrade 2023-06-09 16:27:52 +12:00
Lincoln Stein
6652f3405b merge with main 2023-06-08 21:08:43 -04:00
blessedcoolant
68405910ba Upgrade to Diffusers 0.17.0 2023-06-08 04:42:52 +12:00
Lincoln Stein
04f9757f8d prevent crash when trying to calculate size of missing safety_checker
- Also fixed up order in which logger is created in invokeai-web
  so that handlers are installed after command-line options are
  parsed (and not before!)
2023-06-06 22:57:49 -04:00
Lincoln Stein
1f9e1eb964 merge with main 2023-06-06 22:18:41 -04:00
Lincoln Stein
5f6f38074d merge with main 2023-06-04 13:59:31 -04:00
blessedcoolant
fb06f5b892
Merge branch 'main' into feat_compel_longprompts_and_concat 2023-06-05 04:34:39 +12:00
Damian Stewart
cdcfda164d enable long prompts, upgrade compel to enable .and() (concatenating prompts) 2023-06-04 15:30:54 +02:00
psychedelicious
6bbb5f061a feat(nodes): update controlnet names/descriptions 2023-06-04 22:34:58 +10:00
psychedelicious
d6a959b000 feat(nodes): tidy controlnet processor nodes & improve descriptions 2023-06-04 22:34:58 +10:00
Lincoln Stein
31e97ead2a move invokeai.db to ~/invokeai/databases
- The invokeai.db database file has now been moved into
  `INVOKEAIROOT/databases`. Using plural here for possible
  future with more than one database file.

- Removed a few dangling debug messages that appeared during
  testing.

- Rebuilt frontend to test web.
2023-06-03 20:25:34 -04:00
Lincoln Stein
98773b20ac merge with main 2023-06-01 18:09:49 -04:00
Sergey Borisov
b47786e846 First working TI draft 2023-05-31 02:12:27 +03:00
psychedelicious
a9a2bd90c2 fix(nodes): set min and max for l2l strength 2023-05-30 09:06:13 -04:00
psychedelicious
47ca71a7eb fix(nodes): set cfg_scale min to 1 in latents 2023-05-30 09:06:13 -04:00
Sergey Borisov
420a76ecdd Add lora loader node 2023-05-30 02:12:33 +03:00
Sergey Borisov
79de9047b5 First working lora implementation 2023-05-30 01:11:00 +03:00
psychedelicious
bbb4e8f5ef feat(nodes): add resize image and scale image nodes 2023-05-29 09:07:46 -04:00
psychedelicious
08a14ee6d5 fix(nodes): fix conflicts with controlnet 2023-05-28 20:19:56 -04:00
psychedelicious
160267c71a feat(nodes): refactor image types
- Remove `ImageType` entirely, it is confusing
- Create `ResourceOrigin`, may be `internal` or `external`
- Revamp `ImageCategory`, may be `general`, `mask`, `control`, `user`, `other`. Expect to add more as time goes on
- Update images `list` route to accept `include_categories` OR `exclude_categories` query parameters to afford finer-grained querying. All services are updated to accomodate this change.

The new setup should account for our types of images, including the combinations we couldn't really handle until now:
- Canvas init and masks
- Canvas when saved-to-gallery or merged
2023-05-28 20:19:56 -04:00
user1
9a796364da Fixed controlnet preprocessors and controlnet handling in TextToLatents to work with revised Image services. 2023-05-26 21:44:00 -04:00
user1
d9b1e4a98c Added nodes for float params: ParamFloatInvocation and FloatCollectionOutput. Also added FloatOutput. 2023-05-26 21:44:00 -04:00
user1
a4dec8c1d6 Fixed bug where MediapipFaceProcessorInvocation was ignoring max_faces and min_confidence params. 2023-05-26 21:44:00 -04:00
user1
8960ceb98b Added Mediapipe image processor for use as ControlNet preprocessor.
Also hacked in ability to specify HF subfolder when loading ControlNet models from string.
2023-05-26 21:44:00 -04:00
psychedelicious
be79d088c0 fix(nodes): controlnet input accepts list or single controlnet 2023-05-26 21:44:00 -04:00
psychedelicious
009407ea3f fix(ui): fix node ui type hints 2023-05-26 21:44:00 -04:00
user1
324e9eb74b Extended node-based ControlNet support to LatentsToLatentsInvocation. 2023-05-26 21:44:00 -04:00
user1
56cff40362 Cleaning up after ControlNet refactor in TextToLatentsInvocation 2023-05-26 21:44:00 -04:00
user1
2ba40c5e52 Refactored most of controlnet code into its own method to declutter TextToLatents.invoke(), and make upcoming integration with LatentsToLatents easier. 2023-05-26 21:44:00 -04:00
user1
3ab147204c Fix to work with current stable release of controlnet_aux (v0.0.3). Turned of pre-processor params that were added post v0.0.3. Also change defaults for shuffle. 2023-05-26 21:44:00 -04:00
user1
e4c89cba9c Switched CotrolNet node modelname input from free text to default list of popular ControlNet model names. 2023-05-26 21:44:00 -04:00
user1
322ea84c4e Commented out ZoeDetector. Will re-instate once there's a controlnet-aux release that supports it. 2023-05-26 21:44:00 -04:00
user1
f2b41c60ff Cleaning up prior to submitting ControlNet PR. Mostly turning off diagnostic printing. Also fixed error when there is no controlnet input. 2023-05-26 21:44:00 -04:00
user1
754acec92f Added support for specifying which step iteration to start using
each ControlNet, and which step to end using each controlnet (specified as fraction of total steps)
2023-05-26 21:44:00 -04:00
user1
11fc7e40a5 Refactored ControNet support to consolidate multiple parameters into data struct. Also redid how multiple controlnets are handled. 2023-05-26 21:44:00 -04:00
user1
d15bb88eb2 Removed last bits of dtype and and device hardwiring from controlnet section 2023-05-26 21:44:00 -04:00
user1
70ba36eefc Cleaning up mistakes after rebase. 2023-05-26 21:44:00 -04:00
user1
7e70391c2b Cleaning up TextToLatent arg testing 2023-05-26 21:44:00 -04:00
user1
e2a94be336 Added resizing of controlnet image based on noise latent. Fixes a tensor mismatch issue. 2023-05-26 21:44:00 -04:00
user1
63a86eefb4 Refactored controlnet nodes: split out controlnet stuff into separate node, stripped controlnet stuff form image processing/analysis nodes. 2023-05-26 21:44:00 -04:00
user1
b0727b9d47 Prep for splitting pre-processor and controlnet nodes 2023-05-26 21:44:00 -04:00
user1
d96e727dd5 Added more preprocessor nodes for:
MidasDepth
      ZoeDepth
      MLSD
      NormalBae
      Pidi
      LineartAnime
      ContentShuffle
Removed pil_output options, ControlNet preprocessors should always output as PIL. Removed diagnostics and other general cleanup.
2023-05-26 21:44:00 -04:00
user1
fe480886dc changes to base class for controlnet nodes 2023-05-26 21:44:00 -04:00
user1
8031d1827b Refactored controlnet node to output ControlField that bundles control info. 2023-05-26 21:44:00 -04:00
user1
b5acdb322d Switching to ControlField for output from controlnet nodes. 2023-05-26 21:44:00 -04:00
user1
a4d1fe8819 Initial port of controlnet node support from generator-based TextToImageInvocation node to latent-based TextToLatentsInvocation node 2023-05-26 21:44:00 -04:00
user1
10b7a58887 Added first controlnet preprocessor node for canny edge detection. 2023-05-26 21:44:00 -04:00
user1
aaa093bef1 Fixed use of ControlNet control_weight parameter 2023-05-26 21:44:00 -04:00
user1
bb96543d66 Added support for using multiple control nets. Unfortunately this breaks direct usage of Control node output port ==> TextToLatent control input port -- passing through a Collect node is now required. Working on fixing this... 2023-05-26 21:44:00 -04:00
user1
a2a2cfa765 Added resizing of controlnet image based on noise latent. Fixes a tensor mismatch issue. 2023-05-26 21:44:00 -04:00
user1
18e6a2b410 Refactored controlnet nodes: split out controlnet stuff into separate node, stripped controlnet stuff form image processing/analysis nodes. 2023-05-26 21:44:00 -04:00
user1
db27263bc2 Prep for splitting pre-processor and controlnet nodes 2023-05-26 21:44:00 -04:00
user1
0e027ec3ef Added more preprocessor nodes for:
MidasDepth
      ZoeDepth
      MLSD
      NormalBae
      Pidi
      LineartAnime
      ContentShuffle
Removed pil_output options, ControlNet preprocessors should always output as PIL. Removed diagnostics and other general cleanup.
2023-05-26 21:44:00 -04:00
user1
5acbbeecaa Added HED, LineArt, and OpenPose ControlNet nodes 2023-05-26 21:44:00 -04:00
user1
6ef2168b67 changes to base class for controlnet nodes 2023-05-26 21:44:00 -04:00
user1
6d958a214c Refactored ControlNet nodes so they subclass from PreprocessedControlInvocation, and only need to override run_processor(image) (instead of reimplementing invoke()) 2023-05-26 21:44:00 -04:00
user1
4ae4bf4ff9 Resolving conflicts in rebase to origin/main 2023-05-26 21:44:00 -04:00
user1
fdef53b2de Switching to ControlField for output from controlnet nodes. 2023-05-26 21:44:00 -04:00
user1
11bd038b9d Added first controlnet preprocessor node for canny edge detection. 2023-05-26 21:44:00 -04:00
user1
c4277b0662 Moved to controlnet_aux v0.0.4, reinstated Zoe controlnet preprocessor. Also in pyproject.toml had to specify downgrade of timm to 0.6.13 _after_ controlnet-aux installs timm >= 0.9.2, because timm >0.6.13 breaks Zoe preprocessor. 2023-05-26 21:44:00 -04:00
psychedelicious
020f3ccf07 fix(nodes): controlnet input accepts list or single controlnet 2023-05-26 21:44:00 -04:00
psychedelicious
7467fa5e57 fix(ui): fix node ui type hints 2023-05-26 21:44:00 -04:00
user1
dcebd71381 Extended node-based ControlNet support to LatentsToLatentsInvocation. 2023-05-26 21:44:00 -04:00
user1
d855a65e73 Cleaning up after ControlNet refactor in TextToLatentsInvocation 2023-05-26 21:44:00 -04:00
user1
a9007c7e0f Refactored most of controlnet code into its own method to declutter TextToLatents.invoke(), and make upcoming integration with LatentsToLatents easier. 2023-05-26 21:44:00 -04:00
user1
af60304f97 Fix to work with current stable release of controlnet_aux (v0.0.3). Turned of pre-processor params that were added post v0.0.3. Also change defaults for shuffle. 2023-05-26 21:44:00 -04:00
user1
6de241eead Switched CotrolNet node modelname input from free text to default list of popular ControlNet model names. 2023-05-26 21:44:00 -04:00
user1
51032dc0b2 Commented out ZoeDetector. Will re-instate once there's a controlnet-aux release that supports it. 2023-05-26 21:44:00 -04:00
user1
297931f5d9 Cleaning up prior to submitting ControlNet PR. Mostly turning off diagnostic printing. Also fixed error when there is no controlnet input. 2023-05-26 21:44:00 -04:00
user1
f613c073c1 Added support for specifying which step iteration to start using
each ControlNet, and which step to end using each controlnet (specified as fraction of total steps)
2023-05-26 21:44:00 -04:00
user1
63d248622c Refactored ControNet support to consolidate multiple parameters into data struct. Also redid how multiple controlnets are handled. 2023-05-26 21:44:00 -04:00
user1
48485fe92f Removed last bits of dtype and and device hardwiring from controlnet section 2023-05-26 21:44:00 -04:00
user1
07726af703 Cleaning up mistakes after rebase. 2023-05-26 21:44:00 -04:00
user1
ad1004b485 Cleaning up TextToLatent arg testing 2023-05-26 21:44:00 -04:00
user1
0096fb2790 Added resizing of controlnet image based on noise latent. Fixes a tensor mismatch issue. 2023-05-26 21:44:00 -04:00
user1
9c8c2e49d6 Refactored controlnet nodes: split out controlnet stuff into separate node, stripped controlnet stuff form image processing/analysis nodes. 2023-05-26 21:44:00 -04:00
user1
2005a96847 Prep for splitting pre-processor and controlnet nodes 2023-05-26 21:44:00 -04:00
user1
00a8d60c1b Added more preprocessor nodes for:
MidasDepth
      ZoeDepth
      MLSD
      NormalBae
      Pidi
      LineartAnime
      ContentShuffle
Removed pil_output options, ControlNet preprocessors should always output as PIL. Removed diagnostics and other general cleanup.
2023-05-26 21:44:00 -04:00
user1
3aa182390a changes to base class for controlnet nodes 2023-05-26 21:44:00 -04:00
user1
e44f1d6d4e Refactored controlnet node to output ControlField that bundles control info. 2023-05-26 21:44:00 -04:00
user1
dfdf8e2ead Switching to ControlField for output from controlnet nodes. 2023-05-26 21:44:00 -04:00
user1
3a645c4e80 Initial port of controlnet node support from generator-based TextToImageInvocation node to latent-based TextToLatentsInvocation node 2023-05-26 21:44:00 -04:00
user1
113129daf9 Added first controlnet preprocessor node for canny edge detection. 2023-05-26 21:44:00 -04:00
user1
714ad6dbb8 Fixed use of ControlNet control_weight parameter 2023-05-26 21:44:00 -04:00
user1
c0863fa20f Added support for using multiple control nets. Unfortunately this breaks direct usage of Control node output port ==> TextToLatent control input port -- passing through a Collect node is now required. Working on fixing this... 2023-05-26 21:44:00 -04:00
user1
5d5cdc7716 Added resizing of controlnet image based on noise latent. Fixes a tensor mismatch issue. 2023-05-26 21:44:00 -04:00
user1
93cd818f6a Refactored controlnet nodes: split out controlnet stuff into separate node, stripped controlnet stuff form image processing/analysis nodes. 2023-05-26 21:44:00 -04:00
user1
598a628790 Prep for splitting pre-processor and controlnet nodes 2023-05-26 21:44:00 -04:00
user1
f3666eda63 Added more preprocessor nodes for:
MidasDepth
      ZoeDepth
      MLSD
      NormalBae
      Pidi
      LineartAnime
      ContentShuffle
Removed pil_output options, ControlNet preprocessors should always output as PIL. Removed diagnostics and other general cleanup.
2023-05-26 21:44:00 -04:00
user1
754017b59e Added an additional "raw_processed_image" output port to controlnets, mainly so could route ImageField to a ShowImage node 2023-05-26 21:44:00 -04:00
user1
21251ce12c Added HED, LineArt, and OpenPose ControlNet nodes 2023-05-26 21:44:00 -04:00
user1
dc12fa6cd6 changes to base class for controlnet nodes 2023-05-26 21:44:00 -04:00
user1
f2f4c37f19 Refactored ControlNet nodes so they subclass from PreprocessedControlInvocation, and only need to override run_processor(image) (instead of reimplementing invoke()) 2023-05-26 21:44:00 -04:00
user1
0864fca641 Resolving conflicts in rebase to origin/main 2023-05-26 21:44:00 -04:00
user1
5e4c0217c7 Switching to ControlField for output from controlnet nodes. 2023-05-26 21:44:00 -04:00
user1
78cd106c23 Initial port of controlnet node support from generator-based TextToImageInvocation node to latent-based TextToLatentsInvocation node 2023-05-26 21:44:00 -04:00
user1
6ed0efa938 Added first controlnet preprocessor node for canny edge detection. 2023-05-26 21:44:00 -04:00
user1
ca0669c337 Resolving rebase conflict 2023-05-26 21:44:00 -04:00
Lincoln Stein
1e2db3a17f hook tiled_decode up to configuration 2023-05-25 23:28:15 -04:00
Lincoln Stein
5f8f51436a merge with main; fix conflicts 2023-05-25 22:40:45 -04:00
psychedelicious
33e13820fc feat(nodes): remove meta node field; use individual is_intermediate field instead
as suggested by @Kyle0654
2023-05-25 22:17:14 -04:00
psychedelicious
291e9cf14b fix(nodes): add is_intermediate to all image-outputting nodes 2023-05-25 22:17:14 -04:00
psychedelicious
d2c8a53c55 feat(nodes): change intermediates handling
- `ImageType` is now restricted to `results` and `uploads`.
- Add a reserved `meta` field to nodes to hold the `is_intermediate` boolean. We can extend it in the future to support other node `meta`.
- Add a `is_intermediate` column to the `images` table to hold this. (When `latents`, `conditioning` etc are added to the DB, they will also have this column.)
- All nodes default to `*not* intermediate`. Nodes must explicitly be marked `intermediate` for their outputs to be `intermediate`.
- When building a graph, you can set `node.meta.is_intermediate=True` and it will be handled as an intermediate.
- Add a new `update()` method to the `ImageService`, and a route to call it. Updates have a strict model, currently only `session_id` and `image_category` may be updated.
- Add a new `update()` method to the `ImageRecordStorageService` to update the image record using the model.
2023-05-25 22:17:14 -04:00
Kent Keirsey
05fb0ac2b2 Update latent.py 2023-05-26 10:27:33 +10:00
Kent Keirsey
d4acd49ee3 Update generate.py 2023-05-26 10:27:33 +10:00
psychedelicious
1fb307abf4 feat(nodes): restore canvas functionality (non-latents) 2023-05-24 11:30:47 -04:00
psychedelicious
ad39680feb feat(nodes): wip inpainting nodes prep 2023-05-24 11:30:47 -04:00
psychedelicious
1e0ae8404c feat(nodes): comment out seamless
this will be a model config feature when model manager is ready
2023-05-24 11:30:47 -04:00
psychedelicious
460d555a3d feat(nodes): add image mul, channel, convert nodes
also make img node names consistent
2023-05-24 11:30:47 -04:00
psychedelicious
d2c223de8f feat(nodes): move fully* to new images service
* except i haven't rebuilt inpaint in latents
2023-05-24 11:30:47 -04:00
psychedelicious
dd16f788ed fix(nodes): fix RangeOfSizeInvocation off-by-one error 2023-05-24 11:30:47 -04:00
psychedelicious
b25c1af018 feat(nodes): add RangeOfSizeInvocation
The `RangeInvocation` is a simple wrapper around `range()`, but you must provide `stop > start`.

`RangeOfSizeInvocation` replaces the `stop` parameter with `size`, so that you can just provide the `start` and `step` and get a range of `size` length.
2023-05-24 11:30:47 -04:00
psychedelicious
8f393b64b8 feat(nodes): add seed validator
If `seed>SEED_MAX`, we can still continue if we parse the seed as `seed % SEED_MAX`.
2023-05-24 11:30:47 -04:00
psychedelicious
55b3193629 fix(nodes): add RangeInvocation validator
`stop` must be greater than `start`.
2023-05-24 11:30:47 -04:00
psychedelicious
035425ef24 feat(nodes): address feedback
- Address database feedback:
  - Remove all the extraneous tables. Only an `images` table now:
  - `image_type` and `image_category` are unrestricted strings. When creating images, the provided values are checked to ensure they are a valid type and category.
  - Add `updated_at` and `deleted_at` columns. `deleted_at` is currently unused.
  - Use SQLite's built-in timestamp features to populate these. Add a trigger to update `updated_at` when the row is updated. Currently no way to update a row.
  - Rename the `id` column in `images` to `image_name`
- Rename `ImageCategory.IMAGE` to `ImageCategory.GENERAL`
- Move all exceptions outside their base classes to make them more portable.
- Add `width` and `height` columns to the database. These store the actual dimensions of the image file, whereas the metadata's `width` and `height` refer to the respective generation parameters and are nullable.
- Make `deserialize_image_record` take a `dict` instead of `sqlite3.Row`
- Improve comments throughout
- Tidy up unused code/files and some minor organisation
2023-05-24 11:30:47 -04:00
psychedelicious
5de3c41d19 feat(nodes): add metadata handling 2023-05-24 11:30:47 -04:00
psychedelicious
adde8450bc fix(nodes): remove bad import 2023-05-24 11:30:47 -04:00
psychedelicious
5bf9891553 feat(nodes): it works 2023-05-24 11:30:47 -04:00
psychedelicious
d4aa79acd7 fix(nodes): use save instead of set
`set` is a python builtin
2023-05-24 11:30:47 -04:00
psychedelicious
9c89d3452c feat(nodes): add high-level images service
feat(nodes): add ResultsServiceABC & SqliteResultsService

**Doesn't actually work bc of circular imports. Can't even test it.**

- add a base class for ResultsService and SQLite implementation
- use `graph_execution_manager` `on_changed` callback to keep `results` table in sync

fix(nodes): fix results service bugs

chore(ui): regen api

fix(ui): fix type guards

feat(nodes): add `result_type` to results table, fix types

fix(nodes): do not shadow `list` builtin

feat(nodes): add results router

It doesn't work due to circular imports still

fix(nodes): Result class should use outputs classes, not fields

feat(ui): crude results router

fix(ui): send to canvas in currentimagebuttons not working

feat(nodes): add core metadata builder

feat(nodes): add design doc

feat(nodes): wip latents db stuff

feat(nodes): images_db_service and resources router

feat(nodes): wip images db & router

feat(nodes): update image related names

feat(nodes): update urlservice

feat(nodes): add high-level images service
2023-05-24 11:30:47 -04:00
Lincoln Stein
bdf33f13b3 fix bad merge in compel 2023-05-18 18:08:45 -04:00
Lincoln Stein
d96175d127 resolve some undefined symbols in model_cache 2023-05-18 14:31:47 -04:00
Lincoln Stein
8adff96e29
Merge branch 'main' into lstein/global-configuration 2023-05-17 14:37:09 -04:00
psychedelicious
1d9c115225 feat(nodes): add low and high to RandomIntInvocation 2023-05-16 13:50:52 +10:00
Lincoln Stein
426f4eaf7e adjusted regression tests to work with new SDModelTypes 2023-05-13 22:29:33 -04:00
Lincoln Stein
baf5451fa0
Merge branch 'main' into lstein/new-model-manager 2023-05-13 22:01:34 -04:00
Lincoln Stein
1103ab2844 merge with main 2023-05-13 21:35:19 -04:00
Sergey Borisov
039fa73269 Change SDModelType enum to string, fixes(model unload negative locks count, scheduler load error, saftensors convert, wrong logic in del_model, wrong parse metadata in web) 2023-05-14 03:06:26 +03:00
Lincoln Stein
72967bf118 convert add_model(), del_model(), list_models() etc to use bifurcated names 2023-05-13 14:44:44 -04:00
Sergey Borisov
bc96727cbe Rewrite latent nodes to new model manager 2023-05-13 16:08:03 +03:00
Sergey Borisov
3b2a054f7a Add model loader node; unet, clip, vae fields; change compel node to clip field 2023-05-13 04:37:20 +03:00
Sergey Borisov
4492044d29 Redo compel node to separate model loading 2023-05-12 23:09:33 +03:00
Eugene Brodsky
af060188bd
Merge branch 'main' into lstein/bugfix/compel 2023-05-12 08:22:18 -04:00
Lincoln Stein
2ef79b8bf3 fix bug in persistent model scheme 2023-05-12 00:14:56 -04:00
blessedcoolant
9c1e65f3a3
Merge branch 'main' into feat/nodes/add-randomintinvocation 2023-05-12 15:56:41 +12:00
psychedelicious
ebec200ba6
Remove unused import 2023-05-12 13:56:02 +10:00
Lincoln Stein
11ecf438f5 latents.py converted to use model manager service; events emitted 2023-05-11 23:33:24 -04:00
blessedcoolant
0acb8ed85d
Merge branch 'main' into feat/nodes/add-w-h-latentsoutput 2023-05-12 15:23:29 +12:00
blessedcoolant
8c1c9cd702
Merge branch 'main' into feat/nodes/add-randomintinvocation 2023-05-12 15:21:49 +12:00
blessedcoolant
af95cef7f9
Merge branch 'main' into fix/nodes/fix-imageoutput-optionals 2023-05-12 15:08:19 +12:00
Lincoln Stein
df5b968954 model manager now running as a service 2023-05-11 21:24:29 -04:00
Sergey Borisov
27dc07d95a Set zero eta by default(fix ddim scheduler error) 2023-05-11 18:49:27 +03:00
blessedcoolant
f7dc171c4f Rename default schedulers across the app 2023-05-12 03:44:20 +12:00
blessedcoolant
8a836247c8 Add DPMPP Single, Euler Karras and DPMPP2 Multi Karras Schedulers 2023-05-12 02:23:33 +12:00
blessedcoolant
9a383e456d Codesplit SCHEDULER_MAP for reusage 2023-05-12 00:40:03 +12:00
blessedcoolant
c1e7460d39
Merge branch 'main' into unipc-sched 2023-05-12 00:11:09 +12:00
blessedcoolant
3ffff023b2 Add missing key to scheduler_map
It was breaking coz the sampler was not being reset. So needs a key on each. Will simplify this later.
2023-05-12 00:08:50 +12:00
psychedelicious
6cf308004a fix(nodes): remove Optionals on ImageOutputs 2023-05-11 20:54:57 +10:00
blessedcoolant
d1029138d2 Default to DDIM if scheduler is missing 2023-05-11 22:54:35 +12:00
blessedcoolant
06b5800d28 Add UniPC Scheduler 2023-05-11 22:43:18 +12:00
psychedelicious
483f2ccb56 feat(nodes): add RandomIntInvocation
just outputs a single random int
2023-05-11 20:33:32 +10:00
psychedelicious
93ced0bec6 feat(nodes): add w/h to latents outputs
This reduces the number of nodes needed when working with latents (ie fewer plain integer value nodes)

Also correct a few mistakes in the fields
2023-05-11 20:32:55 +10:00
psychedelicious
4333852c37 fix(nodes): fix missing context arg in LatentsToLatents 2023-05-11 19:28:42 +10:00
Eugene Brodsky
3baa230077
Merge branch 'main' into lstein/bugfix/compel 2023-05-11 00:50:45 -04:00
Lincoln Stein
8ad8c5c67a resolve conflicts with main 2023-05-11 00:19:20 -04:00
Lincoln Stein
4627910c5d added a wrapper model_manager_service and model events 2023-05-11 00:09:19 -04:00
psychedelicious
f488b1a7f2 fix(nodes): fix usage of Optional 2023-05-11 11:55:51 +10:00
psychedelicious
34f3a0f0e3 feat(nodes): improve default model choosing output 2023-05-11 11:55:51 +10:00
psychedelicious
d0bac1675e fix(nodes): fix ImageOutput Config 2023-05-11 11:55:51 +10:00
psychedelicious
4e56c962f4 fix(nodes): fix infill docstrings 2023-05-11 11:55:51 +10:00
psychedelicious
4ef0e43759 fix(nodes): remove dataURL invocation 2023-05-11 11:55:51 +10:00
psychedelicious
a7786d5ff2 fix(nodes): restore seamless to TextToLatents 2023-05-11 11:55:51 +10:00
psychedelicious
6c1de975d9 feat(nodes): add infill nodes 2023-05-11 11:55:51 +10:00
psychedelicious
a1079e455a feat(nodes): cleanup unused params, seed generation 2023-05-11 11:55:51 +10:00
psychedelicious
da4eacdffe feat(nodes): add InfillInvocation 2023-05-11 11:55:51 +10:00
psychedelicious
6102e560ba feat(nodes): add LatentsToImage node (VAE encode) 2023-05-11 11:55:51 +10:00
psychedelicious
206e6b1730 feat(nodes): wip inpaint node 2023-05-11 11:55:51 +10:00
psychedelicious
357cee2849 fix(nodes): fix cfg scale min value 2023-05-11 11:55:51 +10:00
psychedelicious
e0b9b5cc6c feat(nodes): add dataURL to image node 2023-05-11 11:55:51 +10:00
Lincoln Stein
aca4770481 fixed compel.py as requested 2023-05-10 21:40:44 -04:00
Lincoln Stein
fa6a580452 merge with main 2023-05-10 00:03:32 -04:00
Lincoln Stein
4649920074 adjust t2i to work with new model structure 2023-05-07 19:06:49 -04:00
Lincoln Stein
42d938fda5 remove debugging statement 2023-05-06 23:54:11 -04:00
Lincoln Stein
25ce47c44f remove reference to globals in compel.py 2023-05-06 22:49:35 -04:00
Lincoln Stein
a8cfa3565c Merge branch 'lstein/new-model-manager' of github.com:invoke-ai/InvokeAI into lstein/new-model-manager 2023-05-06 08:14:15 -04:00
Lincoln Stein
e0214a32bc mostly ported to new manager API; needs testing 2023-05-06 00:44:12 -04:00
StAlKeR7779
a80fe05e23 Rename compel node 2023-05-05 21:30:16 +03:00
StAlKeR7779
58d7833c5c Review changes 2023-05-05 21:09:29 +03:00
StAlKeR7779
5012f61599 Separate conditionings back to positive and negative 2023-05-05 15:47:51 +03:00
blessedcoolant
85c33823c3
Merge branch 'main' into feat/compel_node 2023-05-05 14:41:45 +12:00
psychedelicious
e04ada1319
Merge branch 'main' into patch-1 2023-05-05 10:38:45 +10:00
StAlKeR7779
81ec476f3a Revert seed field addition 2023-05-04 21:50:40 +03:00
StAlKeR7779
7d221e2518 Combine conditioning to one field(better fits for multiple type conditioning like perp-neg) 2023-05-04 20:14:22 +03:00
StAlKeR7779
56d3cbead0
Merge branch 'main' into feat/compel_node 2023-05-04 00:28:33 +03:00
Lincoln Stein
4687ad4ed6
Merge branch 'main' into enhance/invokeai-logs 2023-05-03 13:36:06 -04:00
psychedelicious
3264d30b44 feat(nodes): allow multiples of 8 for dimensions 2023-05-02 12:01:52 +10:00
psychedelicious
b49d76ebee feat(nodes): fix image to image fit param
it was ignored previously.
2023-05-01 17:28:22 +10:00
Lincoln Stein
974841926d logger is a interchangeable service 2023-04-29 10:48:50 -04:00
Lincoln Stein
8db20e0d95 rename log to logger throughout 2023-04-29 09:43:40 -04:00
Lincoln Stein
6b79e2b407 Merge branch 'main' into enhance/invokeai-logs
- resolve conflicts
- remove unused code identified by pyflakes
2023-04-28 10:09:46 -04:00
psychedelicious
4a924c9b54 feat(nodes): hardcode resize latents downsampling 2023-04-28 09:52:09 +10:00
psychedelicious
0453d60c64 fix(nodes): fix slatents and rlatents bugs 2023-04-28 09:52:09 +10:00
psychedelicious
c4f4f8b1b8 fix(nodes): remove unused width and height from t2l 2023-04-28 09:52:09 +10:00
psychedelicious
3e80eaa342 feat(nodes): add resize and scale latents nodes
- this resize/scale latents is what is needed for hires fix
- also remove unused `seed` from t2l
2023-04-28 09:52:09 +10:00
Andy Luhrs
0f95f7cea3
Fix inpaint node
Seems like this is the only change needed for the existing inpaint node to work.
2023-04-27 11:03:07 -07:00
StAlKeR7779
37916a22ad Use textual inversion manager from pipeline, remove extra conditioning info for uc 2023-04-25 12:53:13 +03:00
StAlKeR7779
8cb2fa8600 Restore log_tokenization check 2023-04-25 04:29:17 +03:00
StAlKeR7779
8f460b92f1 Make latent generation nodes use conditions instead of prompt 2023-04-25 04:21:03 +03:00
StAlKeR7779
d99a08a441 Add compel node and conditioning field type 2023-04-25 03:48:44 +03:00
psychedelicious
5f498e10bd
Partial migration of UI to nodes API (#3195)
* feat(ui): add axios client generator and simple example

* fix(ui): update client & nodes test code w/ new Edge type

* chore(ui): organize generated files

* chore(ui): update .eslintignore, .prettierignore

* chore(ui): update openapi.json

* feat(backend): fixes for nodes/generator

* feat(ui): generate object args for api client

* feat(ui): more nodes api prototyping

* feat(ui): nodes cancel

* chore(ui): regenerate api client

* fix(ui): disable OG web server socket connection

* fix(ui): fix scrollbar styles typing and prop

just noticed the typo, and made the types stronger.

* feat(ui): add socketio types

* feat(ui): wip nodes

- extract api client method arg types instead of manually declaring them
- update example to display images
- general tidy up

* start building out node translations from frontend state and add notes about missing features

* use reference to sampler_name

* use reference to sampler_name

* add optional apiUrl prop

* feat(ui): start hooking up dynamic txt2img node generation, create middleware for session invocation

* feat(ui): write separate nodes socket layer, txt2img generating and rendering w single node

* feat(ui): img2img implementation

* feat(ui): get intermediate images working but types are stubbed out

* chore(ui): add support for package mode

* feat(ui): add nodes mode script

* feat(ui): handle random seeds

* fix(ui): fix middleware types

* feat(ui): add rtk action type guard

* feat(ui): disable NodeAPITest

This was polluting the network/socket logs.

* feat(ui): fix parameters panel border color

This commit should be elsewhere but I don't want to break my flow

* feat(ui): make thunk types more consistent

* feat(ui): add type guards for outputs

* feat(ui): load images on socket connect

Rudimentary

* chore(ui): bump redux-toolkit

* docs(ui): update readme

* chore(ui): regenerate api client

* chore(ui): add typescript as dev dependency

I am having trouble with TS versions after vscode updated and now uses TS 5. `madge` has installed 3.9.10 and for whatever reason my vscode wants to use that. Manually specifying 4.9.5 and then setting vscode to use that as the workspace TS fixes the issue.

* feat(ui): begin migrating gallery to nodes

Along the way, migrate to use RTK `createEntityAdapter` for gallery images, and separate `results` and `uploads` into separate slices. Much cleaner this way.

* feat(ui): clean up & comment results slice

* fix(ui): separate thunk for initial gallery load so it properly gets index 0

* feat(ui): POST upload working

* fix(ui): restore removed type

* feat(ui): patch api generation for headers access

* chore(ui): regenerate api

* feat(ui): wip gallery migration

* feat(ui): wip gallery migration

* chore(ui): regenerate api

* feat(ui): wip refactor socket events

* feat(ui): disable panels based on app props

* feat(ui): invert logic to be disabled

* disable panels when app mounts

* feat(ui): add support to disableTabs

* docs(ui): organise and update docs

* lang(ui): add toast strings

* feat(ui): wip events, comments, and general refactoring

* feat(ui): add optional token for auth

* feat(ui): export StatusIndicator and ModelSelect for header use

* feat(ui) working on making socket URL dynamic

* feat(ui): dynamic middleware loading

* feat(ui): prep for socket jwt

* feat(ui): migrate cancelation

also updated action names to be event-like instead of declaration-like

sorry, i was scattered and this commit has a lot of unrelated stuff in it.

* fix(ui): fix img2img type

* chore(ui): regenerate api client

* feat(ui): improve InvocationCompleteEvent types

* feat(ui): increase StatusIndicator font size

* fix(ui): fix middleware order for multi-node graphs

* feat(ui): add exampleGraphs object w/ iterations example

* feat(ui): generate iterations graph

* feat(ui): update ModelSelect for nodes API

* feat(ui): add hi-res functionality for txt2img generations

* feat(ui): "subscribe" to particular nodes

feels like a dirty hack but oh well it works

* feat(ui): first steps to node editor ui

* fix(ui): disable event subscription

it is not fully baked just yet

* feat(ui): wip node editor

* feat(ui): remove extraneous field types

* feat(ui): nodes before deleting stuff

* feat(ui): cleanup nodes ui stuff

* feat(ui): hook up nodes to redux

* fix(ui): fix handle

* fix(ui): add basic node edges & connection validation

* feat(ui): add connection validation styling

* feat(ui): increase edge width

* feat(ui): it blends

* feat(ui): wip model handling and graph topology validation

* feat(ui): validation connections w/ graphlib

* docs(ui): update nodes doc

* feat(ui): wip node editor

* chore(ui): rebuild api, update types

* add redux-dynamic-middlewares as a dependency

* feat(ui): add url host transformation

* feat(ui): handle already-connected fields

* feat(ui): rewrite SqliteItemStore in sqlalchemy

* fix(ui): fix sqlalchemy dynamic model instantiation

* feat(ui, nodes): metadata wip

* feat(ui, nodes): models

* feat(ui, nodes): more metadata wip

* feat(ui): wip range/iterate

* fix(nodes): fix sqlite typing

* feat(ui): export new type for invoke component

* tests(nodes): fix test instantiation of ImageField

* feat(nodes): fix LoadImageInvocation

* feat(nodes): add `title` ui hint

* feat(nodes): make ImageField attrs optional

* feat(ui): wip nodes etc

* feat(nodes): roll back sqlalchemy

* fix(nodes): partially address feedback

* fix(backend): roll back changes to pngwriter

* feat(nodes): wip address metadata feedback

* feat(nodes): add seeded rng to RandomRange

* feat(nodes): address feedback

* feat(nodes): move GET images error handling to DiskImageStorage

* feat(nodes): move GET images error handling to DiskImageStorage

* fix(nodes): fix image output schema customization

* feat(ui): img2img/txt2img -> linear

- remove txt2img and img2img tabs
- add linear tab
- add initial image selection to linear parameters accordion

* feat(ui): tidy graph builders

* feat(ui): tidy misc

* feat(ui): improve invocation union types

* feat(ui): wip metadata viewer recall

* feat(ui): move fonts to normal deps

* feat(nodes): fix broken upload

* feat(nodes): add metadata module + tests, thumbnails

- `MetadataModule` is stateless and needed in places where the `InvocationContext` is not available, so have not made it a `service`
- Handles loading/parsing/building metadata, and creating png info objects
- added tests for MetadataModule
- Lifted thumbnail stuff to util

* fix(nodes): revert change to RandomRangeInvocation

* feat(nodes): address feedback

- make metadata a service
- rip out pydantic validation, implement metadata parsing as simple functions
- update tests
- address other minor feedback items

* fix(nodes): fix other tests

* fix(nodes): add metadata service to cli

* fix(nodes): fix latents/image field parsing

* feat(nodes): customise LatentsField schema

* feat(nodes): move metadata parsing to frontend

* fix(nodes): fix metadata test

---------

Co-authored-by: maryhipp <maryhipp@gmail.com>
Co-authored-by: Mary Hipp <maryhipp@Marys-MacBook-Air.local>
2023-04-22 13:10:20 +10:00
Lincoln Stein
b164330e3c replaced remaining print statements with log.*() 2023-04-18 20:49:00 -04:00
Eugene
3daaddf15b nodes: remove duplicate LatentsToLatentsInvocation 2023-04-16 09:39:33 -04:00
Kyle Schouviller
23d65e7162
[nodes] Add subgraph library, subgraph usage in CLI, and fix subgraph execution (#3180)
* Add latent to latent (img2img equivalent)
Fix a CLI bug with multiple links per node

* Using "latents" instead of "latent"

* [nodes] In-progress implementation of graph library

* Add linking to CLI for graph nodes (still broken)

* Fix subgraph execution, fix subgraph linking in CLI

* Fix LatentsToLatents
2023-04-14 06:41:06 +00:00
psychedelicious
d923d1d66b fix(nodes): fix naming of CvInvocationConfig 2023-04-11 12:13:53 +10:00
psychedelicious
1f2c1e14db fix(nodes): move InvocationConfig to baseinvocation.py 2023-04-11 12:13:53 +10:00
psychedelicious
07e3a0ec15 feat(nodes): add invocation schema customisation, add model selection
- add invocation schema customisation

done via fastapi's `Config` class and `schema_extra`. when using `Config`, inherit from `InvocationConfig` to get type hints.

where it makes sense - like for all math invocations - define a `MathInvocationConfig` class and have all invocations inherit from it.

this customisation can provide any arbitrary additional data to the UI. currently it provides tags and field type hints.

this is necessary for `model` type fields, which are actually string fields. without something like this, we can't reliably differentiate  `model` fields from normal `string` fields.

can also be used for future field types.

all invocations now have tags, and all `model` fields have ui type hints.

- fix model handling for invocations

added a helper to fall back to the default model if an invalid model name is chosen. model names in graphs now work.

- fix latents progress callback

noticed this wasn't correct while working on everything else.
2023-04-11 12:13:53 +10:00
psychedelicious
427db7c7e2 feat(nodes): fix typo in PasteImageInvocation 2023-04-10 21:33:08 +10:00
psychedelicious
dad3a7f263 fix(nodes): sampler_name --> scheduler
the name of this was changed at some point. nodes still used the old name, so scheduler selection did nothing. simple fix.
2023-04-10 19:54:09 +10:00
psychedelicious
34402cc46a feat(nodes): add list_images endpoint
- add `list_images` endpoint at `GET api/v1/images`
- extend `ImageStorageBase` with `list()` method, implemented it for `DiskImageStorage`
- add `ImageReponse` class to for image responses, which includes urls, metadata
- add `ImageMetadata` class (basically a stub at the moment)
- uploaded images now named `"{uuid}_{timestamp}.png"`
- add `models` modules. besides separating concerns more clearly, this helps to mitigate circular dependencies
- improve thumbnail handling
2023-04-09 13:48:44 +10:00
psychedelicious
111d674e71 fix(nodes): use correct torch device in NoiseInvocation 2023-04-08 12:32:03 +10:00
Kyle Schouviller
85b020f76c
[nodes] Add latent nodes, storage, and fix iteration bugs (#3091)
* Add latents nodes.
* Fix iteration expansion.
* Add collection generator nodes, math nodes.
* Add noise node.
* Add some graph debug commands to the CLI.
* Fix negative id linking in CLI.
* Fix a CLI bug with multiple links per node.
2023-04-06 04:06:05 +00:00
psychedelicious
4221cf7731 fix(nodes): fix schema generation for output classes
All output classes need to have their properties flagged as `required` for the schema generation to work as needed.
2023-03-26 17:20:10 +11:00
psychedelicious
c34ac91ff0 fix(nodes): fix cancel; fix callback for img2img, inpaint 2023-03-26 17:07:40 +11:00
Lincoln Stein
dac3c158a5 Merge branch 'main' into feat/preview_predicted_x0
- resolve conflicts with generate.py invocation
- remove unused symbols that pyflakes complains about
- add **untested** code for passing intermediate latent image to the
  step callback in the format expected.
2023-03-25 16:07:18 -04:00
psychedelicious
67f8f222d9 fix(nodes): fix step_callback + fast latents generation
this depends on the small change in #2957
2023-03-16 20:03:08 +11:00
Kyle Schouviller
0c3b1fe3c4 [nodes] Fixes calls into image to image and inpaint from nodes 2023-03-12 22:12:42 -07:00
Kevin Turner
288cee9611 Merge remote-tracking branch 'origin/main' into feat/preview_predicted_x0
# Conflicts:
#	invokeai/app/invocations/generate.py
2023-03-12 20:56:02 -07:00
Lincoln Stein
8ca91b1774 add restoration services to nodes 2023-03-11 17:00:00 -05:00
Lincoln Stein
d612f11c11 initialize InvokeAIGenerator object with model, not manager 2023-03-11 09:06:46 -05:00