Change tiling strategy to make TiledStableDiffusionRefineInvocation work with more tile shapes and overlaps.

- Fix type errors
- Enable SilenceWarnings to be used as both a context manager and a decorator
- Remove duplicate implementation
- Check the initial verbosity on __enter__() rather than __init__()

Makefile

@@ -18,6 +18,7 @@ help:
 	@echo "frontend-typegen         Generate types for the frontend from the OpenAPI schema"
 	@echo "installer-zip            Build the installer .zip file for the current version"
 	@echo "tag-release              Tag the GitHub repository with the current version (use at release time only!)"
+	@echo "openapi                  Generate the OpenAPI schema for the app, outputting to stdout"
 
 # Runs ruff, fixing any safely-fixable errors and formatting
 ruff:
@@ -70,3 +71,6 @@ installer-zip:
 tag-release:
 	cd installer && ./tag_release.sh
 
+# Generate the OpenAPI Schema for the app
+openapi:
+	python scripts/generate_openapi_schema.py

README.md

@@ -12,7 +12,7 @@
 
 Invoke is a leading creative engine built to empower professionals and enthusiasts alike. Generate and create stunning visual media using the latest AI-driven technologies. Invoke offers an industry leading web-based UI, and serves as the foundation for multiple commercial products.
 
-[Installation][installation docs] - [Documentation and Tutorials][docs home] - [Bug Reports][github issues] - [Contributing][contributing docs]
+[Installation and Updates][installation docs] - [Documentation and Tutorials][docs home] - [Bug Reports][github issues] - [Contributing][contributing docs]
 
 <div align="center">
 

docker/README.md

@@ -64,7 +64,7 @@ GPU_DRIVER=nvidia
 
 Any environment variables supported by InvokeAI can be set here - please see the [Configuration docs](https://invoke-ai.github.io/InvokeAI/features/CONFIGURATION/) for further detail.
 
-## Even Moar Customizing!
+## Even More Customizing!
 
 See the `docker-compose.yml` file. The `command` instruction can be uncommented and used to run arbitrary startup commands. Some examples below.
 

docs/contributing/frontend/WORKFLOWS.md

@@ -117,13 +117,13 @@ Stateless fields do not store their value in the node, so their field instances
 
 "Custom" fields will always be treated as stateless fields.
 
-##### Collection and Scalar Fields
+##### Single and Collection Fields
 
-Field types have a name and two flags which may identify it as a **collection** or **collection or scalar** field.
+Field types have a name and cardinality property which may identify it as a **SINGLE**, **COLLECTION** or **SINGLE_OR_COLLECTION** field.
 
-If a field is annotated in python as a list, its field type is parsed and flagged as a **collection** type (e.g. `list[int]`).
-
-If it is annotated as a union of a type and list, the type will be flagged as a **collection or scalar** type (e.g. `Union[int, list[int]]`). Fields may not be unions of different types (e.g. `Union[int, list[str]]` and `Union[int, str]` are not allowed).
+- If a field is annotated in python as a singular value or class, its field type is parsed as a **SINGLE** type (e.g. `int`, `ImageField`, `str`).
+- If a field is annotated in python as a list, its field type is parsed as a **COLLECTION** type (e.g. `list[int]`).
+- If it is annotated as a union of a type and list, the type will be parsed as a **SINGLE_OR_COLLECTION** type (e.g. `Union[int, list[int]]`). Fields may not be unions of different types (e.g. `Union[int, list[str]]` and `Union[int, str]` are not allowed).
 
 ## Implementation
 
@@ -173,8 +173,7 @@ Field types are represented as structured objects:
 ```ts
 type FieldType = {
   name: string;
-  isCollection: boolean;
-  isCollectionOrScalar: boolean;
+  cardinality: 'SINGLE' | 'COLLECTION' | 'SINGLE_OR_COLLECTION';
 };
 ```
 
@@ -186,7 +185,7 @@ There are 4 general cases for field type parsing.
 
 When a field is annotated as a primitive values (e.g. `int`, `str`, `float`), the field type parsing is fairly straightforward. The field is represented by a simple OpenAPI **schema object**, which has a `type` property.
 
-We create a field type name from this `type` string (e.g. `string` -> `StringField`).
+We create a field type name from this `type` string (e.g. `string` -> `StringField`). The cardinality is `"SINGLE"`.
 
 ##### Complex Types
 
@@ -200,13 +199,13 @@ We need to **dereference** the schema to pull these out. Dereferencing may requi
 
 When a field is annotated as a list of a single type, the schema object has an `items` property. They may be a schema object or reference object and must be parsed to determine the item type.
 
-We use the item type for field type name, adding `isCollection: true` to the field type.
+We use the item type for field type name. The cardinality is `"COLLECTION"`.
 
-##### Collection or Scalar Types
+##### Single or Collection Types
 
 When a field is annotated as a union of a type and list of that type, the schema object has an `anyOf` property, which holds a list of valid types for the union.
 
-After verifying that the union has two members (a type and list of the same type), we use the type for field type name, adding `isCollectionOrScalar: true` to the field type.
+After verifying that the union has two members (a type and list of the same type), we use the type for field type name, with cardinality `"SINGLE_OR_COLLECTION"`.
 
 ##### Optional Fields
 

docs/features/CONFIGURATION.md

@@ -51,13 +51,11 @@ The settings in this file will override the defaults. You only need
 to change this file if the default for a particular setting doesn't
 work for you.
 
+You'll find an example file next to `invokeai.yaml` that shows the default values.
+
 Some settings, like [Model Marketplace API Keys], require the YAML
 to be formatted correctly. Here is a [basic guide to YAML files].
 
-You can fix a broken `invokeai.yaml` by deleting it and running the
-configuration script again -- option [6] in the launcher, "Re-run the
-configure script".
-
 #### Custom Config File Location
 
 You can use any config file with the `--config` CLI arg. Pass in the path to the `invokeai.yaml` file you want to use.

docs/features/CONTROLNET.md

@@ -165,7 +165,7 @@ Additionally, each  section can be expanded with the "Show Advanced" button in o
 There are several ways to install IP-Adapter models with an existing InvokeAI installation:
 
 1. Through the command line interface launched from the invoke.sh / invoke.bat scripts, option [4] to download models.
-2. Through the Model Manager UI with models from the *Tools* section of [www.models.invoke.ai](https://www.models.invoke.ai). To do this, copy the repo ID from the desired model page, and paste it in the Add Model field of the model manager. **Note** Both the IP-Adapter and the Image Encoder must be installed for IP-Adapter to work. For example, the [SD 1.5 IP-Adapter](https://models.invoke.ai/InvokeAI/ip_adapter_plus_sd15) and [SD1.5 Image Encoder](https://models.invoke.ai/InvokeAI/ip_adapter_sd_image_encoder) must be installed to use IP-Adapter with SD1.5 based models.  
+2. Through the Model Manager UI with models from the *Tools* section of [models.invoke.ai](https://models.invoke.ai). To do this, copy the repo ID from the desired model page, and paste it in the Add Model field of the model manager. **Note** Both the IP-Adapter and the Image Encoder must be installed for IP-Adapter to work. For example, the [SD 1.5 IP-Adapter](https://models.invoke.ai/InvokeAI/ip_adapter_plus_sd15) and [SD1.5 Image Encoder](https://models.invoke.ai/InvokeAI/ip_adapter_sd_image_encoder) must be installed to use IP-Adapter with SD1.5 based models.  
 3. **Advanced -- Not recommended ** Manually downloading the IP-Adapter and Image Encoder files - Image Encoder folders shouid be placed in the `models\any\clip_vision` folders. IP Adapter Model folders should be placed in the relevant `ip-adapter` folder of relevant base model folder of Invoke root directory. For example, for the SDXL IP-Adapter, files should be added to the `model/sdxl/ip_adapter/` folder. 
 
 #### Using IP-Adapter

docs/features/TRAINING.md

@@ -4,278 +4,6 @@ title: Training
 
 # :material-file-document: Training
 
-# Textual Inversion Training
-## **Personalizing Text-to-Image Generation**
+Invoke Training has moved to its own repository, with a dedicated UI for accessing common scripts like Textual Inversion and LoRA training.
 
-You may personalize the generated images to provide your own styles or objects
-by training a new LDM checkpoint and introducing a new vocabulary to the fixed
-model as a (.pt) embeddings file. Alternatively, you may use or train
-HuggingFace Concepts embeddings files (.bin) from
-<https://huggingface.co/sd-concepts-library> and its associated
-notebooks.
-
-## **Hardware and Software Requirements**
-
-You will need a GPU to perform training in a reasonable length of
-time, and at least 12 GB of VRAM. We recommend using the [`xformers`
-library](../installation/070_INSTALL_XFORMERS.md) to accelerate the
-training process further. During training, about ~8 GB is temporarily
-needed in order to store intermediate models, checkpoints and logs.
-
-## **Preparing for Training**
-
-To train, prepare a folder that contains 3-5 images that illustrate
-the object or concept. It is good to provide a variety of examples or
-poses to avoid overtraining the system. Format these images as PNG
-(preferred) or JPG. You do not need to resize or crop the images in
-advance, but for more control you may wish to do so.
-
-Place the training images in a directory on the machine InvokeAI runs
-on. We recommend placing them in a subdirectory of the
-`text-inversion-training-data` folder located in the InvokeAI root
-directory, ordinarily `~/invokeai` (Linux/Mac), or
-`C:\Users\your_name\invokeai` (Windows). For example, to create an
-embedding for the "psychedelic" style, you'd place the training images
-into the directory
-`~invokeai/text-inversion-training-data/psychedelic`.
-
-## **Launching Training Using the Console Front End**
-
-InvokeAI 2.3 and higher comes with a text console-based training front
-end. From within the `invoke.sh`/`invoke.bat` Invoke launcher script,
-start training tool selecting choice (3):
-
-```sh
-1 "Generate images with a browser-based interface"
-2 "Explore InvokeAI nodes using a command-line interface"
-3 "Textual inversion training"
-4 "Merge models (diffusers type only)"
-5 "Download and install models"
-6 "Change InvokeAI startup options"
-7 "Re-run the configure script to fix a broken install or to complete a major upgrade"
-8 "Open the developer console"
-9 "Update InvokeAI"
-```
-
-Alternatively, you can select option (8) or from the command line, with the InvokeAI virtual environment active,
-you can then launch the front end with the command `invokeai-ti --gui`.
-
-This will launch a text-based front end that will look like this:
-
-<figure markdown>
-![ti-frontend](../assets/textual-inversion/ti-frontend.png)
-</figure>
-
-The interface is keyboard-based. Move from field to field using
-control-N (^N) to move to the next field and control-P (^P) to the
-previous one. <Tab> and <shift-TAB> work as well. Once a field is
-active, use the cursor keys. In a checkbox group, use the up and down
-cursor keys to move from choice to choice, and <space> to select a
-choice. In a scrollbar, use the left and right cursor keys to increase
-and decrease the value of the scroll. In textfields, type the desired
-values.
-
-The number of parameters may look intimidating, but in most cases the
-predefined defaults work fine. The red circled fields in the above
-illustration are the ones you will adjust most frequently.
-
-### Model Name
-
-This will list all the diffusers models that are currently
-installed. Select the one you wish to use as the basis for your
-embedding. Be aware that if you use a SD-1.X-based model for your
-training, you will only be able to use this embedding with other
-SD-1.X-based models. Similarly, if you train on SD-2.X, you will only
-be able to use the embeddings with models based on SD-2.X.
-
-### Trigger Term
-
-This is the prompt term you will use to trigger the embedding. Type a
-single word or phrase you wish to use as the trigger, example
-"psychedelic" (without angle brackets). Within InvokeAI, you will then
-be able to activate the trigger using the syntax `<psychedelic>`.
-
-### Initializer
-
-This is a single character that is used internally during the training
-process as a placeholder for the trigger term. It defaults to "*" and
-can usually be left alone.
-
-### Resume from last saved checkpoint
-
-As training proceeds, textual inversion will write a series of
-intermediate files that can be used to resume training from where it
-was left off in the case of an interruption. This checkbox will be
-automatically selected if you provide a previously used trigger term
-and at least one checkpoint file is found on disk.
-
-Note that as of 20 January 2023, resume does not seem to be working
-properly due to an issue with the upstream code.
-
-### Data Training Directory
-
-This is the location of the images to be used for training. When you
-select a trigger term like "my-trigger", the frontend will prepopulate
-this field with `~/invokeai/text-inversion-training-data/my-trigger`,
-but you can change the path to wherever you want.
-
-### Output Destination Directory
-
-This is the location of the logs, checkpoint files, and embedding
-files created during training. When you select a trigger term like
-"my-trigger", the frontend will prepopulate this field with
-`~/invokeai/text-inversion-output/my-trigger`, but you can change the
-path to wherever you want.
-
-### Image resolution
-
-The images in the training directory will be automatically scaled to
-the value you use here. For best results, you will want to use the
-same default resolution of the underlying model (512 pixels for
-SD-1.5, 768 for the larger version of SD-2.1).
-
-### Center crop images
-
-If this is selected, your images will be center cropped to make them
-square before resizing them to the desired resolution. Center cropping
-can indiscriminately cut off the top of subjects' heads for portrait
-aspect images, so if you have images like this, you may wish to use a
-photoeditor to manually crop them to a square aspect ratio.
-
-### Mixed precision
-
-Select the floating point precision for the embedding. "no" will
-result in a full 32-bit precision, "fp16" will provide 16-bit
-precision, and "bf16" will provide mixed precision (only available
-when XFormers is used).
-
-### Max training steps
-
-How many steps the training will take before the model converges. Most
-training sets will converge with 2000-3000 steps.
-
-### Batch size
-
-This adjusts how many training images are processed simultaneously in
-each step. Higher values will cause the training process to run more
-quickly, but use more memory. The default size will run with GPUs with
-as little as 12 GB.
-
-### Learning rate
-
-The rate at which the system adjusts its internal weights during
-training. Higher values risk overtraining (getting the same image each
-time), and lower values will take more steps to train a good
-model. The default of 0.0005 is conservative; you may wish to increase
-it to 0.005 to speed up training.
-
-### Scale learning rate by number of GPUs, steps and batch size
-
-If this is selected (the default) the system will adjust the provided
-learning rate to improve performance.
-
-### Use xformers acceleration
-
-This will activate XFormers memory-efficient attention. You need to
-have XFormers installed for this to have an effect.
-
-### Learning rate scheduler
-
-This adjusts how the learning rate changes over the course of
-training. The default "constant" means to use a constant learning rate
-for the entire training session. The other values scale the learning
-rate according to various formulas.
-
-Only "constant" is supported by the XFormers library.
-
-### Gradient accumulation steps
-
-This is a parameter that allows you to use bigger batch sizes than
-your GPU's VRAM would ordinarily accommodate, at the cost of some
-performance.
-
-### Warmup steps
-
-If "constant_with_warmup" is selected in the learning rate scheduler,
-then this provides the number of warmup steps. Warmup steps have a
-very low learning rate, and are one way of preventing early
-overtraining.
-
-## The training run
-
-Start the training run by advancing to the OK button (bottom right)
-and pressing <enter>. A series of progress messages will be displayed
-as the training process proceeds. This may take an hour or two,
-depending on settings and the speed of your system. Various log and
-checkpoint files will be written into the output directory (ordinarily
-`~/invokeai/text-inversion-output/my-model/`)
-
-At the end of successful training, the system will copy the file
-`learned_embeds.bin` into the InvokeAI root directory's `embeddings`
-directory, using a subdirectory named after the trigger token. For
-example, if the trigger token was `psychedelic`, then look for the
-embeddings file in
-`~/invokeai/embeddings/psychedelic/learned_embeds.bin`
-
-You may now launch InvokeAI and try out a prompt that uses the trigger
-term. For example `a plate of banana sushi in <psychedelic> style`.
-
-## **Training with the Command-Line Script**
-
-Training can also be done using a traditional command-line script. It
-can be launched from within the "developer's console", or from the
-command line after activating InvokeAI's virtual environment.
-
-It accepts a large number of arguments, which can be summarized by
-passing the `--help` argument:
-
-```sh
-invokeai-ti --help
-```
-
-Typical usage is shown here:
-```sh
-invokeai-ti \
-       --model=stable-diffusion-1.5 \
-       --resolution=512 \
-       --learnable_property=style \
-       --initializer_token='*' \
-       --placeholder_token='<psychedelic>' \
-       --train_data_dir=/home/lstein/invokeai/training-data/psychedelic \
-       --output_dir=/home/lstein/invokeai/text-inversion-training/psychedelic \
-       --scale_lr \
-       --train_batch_size=8 \
-       --gradient_accumulation_steps=4 \
-       --max_train_steps=3000 \
-       --learning_rate=0.0005 \
-       --resume_from_checkpoint=latest \
-       --lr_scheduler=constant \
-       --mixed_precision=fp16 \
-       --only_save_embeds
-```
-
-## Troubleshooting
-
-### `Cannot load embedding for <trigger>. It was trained on a model with token dimension 1024, but the current model has token dimension 768`
-
-Messages like this indicate you trained the embedding on a different base model than the currently selected one.
-
-For example, in the error above, the training was done on SD2.1 (768x768) but it was used on SD1.5 (512x512).
-
-## Reading
-
-For more information on textual inversion, please see the following
-resources:
-
-* The [textual inversion repository](https://github.com/rinongal/textual_inversion) and
-  associated paper for details and limitations.
-* [HuggingFace's textual inversion training
-  page](https://huggingface.co/docs/diffusers/training/text_inversion)
-* [HuggingFace example script
-  documentation](https://github.com/huggingface/diffusers/tree/main/examples/textual_inversion)
-  (Note that this script is similar to, but not identical, to
-  `textual_inversion`, but produces embed files that are completely compatible.
-
----
-
-copyright (c) 2023, Lincoln Stein and the InvokeAI Development Team
+You can find more by visiting the repo at https://github.com/invoke-ai/invoke-training

docs/help/FAQ.md

@@ -154,6 +154,18 @@ This is caused by an invalid setting in the `invokeai.yaml` configuration file.
 
 Check the [configuration docs] for more detail about the settings and how to specify them.
 
+## `ModuleNotFoundError: No module named 'controlnet_aux'`
+
+`controlnet_aux` is a dependency of Invoke and appears to have been packaged or distributed strangely. Sometimes, it doesn't install correctly. This is outside our control.
+
+If you encounter this error, the solution is to remove the package from the `pip` cache and re-run the Invoke installer so a fresh, working version of `controlnet_aux` can be downloaded and installed:
+
+- Run the Invoke launcher
+- Choose the developer console option
+- Run this command: `pip cache remove controlnet_aux`
+- Close the terminal window
+- Download and run the [installer](https://github.com/invoke-ai/InvokeAI/releases/latest), selecting your current install location
+
 ## Out of Memory Issues
 
 The models are large, VRAM is expensive, and you may find yourself

docs/help/diffusion.md

@@ -20,7 +20,7 @@ When you generate an image using text-to-image, multiple steps occur in latent s
 4. The VAE decodes the final latent image from latent space into image space.
 
 Image-to-image is a similar process, with only step 1 being different:
-1. The input image is encoded from image space into latent space by the VAE. Noise is then added to the input latent image. Denoising Strength dictates how may noise steps are added, and the amount of noise added at each step. A Denoising Strength of 0 means there are 0 steps and no noise added, resulting in an unchanged image, while a Denoising Strength of 1 results in the image being completely replaced with noise and a full set of denoising steps are performance. The process is then the same as steps 2-4 in the text-to-image process. 
+1. The input image is encoded from image space into latent space by the VAE. Noise is then added to the input latent image. Denoising Strength dictates how many noise steps are added, and the amount of noise added at each step. A Denoising Strength of 0 means there are 0 steps and no noise added, resulting in an unchanged image, while a Denoising Strength of 1 results in the image being completely replaced with noise and a full set of denoising steps are performance. The process is then the same as steps 2-4 in the text-to-image process. 
 
 Furthermore, a model provides the CLIP prompt tokenizer, the VAE, and a U-Net (where noise prediction occurs given a prompt and initial noise tensor).
 

docs/installation/010_INSTALL_AUTOMATED.md

@@ -1,8 +1,10 @@
-# Automatic Install
+# Automatic Install & Updates
 
-The installer is used for both new installs and updates.
+**The same packaged installer file can be used for both new installs and updates.**
+Using the installer for updates will leave everything you've added since installation, and just update the core libraries used to run Invoke.
+Simply use the same path you installed to originally.
 
-Both release and pre-release versions can be installed using it. It also supports install a wheel if needed.
+Both release and pre-release versions can be installed using the installer. It also supports install through a wheel if needed.
 
 Be sure to review the [installation requirements] and ensure your system has everything it needs to install Invoke.
 
@@ -96,7 +98,7 @@ Updating is exactly the same as installing - download the latest installer, choo
 
 If you have installation issues, please review the [FAQ]. You can also [create an issue] or ask for help on [discord].
 
-[installation requirements]: INSTALLATION.md#installation-requirements
+[installation requirements]: INSTALL_REQUIREMENTS.md
 [FAQ]: ../help/FAQ.md
 [install some models]: 050_INSTALLING_MODELS.md
 [configuration docs]: ../features/CONFIGURATION.md

docs/installation/020_INSTALL_MANUAL.md

@@ -10,7 +10,7 @@ InvokeAI is distributed as a python package on PyPI, installable with `pip`. The
 
 ### Requirements
 
-Before you start, go through the [installation requirements].
+Before you start, go through the [installation requirements](./INSTALL_REQUIREMENTS.md).
 
 ### Installation Walkthrough
 
@@ -79,7 +79,7 @@ Before you start, go through the [installation requirements].
 
 1. Install the InvokeAI Package. The base command is `pip install InvokeAI --use-pep517`, but you may need to change this depending on your system and the desired features.
 
-    - You may need to provide an [extra index URL]. Select your platform configuration using [this tool on the PyTorch website]. Copy the `--extra-index-url` string from this and append it to your install command.
+    - You may need to provide an [extra index URL](https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-extra-index-url). Select your platform configuration using [this tool on the PyTorch website](https://pytorch.org/get-started/locally/). Copy the `--extra-index-url` string from this and append it to your install command.
 
         !!! example "Install with an extra index URL"
 
@@ -116,4 +116,4 @@ Before you start, go through the [installation requirements].
 
     !!! warning
 
-        If the virtual environment is _not_ inside the root directory, then you _must_ specify the path to the root directory with `--root_dir \path\to\invokeai` or the `INVOKEAI_ROOT` environment variable.
+        If the virtual environment is _not_ inside the root directory, then you _must_ specify the path to the root directory with `--root \path\to\invokeai` or the `INVOKEAI_ROOT` environment variable.

docs/installation/INSTALLATION.md

@@ -1,4 +1,4 @@
-# Installation Overview
+# Installation and Updating Overview
 
 Before installing, review the [installation requirements] to ensure your system is set up properly.
 
@@ -6,14 +6,21 @@ See the [FAQ] for frequently-encountered installation issues.
 
 If you need more help, join our [discord] or [create an issue].
 
-<h2>Automatic Install</h2>
+<h2>Automatic Install & Updates </h2>
 
 ✅ The automatic install is the best way to run InvokeAI. Check out the [installation guide] to get started.
 
+⬆️ The same installer is also the best way to update InvokeAI - Simply rerun it for the same folder you installed to.
+
+The installation process simply manages installation for the core libraries & application dependencies that run Invoke.
+Any models, images, or other assets in the Invoke root folder won't be affected by the installation process.
+
 <h2>Manual Install</h2>
 
 If you are familiar with python and want more control over the packages that are installed, you can [install InvokeAI manually via PyPI].
 
+Updates are managed by reinstalling the latest version through PyPi.
+
 <h2>Developer Install</h2>
 
 If you want to contribute to InvokeAI, consult the [developer install guide].

docs/installation/INSTALL_REQUIREMENTS.md

@@ -37,13 +37,13 @@ Invoke runs best with a dedicated GPU, but will fall back to running on CPU, alb
     === "Nvidia"
 
         ```
-        Any GPU with at least 8GB VRAM. Linux only.
+        Any GPU with at least 8GB VRAM.
         ```
 
     === "AMD"
 
         ```
-        Any GPU with at least 16GB VRAM.
+        Any GPU with at least 16GB VRAM. Linux only.
         ```
 
     === "Mac"

installer/templates/invoke.bat.in

@@ -10,8 +10,7 @@ set INVOKEAI_ROOT=.
 echo Desired action:
 echo 1. Generate images with the browser-based interface
 echo 2. Open the developer console
-echo 3. Run the InvokeAI image database maintenance script
-echo 4. Command-line help
+echo 3. Command-line help
 echo Q - Quit
 echo.
 echo To update, download and run the installer from https://github.com/invoke-ai/InvokeAI/releases/latest.
@@ -34,9 +33,6 @@ IF /I "%choice%" == "1" (
     echo *** Type `exit` to quit this shell and deactivate the Python virtual environment ***
     call cmd /k
 ) ELSE IF /I "%choice%" == "3" (
-   echo Running the db maintenance script...
-   python .venv\Scripts\invokeai-db-maintenance.exe
-) ELSE IF /I "%choice%" == "4" (
     echo Displaying command line help...
     python .venv\Scripts\invokeai-web.exe --help %*
     pause

installer/templates/invoke.sh.in

@@ -47,11 +47,6 @@ do_choice() {
         bash --init-file "$file_name"
         ;;
     3)
-        clear
-        printf "Running the db maintenance script\n"
-        invokeai-db-maintenance --root ${INVOKEAI_ROOT}
-        ;;
-    4)
         clear
         printf "Command-line help\n"
         invokeai-web --help
@@ -71,8 +66,7 @@ do_line_input() {
     printf "What would you like to do?\n"
     printf "1: Generate images using the browser-based interface\n"
     printf "2: Open the developer console\n"
-    printf "3: Run the InvokeAI image database maintenance script\n"
-    printf "4: Command-line help\n"
+    printf "3: Command-line help\n"
     printf "Q: Quit\n\n"
     printf "To update, download and run the installer from https://github.com/invoke-ai/InvokeAI/releases/latest.\n\n"
     read -p "Please enter 1-4, Q: [1] " yn

invokeai/app/api/dependencies.py

@@ -18,6 +18,7 @@ from ..services.boards.boards_default import BoardService
 from ..services.bulk_download.bulk_download_default import BulkDownloadService
 from ..services.config import InvokeAIAppConfig
 from ..services.download import DownloadQueueService
+from ..services.events.events_fastapievents import FastAPIEventService
 from ..services.image_files.image_files_disk import DiskImageFileStorage
 from ..services.image_records.image_records_sqlite import SqliteImageRecordStorage
 from ..services.images.images_default import ImageService
@@ -29,11 +30,10 @@ from ..services.model_images.model_images_default import ModelImageFileStorageDi
 from ..services.model_manager.model_manager_default import ModelManagerService
 from ..services.model_records import ModelRecordServiceSQL
 from ..services.names.names_default import SimpleNameService
-from ..services.session_processor.session_processor_default import DefaultSessionProcessor
+from ..services.session_processor.session_processor_default import DefaultSessionProcessor, DefaultSessionRunner
 from ..services.session_queue.session_queue_sqlite import SqliteSessionQueue
 from ..services.urls.urls_default import LocalUrlService
 from ..services.workflow_records.workflow_records_sqlite import SqliteWorkflowRecordsStorage
-from .events import FastAPIEventService
 
 
 # TODO: is there a better way to achieve this?
@@ -103,7 +103,7 @@ class ApiDependencies:
         )
         names = SimpleNameService()
         performance_statistics = InvocationStatsService()
-        session_processor = DefaultSessionProcessor()
+        session_processor = DefaultSessionProcessor(session_runner=DefaultSessionRunner())
         session_queue = SqliteSessionQueue(db=db)
         urls = LocalUrlService()
         workflow_records = SqliteWorkflowRecordsStorage(db=db)

invokeai/app/api/events.py

@@ -1,52 +0,0 @@
-# Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
-
-import asyncio
-import threading
-from queue import Empty, Queue
-from typing import Any
-
-from fastapi_events.dispatcher import dispatch
-
-from ..services.events.events_base import EventServiceBase
-
-
-class FastAPIEventService(EventServiceBase):
-    event_handler_id: int
-    __queue: Queue
-    __stop_event: threading.Event
-
-    def __init__(self, event_handler_id: int) -> None:
-        self.event_handler_id = event_handler_id
-        self.__queue = Queue()
-        self.__stop_event = threading.Event()
-        asyncio.create_task(self.__dispatch_from_queue(stop_event=self.__stop_event))
-
-        super().__init__()
-
-    def stop(self, *args, **kwargs):
-        self.__stop_event.set()
-        self.__queue.put(None)
-
-    def dispatch(self, event_name: str, payload: Any) -> None:
-        self.__queue.put({"event_name": event_name, "payload": payload})
-
-    async def __dispatch_from_queue(self, stop_event: threading.Event):
-        """Get events on from the queue and dispatch them, from the correct thread"""
-        while not stop_event.is_set():
-            try:
-                event = self.__queue.get(block=False)
-                if not event:  # Probably stopping
-                    continue
-
-                dispatch(
-                    event.get("event_name"),
-                    payload=event.get("payload"),
-                    middleware_id=self.event_handler_id,
-                )
-
-            except Empty:
-                await asyncio.sleep(0.1)
-                pass
-
-            except asyncio.CancelledError as e:
-                raise e  # Raise a proper error

invokeai/app/api/routers/app_info.py

@@ -13,7 +13,6 @@ from pydantic import BaseModel, Field
 from invokeai.app.invocations.upscale import ESRGAN_MODELS
 from invokeai.app.services.invocation_cache.invocation_cache_common import InvocationCacheStatus
 from invokeai.backend.image_util.infill_methods.patchmatch import PatchMatch
-from invokeai.backend.image_util.safety_checker import SafetyChecker
 from invokeai.backend.util.logging import logging
 from invokeai.version import __version__
 
@@ -109,9 +108,7 @@ async def get_config() -> AppConfig:
         upscaling_models.append(str(Path(model).stem))
     upscaler = Upscaler(upscaling_method="esrgan", upscaling_models=upscaling_models)
 
-    nsfw_methods = []
-    if SafetyChecker.safety_checker_available():
-        nsfw_methods.append("nsfw_checker")
+    nsfw_methods = ["nsfw_checker"]
 
     watermarking_methods = ["invisible_watermark"]
 

invokeai/app/api/routers/images.py

@@ -6,13 +6,12 @@ from fastapi import BackgroundTasks, Body, HTTPException, Path, Query, Request,
 from fastapi.responses import FileResponse
 from fastapi.routing import APIRouter
 from PIL import Image
-from pydantic import BaseModel, Field, ValidationError
+from pydantic import BaseModel, Field, JsonValue
 
-from invokeai.app.invocations.fields import MetadataField, MetadataFieldValidator
+from invokeai.app.invocations.fields import MetadataField
 from invokeai.app.services.image_records.image_records_common import ImageCategory, ImageRecordChanges, ResourceOrigin
 from invokeai.app.services.images.images_common import ImageDTO, ImageUrlsDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.workflow_records.workflow_records_common import WorkflowWithoutID, WorkflowWithoutIDValidator
 
 from ..dependencies import ApiDependencies
 
@@ -42,13 +41,17 @@ async def upload_image(
     board_id: Optional[str] = Query(default=None, description="The board to add this image to, if any"),
     session_id: Optional[str] = Query(default=None, description="The session ID associated with this upload, if any"),
     crop_visible: Optional[bool] = Query(default=False, description="Whether to crop the image"),
+    metadata: Optional[JsonValue] = Body(
+        default=None, description="The metadata to associate with the image", embed=True
+    ),
 ) -> ImageDTO:
     """Uploads an image"""
     if not file.content_type or not file.content_type.startswith("image"):
         raise HTTPException(status_code=415, detail="Not an image")
 
-    metadata = None
-    workflow = None
+    _metadata = None
+    _workflow = None
+    _graph = None
 
     contents = await file.read()
     try:
@@ -62,22 +65,28 @@ async def upload_image(
 
     # TODO: retain non-invokeai metadata on upload?
     # attempt to parse metadata from image
-    metadata_raw = pil_image.info.get("invokeai_metadata", None)
-    if metadata_raw:
-        try:
-            metadata = MetadataFieldValidator.validate_json(metadata_raw)
-        except ValidationError:
-            ApiDependencies.invoker.services.logger.warn("Failed to parse metadata for uploaded image")
-            pass
+    metadata_raw = metadata if isinstance(metadata, str) else pil_image.info.get("invokeai_metadata", None)
+    if isinstance(metadata_raw, str):
+        _metadata = metadata_raw
+    else:
+        ApiDependencies.invoker.services.logger.debug("Failed to parse metadata for uploaded image")
+        pass
 
     # attempt to parse workflow from image
     workflow_raw = pil_image.info.get("invokeai_workflow", None)
-    if workflow_raw is not None:
-        try:
-            workflow = WorkflowWithoutIDValidator.validate_json(workflow_raw)
-        except ValidationError:
-            ApiDependencies.invoker.services.logger.warn("Failed to parse metadata for uploaded image")
-            pass
+    if isinstance(workflow_raw, str):
+        _workflow = workflow_raw
+    else:
+        ApiDependencies.invoker.services.logger.debug("Failed to parse workflow for uploaded image")
+        pass
+
+    # attempt to extract graph from image
+    graph_raw = pil_image.info.get("invokeai_graph", None)
+    if isinstance(graph_raw, str):
+        _graph = graph_raw
+    else:
+        ApiDependencies.invoker.services.logger.debug("Failed to parse graph for uploaded image")
+        pass
 
     try:
         image_dto = ApiDependencies.invoker.services.images.create(
@@ -86,8 +95,9 @@ async def upload_image(
             image_category=image_category,
             session_id=session_id,
             board_id=board_id,
-            metadata=metadata,
-            workflow=workflow,
+            metadata=_metadata,
+            workflow=_workflow,
+            graph=_graph,
             is_intermediate=is_intermediate,
         )
 
@@ -185,14 +195,21 @@ async def get_image_metadata(
         raise HTTPException(status_code=404)
 
 
+class WorkflowAndGraphResponse(BaseModel):
+    workflow: Optional[str] = Field(description="The workflow used to generate the image, as stringified JSON")
+    graph: Optional[str] = Field(description="The graph used to generate the image, as stringified JSON")
+
+
 @images_router.get(
-    "/i/{image_name}/workflow", operation_id="get_image_workflow", response_model=Optional[WorkflowWithoutID]
+    "/i/{image_name}/workflow", operation_id="get_image_workflow", response_model=WorkflowAndGraphResponse
 )
 async def get_image_workflow(
     image_name: str = Path(description="The name of image whose workflow to get"),
-) -> Optional[WorkflowWithoutID]:
+) -> WorkflowAndGraphResponse:
     try:
-        return ApiDependencies.invoker.services.images.get_workflow(image_name)
+        workflow = ApiDependencies.invoker.services.images.get_workflow(image_name)
+        graph = ApiDependencies.invoker.services.images.get_graph(image_name)
+        return WorkflowAndGraphResponse(workflow=workflow, graph=graph)
     except Exception:
         raise HTTPException(status_code=404)
 

invokeai/app/api/routers/model_manager.py

@@ -6,7 +6,7 @@ import pathlib
 import shutil
 import traceback
 from copy import deepcopy
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List, Optional, Type
 
 from fastapi import Body, Path, Query, Response, UploadFile
 from fastapi.responses import FileResponse
@@ -16,7 +16,8 @@ from pydantic import AnyHttpUrl, BaseModel, ConfigDict, Field
 from starlette.exceptions import HTTPException
 from typing_extensions import Annotated
 
-from invokeai.app.services.model_install import ModelInstallJob
+from invokeai.app.services.model_images.model_images_common import ModelImageFileNotFoundException
+from invokeai.app.services.model_install.model_install_common import ModelInstallJob
 from invokeai.app.services.model_records import (
     DuplicateModelException,
     InvalidModelException,
@@ -52,6 +53,13 @@ class ModelsList(BaseModel):
     model_config = ConfigDict(use_enum_values=True)
 
 
+def add_cover_image_to_model_config(config: AnyModelConfig, dependencies: Type[ApiDependencies]) -> AnyModelConfig:
+    """Add a cover image URL to a model configuration."""
+    cover_image = dependencies.invoker.services.model_images.get_url(config.key)
+    config.cover_image = cover_image
+    return config
+
+
 ##############################################################################
 # These are example inputs and outputs that are used in places where Swagger
 # is unable to generate a correct example.
@@ -118,8 +126,7 @@ async def list_model_records(
             record_store.search_by_attr(model_type=model_type, model_name=model_name, model_format=model_format)
         )
     for model in found_models:
-        cover_image = ApiDependencies.invoker.services.model_images.get_url(model.key)
-        model.cover_image = cover_image
+        model = add_cover_image_to_model_config(model, ApiDependencies)
     return ModelsList(models=found_models)
 
 
@@ -160,12 +167,9 @@ async def get_model_record(
     key: str = Path(description="Key of the model record to fetch."),
 ) -> AnyModelConfig:
     """Get a model record"""
-    record_store = ApiDependencies.invoker.services.model_manager.store
     try:
-        config: AnyModelConfig = record_store.get_model(key)
-        cover_image = ApiDependencies.invoker.services.model_images.get_url(key)
-        config.cover_image = cover_image
-        return config
+        config = ApiDependencies.invoker.services.model_manager.store.get_model(key)
+        return add_cover_image_to_model_config(config, ApiDependencies)
     except UnknownModelException as e:
         raise HTTPException(status_code=404, detail=str(e))
 
@@ -294,14 +298,15 @@ async def update_model_record(
     installer = ApiDependencies.invoker.services.model_manager.install
     try:
         record_store.update_model(key, changes=changes)
-        model_response: AnyModelConfig = installer.sync_model_path(key)
+        config = installer.sync_model_path(key)
+        config = add_cover_image_to_model_config(config, ApiDependencies)
         logger.info(f"Updated model: {key}")
     except UnknownModelException as e:
         raise HTTPException(status_code=404, detail=str(e))
     except ValueError as e:
         logger.error(str(e))
         raise HTTPException(status_code=409, detail=str(e))
-    return model_response
+    return config
 
 
 @model_manager_router.get(
@@ -648,6 +653,14 @@ async def convert_model(
         logger.error(str(e))
         raise HTTPException(status_code=409, detail=str(e))
 
+    # Update the model image if the model had one
+    try:
+        model_image = ApiDependencies.invoker.services.model_images.get(key)
+        ApiDependencies.invoker.services.model_images.save(model_image, new_key)
+        ApiDependencies.invoker.services.model_images.delete(key)
+    except ModelImageFileNotFoundException:
+        pass
+
     # delete the original safetensors file
     installer.delete(key)
 
@@ -655,7 +668,8 @@ async def convert_model(
     shutil.rmtree(cache_path)
 
     # return the config record for the new diffusers directory
-    new_config: AnyModelConfig = store.get_model(new_key)
+    new_config = store.get_model(new_key)
+    new_config = add_cover_image_to_model_config(new_config, ApiDependencies)
     return new_config
 
 

invokeai/app/api/routers/session_queue.py

@@ -203,6 +203,7 @@ async def get_batch_status(
     responses={
         200: {"model": SessionQueueItem},
     },
+    response_model_exclude_none=True,
 )
 async def get_queue_item(
     queue_id: str = Path(description="The queue id to perform this operation on"),

invokeai/app/api/sockets.py

@@ -1,66 +1,125 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
 
+from typing import Any
+
 from fastapi import FastAPI
-from fastapi_events.handlers.local import local_handler
-from fastapi_events.typing import Event
+from pydantic import BaseModel
 from socketio import ASGIApp, AsyncServer
 
-from ..services.events.events_base import EventServiceBase
+from invokeai.app.services.events.events_common import (
+    BatchEnqueuedEvent,
+    BulkDownloadCompleteEvent,
+    BulkDownloadErrorEvent,
+    BulkDownloadEventBase,
+    BulkDownloadStartedEvent,
+    DownloadCancelledEvent,
+    DownloadCompleteEvent,
+    DownloadErrorEvent,
+    DownloadEventBase,
+    DownloadProgressEvent,
+    DownloadStartedEvent,
+    FastAPIEvent,
+    InvocationCompleteEvent,
+    InvocationDenoiseProgressEvent,
+    InvocationErrorEvent,
+    InvocationStartedEvent,
+    ModelEventBase,
+    ModelInstallCancelledEvent,
+    ModelInstallCompleteEvent,
+    ModelInstallDownloadProgressEvent,
+    ModelInstallDownloadsCompleteEvent,
+    ModelInstallErrorEvent,
+    ModelInstallStartedEvent,
+    ModelLoadCompleteEvent,
+    ModelLoadStartedEvent,
+    QueueClearedEvent,
+    QueueEventBase,
+    QueueItemStatusChangedEvent,
+    register_events,
+)
+
+
+class QueueSubscriptionEvent(BaseModel):
+    """Event data for subscribing to the socket.io queue room.
+    This is a pydantic model to ensure the data is in the correct format."""
+
+    queue_id: str
+
+
+class BulkDownloadSubscriptionEvent(BaseModel):
+    """Event data for subscribing to the socket.io bulk downloads room.
+    This is a pydantic model to ensure the data is in the correct format."""
+
+    bulk_download_id: str
+
+
+QUEUE_EVENTS = {
+    InvocationStartedEvent,
+    InvocationDenoiseProgressEvent,
+    InvocationCompleteEvent,
+    InvocationErrorEvent,
+    QueueItemStatusChangedEvent,
+    BatchEnqueuedEvent,
+    QueueClearedEvent,
+}
+
+MODEL_EVENTS = {
+    DownloadCancelledEvent,
+    DownloadCompleteEvent,
+    DownloadErrorEvent,
+    DownloadProgressEvent,
+    DownloadStartedEvent,
+    ModelLoadStartedEvent,
+    ModelLoadCompleteEvent,
+    ModelInstallDownloadProgressEvent,
+    ModelInstallDownloadsCompleteEvent,
+    ModelInstallStartedEvent,
+    ModelInstallCompleteEvent,
+    ModelInstallCancelledEvent,
+    ModelInstallErrorEvent,
+}
+
+BULK_DOWNLOAD_EVENTS = {BulkDownloadStartedEvent, BulkDownloadCompleteEvent, BulkDownloadErrorEvent}
 
 
 class SocketIO:
-    __sio: AsyncServer
-    __app: ASGIApp
+    _sub_queue = "subscribe_queue"
+    _unsub_queue = "unsubscribe_queue"
 
-    __sub_queue: str = "subscribe_queue"
-    __unsub_queue: str = "unsubscribe_queue"
-
-    __sub_bulk_download: str = "subscribe_bulk_download"
-    __unsub_bulk_download: str = "unsubscribe_bulk_download"
+    _sub_bulk_download = "subscribe_bulk_download"
+    _unsub_bulk_download = "unsubscribe_bulk_download"
 
     def __init__(self, app: FastAPI):
-        self.__sio = AsyncServer(async_mode="asgi", cors_allowed_origins="*")
-        self.__app = ASGIApp(socketio_server=self.__sio, socketio_path="/ws/socket.io")
-        app.mount("/ws", self.__app)
+        self._sio = AsyncServer(async_mode="asgi", cors_allowed_origins="*")
+        self._app = ASGIApp(socketio_server=self._sio, socketio_path="/ws/socket.io")
+        app.mount("/ws", self._app)
 
-        self.__sio.on(self.__sub_queue, handler=self._handle_sub_queue)
-        self.__sio.on(self.__unsub_queue, handler=self._handle_unsub_queue)
-        local_handler.register(event_name=EventServiceBase.queue_event, _func=self._handle_queue_event)
-        local_handler.register(event_name=EventServiceBase.model_event, _func=self._handle_model_event)
+        self._sio.on(self._sub_queue, handler=self._handle_sub_queue)
+        self._sio.on(self._unsub_queue, handler=self._handle_unsub_queue)
+        self._sio.on(self._sub_bulk_download, handler=self._handle_sub_bulk_download)
+        self._sio.on(self._unsub_bulk_download, handler=self._handle_unsub_bulk_download)
 
-        self.__sio.on(self.__sub_bulk_download, handler=self._handle_sub_bulk_download)
-        self.__sio.on(self.__unsub_bulk_download, handler=self._handle_unsub_bulk_download)
-        local_handler.register(event_name=EventServiceBase.bulk_download_event, _func=self._handle_bulk_download_event)
+        register_events(QUEUE_EVENTS, self._handle_queue_event)
+        register_events(MODEL_EVENTS, self._handle_model_event)
+        register_events(BULK_DOWNLOAD_EVENTS, self._handle_bulk_image_download_event)
 
-    async def _handle_queue_event(self, event: Event):
-        await self.__sio.emit(
-            event=event[1]["event"],
-            data=event[1]["data"],
-            room=event[1]["data"]["queue_id"],
-        )
+    async def _handle_sub_queue(self, sid: str, data: Any) -> None:
+        await self._sio.enter_room(sid, QueueSubscriptionEvent(**data).queue_id)
 
-    async def _handle_sub_queue(self, sid, data, *args, **kwargs) -> None:
-        if "queue_id" in data:
-            await self.__sio.enter_room(sid, data["queue_id"])
+    async def _handle_unsub_queue(self, sid: str, data: Any) -> None:
+        await self._sio.leave_room(sid, QueueSubscriptionEvent(**data).queue_id)
 
-    async def _handle_unsub_queue(self, sid, data, *args, **kwargs) -> None:
-        if "queue_id" in data:
-            await self.__sio.leave_room(sid, data["queue_id"])
+    async def _handle_sub_bulk_download(self, sid: str, data: Any) -> None:
+        await self._sio.enter_room(sid, BulkDownloadSubscriptionEvent(**data).bulk_download_id)
 
-    async def _handle_model_event(self, event: Event) -> None:
-        await self.__sio.emit(event=event[1]["event"], data=event[1]["data"])
+    async def _handle_unsub_bulk_download(self, sid: str, data: Any) -> None:
+        await self._sio.leave_room(sid, BulkDownloadSubscriptionEvent(**data).bulk_download_id)
 
-    async def _handle_bulk_download_event(self, event: Event):
-        await self.__sio.emit(
-            event=event[1]["event"],
-            data=event[1]["data"],
-            room=event[1]["data"]["bulk_download_id"],
-        )
+    async def _handle_queue_event(self, event: FastAPIEvent[QueueEventBase]):
+        await self._sio.emit(event=event[0], data=event[1].model_dump(mode="json"), room=event[1].queue_id)
 
-    async def _handle_sub_bulk_download(self, sid, data, *args, **kwargs):
-        if "bulk_download_id" in data:
-            await self.__sio.enter_room(sid, data["bulk_download_id"])
+    async def _handle_model_event(self, event: FastAPIEvent[ModelEventBase | DownloadEventBase]) -> None:
+        await self._sio.emit(event=event[0], data=event[1].model_dump(mode="json"))
 
-    async def _handle_unsub_bulk_download(self, sid, data, *args, **kwargs):
-        if "bulk_download_id" in data:
-            await self.__sio.leave_room(sid, data["bulk_download_id"])
+    async def _handle_bulk_image_download_event(self, event: FastAPIEvent[BulkDownloadEventBase]) -> None:
+        await self._sio.emit(event=event[0], data=event[1].model_dump(mode="json"), room=event[1].bulk_download_id)

invokeai/app/api_app.py

@@ -3,9 +3,7 @@ import logging
 import mimetypes
 import socket
 from contextlib import asynccontextmanager
-from inspect import signature
 from pathlib import Path
-from typing import Any
 
 import torch
 import uvicorn
@@ -13,11 +11,9 @@ from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.middleware.gzip import GZipMiddleware
 from fastapi.openapi.docs import get_redoc_html, get_swagger_ui_html
-from fastapi.openapi.utils import get_openapi
 from fastapi.responses import HTMLResponse
 from fastapi_events.handlers.local import local_handler
 from fastapi_events.middleware import EventHandlerASGIMiddleware
-from pydantic.json_schema import models_json_schema
 from torch.backends.mps import is_available as is_mps_available
 
 # for PyCharm:
@@ -25,9 +21,8 @@ from torch.backends.mps import is_available as is_mps_available
 import invokeai.backend.util.hotfixes  # noqa: F401 (monkeypatching on import)
 import invokeai.frontend.web as web_dir
 from invokeai.app.api.no_cache_staticfiles import NoCacheStaticFiles
-from invokeai.app.invocations.model import ModelIdentifierField
 from invokeai.app.services.config.config_default import get_config
-from invokeai.app.services.session_processor.session_processor_common import ProgressImage
+from invokeai.app.util.custom_openapi import get_openapi_func
 from invokeai.backend.util.devices import TorchDevice
 
 from ..backend.util.logging import InvokeAILogger
@@ -44,11 +39,6 @@ from .api.routers import (
     workflows,
 )
 from .api.sockets import SocketIO
-from .invocations.baseinvocation import (
-    BaseInvocation,
-    UIConfigBase,
-)
-from .invocations.fields import InputFieldJSONSchemaExtra, OutputFieldJSONSchemaExtra
 
 app_config = get_config()
 
@@ -118,85 +108,7 @@ app.include_router(app_info.app_router, prefix="/api")
 app.include_router(session_queue.session_queue_router, prefix="/api")
 app.include_router(workflows.workflows_router, prefix="/api")
 
-
-# Build a custom OpenAPI to include all outputs
-# TODO: can outputs be included on metadata of invocation schemas somehow?
-def custom_openapi() -> dict[str, Any]:
-    if app.openapi_schema:
-        return app.openapi_schema
-    openapi_schema = get_openapi(
-        title=app.title,
-        description="An API for invoking AI image operations",
-        version="1.0.0",
-        routes=app.routes,
-        separate_input_output_schemas=False,  # https://fastapi.tiangolo.com/how-to/separate-openapi-schemas/
-    )
-
-    # Add all outputs
-    all_invocations = BaseInvocation.get_invocations()
-    output_types = set()
-    output_type_titles = {}
-    for invoker in all_invocations:
-        output_type = signature(invoker.invoke).return_annotation
-        output_types.add(output_type)
-
-    output_schemas = models_json_schema(
-        models=[(o, "serialization") for o in output_types], ref_template="#/components/schemas/{model}"
-    )
-    for schema_key, output_schema in output_schemas[1]["$defs"].items():
-        # TODO: note that we assume the schema_key here is the TYPE.__name__
-        # This could break in some cases, figure out a better way to do it
-        output_type_titles[schema_key] = output_schema["title"]
-        openapi_schema["components"]["schemas"][schema_key] = output_schema
-        openapi_schema["components"]["schemas"][schema_key]["class"] = "output"
-
-    # Some models don't end up in the schemas as standalone definitions
-    additional_schemas = models_json_schema(
-        [
-            (UIConfigBase, "serialization"),
-            (InputFieldJSONSchemaExtra, "serialization"),
-            (OutputFieldJSONSchemaExtra, "serialization"),
-            (ModelIdentifierField, "serialization"),
-            (ProgressImage, "serialization"),
-        ],
-        ref_template="#/components/schemas/{model}",
-    )
-    for schema_key, schema_json in additional_schemas[1]["$defs"].items():
-        openapi_schema["components"]["schemas"][schema_key] = schema_json
-
-    # Add a reference to the output type to additionalProperties of the invoker schema
-    for invoker in all_invocations:
-        invoker_name = invoker.__name__  # type: ignore [attr-defined] # this is a valid attribute
-        output_type = signature(obj=invoker.invoke).return_annotation
-        output_type_title = output_type_titles[output_type.__name__]
-        invoker_schema = openapi_schema["components"]["schemas"][f"{invoker_name}"]
-        outputs_ref = {"$ref": f"#/components/schemas/{output_type_title}"}
-        invoker_schema["output"] = outputs_ref
-        invoker_schema["class"] = "invocation"
-
-    # This code no longer seems to be necessary?
-    # Leave it here just in case
-    #
-    # from invokeai.backend.model_manager import get_model_config_formats
-    # formats = get_model_config_formats()
-    # for model_config_name, enum_set in formats.items():
-
-    #     if model_config_name in openapi_schema["components"]["schemas"]:
-    #         # print(f"Config with name {name} already defined")
-    #         continue
-
-    #     openapi_schema["components"]["schemas"][model_config_name] = {
-    #         "title": model_config_name,
-    #         "description": "An enumeration.",
-    #         "type": "string",
-    #         "enum": [v.value for v in enum_set],
-    #     }
-
-    app.openapi_schema = openapi_schema
-    return app.openapi_schema
-
-
-app.openapi = custom_openapi  # type: ignore [method-assign] # this is a valid assignment
+app.openapi = get_openapi_func(app)
 
 
 @app.get("/docs", include_in_schema=False)

invokeai/app/invocations/baseinvocation.py

@@ -98,11 +98,13 @@ class BaseInvocationOutput(BaseModel):
 
     _output_classes: ClassVar[set[BaseInvocationOutput]] = set()
     _typeadapter: ClassVar[Optional[TypeAdapter[Any]]] = None
+    _typeadapter_needs_update: ClassVar[bool] = False
 
     @classmethod
     def register_output(cls, output: BaseInvocationOutput) -> None:
         """Registers an invocation output."""
         cls._output_classes.add(output)
+        cls._typeadapter_needs_update = True
 
     @classmethod
     def get_outputs(cls) -> Iterable[BaseInvocationOutput]:
@@ -112,11 +114,12 @@ class BaseInvocationOutput(BaseModel):
     @classmethod
     def get_typeadapter(cls) -> TypeAdapter[Any]:
         """Gets a pydantc TypeAdapter for the union of all invocation output types."""
-        if not cls._typeadapter:
-            InvocationOutputsUnion = TypeAliasType(
-                "InvocationOutputsUnion", Annotated[Union[tuple(cls._output_classes)], Field(discriminator="type")]
+        if not cls._typeadapter or cls._typeadapter_needs_update:
+            AnyInvocationOutput = TypeAliasType(
+                "AnyInvocationOutput", Annotated[Union[tuple(cls._output_classes)], Field(discriminator="type")]
             )
-            cls._typeadapter = TypeAdapter(InvocationOutputsUnion)
+            cls._typeadapter = TypeAdapter(AnyInvocationOutput)
+            cls._typeadapter_needs_update = False
         return cls._typeadapter
 
     @classmethod
@@ -125,12 +128,13 @@ class BaseInvocationOutput(BaseModel):
         return (i.get_type() for i in BaseInvocationOutput.get_outputs())
 
     @staticmethod
-    def json_schema_extra(schema: dict[str, Any], model_class: Type[BaseModel]) -> None:
+    def json_schema_extra(schema: dict[str, Any], model_class: Type[BaseInvocationOutput]) -> None:
         """Adds various UI-facing attributes to the invocation output's OpenAPI schema."""
         # Because we use a pydantic Literal field with default value for the invocation type,
         # it will be typed as optional in the OpenAPI schema. Make it required manually.
         if "required" not in schema or not isinstance(schema["required"], list):
             schema["required"] = []
+        schema["class"] = "output"
         schema["required"].extend(["type"])
 
     @classmethod
@@ -167,6 +171,7 @@ class BaseInvocation(ABC, BaseModel):
 
     _invocation_classes: ClassVar[set[BaseInvocation]] = set()
     _typeadapter: ClassVar[Optional[TypeAdapter[Any]]] = None
+    _typeadapter_needs_update: ClassVar[bool] = False
 
     @classmethod
     def get_type(cls) -> str:
@@ -177,15 +182,17 @@ class BaseInvocation(ABC, BaseModel):
     def register_invocation(cls, invocation: BaseInvocation) -> None:
         """Registers an invocation."""
         cls._invocation_classes.add(invocation)
+        cls._typeadapter_needs_update = True
 
     @classmethod
     def get_typeadapter(cls) -> TypeAdapter[Any]:
         """Gets a pydantc TypeAdapter for the union of all invocation types."""
-        if not cls._typeadapter:
-            InvocationsUnion = TypeAliasType(
-                "InvocationsUnion", Annotated[Union[tuple(cls._invocation_classes)], Field(discriminator="type")]
+        if not cls._typeadapter or cls._typeadapter_needs_update:
+            AnyInvocation = TypeAliasType(
+                "AnyInvocation", Annotated[Union[tuple(cls._invocation_classes)], Field(discriminator="type")]
             )
-            cls._typeadapter = TypeAdapter(InvocationsUnion)
+            cls._typeadapter = TypeAdapter(AnyInvocation)
+            cls._typeadapter_needs_update = False
         return cls._typeadapter
 
     @classmethod
@@ -221,7 +228,7 @@ class BaseInvocation(ABC, BaseModel):
         return signature(cls.invoke).return_annotation
 
     @staticmethod
-    def json_schema_extra(schema: dict[str, Any], model_class: Type[BaseModel], *args, **kwargs) -> None:
+    def json_schema_extra(schema: dict[str, Any], model_class: Type[BaseInvocation]) -> None:
         """Adds various UI-facing attributes to the invocation's OpenAPI schema."""
         uiconfig = cast(UIConfigBase | None, getattr(model_class, "UIConfig", None))
         if uiconfig is not None:
@@ -237,6 +244,7 @@ class BaseInvocation(ABC, BaseModel):
             schema["version"] = uiconfig.version
         if "required" not in schema or not isinstance(schema["required"], list):
             schema["required"] = []
+        schema["class"] = "invocation"
         schema["required"].extend(["type", "id"])
 
     @abstractmethod
@@ -310,7 +318,7 @@ class BaseInvocation(ABC, BaseModel):
         protected_namespaces=(),
         validate_assignment=True,
         json_schema_extra=json_schema_extra,
-        json_schema_serialization_defaults_required=True,
+        json_schema_serialization_defaults_required=False,
         coerce_numbers_to_str=True,
     )
 

invokeai/app/invocations/blend_latents.py

@@ -0,0 +1,98 @@
+from typing import Any, Union
+
+import numpy as np
+import numpy.typing as npt
+import torch
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, LatentsField
+from invokeai.app.invocations.primitives import LatentsOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.util.devices import TorchDevice
+
+
+@invocation(
+    "lblend",
+    title="Blend Latents",
+    tags=["latents", "blend"],
+    category="latents",
+    version="1.0.3",
+)
+class BlendLatentsInvocation(BaseInvocation):
+    """Blend two latents using a given alpha. Latents must have same size."""
+
+    latents_a: LatentsField = InputField(
+        description=FieldDescriptions.latents,
+        input=Input.Connection,
+    )
+    latents_b: LatentsField = InputField(
+        description=FieldDescriptions.latents,
+        input=Input.Connection,
+    )
+    alpha: float = InputField(default=0.5, description=FieldDescriptions.blend_alpha)
+
+    def invoke(self, context: InvocationContext) -> LatentsOutput:
+        latents_a = context.tensors.load(self.latents_a.latents_name)
+        latents_b = context.tensors.load(self.latents_b.latents_name)
+
+        if latents_a.shape != latents_b.shape:
+            raise Exception("Latents to blend must be the same size.")
+
+        device = TorchDevice.choose_torch_device()
+
+        def slerp(
+            t: Union[float, npt.NDArray[Any]],  # FIXME: maybe use np.float32 here?
+            v0: Union[torch.Tensor, npt.NDArray[Any]],
+            v1: Union[torch.Tensor, npt.NDArray[Any]],
+            DOT_THRESHOLD: float = 0.9995,
+        ) -> Union[torch.Tensor, npt.NDArray[Any]]:
+            """
+            Spherical linear interpolation
+            Args:
+                t (float/np.ndarray): Float value between 0.0 and 1.0
+                v0 (np.ndarray): Starting vector
+                v1 (np.ndarray): Final vector
+                DOT_THRESHOLD (float): Threshold for considering the two vectors as
+                                    colineal. Not recommended to alter this.
+            Returns:
+                v2 (np.ndarray): Interpolation vector between v0 and v1
+            """
+            inputs_are_torch = False
+            if not isinstance(v0, np.ndarray):
+                inputs_are_torch = True
+                v0 = v0.detach().cpu().numpy()
+            if not isinstance(v1, np.ndarray):
+                inputs_are_torch = True
+                v1 = v1.detach().cpu().numpy()
+
+            dot = np.sum(v0 * v1 / (np.linalg.norm(v0) * np.linalg.norm(v1)))
+            if np.abs(dot) > DOT_THRESHOLD:
+                v2 = (1 - t) * v0 + t * v1
+            else:
+                theta_0 = np.arccos(dot)
+                sin_theta_0 = np.sin(theta_0)
+                theta_t = theta_0 * t
+                sin_theta_t = np.sin(theta_t)
+                s0 = np.sin(theta_0 - theta_t) / sin_theta_0
+                s1 = sin_theta_t / sin_theta_0
+                v2 = s0 * v0 + s1 * v1
+
+            if inputs_are_torch:
+                v2_torch: torch.Tensor = torch.from_numpy(v2).to(device)
+                return v2_torch
+            else:
+                assert isinstance(v2, np.ndarray)
+                return v2
+
+        # blend
+        bl = slerp(self.alpha, latents_a, latents_b)
+        assert isinstance(bl, torch.Tensor)
+        blended_latents: torch.Tensor = bl  # for type checking convenience
+
+        # https://discuss.huggingface.co/t/memory-usage-by-later-pipeline-stages/23699
+        blended_latents = blended_latents.to("cpu")
+
+        TorchDevice.empty_cache()
+
+        name = context.tensors.save(tensor=blended_latents)
+        return LatentsOutput.build(latents_name=name, latents=blended_latents, seed=self.latents_a.seed)

invokeai/app/invocations/compel.py

@@ -65,11 +65,7 @@ class CompelInvocation(BaseInvocation):
     @torch.no_grad()
     def invoke(self, context: InvocationContext) -> ConditioningOutput:
         tokenizer_info = context.models.load(self.clip.tokenizer)
-        tokenizer_model = tokenizer_info.model
-        assert isinstance(tokenizer_model, CLIPTokenizer)
         text_encoder_info = context.models.load(self.clip.text_encoder)
-        text_encoder_model = text_encoder_info.model
-        assert isinstance(text_encoder_model, CLIPTextModel)
 
         def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
             for lora in self.clip.loras:
@@ -84,19 +80,21 @@ class CompelInvocation(BaseInvocation):
         ti_list = generate_ti_list(self.prompt, text_encoder_info.config.base, context)
 
         with (
-            ModelPatcher.apply_ti(tokenizer_model, text_encoder_model, ti_list) as (
-                tokenizer,
-                ti_manager,
-            ),
+            # apply all patches while the model is on the target device
             text_encoder_info as text_encoder,
-            # Apply the LoRA after text_encoder has been moved to its target device for faster patching.
+            tokenizer_info as tokenizer,
             ModelPatcher.apply_lora_text_encoder(text_encoder, _lora_loader()),
             # Apply CLIP Skip after LoRA to prevent LoRA application from failing on skipped layers.
-            ModelPatcher.apply_clip_skip(text_encoder_model, self.clip.skipped_layers),
+            ModelPatcher.apply_clip_skip(text_encoder, self.clip.skipped_layers),
+            ModelPatcher.apply_ti(tokenizer, text_encoder, ti_list) as (
+                patched_tokenizer,
+                ti_manager,
+            ),
         ):
             assert isinstance(text_encoder, CLIPTextModel)
+            assert isinstance(tokenizer, CLIPTokenizer)
             compel = Compel(
-                tokenizer=tokenizer,
+                tokenizer=patched_tokenizer,
                 text_encoder=text_encoder,
                 textual_inversion_manager=ti_manager,
                 dtype_for_device_getter=TorchDevice.choose_torch_dtype,
@@ -106,7 +104,7 @@ class CompelInvocation(BaseInvocation):
             conjunction = Compel.parse_prompt_string(self.prompt)
 
             if context.config.get().log_tokenization:
-                log_tokenization_for_conjunction(conjunction, tokenizer)
+                log_tokenization_for_conjunction(conjunction, patched_tokenizer)
 
             c, _options = compel.build_conditioning_tensor_for_conjunction(conjunction)
 
@@ -136,11 +134,7 @@ class SDXLPromptInvocationBase:
         zero_on_empty: bool,
     ) -> Tuple[torch.Tensor, Optional[torch.Tensor]]:
         tokenizer_info = context.models.load(clip_field.tokenizer)
-        tokenizer_model = tokenizer_info.model
-        assert isinstance(tokenizer_model, CLIPTokenizer)
         text_encoder_info = context.models.load(clip_field.text_encoder)
-        text_encoder_model = text_encoder_info.model
-        assert isinstance(text_encoder_model, (CLIPTextModel, CLIPTextModelWithProjection))
 
         # return zero on empty
         if prompt == "" and zero_on_empty:
@@ -177,20 +171,23 @@ class SDXLPromptInvocationBase:
         ti_list = generate_ti_list(prompt, text_encoder_info.config.base, context)
 
         with (
-            ModelPatcher.apply_ti(tokenizer_model, text_encoder_model, ti_list) as (
-                tokenizer,
-                ti_manager,
-            ),
+            # apply all patches while the model is on the target device
             text_encoder_info as text_encoder,
-            # Apply the LoRA after text_encoder has been moved to its target device for faster patching.
+            tokenizer_info as tokenizer,
             ModelPatcher.apply_lora(text_encoder, _lora_loader(), lora_prefix),
             # Apply CLIP Skip after LoRA to prevent LoRA application from failing on skipped layers.
-            ModelPatcher.apply_clip_skip(text_encoder_model, clip_field.skipped_layers),
+            ModelPatcher.apply_clip_skip(text_encoder, clip_field.skipped_layers),
+            ModelPatcher.apply_ti(tokenizer, text_encoder, ti_list) as (
+                patched_tokenizer,
+                ti_manager,
+            ),
         ):
             assert isinstance(text_encoder, (CLIPTextModel, CLIPTextModelWithProjection))
+            assert isinstance(tokenizer, CLIPTokenizer)
+
             text_encoder = cast(CLIPTextModel, text_encoder)
             compel = Compel(
-                tokenizer=tokenizer,
+                tokenizer=patched_tokenizer,
                 text_encoder=text_encoder,
                 textual_inversion_manager=ti_manager,
                 dtype_for_device_getter=TorchDevice.choose_torch_dtype,
@@ -203,7 +200,7 @@ class SDXLPromptInvocationBase:
 
             if context.config.get().log_tokenization:
                 # TODO: better logging for and syntax
-                log_tokenization_for_conjunction(conjunction, tokenizer)
+                log_tokenization_for_conjunction(conjunction, patched_tokenizer)
 
             # TODO: ask for optimizations? to not run text_encoder twice
             c, _options = compel.build_conditioning_tensor_for_conjunction(conjunction)

invokeai/app/invocations/constants.py

@@ -1,6 +1,7 @@
 from typing import Literal
 
 from invokeai.backend.stable_diffusion.schedulers import SCHEDULER_MAP
+from invokeai.backend.util.devices import TorchDevice
 
 LATENT_SCALE_FACTOR = 8
 """
@@ -15,3 +16,5 @@ SCHEDULER_NAME_VALUES = Literal[tuple(SCHEDULER_MAP.keys())]
 
 IMAGE_MODES = Literal["L", "RGB", "RGBA", "CMYK", "YCbCr", "LAB", "HSV", "I", "F"]
 """A literal type for PIL image modes supported by Invoke"""
+
+DEFAULT_PRECISION = TorchDevice.choose_torch_dtype()

invokeai/app/invocations/controlnet_image_processors.py

@@ -24,7 +24,6 @@ from pydantic import BaseModel, Field, field_validator, model_validator
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     ImageField,
-    Input,
     InputField,
     OutputField,
     UIType,
@@ -35,15 +34,16 @@ from invokeai.app.invocations.model import ModelIdentifierField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.invocations.util import validate_begin_end_step, validate_weights
 from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES
+from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES, heuristic_resize
 from invokeai.backend.image_util.canny import get_canny_edges
 from invokeai.backend.image_util.depth_anything import DepthAnythingDetector
 from invokeai.backend.image_util.dw_openpose import DWOpenposeDetector
 from invokeai.backend.image_util.hed import HEDProcessor
 from invokeai.backend.image_util.lineart import LineartProcessor
 from invokeai.backend.image_util.lineart_anime import LineartAnimeProcessor
+from invokeai.backend.image_util.util import np_to_pil, pil_to_np
 
-from .baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
+from .baseinvocation import BaseInvocation, BaseInvocationOutput, Classification, invocation, invocation_output
 
 
 class ControlField(BaseModel):
@@ -79,13 +79,13 @@ class ControlOutput(BaseInvocationOutput):
     control: ControlField = OutputField(description=FieldDescriptions.control)
 
 
-@invocation("controlnet", title="ControlNet", tags=["controlnet"], category="controlnet", version="1.1.1")
+@invocation("controlnet", title="ControlNet", tags=["controlnet"], category="controlnet", version="1.1.2")
 class ControlNetInvocation(BaseInvocation):
     """Collects ControlNet info to pass to other nodes"""
 
     image: ImageField = InputField(description="The control image")
     control_model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.controlnet_model, input=Input.Direct, ui_type=UIType.ControlNetModel
+        description=FieldDescriptions.controlnet_model, ui_type=UIType.ControlNetModel
     )
     control_weight: Union[float, List[float]] = InputField(
         default=1.0, ge=-1, le=2, description="The weight given to the ControlNet"
@@ -164,13 +164,13 @@ class ImageProcessorInvocation(BaseInvocation, WithMetadata, WithBoard):
     title="Canny Processor",
     tags=["controlnet", "canny"],
     category="controlnet",
-    version="1.3.2",
+    version="1.3.3",
 )
 class CannyImageProcessorInvocation(ImageProcessorInvocation):
     """Canny edge detection for ControlNet"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
     low_threshold: int = InputField(
         default=100, ge=0, le=255, description="The low threshold of the Canny pixel gradient (0-255)"
     )
@@ -198,13 +198,13 @@ class CannyImageProcessorInvocation(ImageProcessorInvocation):
     title="HED (softedge) Processor",
     tags=["controlnet", "hed", "softedge"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class HedImageProcessorInvocation(ImageProcessorInvocation):
     """Applies HED edge detection to image"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
     # safe not supported in controlnet_aux v0.0.3
     # safe: bool = InputField(default=False, description=FieldDescriptions.safe_mode)
     scribble: bool = InputField(default=False, description=FieldDescriptions.scribble_mode)
@@ -227,13 +227,13 @@ class HedImageProcessorInvocation(ImageProcessorInvocation):
     title="Lineart Processor",
     tags=["controlnet", "lineart"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class LineartImageProcessorInvocation(ImageProcessorInvocation):
     """Applies line art processing to image"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
     coarse: bool = InputField(default=False, description="Whether to use coarse mode")
 
     def run_processor(self, image: Image.Image) -> Image.Image:
@@ -249,13 +249,13 @@ class LineartImageProcessorInvocation(ImageProcessorInvocation):
     title="Lineart Anime Processor",
     tags=["controlnet", "lineart", "anime"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class LineartAnimeImageProcessorInvocation(ImageProcessorInvocation):
     """Applies line art anime processing to image"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
     def run_processor(self, image: Image.Image) -> Image.Image:
         processor = LineartAnimeProcessor()
@@ -272,15 +272,15 @@ class LineartAnimeImageProcessorInvocation(ImageProcessorInvocation):
     title="Midas Depth Processor",
     tags=["controlnet", "midas"],
     category="controlnet",
-    version="1.2.3",
+    version="1.2.4",
 )
 class MidasDepthImageProcessorInvocation(ImageProcessorInvocation):
     """Applies Midas depth processing to image"""
 
     a_mult: float = InputField(default=2.0, ge=0, description="Midas parameter `a_mult` (a = a_mult * PI)")
     bg_th: float = InputField(default=0.1, ge=0, description="Midas parameter `bg_th`")
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
     # depth_and_normal not supported in controlnet_aux v0.0.3
     # depth_and_normal: bool = InputField(default=False, description="whether to use depth and normal mode")
 
@@ -303,13 +303,13 @@ class MidasDepthImageProcessorInvocation(ImageProcessorInvocation):
     title="Normal BAE Processor",
     tags=["controlnet"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class NormalbaeImageProcessorInvocation(ImageProcessorInvocation):
     """Applies NormalBae processing to image"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
     def run_processor(self, image):
         normalbae_processor = NormalBaeDetector.from_pretrained("lllyasviel/Annotators")
@@ -320,13 +320,13 @@ class NormalbaeImageProcessorInvocation(ImageProcessorInvocation):
 
 
 @invocation(
-    "mlsd_image_processor", title="MLSD Processor", tags=["controlnet", "mlsd"], category="controlnet", version="1.2.2"
+    "mlsd_image_processor", title="MLSD Processor", tags=["controlnet", "mlsd"], category="controlnet", version="1.2.3"
 )
 class MlsdImageProcessorInvocation(ImageProcessorInvocation):
     """Applies MLSD processing to image"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
     thr_v: float = InputField(default=0.1, ge=0, description="MLSD parameter `thr_v`")
     thr_d: float = InputField(default=0.1, ge=0, description="MLSD parameter `thr_d`")
 
@@ -343,13 +343,13 @@ class MlsdImageProcessorInvocation(ImageProcessorInvocation):
 
 
 @invocation(
-    "pidi_image_processor", title="PIDI Processor", tags=["controlnet", "pidi"], category="controlnet", version="1.2.2"
+    "pidi_image_processor", title="PIDI Processor", tags=["controlnet", "pidi"], category="controlnet", version="1.2.3"
 )
 class PidiImageProcessorInvocation(ImageProcessorInvocation):
     """Applies PIDI processing to image"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
     safe: bool = InputField(default=False, description=FieldDescriptions.safe_mode)
     scribble: bool = InputField(default=False, description=FieldDescriptions.scribble_mode)
 
@@ -370,13 +370,13 @@ class PidiImageProcessorInvocation(ImageProcessorInvocation):
     title="Content Shuffle Processor",
     tags=["controlnet", "contentshuffle"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class ContentShuffleImageProcessorInvocation(ImageProcessorInvocation):
     """Applies content shuffle processing to image"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
     h: int = InputField(default=512, ge=0, description="Content shuffle `h` parameter")
     w: int = InputField(default=512, ge=0, description="Content shuffle `w` parameter")
     f: int = InputField(default=256, ge=0, description="Content shuffle `f` parameter")
@@ -400,7 +400,7 @@ class ContentShuffleImageProcessorInvocation(ImageProcessorInvocation):
     title="Zoe (Depth) Processor",
     tags=["controlnet", "zoe", "depth"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class ZoeDepthImageProcessorInvocation(ImageProcessorInvocation):
     """Applies Zoe depth processing to image"""
@@ -416,15 +416,15 @@ class ZoeDepthImageProcessorInvocation(ImageProcessorInvocation):
     title="Mediapipe Face Processor",
     tags=["controlnet", "mediapipe", "face"],
     category="controlnet",
-    version="1.2.3",
+    version="1.2.4",
 )
 class MediapipeFaceProcessorInvocation(ImageProcessorInvocation):
     """Applies mediapipe face processing to image"""
 
     max_faces: int = InputField(default=1, ge=1, description="Maximum number of faces to detect")
     min_confidence: float = InputField(default=0.5, ge=0, le=1, description="Minimum confidence for face detection")
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
     def run_processor(self, image):
         mediapipe_face_processor = MediapipeFaceDetector()
@@ -443,7 +443,7 @@ class MediapipeFaceProcessorInvocation(ImageProcessorInvocation):
     title="Leres (Depth) Processor",
     tags=["controlnet", "leres", "depth"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class LeresImageProcessorInvocation(ImageProcessorInvocation):
     """Applies leres processing to image"""
@@ -451,8 +451,8 @@ class LeresImageProcessorInvocation(ImageProcessorInvocation):
     thr_a: float = InputField(default=0, description="Leres parameter `thr_a`")
     thr_b: float = InputField(default=0, description="Leres parameter `thr_b`")
     boost: bool = InputField(default=False, description="Whether to use boost mode")
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
     def run_processor(self, image):
         leres_processor = LeresDetector.from_pretrained("lllyasviel/Annotators")
@@ -472,7 +472,7 @@ class LeresImageProcessorInvocation(ImageProcessorInvocation):
     title="Tile Resample Processor",
     tags=["controlnet", "tile"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class TileResamplerProcessorInvocation(ImageProcessorInvocation):
     """Tile resampler processor"""
@@ -512,13 +512,13 @@ class TileResamplerProcessorInvocation(ImageProcessorInvocation):
     title="Segment Anything Processor",
     tags=["controlnet", "segmentanything"],
     category="controlnet",
-    version="1.2.3",
+    version="1.2.4",
 )
 class SegmentAnythingProcessorInvocation(ImageProcessorInvocation):
     """Applies segment anything processing to image"""
 
-    detect_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.detect_res)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
     def run_processor(self, image):
         # segment_anything_processor = SamDetector.from_pretrained("ybelkada/segment-anything", subfolder="checkpoints")
@@ -559,12 +559,12 @@ class SamDetectorReproducibleColors(SamDetector):
     title="Color Map Processor",
     tags=["controlnet"],
     category="controlnet",
-    version="1.2.2",
+    version="1.2.3",
 )
 class ColorMapImageProcessorInvocation(ImageProcessorInvocation):
     """Generates a color map from the provided image"""
 
-    color_map_tile_size: int = InputField(default=64, ge=0, description=FieldDescriptions.tile_size)
+    color_map_tile_size: int = InputField(default=64, ge=1, description=FieldDescriptions.tile_size)
 
     def run_processor(self, image: Image.Image):
         np_image = np.array(image, dtype=np.uint8)
@@ -591,7 +591,7 @@ DEPTH_ANYTHING_MODEL_SIZES = Literal["large", "base", "small"]
     title="Depth Anything Processor",
     tags=["controlnet", "depth", "depth anything"],
     category="controlnet",
-    version="1.1.1",
+    version="1.1.2",
 )
 class DepthAnythingImageProcessorInvocation(ImageProcessorInvocation):
     """Generates a depth map based on the Depth Anything algorithm"""
@@ -599,7 +599,7 @@ class DepthAnythingImageProcessorInvocation(ImageProcessorInvocation):
     model_size: DEPTH_ANYTHING_MODEL_SIZES = InputField(
         default="small", description="The size of the depth model to use"
     )
-    resolution: int = InputField(default=512, ge=64, multiple_of=64, description=FieldDescriptions.image_res)
+    resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
     def run_processor(self, image: Image.Image):
         depth_anything_detector = DepthAnythingDetector()
@@ -614,7 +614,7 @@ class DepthAnythingImageProcessorInvocation(ImageProcessorInvocation):
     title="DW Openpose Image Processor",
     tags=["controlnet", "dwpose", "openpose"],
     category="controlnet",
-    version="1.1.0",
+    version="1.1.1",
 )
 class DWOpenposeImageProcessorInvocation(ImageProcessorInvocation):
     """Generates an openpose pose from an image using DWPose"""
@@ -622,7 +622,7 @@ class DWOpenposeImageProcessorInvocation(ImageProcessorInvocation):
     draw_body: bool = InputField(default=True)
     draw_face: bool = InputField(default=False)
     draw_hands: bool = InputField(default=False)
-    image_resolution: int = InputField(default=512, ge=0, description=FieldDescriptions.image_res)
+    image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
     def run_processor(self, image: Image.Image):
         dw_openpose = DWOpenposeDetector()
@@ -634,3 +634,27 @@ class DWOpenposeImageProcessorInvocation(ImageProcessorInvocation):
             resolution=self.image_resolution,
         )
         return processed_image
+
+
+@invocation(
+    "heuristic_resize",
+    title="Heuristic Resize",
+    tags=["image, controlnet"],
+    category="image",
+    version="1.0.1",
+    classification=Classification.Prototype,
+)
+class HeuristicResizeInvocation(BaseInvocation):
+    """Resize an image using a heuristic method. Preserves edge maps."""
+
+    image: ImageField = InputField(description="The image to resize")
+    width: int = InputField(default=512, ge=1, description="The width to resize to (px)")
+    height: int = InputField(default=512, ge=1, description="The height to resize to (px)")
+
+    def invoke(self, context: InvocationContext) -> ImageOutput:
+        image = context.images.get_pil(self.image.image_name, "RGB")
+        np_img = pil_to_np(image)
+        np_resized = heuristic_resize(np_img, (self.width, self.height))
+        resized = np_to_pil(np_resized)
+        image_dto = context.images.save(image=resized)
+        return ImageOutput.build(image_dto)

invokeai/app/invocations/create_denoise_mask.py

@@ -0,0 +1,80 @@
+from typing import Optional
+
+import torch
+import torchvision.transforms as T
+from PIL import Image
+from torchvision.transforms.functional import resize as tv_resize
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import DEFAULT_PRECISION
+from invokeai.app.invocations.fields import FieldDescriptions, ImageField, Input, InputField
+from invokeai.app.invocations.image_to_latents import ImageToLatentsInvocation
+from invokeai.app.invocations.model import VAEField
+from invokeai.app.invocations.primitives import DenoiseMaskOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.stable_diffusion.diffusers_pipeline import image_resized_to_grid_as_tensor
+
+
+@invocation(
+    "create_denoise_mask",
+    title="Create Denoise Mask",
+    tags=["mask", "denoise"],
+    category="latents",
+    version="1.0.2",
+)
+class CreateDenoiseMaskInvocation(BaseInvocation):
+    """Creates mask for denoising model run."""
+
+    vae: VAEField = InputField(description=FieldDescriptions.vae, input=Input.Connection, ui_order=0)
+    image: Optional[ImageField] = InputField(default=None, description="Image which will be masked", ui_order=1)
+    mask: ImageField = InputField(description="The mask to use when pasting", ui_order=2)
+    tiled: bool = InputField(default=False, description=FieldDescriptions.tiled, ui_order=3)
+    fp32: bool = InputField(
+        default=DEFAULT_PRECISION == torch.float32,
+        description=FieldDescriptions.fp32,
+        ui_order=4,
+    )
+
+    def prep_mask_tensor(self, mask_image: Image.Image) -> torch.Tensor:
+        if mask_image.mode != "L":
+            mask_image = mask_image.convert("L")
+        mask_tensor: torch.Tensor = image_resized_to_grid_as_tensor(mask_image, normalize=False)
+        if mask_tensor.dim() == 3:
+            mask_tensor = mask_tensor.unsqueeze(0)
+        # if shape is not None:
+        #    mask_tensor = tv_resize(mask_tensor, shape, T.InterpolationMode.BILINEAR)
+        return mask_tensor
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> DenoiseMaskOutput:
+        if self.image is not None:
+            image = context.images.get_pil(self.image.image_name)
+            image_tensor = image_resized_to_grid_as_tensor(image.convert("RGB"))
+            if image_tensor.dim() == 3:
+                image_tensor = image_tensor.unsqueeze(0)
+        else:
+            image_tensor = None
+
+        mask = self.prep_mask_tensor(
+            context.images.get_pil(self.mask.image_name),
+        )
+
+        if image_tensor is not None:
+            vae_info = context.models.load(self.vae.vae)
+
+            img_mask = tv_resize(mask, image_tensor.shape[-2:], T.InterpolationMode.BILINEAR, antialias=False)
+            masked_image = image_tensor * torch.where(img_mask < 0.5, 0.0, 1.0)
+            # TODO:
+            masked_latents = ImageToLatentsInvocation.vae_encode(vae_info, self.fp32, self.tiled, masked_image.clone())
+
+            masked_latents_name = context.tensors.save(tensor=masked_latents)
+        else:
+            masked_latents_name = None
+
+        mask_name = context.tensors.save(tensor=mask)
+
+        return DenoiseMaskOutput.build(
+            mask_name=mask_name,
+            masked_latents_name=masked_latents_name,
+            gradient=False,
+        )

invokeai/app/invocations/create_gradient_mask.py

@@ -0,0 +1,138 @@
+from typing import Literal, Optional
+
+import numpy as np
+import torch
+import torchvision.transforms as T
+from PIL import Image, ImageFilter
+from torchvision.transforms.functional import resize as tv_resize
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
+from invokeai.app.invocations.constants import DEFAULT_PRECISION
+from invokeai.app.invocations.fields import (
+    DenoiseMaskField,
+    FieldDescriptions,
+    ImageField,
+    Input,
+    InputField,
+    OutputField,
+)
+from invokeai.app.invocations.image_to_latents import ImageToLatentsInvocation
+from invokeai.app.invocations.model import UNetField, VAEField
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.model_manager import LoadedModel
+from invokeai.backend.model_manager.config import MainConfigBase, ModelVariantType
+from invokeai.backend.stable_diffusion.diffusers_pipeline import image_resized_to_grid_as_tensor
+
+
+@invocation_output("gradient_mask_output")
+class GradientMaskOutput(BaseInvocationOutput):
+    """Outputs a denoise mask and an image representing the total gradient of the mask."""
+
+    denoise_mask: DenoiseMaskField = OutputField(description="Mask for denoise model run")
+    expanded_mask_area: ImageField = OutputField(
+        description="Image representing the total gradient area of the mask. For paste-back purposes."
+    )
+
+
+@invocation(
+    "create_gradient_mask",
+    title="Create Gradient Mask",
+    tags=["mask", "denoise"],
+    category="latents",
+    version="1.1.0",
+)
+class CreateGradientMaskInvocation(BaseInvocation):
+    """Creates mask for denoising model run."""
+
+    mask: ImageField = InputField(default=None, description="Image which will be masked", ui_order=1)
+    edge_radius: int = InputField(
+        default=16, ge=0, description="How far to blur/expand the edges of the mask", ui_order=2
+    )
+    coherence_mode: Literal["Gaussian Blur", "Box Blur", "Staged"] = InputField(default="Gaussian Blur", ui_order=3)
+    minimum_denoise: float = InputField(
+        default=0.0, ge=0, le=1, description="Minimum denoise level for the coherence region", ui_order=4
+    )
+    image: Optional[ImageField] = InputField(
+        default=None,
+        description="OPTIONAL: Only connect for specialized Inpainting models, masked_latents will be generated from the image with the VAE",
+        title="[OPTIONAL] Image",
+        ui_order=6,
+    )
+    unet: Optional[UNetField] = InputField(
+        description="OPTIONAL: If the Unet is a specialized Inpainting model, masked_latents will be generated from the image with the VAE",
+        default=None,
+        input=Input.Connection,
+        title="[OPTIONAL] UNet",
+        ui_order=5,
+    )
+    vae: Optional[VAEField] = InputField(
+        default=None,
+        description="OPTIONAL: Only connect for specialized Inpainting models, masked_latents will be generated from the image with the VAE",
+        title="[OPTIONAL] VAE",
+        input=Input.Connection,
+        ui_order=7,
+    )
+    tiled: bool = InputField(default=False, description=FieldDescriptions.tiled, ui_order=8)
+    fp32: bool = InputField(
+        default=DEFAULT_PRECISION == torch.float32,
+        description=FieldDescriptions.fp32,
+        ui_order=9,
+    )
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> GradientMaskOutput:
+        mask_image = context.images.get_pil(self.mask.image_name, mode="L")
+        if self.edge_radius > 0:
+            if self.coherence_mode == "Box Blur":
+                blur_mask = mask_image.filter(ImageFilter.BoxBlur(self.edge_radius))
+            else:  # Gaussian Blur OR Staged
+                # Gaussian Blur uses standard deviation. 1/2 radius is a good approximation
+                blur_mask = mask_image.filter(ImageFilter.GaussianBlur(self.edge_radius / 2))
+
+            blur_tensor: torch.Tensor = image_resized_to_grid_as_tensor(blur_mask, normalize=False)
+
+            # redistribute blur so that the original edges are 0 and blur outwards to 1
+            blur_tensor = (blur_tensor - 0.5) * 2
+
+            threshold = 1 - self.minimum_denoise
+
+            if self.coherence_mode == "Staged":
+                # wherever the blur_tensor is less than fully masked, convert it to threshold
+                blur_tensor = torch.where((blur_tensor < 1) & (blur_tensor > 0), threshold, blur_tensor)
+            else:
+                # wherever the blur_tensor is above threshold but less than 1, drop it to threshold
+                blur_tensor = torch.where((blur_tensor > threshold) & (blur_tensor < 1), threshold, blur_tensor)
+
+        else:
+            blur_tensor: torch.Tensor = image_resized_to_grid_as_tensor(mask_image, normalize=False)
+
+        mask_name = context.tensors.save(tensor=blur_tensor.unsqueeze(1))
+
+        # compute a [0, 1] mask from the blur_tensor
+        expanded_mask = torch.where((blur_tensor < 1), 0, 1)
+        expanded_mask_image = Image.fromarray((expanded_mask.squeeze(0).numpy() * 255).astype(np.uint8), mode="L")
+        expanded_image_dto = context.images.save(expanded_mask_image)
+
+        masked_latents_name = None
+        if self.unet is not None and self.vae is not None and self.image is not None:
+            # all three fields must be present at the same time
+            main_model_config = context.models.get_config(self.unet.unet.key)
+            assert isinstance(main_model_config, MainConfigBase)
+            if main_model_config.variant is ModelVariantType.Inpaint:
+                mask = blur_tensor
+                vae_info: LoadedModel = context.models.load(self.vae.vae)
+                image = context.images.get_pil(self.image.image_name)
+                image_tensor = image_resized_to_grid_as_tensor(image.convert("RGB"))
+                if image_tensor.dim() == 3:
+                    image_tensor = image_tensor.unsqueeze(0)
+                img_mask = tv_resize(mask, image_tensor.shape[-2:], T.InterpolationMode.BILINEAR, antialias=False)
+                masked_image = image_tensor * torch.where(img_mask < 0.5, 0.0, 1.0)
+                masked_latents = ImageToLatentsInvocation.vae_encode(
+                    vae_info, self.fp32, self.tiled, masked_image.clone()
+                )
+                masked_latents_name = context.tensors.save(tensor=masked_latents)
+
+        return GradientMaskOutput(
+            denoise_mask=DenoiseMaskField(mask_name=mask_name, masked_latents_name=masked_latents_name, gradient=True),
+            expanded_mask_area=ImageField(image_name=expanded_image_dto.image_name),
+        )

invokeai/app/invocations/crop_latents.py

@@ -0,0 +1,61 @@
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
+from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, LatentsField
+from invokeai.app.invocations.primitives import LatentsOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+
+
+# The Crop Latents node was copied from @skunkworxdark's implementation here:
+# https://github.com/skunkworxdark/XYGrid_nodes/blob/74647fa9c1fa57d317a94bd43ca689af7f0aae5e/images_to_grids.py#L1117C1-L1167C80
+@invocation(
+    "crop_latents",
+    title="Crop Latents",
+    tags=["latents", "crop"],
+    category="latents",
+    version="1.0.2",
+)
+# TODO(ryand): Named `CropLatentsCoreInvocation` to prevent a conflict with custom node `CropLatentsInvocation`.
+# Currently, if the class names conflict then 'GET /openapi.json' fails.
+class CropLatentsCoreInvocation(BaseInvocation):
+    """Crops a latent-space tensor to a box specified in image-space. The box dimensions and coordinates must be
+    divisible by the latent scale factor of 8.
+    """
+
+    latents: LatentsField = InputField(
+        description=FieldDescriptions.latents,
+        input=Input.Connection,
+    )
+    x: int = InputField(
+        ge=0,
+        multiple_of=LATENT_SCALE_FACTOR,
+        description="The left x coordinate (in px) of the crop rectangle in image space. This value will be converted to a dimension in latent space.",
+    )
+    y: int = InputField(
+        ge=0,
+        multiple_of=LATENT_SCALE_FACTOR,
+        description="The top y coordinate (in px) of the crop rectangle in image space. This value will be converted to a dimension in latent space.",
+    )
+    width: int = InputField(
+        ge=1,
+        multiple_of=LATENT_SCALE_FACTOR,
+        description="The width (in px) of the crop rectangle in image space. This value will be converted to a dimension in latent space.",
+    )
+    height: int = InputField(
+        ge=1,
+        multiple_of=LATENT_SCALE_FACTOR,
+        description="The height (in px) of the crop rectangle in image space. This value will be converted to a dimension in latent space.",
+    )
+
+    def invoke(self, context: InvocationContext) -> LatentsOutput:
+        latents = context.tensors.load(self.latents.latents_name)
+
+        x1 = self.x // LATENT_SCALE_FACTOR
+        y1 = self.y // LATENT_SCALE_FACTOR
+        x2 = x1 + (self.width // LATENT_SCALE_FACTOR)
+        y2 = y1 + (self.height // LATENT_SCALE_FACTOR)
+
+        cropped_latents = latents[..., y1:y2, x1:x2]
+
+        name = context.tensors.save(tensor=cropped_latents)
+
+        return LatentsOutput.build(latents_name=name, latents=cropped_latents)

invokeai/app/invocations/ideal_size.py

@@ -0,0 +1,65 @@
+import math
+from typing import Tuple
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
+from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
+from invokeai.app.invocations.fields import FieldDescriptions, InputField, OutputField
+from invokeai.app.invocations.model import UNetField
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.model_manager.config import BaseModelType
+
+
+@invocation_output("ideal_size_output")
+class IdealSizeOutput(BaseInvocationOutput):
+    """Base class for invocations that output an image"""
+
+    width: int = OutputField(description="The ideal width of the image (in pixels)")
+    height: int = OutputField(description="The ideal height of the image (in pixels)")
+
+
+@invocation(
+    "ideal_size",
+    title="Ideal Size",
+    tags=["latents", "math", "ideal_size"],
+    version="1.0.3",
+)
+class IdealSizeInvocation(BaseInvocation):
+    """Calculates the ideal size for generation to avoid duplication"""
+
+    width: int = InputField(default=1024, description="Final image width")
+    height: int = InputField(default=576, description="Final image height")
+    unet: UNetField = InputField(default=None, description=FieldDescriptions.unet)
+    multiplier: float = InputField(
+        default=1.0,
+        description="Amount to multiply the model's dimensions by when calculating the ideal size (may result in "
+        "initial generation artifacts if too large)",
+    )
+
+    def trim_to_multiple_of(self, *args: int, multiple_of: int = LATENT_SCALE_FACTOR) -> Tuple[int, ...]:
+        return tuple((x - x % multiple_of) for x in args)
+
+    def invoke(self, context: InvocationContext) -> IdealSizeOutput:
+        unet_config = context.models.get_config(self.unet.unet.key)
+        aspect = self.width / self.height
+        dimension: float = 512
+        if unet_config.base == BaseModelType.StableDiffusion2:
+            dimension = 768
+        elif unet_config.base == BaseModelType.StableDiffusionXL:
+            dimension = 1024
+        dimension = dimension * self.multiplier
+        min_dimension = math.floor(dimension * 0.5)
+        model_area = dimension * dimension  # hardcoded for now since all models are trained on square images
+
+        if aspect > 1.0:
+            init_height = max(min_dimension, math.sqrt(model_area / aspect))
+            init_width = init_height * aspect
+        else:
+            init_width = max(min_dimension, math.sqrt(model_area * aspect))
+            init_height = init_width / aspect
+
+        scaled_width, scaled_height = self.trim_to_multiple_of(
+            math.floor(init_width),
+            math.floor(init_height),
+        )
+
+        return IdealSizeOutput(width=scaled_width, height=scaled_height)

invokeai/app/invocations/image.py

@@ -1,6 +1,5 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
 
-from pathlib import Path
 from typing import Literal, Optional
 
 import cv2
@@ -504,7 +503,7 @@ class ImageInverseLerpInvocation(BaseInvocation, WithMetadata, WithBoard):
     title="Blur NSFW Image",
     tags=["image", "nsfw"],
     category="image",
-    version="1.2.2",
+    version="1.2.3",
 )
 class ImageNSFWBlurInvocation(BaseInvocation, WithMetadata, WithBoard):
     """Add blur to NSFW-flagged images"""
@@ -516,23 +515,12 @@ class ImageNSFWBlurInvocation(BaseInvocation, WithMetadata, WithBoard):
 
         logger = context.logger
         logger.debug("Running NSFW checker")
-        if SafetyChecker.has_nsfw_concept(image):
-            logger.info("A potentially NSFW image has been detected. Image will be blurred.")
-            blurry_image = image.filter(filter=ImageFilter.GaussianBlur(radius=32))
-            caution = self._get_caution_img()
-            blurry_image.paste(caution, (0, 0), caution)
-            image = blurry_image
+        image = SafetyChecker.blur_if_nsfw(image)
 
         image_dto = context.images.save(image=image)
 
         return ImageOutput.build(image_dto)
 
-    def _get_caution_img(self) -> Image.Image:
-        import invokeai.app.assets.images as image_assets
-
-        caution = Image.open(Path(image_assets.__path__[0]) / "caution.png")
-        return caution.resize((caution.width // 2, caution.height // 2))
-
 
 @invocation(
     "img_watermark",

invokeai/app/invocations/image_to_latents.py

@@ -0,0 +1,125 @@
+from functools import singledispatchmethod
+
+import einops
+import torch
+from diffusers.models.attention_processor import (
+    AttnProcessor2_0,
+    LoRAAttnProcessor2_0,
+    LoRAXFormersAttnProcessor,
+    XFormersAttnProcessor,
+)
+from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL
+from diffusers.models.autoencoders.autoencoder_tiny import AutoencoderTiny
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import DEFAULT_PRECISION
+from invokeai.app.invocations.fields import (
+    FieldDescriptions,
+    ImageField,
+    Input,
+    InputField,
+)
+from invokeai.app.invocations.model import VAEField
+from invokeai.app.invocations.primitives import LatentsOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.model_manager import LoadedModel
+from invokeai.backend.stable_diffusion.diffusers_pipeline import image_resized_to_grid_as_tensor
+
+
+@invocation(
+    "i2l",
+    title="Image to Latents",
+    tags=["latents", "image", "vae", "i2l"],
+    category="latents",
+    version="1.0.2",
+)
+class ImageToLatentsInvocation(BaseInvocation):
+    """Encodes an image into latents."""
+
+    image: ImageField = InputField(
+        description="The image to encode",
+    )
+    vae: VAEField = InputField(
+        description=FieldDescriptions.vae,
+        input=Input.Connection,
+    )
+    tiled: bool = InputField(default=False, description=FieldDescriptions.tiled)
+    fp32: bool = InputField(default=DEFAULT_PRECISION == torch.float32, description=FieldDescriptions.fp32)
+
+    @staticmethod
+    def vae_encode(vae_info: LoadedModel, upcast: bool, tiled: bool, image_tensor: torch.Tensor) -> torch.Tensor:
+        with vae_info as vae:
+            assert isinstance(vae, torch.nn.Module)
+            orig_dtype = vae.dtype
+            if upcast:
+                vae.to(dtype=torch.float32)
+
+                use_torch_2_0_or_xformers = hasattr(vae.decoder, "mid_block") and isinstance(
+                    vae.decoder.mid_block.attentions[0].processor,
+                    (
+                        AttnProcessor2_0,
+                        XFormersAttnProcessor,
+                        LoRAXFormersAttnProcessor,
+                        LoRAAttnProcessor2_0,
+                    ),
+                )
+                # if xformers or torch_2_0 is used attention block does not need
+                # to be in float32 which can save lots of memory
+                if use_torch_2_0_or_xformers:
+                    vae.post_quant_conv.to(orig_dtype)
+                    vae.decoder.conv_in.to(orig_dtype)
+                    vae.decoder.mid_block.to(orig_dtype)
+                # else:
+                #    latents = latents.float()
+
+            else:
+                vae.to(dtype=torch.float16)
+                # latents = latents.half()
+
+            if tiled:
+                vae.enable_tiling()
+            else:
+                vae.disable_tiling()
+
+            # non_noised_latents_from_image
+            image_tensor = image_tensor.to(device=vae.device, dtype=vae.dtype)
+            with torch.inference_mode():
+                latents = ImageToLatentsInvocation._encode_to_tensor(vae, image_tensor)
+
+            latents = vae.config.scaling_factor * latents
+            latents = latents.to(dtype=orig_dtype)
+
+        return latents
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> LatentsOutput:
+        image = context.images.get_pil(self.image.image_name)
+
+        vae_info = context.models.load(self.vae.vae)
+
+        image_tensor = image_resized_to_grid_as_tensor(image.convert("RGB"))
+        if image_tensor.dim() == 3:
+            image_tensor = einops.rearrange(image_tensor, "c h w -> 1 c h w")
+
+        latents = self.vae_encode(vae_info, self.fp32, self.tiled, image_tensor)
+
+        latents = latents.to("cpu")
+        name = context.tensors.save(tensor=latents)
+        return LatentsOutput.build(latents_name=name, latents=latents, seed=None)
+
+    @singledispatchmethod
+    @staticmethod
+    def _encode_to_tensor(vae: AutoencoderKL, image_tensor: torch.FloatTensor) -> torch.FloatTensor:
+        assert isinstance(vae, torch.nn.Module)
+        image_tensor_dist = vae.encode(image_tensor).latent_dist
+        latents: torch.Tensor = image_tensor_dist.sample().to(
+            dtype=vae.dtype
+        )  # FIXME: uses torch.randn. make reproducible!
+        return latents
+
+    @_encode_to_tensor.register
+    @staticmethod
+    def _(vae: AutoencoderTiny, image_tensor: torch.FloatTensor) -> torch.FloatTensor:
+        assert isinstance(vae, torch.nn.Module)
+        latents: torch.FloatTensor = vae.encode(image_tensor).latents
+        return latents

invokeai/app/invocations/ip_adapter.py

@@ -5,7 +5,7 @@ from pydantic import BaseModel, Field, field_validator, model_validator
 from typing_extensions import Self
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
-from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, TensorField, UIType
+from invokeai.app.invocations.fields import FieldDescriptions, InputField, OutputField, TensorField, UIType
 from invokeai.app.invocations.model import ModelIdentifierField
 from invokeai.app.invocations.primitives import ImageField
 from invokeai.app.invocations.util import validate_begin_end_step, validate_weights
@@ -58,7 +58,7 @@ class IPAdapterOutput(BaseInvocationOutput):
 CLIP_VISION_MODEL_MAP = {"ViT-H": "ip_adapter_sd_image_encoder", "ViT-G": "ip_adapter_sdxl_image_encoder"}
 
 
-@invocation("ip_adapter", title="IP-Adapter", tags=["ip_adapter", "control"], category="ip_adapter", version="1.4.0")
+@invocation("ip_adapter", title="IP-Adapter", tags=["ip_adapter", "control"], category="ip_adapter", version="1.4.1")
 class IPAdapterInvocation(BaseInvocation):
     """Collects IP-Adapter info to pass to other nodes."""
 
@@ -67,7 +67,6 @@ class IPAdapterInvocation(BaseInvocation):
     ip_adapter_model: ModelIdentifierField = InputField(
         description="The IP-Adapter model.",
         title="IP-Adapter Model",
-        input=Input.Direct,
         ui_order=-1,
         ui_type=UIType.IPAdapterModel,
     )

invokeai/app/invocations/latents_to_image.py

@@ -0,0 +1,127 @@
+import torch
+from diffusers.image_processor import VaeImageProcessor
+from diffusers.models.attention_processor import (
+    AttnProcessor2_0,
+    LoRAAttnProcessor2_0,
+    LoRAXFormersAttnProcessor,
+    XFormersAttnProcessor,
+)
+from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL
+from diffusers.models.autoencoders.autoencoder_tiny import AutoencoderTiny
+from PIL import Image
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import DEFAULT_PRECISION
+from invokeai.app.invocations.fields import (
+    FieldDescriptions,
+    Input,
+    InputField,
+    LatentsField,
+    WithBoard,
+    WithMetadata,
+)
+from invokeai.app.invocations.model import VAEField
+from invokeai.app.invocations.primitives import ImageOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.model_manager.load.load_base import LoadedModel
+from invokeai.backend.stable_diffusion import set_seamless
+from invokeai.backend.util.devices import TorchDevice
+
+
+@invocation(
+    "l2i",
+    title="Latents to Image",
+    tags=["latents", "image", "vae", "l2i"],
+    category="latents",
+    version="1.2.2",
+)
+class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
+    """Generates an image from latents."""
+
+    latents: LatentsField = InputField(
+        description=FieldDescriptions.latents,
+        input=Input.Connection,
+    )
+    vae: VAEField = InputField(
+        description=FieldDescriptions.vae,
+        input=Input.Connection,
+    )
+    tiled: bool = InputField(default=False, description=FieldDescriptions.tiled)
+    fp32: bool = InputField(default=DEFAULT_PRECISION == torch.float32, description=FieldDescriptions.fp32)
+
+    @staticmethod
+    def vae_decode(
+        context: InvocationContext,
+        vae_info: LoadedModel,
+        seamless_axes: list[str],
+        latents: torch.Tensor,
+        use_fp32: bool,
+        use_tiling: bool,
+    ) -> Image.Image:
+        assert isinstance(vae_info.model, (AutoencoderKL, AutoencoderTiny))
+        with set_seamless(vae_info.model, seamless_axes), vae_info as vae:
+            assert isinstance(vae, (AutoencoderKL, AutoencoderTiny))
+            latents = latents.to(vae.device)
+            if use_fp32:
+                vae.to(dtype=torch.float32)
+
+                use_torch_2_0_or_xformers = hasattr(vae.decoder, "mid_block") and isinstance(
+                    vae.decoder.mid_block.attentions[0].processor,
+                    (
+                        AttnProcessor2_0,
+                        XFormersAttnProcessor,
+                        LoRAXFormersAttnProcessor,
+                        LoRAAttnProcessor2_0,
+                    ),
+                )
+                # if xformers or torch_2_0 is used attention block does not need
+                # to be in float32 which can save lots of memory
+                if use_torch_2_0_or_xformers:
+                    vae.post_quant_conv.to(latents.dtype)
+                    vae.decoder.conv_in.to(latents.dtype)
+                    vae.decoder.mid_block.to(latents.dtype)
+                else:
+                    latents = latents.float()
+
+            else:
+                vae.to(dtype=torch.float16)
+                latents = latents.half()
+
+            if use_tiling or context.config.get().force_tiled_decode:
+                vae.enable_tiling()
+            else:
+                vae.disable_tiling()
+
+            # clear memory as vae decode can request a lot
+            TorchDevice.empty_cache()
+
+            with torch.inference_mode():
+                # copied from diffusers pipeline
+                latents = latents / vae.config.scaling_factor
+                image = vae.decode(latents, return_dict=False)[0]
+                image = (image / 2 + 0.5).clamp(0, 1)  # denormalize
+                # we always cast to float32 as this does not cause significant overhead and is compatible with bfloat16
+                np_image = image.cpu().permute(0, 2, 3, 1).float().numpy()
+
+                image = VaeImageProcessor.numpy_to_pil(np_image)[0]
+
+        TorchDevice.empty_cache()
+
+        return image
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> ImageOutput:
+        latents = context.tensors.load(self.latents.latents_name)
+        vae_info = context.models.load(self.vae.vae)
+
+        image = self.vae_decode(
+            context=context,
+            vae_info=vae_info,
+            seamless_axes=self.vae.seamless_axes,
+            latents=latents,
+            use_fp32=self.fp32,
+            use_tiling=self.tiled,
+        )
+        image_dto = context.images.save(image=image)
+
+        return ImageOutput.build(image_dto)

invokeai/app/invocations/model.py

@@ -11,6 +11,7 @@ from invokeai.backend.model_manager.config import AnyModelConfig, BaseModelType,
 from .baseinvocation import (
     BaseInvocation,
     BaseInvocationOutput,
+    Classification,
     invocation,
     invocation_output,
 )
@@ -93,19 +94,46 @@ class ModelLoaderOutput(UNetOutput, CLIPOutput, VAEOutput):
     pass
 
 
+@invocation_output("model_identifier_output")
+class ModelIdentifierOutput(BaseInvocationOutput):
+    """Model identifier output"""
+
+    model: ModelIdentifierField = OutputField(description="Model identifier", title="Model")
+
+
+@invocation(
+    "model_identifier",
+    title="Model identifier",
+    tags=["model"],
+    category="model",
+    version="1.0.0",
+    classification=Classification.Prototype,
+)
+class ModelIdentifierInvocation(BaseInvocation):
+    """Selects any model, outputting it its identifier. Be careful with this one! The identifier will be accepted as
+    input for any model, even if the model types don't match. If you connect this to a mismatched input, you'll get an
+    error."""
+
+    model: ModelIdentifierField = InputField(description="The model to select", title="Model")
+
+    def invoke(self, context: InvocationContext) -> ModelIdentifierOutput:
+        if not context.models.exists(self.model.key):
+            raise Exception(f"Unknown model {self.model.key}")
+
+        return ModelIdentifierOutput(model=self.model)
+
+
 @invocation(
     "main_model_loader",
     title="Main Model",
     tags=["model"],
     category="model",
-    version="1.0.2",
+    version="1.0.3",
 )
 class MainModelLoaderInvocation(BaseInvocation):
     """Loads a main model, outputting its submodels."""
 
-    model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.main_model, input=Input.Direct, ui_type=UIType.MainModel
-    )
+    model: ModelIdentifierField = InputField(description=FieldDescriptions.main_model, ui_type=UIType.MainModel)
     # TODO: precision?
 
     def invoke(self, context: InvocationContext) -> ModelLoaderOutput:
@@ -134,12 +162,12 @@ class LoRALoaderOutput(BaseInvocationOutput):
     clip: Optional[CLIPField] = OutputField(default=None, description=FieldDescriptions.clip, title="CLIP")
 
 
-@invocation("lora_loader", title="LoRA", tags=["model"], category="model", version="1.0.2")
+@invocation("lora_loader", title="LoRA", tags=["model"], category="model", version="1.0.3")
 class LoRALoaderInvocation(BaseInvocation):
     """Apply selected lora to unet and text_encoder."""
 
     lora: ModelIdentifierField = InputField(
-        description=FieldDescriptions.lora_model, input=Input.Direct, title="LoRA", ui_type=UIType.LoRAModel
+        description=FieldDescriptions.lora_model, title="LoRA", ui_type=UIType.LoRAModel
     )
     weight: float = InputField(default=0.75, description=FieldDescriptions.lora_weight)
     unet: Optional[UNetField] = InputField(
@@ -190,6 +218,75 @@ class LoRALoaderInvocation(BaseInvocation):
         return output
 
 
+@invocation_output("lora_selector_output")
+class LoRASelectorOutput(BaseInvocationOutput):
+    """Model loader output"""
+
+    lora: LoRAField = OutputField(description="LoRA model and weight", title="LoRA")
+
+
+@invocation("lora_selector", title="LoRA Selector", tags=["model"], category="model", version="1.0.1")
+class LoRASelectorInvocation(BaseInvocation):
+    """Selects a LoRA model and weight."""
+
+    lora: ModelIdentifierField = InputField(
+        description=FieldDescriptions.lora_model, title="LoRA", ui_type=UIType.LoRAModel
+    )
+    weight: float = InputField(default=0.75, description=FieldDescriptions.lora_weight)
+
+    def invoke(self, context: InvocationContext) -> LoRASelectorOutput:
+        return LoRASelectorOutput(lora=LoRAField(lora=self.lora, weight=self.weight))
+
+
+@invocation("lora_collection_loader", title="LoRA Collection Loader", tags=["model"], category="model", version="1.0.0")
+class LoRACollectionLoader(BaseInvocation):
+    """Applies a collection of LoRAs to the provided UNet and CLIP models."""
+
+    loras: LoRAField | list[LoRAField] = InputField(
+        description="LoRA models and weights. May be a single LoRA or collection.", title="LoRAs"
+    )
+    unet: Optional[UNetField] = InputField(
+        default=None,
+        description=FieldDescriptions.unet,
+        input=Input.Connection,
+        title="UNet",
+    )
+    clip: Optional[CLIPField] = InputField(
+        default=None,
+        description=FieldDescriptions.clip,
+        input=Input.Connection,
+        title="CLIP",
+    )
+
+    def invoke(self, context: InvocationContext) -> LoRALoaderOutput:
+        output = LoRALoaderOutput()
+        loras = self.loras if isinstance(self.loras, list) else [self.loras]
+        added_loras: list[str] = []
+
+        for lora in loras:
+            if lora.lora.key in added_loras:
+                continue
+
+            if not context.models.exists(lora.lora.key):
+                raise Exception(f"Unknown lora: {lora.lora.key}!")
+
+            assert lora.lora.base in (BaseModelType.StableDiffusion1, BaseModelType.StableDiffusion2)
+
+            added_loras.append(lora.lora.key)
+
+            if self.unet is not None:
+                if output.unet is None:
+                    output.unet = self.unet.model_copy(deep=True)
+                output.unet.loras.append(lora)
+
+            if self.clip is not None:
+                if output.clip is None:
+                    output.clip = self.clip.model_copy(deep=True)
+                output.clip.loras.append(lora)
+
+        return output
+
+
 @invocation_output("sdxl_lora_loader_output")
 class SDXLLoRALoaderOutput(BaseInvocationOutput):
     """SDXL LoRA Loader Output"""
@@ -204,13 +301,13 @@ class SDXLLoRALoaderOutput(BaseInvocationOutput):
     title="SDXL LoRA",
     tags=["lora", "model"],
     category="model",
-    version="1.0.2",
+    version="1.0.3",
 )
 class SDXLLoRALoaderInvocation(BaseInvocation):
     """Apply selected lora to unet and text_encoder."""
 
     lora: ModelIdentifierField = InputField(
-        description=FieldDescriptions.lora_model, input=Input.Direct, title="LoRA", ui_type=UIType.LoRAModel
+        description=FieldDescriptions.lora_model, title="LoRA", ui_type=UIType.LoRAModel
     )
     weight: float = InputField(default=0.75, description=FieldDescriptions.lora_weight)
     unet: Optional[UNetField] = InputField(
@@ -279,12 +376,78 @@ class SDXLLoRALoaderInvocation(BaseInvocation):
         return output
 
 
-@invocation("vae_loader", title="VAE", tags=["vae", "model"], category="model", version="1.0.2")
+@invocation(
+    "sdxl_lora_collection_loader",
+    title="SDXL LoRA Collection Loader",
+    tags=["model"],
+    category="model",
+    version="1.0.0",
+)
+class SDXLLoRACollectionLoader(BaseInvocation):
+    """Applies a collection of SDXL LoRAs to the provided UNet and CLIP models."""
+
+    loras: LoRAField | list[LoRAField] = InputField(
+        description="LoRA models and weights. May be a single LoRA or collection.", title="LoRAs"
+    )
+    unet: Optional[UNetField] = InputField(
+        default=None,
+        description=FieldDescriptions.unet,
+        input=Input.Connection,
+        title="UNet",
+    )
+    clip: Optional[CLIPField] = InputField(
+        default=None,
+        description=FieldDescriptions.clip,
+        input=Input.Connection,
+        title="CLIP",
+    )
+    clip2: Optional[CLIPField] = InputField(
+        default=None,
+        description=FieldDescriptions.clip,
+        input=Input.Connection,
+        title="CLIP 2",
+    )
+
+    def invoke(self, context: InvocationContext) -> SDXLLoRALoaderOutput:
+        output = SDXLLoRALoaderOutput()
+        loras = self.loras if isinstance(self.loras, list) else [self.loras]
+        added_loras: list[str] = []
+
+        for lora in loras:
+            if lora.lora.key in added_loras:
+                continue
+
+            if not context.models.exists(lora.lora.key):
+                raise Exception(f"Unknown lora: {lora.lora.key}!")
+
+            assert lora.lora.base is BaseModelType.StableDiffusionXL
+
+            added_loras.append(lora.lora.key)
+
+            if self.unet is not None:
+                if output.unet is None:
+                    output.unet = self.unet.model_copy(deep=True)
+                output.unet.loras.append(lora)
+
+            if self.clip is not None:
+                if output.clip is None:
+                    output.clip = self.clip.model_copy(deep=True)
+                output.clip.loras.append(lora)
+
+            if self.clip2 is not None:
+                if output.clip2 is None:
+                    output.clip2 = self.clip2.model_copy(deep=True)
+                output.clip2.loras.append(lora)
+
+        return output
+
+
+@invocation("vae_loader", title="VAE", tags=["vae", "model"], category="model", version="1.0.3")
 class VAELoaderInvocation(BaseInvocation):
     """Loads a VAE model, outputting a VaeLoaderOutput"""
 
     vae_model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.vae_model, input=Input.Direct, title="VAE", ui_type=UIType.VAEModel
+        description=FieldDescriptions.vae_model, title="VAE", ui_type=UIType.VAEModel
     )
 
     def invoke(self, context: InvocationContext) -> VAEOutput:

invokeai/app/invocations/resize_latents.py

@@ -0,0 +1,103 @@
+from typing import Literal
+
+import torch
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
+from invokeai.app.invocations.fields import (
+    FieldDescriptions,
+    Input,
+    InputField,
+    LatentsField,
+)
+from invokeai.app.invocations.primitives import LatentsOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.util.devices import TorchDevice
+
+LATENTS_INTERPOLATION_MODE = Literal["nearest", "linear", "bilinear", "bicubic", "trilinear", "area", "nearest-exact"]
+
+
+@invocation(
+    "lresize",
+    title="Resize Latents",
+    tags=["latents", "resize"],
+    category="latents",
+    version="1.0.2",
+)
+class ResizeLatentsInvocation(BaseInvocation):
+    """Resizes latents to explicit width/height (in pixels). Provided dimensions are floor-divided by 8."""
+
+    latents: LatentsField = InputField(
+        description=FieldDescriptions.latents,
+        input=Input.Connection,
+    )
+    width: int = InputField(
+        ge=64,
+        multiple_of=LATENT_SCALE_FACTOR,
+        description=FieldDescriptions.width,
+    )
+    height: int = InputField(
+        ge=64,
+        multiple_of=LATENT_SCALE_FACTOR,
+        description=FieldDescriptions.width,
+    )
+    mode: LATENTS_INTERPOLATION_MODE = InputField(default="bilinear", description=FieldDescriptions.interp_mode)
+    antialias: bool = InputField(default=False, description=FieldDescriptions.torch_antialias)
+
+    def invoke(self, context: InvocationContext) -> LatentsOutput:
+        latents = context.tensors.load(self.latents.latents_name)
+        device = TorchDevice.choose_torch_device()
+
+        resized_latents = torch.nn.functional.interpolate(
+            latents.to(device),
+            size=(self.height // LATENT_SCALE_FACTOR, self.width // LATENT_SCALE_FACTOR),
+            mode=self.mode,
+            antialias=self.antialias if self.mode in ["bilinear", "bicubic"] else False,
+        )
+
+        # https://discuss.huggingface.co/t/memory-usage-by-later-pipeline-stages/23699
+        resized_latents = resized_latents.to("cpu")
+
+        TorchDevice.empty_cache()
+
+        name = context.tensors.save(tensor=resized_latents)
+        return LatentsOutput.build(latents_name=name, latents=resized_latents, seed=self.latents.seed)
+
+
+@invocation(
+    "lscale",
+    title="Scale Latents",
+    tags=["latents", "resize"],
+    category="latents",
+    version="1.0.2",
+)
+class ScaleLatentsInvocation(BaseInvocation):
+    """Scales latents by a given factor."""
+
+    latents: LatentsField = InputField(
+        description=FieldDescriptions.latents,
+        input=Input.Connection,
+    )
+    scale_factor: float = InputField(gt=0, description=FieldDescriptions.scale_factor)
+    mode: LATENTS_INTERPOLATION_MODE = InputField(default="bilinear", description=FieldDescriptions.interp_mode)
+    antialias: bool = InputField(default=False, description=FieldDescriptions.torch_antialias)
+
+    def invoke(self, context: InvocationContext) -> LatentsOutput:
+        latents = context.tensors.load(self.latents.latents_name)
+
+        device = TorchDevice.choose_torch_device()
+
+        # resizing
+        resized_latents = torch.nn.functional.interpolate(
+            latents.to(device),
+            scale_factor=self.scale_factor,
+            mode=self.mode,
+            antialias=self.antialias if self.mode in ["bilinear", "bicubic"] else False,
+        )
+
+        # https://discuss.huggingface.co/t/memory-usage-by-later-pipeline-stages/23699
+        resized_latents = resized_latents.to("cpu")
+        TorchDevice.empty_cache()
+
+        name = context.tensors.save(tensor=resized_latents)
+        return LatentsOutput.build(latents_name=name, latents=resized_latents, seed=self.latents.seed)

invokeai/app/invocations/scheduler.py

@@ -0,0 +1,34 @@
+from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
+from invokeai.app.invocations.constants import SCHEDULER_NAME_VALUES
+from invokeai.app.invocations.fields import (
+    FieldDescriptions,
+    InputField,
+    OutputField,
+    UIType,
+)
+from invokeai.app.services.shared.invocation_context import InvocationContext
+
+
+@invocation_output("scheduler_output")
+class SchedulerOutput(BaseInvocationOutput):
+    scheduler: SCHEDULER_NAME_VALUES = OutputField(description=FieldDescriptions.scheduler, ui_type=UIType.Scheduler)
+
+
+@invocation(
+    "scheduler",
+    title="Scheduler",
+    tags=["scheduler"],
+    category="latents",
+    version="1.0.0",
+)
+class SchedulerInvocation(BaseInvocation):
+    """Selects a scheduler."""
+
+    scheduler: SCHEDULER_NAME_VALUES = InputField(
+        default="euler",
+        description=FieldDescriptions.scheduler,
+        ui_type=UIType.Scheduler,
+    )
+
+    def invoke(self, context: InvocationContext) -> SchedulerOutput:
+        return SchedulerOutput(scheduler=self.scheduler)

invokeai/app/invocations/sdxl.py

@@ -1,4 +1,4 @@
-from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, UIType
+from invokeai.app.invocations.fields import FieldDescriptions, InputField, OutputField, UIType
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.model_manager import SubModelType
 
@@ -30,12 +30,12 @@ class SDXLRefinerModelLoaderOutput(BaseInvocationOutput):
     vae: VAEField = OutputField(description=FieldDescriptions.vae, title="VAE")
 
 
-@invocation("sdxl_model_loader", title="SDXL Main Model", tags=["model", "sdxl"], category="model", version="1.0.2")
+@invocation("sdxl_model_loader", title="SDXL Main Model", tags=["model", "sdxl"], category="model", version="1.0.3")
 class SDXLModelLoaderInvocation(BaseInvocation):
     """Loads an sdxl base model, outputting its submodels."""
 
     model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.sdxl_main_model, input=Input.Direct, ui_type=UIType.SDXLMainModel
+        description=FieldDescriptions.sdxl_main_model, ui_type=UIType.SDXLMainModel
     )
     # TODO: precision?
 
@@ -67,13 +67,13 @@ class SDXLModelLoaderInvocation(BaseInvocation):
     title="SDXL Refiner Model",
     tags=["model", "sdxl", "refiner"],
     category="model",
-    version="1.0.2",
+    version="1.0.3",
 )
 class SDXLRefinerModelLoaderInvocation(BaseInvocation):
     """Loads an sdxl refiner model, outputting its submodels."""
 
     model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.sdxl_refiner_model, input=Input.Direct, ui_type=UIType.SDXLRefinerModel
+        description=FieldDescriptions.sdxl_refiner_model, ui_type=UIType.SDXLRefinerModel
     )
     # TODO: precision?
 

invokeai/app/invocations/t2i_adapter.py

@@ -8,7 +8,7 @@ from invokeai.app.invocations.baseinvocation import (
     invocation,
     invocation_output,
 )
-from invokeai.app.invocations.fields import FieldDescriptions, ImageField, Input, InputField, OutputField, UIType
+from invokeai.app.invocations.fields import FieldDescriptions, ImageField, InputField, OutputField, UIType
 from invokeai.app.invocations.model import ModelIdentifierField
 from invokeai.app.invocations.util import validate_begin_end_step, validate_weights
 from invokeai.app.services.shared.invocation_context import InvocationContext
@@ -45,7 +45,7 @@ class T2IAdapterOutput(BaseInvocationOutput):
 
 
 @invocation(
-    "t2i_adapter", title="T2I-Adapter", tags=["t2i_adapter", "control"], category="t2i_adapter", version="1.0.2"
+    "t2i_adapter", title="T2I-Adapter", tags=["t2i_adapter", "control"], category="t2i_adapter", version="1.0.3"
 )
 class T2IAdapterInvocation(BaseInvocation):
     """Collects T2I-Adapter info to pass to other nodes."""
@@ -55,7 +55,6 @@ class T2IAdapterInvocation(BaseInvocation):
     t2i_adapter_model: ModelIdentifierField = InputField(
         description="The T2I-Adapter model.",
         title="T2I-Adapter Model",
-        input=Input.Direct,
         ui_order=-1,
         ui_type=UIType.T2IAdapterModel,
     )

invokeai/app/invocations/tiled_stable_diffusion_refine.py

@@ -0,0 +1,384 @@
+from contextlib import ExitStack
+from typing import Iterator, Tuple
+
+import numpy as np
+import numpy.typing as npt
+import torch
+from diffusers.models.unets.unet_2d_condition import UNet2DConditionModel
+from PIL import Image
+from pydantic import field_validator
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import DEFAULT_PRECISION, LATENT_SCALE_FACTOR, SCHEDULER_NAME_VALUES
+from invokeai.app.invocations.fields import (
+    ConditioningField,
+    FieldDescriptions,
+    ImageField,
+    Input,
+    InputField,
+    UIType,
+)
+from invokeai.app.invocations.image_to_latents import ImageToLatentsInvocation
+from invokeai.app.invocations.latent import DenoiseLatentsInvocation, get_scheduler
+from invokeai.app.invocations.latents_to_image import LatentsToImageInvocation
+from invokeai.app.invocations.model import ModelIdentifierField, UNetField, VAEField
+from invokeai.app.invocations.noise import get_noise
+from invokeai.app.invocations.primitives import ImageOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES, prepare_control_image
+from invokeai.backend.lora import LoRAModelRaw
+from invokeai.backend.model_patcher import ModelPatcher
+from invokeai.backend.stable_diffusion.diffusers_pipeline import ControlNetData, image_resized_to_grid_as_tensor
+from invokeai.backend.tiles.tiles import calc_tiles_with_overlap, merge_tiles_with_linear_blending
+from invokeai.backend.tiles.utils import Tile
+from invokeai.backend.util.devices import TorchDevice
+from invokeai.backend.util.hotfixes import ControlNetModel
+
+
+@invocation(
+    "tiled_stable_diffusion_refine",
+    title="Tiled Stable Diffusion Refine",
+    tags=["upscale", "denoise"],
+    category="latents",
+    version="1.0.0",
+)
+class TiledStableDiffusionRefineInvocation(BaseInvocation):
+    """A tiled Stable Diffusion pipeline for refining high resolution images. This invocation is intended to be used to
+    refine an image after upscaling i.e. it is the second step in a typical "tiled upscaling" workflow.
+    """
+
+    image: ImageField = InputField(description="Image to be refined.")
+
+    positive_conditioning: ConditioningField = InputField(
+        description=FieldDescriptions.positive_cond, input=Input.Connection
+    )
+    negative_conditioning: ConditioningField = InputField(
+        description=FieldDescriptions.negative_cond, input=Input.Connection
+    )
+    # TODO(ryand): Add multiple-of validation.
+    tile_height: int = InputField(default=512, gt=0, description="Height of the tiles.")
+    tile_width: int = InputField(default=512, gt=0, description="Width of the tiles.")
+    tile_overlap: int = InputField(
+        default=16,
+        gt=0,
+        description="Target overlap between adjacent tiles (the last row/column may overlap more than this).",
+    )
+    steps: int = InputField(default=18, gt=0, description=FieldDescriptions.steps)
+    cfg_scale: float | list[float] = InputField(default=6.0, description=FieldDescriptions.cfg_scale, title="CFG Scale")
+    denoising_start: float = InputField(
+        default=0.65,
+        ge=0,
+        le=1,
+        description=FieldDescriptions.denoising_start,
+    )
+    denoising_end: float = InputField(default=1.0, ge=0, le=1, description=FieldDescriptions.denoising_end)
+    scheduler: SCHEDULER_NAME_VALUES = InputField(
+        default="euler",
+        description=FieldDescriptions.scheduler,
+        ui_type=UIType.Scheduler,
+    )
+    unet: UNetField = InputField(
+        description=FieldDescriptions.unet,
+        input=Input.Connection,
+        title="UNet",
+    )
+    cfg_rescale_multiplier: float = InputField(
+        title="CFG Rescale Multiplier", default=0, ge=0, lt=1, description=FieldDescriptions.cfg_rescale_multiplier
+    )
+    vae: VAEField = InputField(
+        description=FieldDescriptions.vae,
+        input=Input.Connection,
+    )
+    vae_fp32: bool = InputField(
+        default=DEFAULT_PRECISION == torch.float32, description="Whether to use float32 precision when running the VAE."
+    )
+    # HACK(ryand): We probably want to allow the user to control all of the parameters in ControlField. But, we akwardly
+    # don't want to use the image field. Figure out how best to handle this.
+    # TODO(ryand): Currently, there is no ControlNet preprocessor applied to the tile images. In other words, we pretty
+    # much assume that it is a tile ControlNet. We need to decide how we want to handle this. E.g. find a way to support
+    # CN preprocessors, raise a clear warning when a non-tile CN model is selected, hardcode the supported CN models,
+    # etc.
+    control_model: ModelIdentifierField = InputField(
+        description=FieldDescriptions.controlnet_model, ui_type=UIType.ControlNetModel
+    )
+    control_weight: float = InputField(default=0.6)
+
+    @field_validator("cfg_scale")
+    def ge_one(cls, v: list[float] | float) -> list[float] | float:
+        """Validate that all cfg_scale values are >= 1"""
+        if isinstance(v, list):
+            for i in v:
+                if i < 1:
+                    raise ValueError("cfg_scale must be greater than 1")
+        else:
+            if v < 1:
+                raise ValueError("cfg_scale must be greater than 1")
+        return v
+
+    @staticmethod
+    def crop_latents_to_tile(latents: torch.Tensor, image_tile: Tile) -> torch.Tensor:
+        """Crop the latent-space tensor to the area corresponding to the image-space tile.
+        The tile coordinates must be divisible by the LATENT_SCALE_FACTOR.
+        """
+        for coord in [image_tile.coords.top, image_tile.coords.left, image_tile.coords.right, image_tile.coords.bottom]:
+            if coord % LATENT_SCALE_FACTOR != 0:
+                raise ValueError(
+                    f"The tile coordinates must all be divisible by the latent scale factor"
+                    f" ({LATENT_SCALE_FACTOR}). {image_tile.coords=}."
+                )
+        assert latents.dim() == 4  # We expect: (batch_size, channels, height, width).
+
+        top = image_tile.coords.top // LATENT_SCALE_FACTOR
+        left = image_tile.coords.left // LATENT_SCALE_FACTOR
+        bottom = image_tile.coords.bottom // LATENT_SCALE_FACTOR
+        right = image_tile.coords.right // LATENT_SCALE_FACTOR
+        return latents[..., top:bottom, left:right]
+
+    def run_controlnet(
+        self,
+        image: Image.Image,
+        controlnet_model: ControlNetModel,
+        weight: float,
+        do_classifier_free_guidance: bool,
+        width: int,
+        height: int,
+        device: torch.device,
+        dtype: torch.dtype,
+        control_mode: CONTROLNET_MODE_VALUES = "balanced",
+        resize_mode: CONTROLNET_RESIZE_VALUES = "just_resize_simple",
+    ) -> ControlNetData:
+        control_image = prepare_control_image(
+            image=image,
+            do_classifier_free_guidance=do_classifier_free_guidance,
+            width=width,
+            height=height,
+            device=device,
+            dtype=dtype,
+            control_mode=control_mode,
+            resize_mode=resize_mode,
+        )
+        return ControlNetData(
+            model=controlnet_model,
+            image_tensor=control_image,
+            weight=weight,
+            begin_step_percent=0.0,
+            end_step_percent=1.0,
+            control_mode=control_mode,
+            # Any resizing needed should currently be happening in prepare_control_image(), but adding resize_mode to
+            # ControlNetData in case needed in the future.
+            resize_mode=resize_mode,
+        )
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> ImageOutput:
+        # TODO(ryand): Expose the seed parameter.
+        seed = 0
+
+        # Load the input image.
+        input_image = context.images.get_pil(self.image.image_name)
+
+        # Calculate the tile locations to cover the image.
+        # We have selected this tiling strategy to make it easy to achieve tile coords that are multiples of 8. This
+        # facilitates conversions between image space and latent space.
+        # TODO(ryand): Expose these tiling parameters. (Keep in mind the multiple-of constraints on these params.)
+        tiles = calc_tiles_with_overlap(
+            image_height=input_image.height,
+            image_width=input_image.width,
+            tile_height=self.tile_height,
+            tile_width=self.tile_width,
+            overlap=self.tile_overlap,
+        )
+
+        # Convert the input image to a torch.Tensor.
+        input_image_torch = image_resized_to_grid_as_tensor(input_image.convert("RGB"), multiple_of=LATENT_SCALE_FACTOR)
+        input_image_torch = input_image_torch.unsqueeze(0)  # Add a batch dimension.
+        # Validate our assumptions about the shape of input_image_torch.
+        assert input_image_torch.dim() == 4  # We expect: (batch_size, channels, height, width).
+        assert input_image_torch.shape[:2] == (1, 3)
+
+        # Split the input image into tiles in torch.Tensor format.
+        image_tiles_torch: list[torch.Tensor] = []
+        for tile in tiles:
+            image_tile = input_image_torch[
+                :,
+                :,
+                tile.coords.top : tile.coords.bottom,
+                tile.coords.left : tile.coords.right,
+            ]
+            image_tiles_torch.append(image_tile)
+
+        # Split the input image into tiles in numpy format.
+        # TODO(ryand): We currently maintain both np.ndarray and torch.Tensor tiles. Ideally, all operations should work
+        # with torch.Tensor tiles.
+        input_image_np = np.array(input_image)
+        image_tiles_np: list[npt.NDArray[np.uint8]] = []
+        for tile in tiles:
+            image_tile_np = input_image_np[
+                tile.coords.top : tile.coords.bottom,
+                tile.coords.left : tile.coords.right,
+                :,
+            ]
+            image_tiles_np.append(image_tile_np)
+
+        # VAE-encode each image tile independently.
+        # TODO(ryand): Is there any advantage to VAE-encoding the entire image before splitting it into tiles? What
+        # about for decoding?
+        vae_info = context.models.load(self.vae.vae)
+        latent_tiles: list[torch.Tensor] = []
+        for image_tile_torch in image_tiles_torch:
+            latent_tiles.append(
+                ImageToLatentsInvocation.vae_encode(
+                    vae_info=vae_info, upcast=self.vae_fp32, tiled=False, image_tensor=image_tile_torch
+                )
+            )
+
+        # Generate noise with dimensions corresponding to the full image in latent space.
+        # It is important that the noise tensor is generated at the full image dimension and then tiled, rather than
+        # generating for each tile independently. This ensures that overlapping regions between tiles use the same
+        # noise.
+        assert input_image_torch.shape[2] % LATENT_SCALE_FACTOR == 0
+        assert input_image_torch.shape[3] % LATENT_SCALE_FACTOR == 0
+        global_noise = get_noise(
+            width=input_image_torch.shape[3],
+            height=input_image_torch.shape[2],
+            device=TorchDevice.choose_torch_device(),
+            seed=seed,
+            downsampling_factor=LATENT_SCALE_FACTOR,
+            use_cpu=True,
+        )
+
+        # Crop the global noise into tiles.
+        noise_tiles = [self.crop_latents_to_tile(latents=global_noise, image_tile=t) for t in tiles]
+
+        # Prepare an iterator that yields the UNet's LoRA models and their weights.
+        def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
+            for lora in self.unet.loras:
+                lora_info = context.models.load(lora.lora)
+                assert isinstance(lora_info.model, LoRAModelRaw)
+                yield (lora_info.model, lora.weight)
+                del lora_info
+
+        # Load the UNet model.
+        unet_info = context.models.load(self.unet.unet)
+
+        refined_latent_tiles: list[torch.Tensor] = []
+        with ExitStack() as exit_stack, unet_info as unet, ModelPatcher.apply_lora_unet(unet, _lora_loader()):
+            assert isinstance(unet, UNet2DConditionModel)
+            scheduler = get_scheduler(
+                context=context,
+                scheduler_info=self.unet.scheduler,
+                scheduler_name=self.scheduler,
+                seed=seed,
+            )
+            pipeline = DenoiseLatentsInvocation.create_pipeline(unet=unet, scheduler=scheduler)
+
+            # Prepare the prompt conditioning data. The same prompt conditioning is applied to all tiles.
+            # Assume that all tiles have the same shape.
+            _, _, latent_height, latent_width = latent_tiles[0].shape
+            conditioning_data = DenoiseLatentsInvocation.get_conditioning_data(
+                context=context,
+                positive_conditioning_field=self.positive_conditioning,
+                negative_conditioning_field=self.negative_conditioning,
+                unet=unet,
+                latent_height=latent_height,
+                latent_width=latent_width,
+                cfg_scale=self.cfg_scale,
+                steps=self.steps,
+                cfg_rescale_multiplier=self.cfg_rescale_multiplier,
+            )
+
+            # Load the ControlNet model.
+            # TODO(ryand): Support multiple ControlNet models.
+            controlnet_model = exit_stack.enter_context(context.models.load(self.control_model))
+            assert isinstance(controlnet_model, ControlNetModel)
+
+            # Denoise (i.e. "refine") each tile independently.
+            for image_tile_np, latent_tile, noise_tile in zip(image_tiles_np, latent_tiles, noise_tiles, strict=True):
+                assert latent_tile.shape == noise_tile.shape
+
+                # Prepare a PIL Image for ControlNet processing.
+                # TODO(ryand): This is a bit awkward that we have to prepare both torch.Tensor and PIL.Image versions of
+                # the tiles. Ideally, the ControlNet code should be able to work with Tensors.
+                image_tile_pil = Image.fromarray(image_tile_np)
+
+                # Run the ControlNet on the image tile.
+                height, width, _ = image_tile_np.shape
+                # The height and width must be evenly divisible by LATENT_SCALE_FACTOR. This is enforced earlier, but we
+                # validate this assumption here.
+                assert height % LATENT_SCALE_FACTOR == 0
+                assert width % LATENT_SCALE_FACTOR == 0
+                controlnet_data = self.run_controlnet(
+                    image=image_tile_pil,
+                    controlnet_model=controlnet_model,
+                    weight=self.control_weight,
+                    do_classifier_free_guidance=True,
+                    width=width,
+                    height=height,
+                    device=controlnet_model.device,
+                    dtype=controlnet_model.dtype,
+                    control_mode="balanced",
+                    resize_mode="just_resize_simple",
+                )
+
+                num_inference_steps, timesteps, init_timestep, scheduler_step_kwargs = (
+                    DenoiseLatentsInvocation.init_scheduler(
+                        scheduler,
+                        device=unet.device,
+                        steps=self.steps,
+                        denoising_start=self.denoising_start,
+                        denoising_end=self.denoising_end,
+                        seed=seed,
+                    )
+                )
+
+                # TODO(ryand): Think about when/if latents/noise should be moved off of the device to save VRAM.
+                latent_tile = latent_tile.to(device=unet.device, dtype=unet.dtype)
+                noise_tile = noise_tile.to(device=unet.device, dtype=unet.dtype)
+                refined_latent_tile = pipeline.latents_from_embeddings(
+                    latents=latent_tile,
+                    timesteps=timesteps,
+                    init_timestep=init_timestep,
+                    noise=noise_tile,
+                    seed=seed,
+                    mask=None,
+                    masked_latents=None,
+                    gradient_mask=None,
+                    num_inference_steps=num_inference_steps,
+                    scheduler_step_kwargs=scheduler_step_kwargs,
+                    conditioning_data=conditioning_data,
+                    control_data=[controlnet_data],
+                    ip_adapter_data=None,
+                    t2i_adapter_data=None,
+                    callback=lambda x: None,
+                )
+                refined_latent_tiles.append(refined_latent_tile)
+
+        # VAE-decode each refined latent tile independently.
+        refined_image_tiles: list[Image.Image] = []
+        for refined_latent_tile in refined_latent_tiles:
+            refined_image_tile = LatentsToImageInvocation.vae_decode(
+                context=context,
+                vae_info=vae_info,
+                seamless_axes=self.vae.seamless_axes,
+                latents=refined_latent_tile,
+                use_fp32=self.vae_fp32,
+                use_tiling=False,
+            )
+            refined_image_tiles.append(refined_image_tile)
+
+        # TODO(ryand): I copied this from DenoiseLatentsInvocation. I'm not sure if it's actually important.
+        TorchDevice.empty_cache()
+
+        # Merge the refined image tiles back into a single image.
+        refined_image_tiles_np = [np.array(t) for t in refined_image_tiles]
+        merged_image_np = np.zeros(shape=(input_image.height, input_image.width, 3), dtype=np.uint8)
+        # TODO(ryand): Tune the blend_amount. Should this be exposed as a parameter?
+        merge_tiles_with_linear_blending(
+            dst_image=merged_image_np, tiles=tiles, tile_images=refined_image_tiles_np, blend_amount=self.tile_overlap
+        )
+
+        # Save the refined image and return its reference.
+        merged_image_pil = Image.fromarray(merged_image_np)
+        image_dto = context.images.save(image=merged_image_pil)
+
+        return ImageOutput.build(image_dto)

invokeai/app/services/bulk_download/bulk_download_default.py

@@ -106,9 +106,7 @@ class BulkDownloadService(BulkDownloadBase):
         if self._invoker:
             assert bulk_download_id is not None
             self._invoker.services.events.emit_bulk_download_started(
-                bulk_download_id=bulk_download_id,
-                bulk_download_item_id=bulk_download_item_id,
-                bulk_download_item_name=bulk_download_item_name,
+                bulk_download_id, bulk_download_item_id, bulk_download_item_name
             )
 
     def _signal_job_completed(
@@ -118,10 +116,8 @@ class BulkDownloadService(BulkDownloadBase):
         if self._invoker:
             assert bulk_download_id is not None
             assert bulk_download_item_name is not None
-            self._invoker.services.events.emit_bulk_download_completed(
-                bulk_download_id=bulk_download_id,
-                bulk_download_item_id=bulk_download_item_id,
-                bulk_download_item_name=bulk_download_item_name,
+            self._invoker.services.events.emit_bulk_download_complete(
+                bulk_download_id, bulk_download_item_id, bulk_download_item_name
             )
 
     def _signal_job_failed(
@@ -131,11 +127,8 @@ class BulkDownloadService(BulkDownloadBase):
         if self._invoker:
             assert bulk_download_id is not None
             assert exception is not None
-            self._invoker.services.events.emit_bulk_download_failed(
-                bulk_download_id=bulk_download_id,
-                bulk_download_item_id=bulk_download_item_id,
-                bulk_download_item_name=bulk_download_item_name,
-                error=str(exception),
+            self._invoker.services.events.emit_bulk_download_error(
+                bulk_download_id, bulk_download_item_id, bulk_download_item_name, str(exception)
             )
 
     def stop(self, *args, **kwargs):

invokeai/app/services/download/download_default.py

@@ -8,14 +8,13 @@ import time
 import traceback
 from pathlib import Path
 from queue import Empty, PriorityQueue
-from typing import Any, Dict, List, Optional, Set
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set
 
 import requests
 from pydantic.networks import AnyHttpUrl
 from requests import HTTPError
 from tqdm import tqdm
 
-from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.util.misc import get_iso_timestamp
 from invokeai.backend.util.logging import InvokeAILogger
 
@@ -30,6 +29,9 @@ from .download_base import (
     UnknownJobIDException,
 )
 
+if TYPE_CHECKING:
+    from invokeai.app.services.events.events_base import EventServiceBase
+
 # Maximum number of bytes to download during each call to requests.iter_content()
 DOWNLOAD_CHUNK_SIZE = 100000
 
@@ -40,7 +42,7 @@ class DownloadQueueService(DownloadQueueServiceBase):
     def __init__(
         self,
         max_parallel_dl: int = 5,
-        event_bus: Optional[EventServiceBase] = None,
+        event_bus: Optional["EventServiceBase"] = None,
         requests_session: Optional[requests.sessions.Session] = None,
     ):
         """
@@ -318,10 +320,8 @@ class DownloadQueueService(DownloadQueueServiceBase):
         in_progress_path.rename(job.download_path)
 
     def _validate_filename(self, directory: str, filename: str) -> bool:
-        pc_name_max = os.pathconf(directory, "PC_NAME_MAX") if hasattr(os, "pathconf") else 260  # hardcoded for windows
-        pc_path_max = (
-            os.pathconf(directory, "PC_PATH_MAX") if hasattr(os, "pathconf") else 32767
-        )  # hardcoded for windows with long names enabled
+        pc_name_max = get_pc_name_max(directory)
+        pc_path_max = get_pc_path_max(directory)
         if "/" in filename:
             return False
         if filename.startswith(".."):
@@ -345,8 +345,7 @@ class DownloadQueueService(DownloadQueueServiceBase):
                     f"An error occurred while processing the on_start callback: {traceback.format_exception(e)}"
                 )
         if self._event_bus:
-            assert job.download_path
-            self._event_bus.emit_download_started(str(job.source), job.download_path.as_posix())
+            self._event_bus.emit_download_started(job)
 
     def _signal_job_progress(self, job: DownloadJob) -> None:
         if job.on_progress:
@@ -357,13 +356,7 @@ class DownloadQueueService(DownloadQueueServiceBase):
                     f"An error occurred while processing the on_progress callback: {traceback.format_exception(e)}"
                 )
         if self._event_bus:
-            assert job.download_path
-            self._event_bus.emit_download_progress(
-                str(job.source),
-                download_path=job.download_path.as_posix(),
-                current_bytes=job.bytes,
-                total_bytes=job.total_bytes,
-            )
+            self._event_bus.emit_download_progress(job)
 
     def _signal_job_complete(self, job: DownloadJob) -> None:
         job.status = DownloadJobStatus.COMPLETED
@@ -375,10 +368,7 @@ class DownloadQueueService(DownloadQueueServiceBase):
                     f"An error occurred while processing the on_complete callback: {traceback.format_exception(e)}"
                 )
         if self._event_bus:
-            assert job.download_path
-            self._event_bus.emit_download_complete(
-                str(job.source), download_path=job.download_path.as_posix(), total_bytes=job.total_bytes
-            )
+            self._event_bus.emit_download_complete(job)
 
     def _signal_job_cancelled(self, job: DownloadJob) -> None:
         if job.status not in [DownloadJobStatus.RUNNING, DownloadJobStatus.WAITING]:
@@ -392,7 +382,7 @@ class DownloadQueueService(DownloadQueueServiceBase):
                     f"An error occurred while processing the on_cancelled callback: {traceback.format_exception(e)}"
                 )
         if self._event_bus:
-            self._event_bus.emit_download_cancelled(str(job.source))
+            self._event_bus.emit_download_cancelled(job)
 
     def _signal_job_error(self, job: DownloadJob, excp: Optional[Exception] = None) -> None:
         job.status = DownloadJobStatus.ERROR
@@ -405,9 +395,7 @@ class DownloadQueueService(DownloadQueueServiceBase):
                     f"An error occurred while processing the on_error callback: {traceback.format_exception(e)}"
                 )
         if self._event_bus:
-            assert job.error_type
-            assert job.error
-            self._event_bus.emit_download_error(str(job.source), error_type=job.error_type, error=job.error)
+            self._event_bus.emit_download_error(job)
 
     def _cleanup_cancelled_job(self, job: DownloadJob) -> None:
         self._logger.debug(f"Cleaning up leftover files from cancelled download job {job.download_path}")
@@ -419,6 +407,26 @@ class DownloadQueueService(DownloadQueueServiceBase):
             self._logger.warning(excp)
 
 
+def get_pc_name_max(directory: str) -> int:
+    if hasattr(os, "pathconf"):
+        try:
+            return os.pathconf(directory, "PC_NAME_MAX")
+        except OSError:
+            # macOS w/ external drives raise OSError
+            pass
+    return 260  # hardcoded for windows
+
+
+def get_pc_path_max(directory: str) -> int:
+    if hasattr(os, "pathconf"):
+        try:
+            return os.pathconf(directory, "PC_PATH_MAX")
+        except OSError:
+            # some platforms may not have this value
+            pass
+    return 32767  # hardcoded for windows with long names enabled
+
+
 # Example on_progress event handler to display a TQDM status bar
 # Activate with:
 #   download_service.download(DownloadJob('http://foo.bar/baz', '/tmp', on_progress=TqdmProgress().update))

invokeai/app/services/events/events_base.py

@@ -1,486 +1,195 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
 
 
-from typing import Any, Dict, List, Optional, Union
+from typing import TYPE_CHECKING, Optional
 
-from invokeai.app.services.session_processor.session_processor_common import ProgressImage
-from invokeai.app.services.session_queue.session_queue_common import (
-    BatchStatus,
-    EnqueueBatchResult,
-    SessionQueueItem,
-    SessionQueueStatus,
+from invokeai.app.services.events.events_common import (
+    BatchEnqueuedEvent,
+    BulkDownloadCompleteEvent,
+    BulkDownloadErrorEvent,
+    BulkDownloadStartedEvent,
+    DownloadCancelledEvent,
+    DownloadCompleteEvent,
+    DownloadErrorEvent,
+    DownloadProgressEvent,
+    DownloadStartedEvent,
+    EventBase,
+    InvocationCompleteEvent,
+    InvocationDenoiseProgressEvent,
+    InvocationErrorEvent,
+    InvocationStartedEvent,
+    ModelInstallCancelledEvent,
+    ModelInstallCompleteEvent,
+    ModelInstallDownloadProgressEvent,
+    ModelInstallDownloadsCompleteEvent,
+    ModelInstallErrorEvent,
+    ModelInstallStartedEvent,
+    ModelLoadCompleteEvent,
+    ModelLoadStartedEvent,
+    QueueClearedEvent,
+    QueueItemStatusChangedEvent,
 )
-from invokeai.app.util.misc import get_timestamp
-from invokeai.backend.model_manager import AnyModelConfig
-from invokeai.backend.model_manager.config import SubModelType
+from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineIntermediateState
+
+if TYPE_CHECKING:
+    from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput
+    from invokeai.app.services.download.download_base import DownloadJob
+    from invokeai.app.services.events.events_common import EventBase
+    from invokeai.app.services.model_install.model_install_common import ModelInstallJob
+    from invokeai.app.services.session_processor.session_processor_common import ProgressImage
+    from invokeai.app.services.session_queue.session_queue_common import (
+        BatchStatus,
+        EnqueueBatchResult,
+        SessionQueueItem,
+        SessionQueueStatus,
+    )
+    from invokeai.backend.model_manager.config import AnyModelConfig, SubModelType
 
 
 class EventServiceBase:
-    queue_event: str = "queue_event"
-    bulk_download_event: str = "bulk_download_event"
-    download_event: str = "download_event"
-    model_event: str = "model_event"
-
     """Basic event bus, to have an empty stand-in when not needed"""
 
-    def dispatch(self, event_name: str, payload: Any) -> None:
+    def dispatch(self, event: "EventBase") -> None:
         pass
 
-    def _emit_bulk_download_event(self, event_name: str, payload: dict) -> None:
-        """Bulk download events are emitted to a room with queue_id as the room name"""
-        payload["timestamp"] = get_timestamp()
-        self.dispatch(
-            event_name=EventServiceBase.bulk_download_event,
-            payload={"event": event_name, "data": payload},
-        )
+    # region: Invocation
 
-    def __emit_queue_event(self, event_name: str, payload: dict) -> None:
-        """Queue events are emitted to a room with queue_id as the room name"""
-        payload["timestamp"] = get_timestamp()
-        self.dispatch(
-            event_name=EventServiceBase.queue_event,
-            payload={"event": event_name, "data": payload},
-        )
+    def emit_invocation_started(self, queue_item: "SessionQueueItem", invocation: "BaseInvocation") -> None:
+        """Emitted when an invocation is started"""
+        self.dispatch(InvocationStartedEvent.build(queue_item, invocation))
 
-    def __emit_download_event(self, event_name: str, payload: dict) -> None:
-        payload["timestamp"] = get_timestamp()
-        self.dispatch(
-            event_name=EventServiceBase.download_event,
-            payload={"event": event_name, "data": payload},
-        )
-
-    def __emit_model_event(self, event_name: str, payload: dict) -> None:
-        payload["timestamp"] = get_timestamp()
-        self.dispatch(
-            event_name=EventServiceBase.model_event,
-            payload={"event": event_name, "data": payload},
-        )
-
-    # Define events here for every event in the system.
-    # This will make them easier to integrate until we find a schema generator.
-    def emit_generator_progress(
+    def emit_invocation_denoise_progress(
         self,
-        queue_id: str,
-        queue_item_id: int,
-        queue_batch_id: str,
-        graph_execution_state_id: str,
-        node_id: str,
-        source_node_id: str,
-        progress_image: Optional[ProgressImage],
-        step: int,
-        order: int,
-        total_steps: int,
+        queue_item: "SessionQueueItem",
+        invocation: "BaseInvocation",
+        intermediate_state: PipelineIntermediateState,
+        progress_image: "ProgressImage",
     ) -> None:
-        """Emitted when there is generation progress"""
-        self.__emit_queue_event(
-            event_name="generator_progress",
-            payload={
-                "queue_id": queue_id,
-                "queue_item_id": queue_item_id,
-                "queue_batch_id": queue_batch_id,
-                "graph_execution_state_id": graph_execution_state_id,
-                "node_id": node_id,
-                "source_node_id": source_node_id,
-                "progress_image": progress_image.model_dump(mode="json") if progress_image is not None else None,
-                "step": step,
-                "order": order,
-                "total_steps": total_steps,
-            },
-        )
+        """Emitted at each step during denoising of an invocation."""
+        self.dispatch(InvocationDenoiseProgressEvent.build(queue_item, invocation, intermediate_state, progress_image))
 
     def emit_invocation_complete(
-        self,
-        queue_id: str,
-        queue_item_id: int,
-        queue_batch_id: str,
-        graph_execution_state_id: str,
-        result: dict,
-        node: dict,
-        source_node_id: str,
+        self, queue_item: "SessionQueueItem", invocation: "BaseInvocation", output: "BaseInvocationOutput"
     ) -> None:
-        """Emitted when an invocation has completed"""
-        self.__emit_queue_event(
-            event_name="invocation_complete",
-            payload={
-                "queue_id": queue_id,
-                "queue_item_id": queue_item_id,
-                "queue_batch_id": queue_batch_id,
-                "graph_execution_state_id": graph_execution_state_id,
-                "node": node,
-                "source_node_id": source_node_id,
-                "result": result,
-            },
-        )
+        """Emitted when an invocation is complete"""
+        self.dispatch(InvocationCompleteEvent.build(queue_item, invocation, output))
 
     def emit_invocation_error(
         self,
-        queue_id: str,
-        queue_item_id: int,
-        queue_batch_id: str,
-        graph_execution_state_id: str,
-        node: dict,
-        source_node_id: str,
+        queue_item: "SessionQueueItem",
+        invocation: "BaseInvocation",
         error_type: str,
-        error: str,
+        error_message: str,
+        error_traceback: str,
     ) -> None:
-        """Emitted when an invocation has completed"""
-        self.__emit_queue_event(
-            event_name="invocation_error",
-            payload={
-                "queue_id": queue_id,
-                "queue_item_id": queue_item_id,
-                "queue_batch_id": queue_batch_id,
-                "graph_execution_state_id": graph_execution_state_id,
-                "node": node,
-                "source_node_id": source_node_id,
-                "error_type": error_type,
-                "error": error,
-            },
-        )
+        """Emitted when an invocation encounters an error"""
+        self.dispatch(InvocationErrorEvent.build(queue_item, invocation, error_type, error_message, error_traceback))
 
-    def emit_invocation_started(
-        self,
-        queue_id: str,
-        queue_item_id: int,
-        queue_batch_id: str,
-        graph_execution_state_id: str,
-        node: dict,
-        source_node_id: str,
-    ) -> None:
-        """Emitted when an invocation has started"""
-        self.__emit_queue_event(
-            event_name="invocation_started",
-            payload={
-                "queue_id": queue_id,
-                "queue_item_id": queue_item_id,
-                "queue_batch_id": queue_batch_id,
-                "graph_execution_state_id": graph_execution_state_id,
-                "node": node,
-                "source_node_id": source_node_id,
-            },
-        )
+    # endregion
 
-    def emit_graph_execution_complete(
-        self, queue_id: str, queue_item_id: int, queue_batch_id: str, graph_execution_state_id: str
-    ) -> None:
-        """Emitted when a session has completed all invocations"""
-        self.__emit_queue_event(
-            event_name="graph_execution_state_complete",
-            payload={
-                "queue_id": queue_id,
-                "queue_item_id": queue_item_id,
-                "queue_batch_id": queue_batch_id,
-                "graph_execution_state_id": graph_execution_state_id,
-            },
-        )
-
-    def emit_model_load_started(
-        self,
-        queue_id: str,
-        queue_item_id: int,
-        queue_batch_id: str,
-        graph_execution_state_id: str,
-        model_config: AnyModelConfig,
-        submodel_type: Optional[SubModelType] = None,
-    ) -> None:
-        """Emitted when a model is requested"""
-        self.__emit_queue_event(
-            event_name="model_load_started",
-            payload={
-                "queue_id": queue_id,
-                "queue_item_id": queue_item_id,
-                "queue_batch_id": queue_batch_id,
-                "graph_execution_state_id": graph_execution_state_id,
-                "model_config": model_config.model_dump(mode="json"),
-                "submodel_type": submodel_type,
-            },
-        )
-
-    def emit_model_load_completed(
-        self,
-        queue_id: str,
-        queue_item_id: int,
-        queue_batch_id: str,
-        graph_execution_state_id: str,
-        model_config: AnyModelConfig,
-        submodel_type: Optional[SubModelType] = None,
-    ) -> None:
-        """Emitted when a model is correctly loaded (returns model info)"""
-        self.__emit_queue_event(
-            event_name="model_load_completed",
-            payload={
-                "queue_id": queue_id,
-                "queue_item_id": queue_item_id,
-                "queue_batch_id": queue_batch_id,
-                "graph_execution_state_id": graph_execution_state_id,
-                "model_config": model_config.model_dump(mode="json"),
-                "submodel_type": submodel_type,
-            },
-        )
-
-    def emit_session_canceled(
-        self,
-        queue_id: str,
-        queue_item_id: int,
-        queue_batch_id: str,
-        graph_execution_state_id: str,
-    ) -> None:
-        """Emitted when a session is canceled"""
-        self.__emit_queue_event(
-            event_name="session_canceled",
-            payload={
-                "queue_id": queue_id,
-                "queue_item_id": queue_item_id,
-                "queue_batch_id": queue_batch_id,
-                "graph_execution_state_id": graph_execution_state_id,
-            },
-        )
+    # region Queue
 
     def emit_queue_item_status_changed(
-        self,
-        session_queue_item: SessionQueueItem,
-        batch_status: BatchStatus,
-        queue_status: SessionQueueStatus,
+        self, queue_item: "SessionQueueItem", batch_status: "BatchStatus", queue_status: "SessionQueueStatus"
     ) -> None:
         """Emitted when a queue item's status changes"""
-        self.__emit_queue_event(
-            event_name="queue_item_status_changed",
-            payload={
-                "queue_id": queue_status.queue_id,
-                "queue_item": {
-                    "queue_id": session_queue_item.queue_id,
-                    "item_id": session_queue_item.item_id,
-                    "status": session_queue_item.status,
-                    "batch_id": session_queue_item.batch_id,
-                    "session_id": session_queue_item.session_id,
-                    "error": session_queue_item.error,
-                    "created_at": str(session_queue_item.created_at) if session_queue_item.created_at else None,
-                    "updated_at": str(session_queue_item.updated_at) if session_queue_item.updated_at else None,
-                    "started_at": str(session_queue_item.started_at) if session_queue_item.started_at else None,
-                    "completed_at": str(session_queue_item.completed_at) if session_queue_item.completed_at else None,
-                },
-                "batch_status": batch_status.model_dump(mode="json"),
-                "queue_status": queue_status.model_dump(mode="json"),
-            },
-        )
+        self.dispatch(QueueItemStatusChangedEvent.build(queue_item, batch_status, queue_status))
 
-    def emit_batch_enqueued(self, enqueue_result: EnqueueBatchResult) -> None:
+    def emit_batch_enqueued(self, enqueue_result: "EnqueueBatchResult") -> None:
         """Emitted when a batch is enqueued"""
-        self.__emit_queue_event(
-            event_name="batch_enqueued",
-            payload={
-                "queue_id": enqueue_result.queue_id,
-                "batch_id": enqueue_result.batch.batch_id,
-                "enqueued": enqueue_result.enqueued,
-            },
-        )
+        self.dispatch(BatchEnqueuedEvent.build(enqueue_result))
 
     def emit_queue_cleared(self, queue_id: str) -> None:
-        """Emitted when the queue is cleared"""
-        self.__emit_queue_event(
-            event_name="queue_cleared",
-            payload={"queue_id": queue_id},
-        )
+        """Emitted when a queue is cleared"""
+        self.dispatch(QueueClearedEvent.build(queue_id))
 
-    def emit_download_started(self, source: str, download_path: str) -> None:
-        """
-        Emit when a download job is started.
+    # endregion
 
-        :param url: The downloaded url
-        """
-        self.__emit_download_event(
-            event_name="download_started",
-            payload={"source": source, "download_path": download_path},
-        )
+    # region Download
 
-    def emit_download_progress(self, source: str, download_path: str, current_bytes: int, total_bytes: int) -> None:
-        """
-        Emit "download_progress" events at regular intervals during a download job.
+    def emit_download_started(self, job: "DownloadJob") -> None:
+        """Emitted when a download is started"""
+        self.dispatch(DownloadStartedEvent.build(job))
 
-        :param source: The downloaded source
-        :param download_path: The local downloaded file
-        :param current_bytes: Number of bytes downloaded so far
-        :param total_bytes: The size of the file being downloaded (if known)
-        """
-        self.__emit_download_event(
-            event_name="download_progress",
-            payload={
-                "source": source,
-                "download_path": download_path,
-                "current_bytes": current_bytes,
-                "total_bytes": total_bytes,
-            },
-        )
+    def emit_download_progress(self, job: "DownloadJob") -> None:
+        """Emitted at intervals during a download"""
+        self.dispatch(DownloadProgressEvent.build(job))
 
-    def emit_download_complete(self, source: str, download_path: str, total_bytes: int) -> None:
-        """
-        Emit a "download_complete" event at the end of a successful download.
+    def emit_download_complete(self, job: "DownloadJob") -> None:
+        """Emitted when a download is completed"""
+        self.dispatch(DownloadCompleteEvent.build(job))
 
-        :param source: Source URL
-        :param download_path: Path to the locally downloaded file
-        :param total_bytes: The size of the downloaded file
-        """
-        self.__emit_download_event(
-            event_name="download_complete",
-            payload={
-                "source": source,
-                "download_path": download_path,
-                "total_bytes": total_bytes,
-            },
-        )
+    def emit_download_cancelled(self, job: "DownloadJob") -> None:
+        """Emitted when a download is cancelled"""
+        self.dispatch(DownloadCancelledEvent.build(job))
 
-    def emit_download_cancelled(self, source: str) -> None:
-        """Emit a "download_cancelled" event in the event that the download was cancelled by user."""
-        self.__emit_download_event(
-            event_name="download_cancelled",
-            payload={
-                "source": source,
-            },
-        )
+    def emit_download_error(self, job: "DownloadJob") -> None:
+        """Emitted when a download encounters an error"""
+        self.dispatch(DownloadErrorEvent.build(job))
 
-    def emit_download_error(self, source: str, error_type: str, error: str) -> None:
-        """
-        Emit a "download_error" event when an download job encounters an exception.
+    # endregion
 
-        :param source: Source URL
-        :param error_type: The name of the exception that raised the error
-        :param error: The traceback from this error
-        """
-        self.__emit_download_event(
-            event_name="download_error",
-            payload={
-                "source": source,
-                "error_type": error_type,
-                "error": error,
-            },
-        )
+    # region Model loading
 
-    def emit_model_install_downloading(
-        self,
-        source: str,
-        local_path: str,
-        bytes: int,
-        total_bytes: int,
-        parts: List[Dict[str, Union[str, int]]],
-        id: int,
+    def emit_model_load_started(self, config: "AnyModelConfig", submodel_type: Optional["SubModelType"] = None) -> None:
+        """Emitted when a model load is started."""
+        self.dispatch(ModelLoadStartedEvent.build(config, submodel_type))
+
+    def emit_model_load_complete(
+        self, config: "AnyModelConfig", submodel_type: Optional["SubModelType"] = None
     ) -> None:
-        """
-        Emit at intervals while the install job is in progress (remote models only).
+        """Emitted when a model load is complete."""
+        self.dispatch(ModelLoadCompleteEvent.build(config, submodel_type))
 
-        :param source: Source of the model
-        :param local_path: Where model is downloading to
-        :param parts: Progress of downloading URLs that comprise the model, if any.
-        :param bytes: Number of bytes downloaded so far.
-        :param total_bytes: Total size of download, including all files.
-        This emits a Dict with keys "source", "local_path", "bytes" and "total_bytes".
-        """
-        self.__emit_model_event(
-            event_name="model_install_downloading",
-            payload={
-                "source": source,
-                "local_path": local_path,
-                "bytes": bytes,
-                "total_bytes": total_bytes,
-                "parts": parts,
-                "id": id,
-            },
-        )
+    # endregion
 
-    def emit_model_install_downloads_done(self, source: str) -> None:
-        """
-        Emit once when all parts are downloaded, but before the probing and registration start.
+    # region Model install
 
-        :param source: Source of the model; local path, repo_id or url
-        """
-        self.__emit_model_event(
-            event_name="model_install_downloads_done",
-            payload={"source": source},
-        )
+    def emit_model_install_download_progress(self, job: "ModelInstallJob") -> None:
+        """Emitted at intervals while the install job is in progress (remote models only)."""
+        self.dispatch(ModelInstallDownloadProgressEvent.build(job))
 
-    def emit_model_install_running(self, source: str) -> None:
-        """
-        Emit once when an install job becomes active.
+    def emit_model_install_downloads_complete(self, job: "ModelInstallJob") -> None:
+        self.dispatch(ModelInstallDownloadsCompleteEvent.build(job))
 
-        :param source: Source of the model; local path, repo_id or url
-        """
-        self.__emit_model_event(
-            event_name="model_install_running",
-            payload={"source": source},
-        )
+    def emit_model_install_started(self, job: "ModelInstallJob") -> None:
+        """Emitted once when an install job is started (after any download)."""
+        self.dispatch(ModelInstallStartedEvent.build(job))
 
-    def emit_model_install_completed(self, source: str, key: str, id: int, total_bytes: Optional[int] = None) -> None:
-        """
-        Emit when an install job is completed successfully.
+    def emit_model_install_complete(self, job: "ModelInstallJob") -> None:
+        """Emitted when an install job is completed successfully."""
+        self.dispatch(ModelInstallCompleteEvent.build(job))
 
-        :param source: Source of the model; local path, repo_id or url
-        :param key: Model config record key
-        :param total_bytes: Size of the model (may be None for installation of a local path)
-        """
-        self.__emit_model_event(
-            event_name="model_install_completed",
-            payload={"source": source, "total_bytes": total_bytes, "key": key, "id": id},
-        )
+    def emit_model_install_cancelled(self, job: "ModelInstallJob") -> None:
+        """Emitted when an install job is cancelled."""
+        self.dispatch(ModelInstallCancelledEvent.build(job))
 
-    def emit_model_install_cancelled(self, source: str, id: int) -> None:
-        """
-        Emit when an install job is cancelled.
+    def emit_model_install_error(self, job: "ModelInstallJob") -> None:
+        """Emitted when an install job encounters an exception."""
+        self.dispatch(ModelInstallErrorEvent.build(job))
 
-        :param source: Source of the model; local path, repo_id or url
-        """
-        self.__emit_model_event(
-            event_name="model_install_cancelled",
-            payload={"source": source, "id": id},
-        )
+    # endregion
 
-    def emit_model_install_error(self, source: str, error_type: str, error: str, id: int) -> None:
-        """
-        Emit when an install job encounters an exception.
-
-        :param source: Source of the model
-        :param error_type: The name of the exception
-        :param error: A text description of the exception
-        """
-        self.__emit_model_event(
-            event_name="model_install_error",
-            payload={"source": source, "error_type": error_type, "error": error, "id": id},
-        )
+    # region Bulk image download
 
     def emit_bulk_download_started(
         self, bulk_download_id: str, bulk_download_item_id: str, bulk_download_item_name: str
     ) -> None:
-        """Emitted when a bulk download starts"""
-        self._emit_bulk_download_event(
-            event_name="bulk_download_started",
-            payload={
-                "bulk_download_id": bulk_download_id,
-                "bulk_download_item_id": bulk_download_item_id,
-                "bulk_download_item_name": bulk_download_item_name,
-            },
-        )
+        """Emitted when a bulk image download is started"""
+        self.dispatch(BulkDownloadStartedEvent.build(bulk_download_id, bulk_download_item_id, bulk_download_item_name))
 
-    def emit_bulk_download_completed(
+    def emit_bulk_download_complete(
         self, bulk_download_id: str, bulk_download_item_id: str, bulk_download_item_name: str
     ) -> None:
-        """Emitted when a bulk download completes"""
-        self._emit_bulk_download_event(
-            event_name="bulk_download_completed",
-            payload={
-                "bulk_download_id": bulk_download_id,
-                "bulk_download_item_id": bulk_download_item_id,
-                "bulk_download_item_name": bulk_download_item_name,
-            },
-        )
+        """Emitted when a bulk image download is complete"""
+        self.dispatch(BulkDownloadCompleteEvent.build(bulk_download_id, bulk_download_item_id, bulk_download_item_name))
 
-    def emit_bulk_download_failed(
+    def emit_bulk_download_error(
         self, bulk_download_id: str, bulk_download_item_id: str, bulk_download_item_name: str, error: str
     ) -> None:
-        """Emitted when a bulk download fails"""
-        self._emit_bulk_download_event(
-            event_name="bulk_download_failed",
-            payload={
-                "bulk_download_id": bulk_download_id,
-                "bulk_download_item_id": bulk_download_item_id,
-                "bulk_download_item_name": bulk_download_item_name,
-                "error": error,
-            },
+        """Emitted when a bulk image download has an error"""
+        self.dispatch(
+            BulkDownloadErrorEvent.build(bulk_download_id, bulk_download_item_id, bulk_download_item_name, error)
         )
+
+    # endregion

invokeai/app/services/events/events_common.py

@@ -0,0 +1,592 @@
+from math import floor
+from typing import TYPE_CHECKING, Any, ClassVar, Coroutine, Generic, Optional, Protocol, TypeAlias, TypeVar
+
+from fastapi_events.handlers.local import local_handler
+from fastapi_events.registry.payload_schema import registry as payload_schema
+from pydantic import BaseModel, ConfigDict, Field
+
+from invokeai.app.services.session_processor.session_processor_common import ProgressImage
+from invokeai.app.services.session_queue.session_queue_common import (
+    QUEUE_ITEM_STATUS,
+    BatchStatus,
+    EnqueueBatchResult,
+    SessionQueueItem,
+    SessionQueueStatus,
+)
+from invokeai.app.services.shared.graph import AnyInvocation, AnyInvocationOutput
+from invokeai.app.util.misc import get_timestamp
+from invokeai.backend.model_manager.config import AnyModelConfig, SubModelType
+from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineIntermediateState
+
+if TYPE_CHECKING:
+    from invokeai.app.services.download.download_base import DownloadJob
+    from invokeai.app.services.model_install.model_install_common import ModelInstallJob
+
+
+class EventBase(BaseModel):
+    """Base class for all events. All events must inherit from this class.
+
+    Events must define a class attribute `__event_name__` to identify the event.
+
+    All other attributes should be defined as normal for a pydantic model.
+
+    A timestamp is automatically added to the event when it is created.
+    """
+
+    __event_name__: ClassVar[str]
+    timestamp: int = Field(description="The timestamp of the event", default_factory=get_timestamp)
+
+    model_config = ConfigDict(json_schema_serialization_defaults_required=True)
+
+    @classmethod
+    def get_events(cls) -> set[type["EventBase"]]:
+        """Get a set of all event models."""
+
+        event_subclasses: set[type["EventBase"]] = set()
+        for subclass in cls.__subclasses__():
+            # We only want to include subclasses that are event models, not intermediary classes
+            if hasattr(subclass, "__event_name__"):
+                event_subclasses.add(subclass)
+            event_subclasses.update(subclass.get_events())
+
+        return event_subclasses
+
+
+TEvent = TypeVar("TEvent", bound=EventBase, contravariant=True)
+
+FastAPIEvent: TypeAlias = tuple[str, TEvent]
+"""
+A tuple representing a `fastapi-events` event, with the event name and payload.
+Provide a generic type to `TEvent` to specify the payload type.
+"""
+
+
+class FastAPIEventFunc(Protocol, Generic[TEvent]):
+    def __call__(self, event: FastAPIEvent[TEvent]) -> Optional[Coroutine[Any, Any, None]]: ...
+
+
+def register_events(events: set[type[TEvent]] | type[TEvent], func: FastAPIEventFunc[TEvent]) -> None:
+    """Register a function to handle specific events.
+
+    :param events: An event or set of events to handle
+    :param func: The function to handle the events
+    """
+    events = events if isinstance(events, set) else {events}
+    for event in events:
+        assert hasattr(event, "__event_name__")
+        local_handler.register(event_name=event.__event_name__, _func=func)  # pyright: ignore [reportUnknownMemberType, reportUnknownArgumentType, reportAttributeAccessIssue]
+
+
+class QueueEventBase(EventBase):
+    """Base class for queue events"""
+
+    queue_id: str = Field(description="The ID of the queue")
+
+
+class QueueItemEventBase(QueueEventBase):
+    """Base class for queue item events"""
+
+    item_id: int = Field(description="The ID of the queue item")
+    batch_id: str = Field(description="The ID of the queue batch")
+
+
+class InvocationEventBase(QueueItemEventBase):
+    """Base class for invocation events"""
+
+    session_id: str = Field(description="The ID of the session (aka graph execution state)")
+    queue_id: str = Field(description="The ID of the queue")
+    item_id: int = Field(description="The ID of the queue item")
+    batch_id: str = Field(description="The ID of the queue batch")
+    session_id: str = Field(description="The ID of the session (aka graph execution state)")
+    invocation: AnyInvocation = Field(description="The ID of the invocation")
+    invocation_source_id: str = Field(description="The ID of the prepared invocation's source node")
+
+
+@payload_schema.register
+class InvocationStartedEvent(InvocationEventBase):
+    """Event model for invocation_started"""
+
+    __event_name__ = "invocation_started"
+
+    @classmethod
+    def build(cls, queue_item: SessionQueueItem, invocation: AnyInvocation) -> "InvocationStartedEvent":
+        return cls(
+            queue_id=queue_item.queue_id,
+            item_id=queue_item.item_id,
+            batch_id=queue_item.batch_id,
+            session_id=queue_item.session_id,
+            invocation=invocation,
+            invocation_source_id=queue_item.session.prepared_source_mapping[invocation.id],
+        )
+
+
+@payload_schema.register
+class InvocationDenoiseProgressEvent(InvocationEventBase):
+    """Event model for invocation_denoise_progress"""
+
+    __event_name__ = "invocation_denoise_progress"
+
+    progress_image: ProgressImage = Field(description="The progress image sent at each step during processing")
+    step: int = Field(description="The current step of the invocation")
+    total_steps: int = Field(description="The total number of steps in the invocation")
+    order: int = Field(description="The order of the invocation in the session")
+    percentage: float = Field(description="The percentage of completion of the invocation")
+
+    @classmethod
+    def build(
+        cls,
+        queue_item: SessionQueueItem,
+        invocation: AnyInvocation,
+        intermediate_state: PipelineIntermediateState,
+        progress_image: ProgressImage,
+    ) -> "InvocationDenoiseProgressEvent":
+        step = intermediate_state.step
+        total_steps = intermediate_state.total_steps
+        order = intermediate_state.order
+        return cls(
+            queue_id=queue_item.queue_id,
+            item_id=queue_item.item_id,
+            batch_id=queue_item.batch_id,
+            session_id=queue_item.session_id,
+            invocation=invocation,
+            invocation_source_id=queue_item.session.prepared_source_mapping[invocation.id],
+            progress_image=progress_image,
+            step=step,
+            total_steps=total_steps,
+            order=order,
+            percentage=cls.calc_percentage(step, total_steps, order),
+        )
+
+    @staticmethod
+    def calc_percentage(step: int, total_steps: int, scheduler_order: float) -> float:
+        """Calculate the percentage of completion of denoising."""
+        if total_steps == 0:
+            return 0.0
+        if scheduler_order == 2:
+            return floor((step + 1 + 1) / 2) / floor((total_steps + 1) / 2)
+        # order == 1
+        return (step + 1 + 1) / (total_steps + 1)
+
+
+@payload_schema.register
+class InvocationCompleteEvent(InvocationEventBase):
+    """Event model for invocation_complete"""
+
+    __event_name__ = "invocation_complete"
+
+    result: AnyInvocationOutput = Field(description="The result of the invocation")
+
+    @classmethod
+    def build(
+        cls, queue_item: SessionQueueItem, invocation: AnyInvocation, result: AnyInvocationOutput
+    ) -> "InvocationCompleteEvent":
+        return cls(
+            queue_id=queue_item.queue_id,
+            item_id=queue_item.item_id,
+            batch_id=queue_item.batch_id,
+            session_id=queue_item.session_id,
+            invocation=invocation,
+            invocation_source_id=queue_item.session.prepared_source_mapping[invocation.id],
+            result=result,
+        )
+
+
+@payload_schema.register
+class InvocationErrorEvent(InvocationEventBase):
+    """Event model for invocation_error"""
+
+    __event_name__ = "invocation_error"
+
+    error_type: str = Field(description="The error type")
+    error_message: str = Field(description="The error message")
+    error_traceback: str = Field(description="The error traceback")
+    user_id: Optional[str] = Field(default=None, description="The ID of the user who created the invocation")
+    project_id: Optional[str] = Field(default=None, description="The ID of the user who created the invocation")
+
+    @classmethod
+    def build(
+        cls,
+        queue_item: SessionQueueItem,
+        invocation: AnyInvocation,
+        error_type: str,
+        error_message: str,
+        error_traceback: str,
+    ) -> "InvocationErrorEvent":
+        return cls(
+            queue_id=queue_item.queue_id,
+            item_id=queue_item.item_id,
+            batch_id=queue_item.batch_id,
+            session_id=queue_item.session_id,
+            invocation=invocation,
+            invocation_source_id=queue_item.session.prepared_source_mapping[invocation.id],
+            error_type=error_type,
+            error_message=error_message,
+            error_traceback=error_traceback,
+            user_id=getattr(queue_item, "user_id", None),
+            project_id=getattr(queue_item, "project_id", None),
+        )
+
+
+@payload_schema.register
+class QueueItemStatusChangedEvent(QueueItemEventBase):
+    """Event model for queue_item_status_changed"""
+
+    __event_name__ = "queue_item_status_changed"
+
+    status: QUEUE_ITEM_STATUS = Field(description="The new status of the queue item")
+    error_type: Optional[str] = Field(default=None, description="The error type, if any")
+    error_message: Optional[str] = Field(default=None, description="The error message, if any")
+    error_traceback: Optional[str] = Field(default=None, description="The error traceback, if any")
+    created_at: Optional[str] = Field(default=None, description="The timestamp when the queue item was created")
+    updated_at: Optional[str] = Field(default=None, description="The timestamp when the queue item was last updated")
+    started_at: Optional[str] = Field(default=None, description="The timestamp when the queue item was started")
+    completed_at: Optional[str] = Field(default=None, description="The timestamp when the queue item was completed")
+    batch_status: BatchStatus = Field(description="The status of the batch")
+    queue_status: SessionQueueStatus = Field(description="The status of the queue")
+    session_id: str = Field(description="The ID of the session (aka graph execution state)")
+
+    @classmethod
+    def build(
+        cls, queue_item: SessionQueueItem, batch_status: BatchStatus, queue_status: SessionQueueStatus
+    ) -> "QueueItemStatusChangedEvent":
+        return cls(
+            queue_id=queue_item.queue_id,
+            item_id=queue_item.item_id,
+            batch_id=queue_item.batch_id,
+            session_id=queue_item.session_id,
+            status=queue_item.status,
+            error_type=queue_item.error_type,
+            error_message=queue_item.error_message,
+            error_traceback=queue_item.error_traceback,
+            created_at=str(queue_item.created_at) if queue_item.created_at else None,
+            updated_at=str(queue_item.updated_at) if queue_item.updated_at else None,
+            started_at=str(queue_item.started_at) if queue_item.started_at else None,
+            completed_at=str(queue_item.completed_at) if queue_item.completed_at else None,
+            batch_status=batch_status,
+            queue_status=queue_status,
+        )
+
+
+@payload_schema.register
+class BatchEnqueuedEvent(QueueEventBase):
+    """Event model for batch_enqueued"""
+
+    __event_name__ = "batch_enqueued"
+
+    batch_id: str = Field(description="The ID of the batch")
+    enqueued: int = Field(description="The number of invocations enqueued")
+    requested: int = Field(
+        description="The number of invocations initially requested to be enqueued (may be less than enqueued if queue was full)"
+    )
+    priority: int = Field(description="The priority of the batch")
+
+    @classmethod
+    def build(cls, enqueue_result: EnqueueBatchResult) -> "BatchEnqueuedEvent":
+        return cls(
+            queue_id=enqueue_result.queue_id,
+            batch_id=enqueue_result.batch.batch_id,
+            enqueued=enqueue_result.enqueued,
+            requested=enqueue_result.requested,
+            priority=enqueue_result.priority,
+        )
+
+
+@payload_schema.register
+class QueueClearedEvent(QueueEventBase):
+    """Event model for queue_cleared"""
+
+    __event_name__ = "queue_cleared"
+
+    @classmethod
+    def build(cls, queue_id: str) -> "QueueClearedEvent":
+        return cls(queue_id=queue_id)
+
+
+class DownloadEventBase(EventBase):
+    """Base class for events associated with a download"""
+
+    source: str = Field(description="The source of the download")
+
+
+@payload_schema.register
+class DownloadStartedEvent(DownloadEventBase):
+    """Event model for download_started"""
+
+    __event_name__ = "download_started"
+
+    download_path: str = Field(description="The local path where the download is saved")
+
+    @classmethod
+    def build(cls, job: "DownloadJob") -> "DownloadStartedEvent":
+        assert job.download_path
+        return cls(source=str(job.source), download_path=job.download_path.as_posix())
+
+
+@payload_schema.register
+class DownloadProgressEvent(DownloadEventBase):
+    """Event model for download_progress"""
+
+    __event_name__ = "download_progress"
+
+    download_path: str = Field(description="The local path where the download is saved")
+    current_bytes: int = Field(description="The number of bytes downloaded so far")
+    total_bytes: int = Field(description="The total number of bytes to be downloaded")
+
+    @classmethod
+    def build(cls, job: "DownloadJob") -> "DownloadProgressEvent":
+        assert job.download_path
+        return cls(
+            source=str(job.source),
+            download_path=job.download_path.as_posix(),
+            current_bytes=job.bytes,
+            total_bytes=job.total_bytes,
+        )
+
+
+@payload_schema.register
+class DownloadCompleteEvent(DownloadEventBase):
+    """Event model for download_complete"""
+
+    __event_name__ = "download_complete"
+
+    download_path: str = Field(description="The local path where the download is saved")
+    total_bytes: int = Field(description="The total number of bytes downloaded")
+
+    @classmethod
+    def build(cls, job: "DownloadJob") -> "DownloadCompleteEvent":
+        assert job.download_path
+        return cls(source=str(job.source), download_path=job.download_path.as_posix(), total_bytes=job.total_bytes)
+
+
+@payload_schema.register
+class DownloadCancelledEvent(DownloadEventBase):
+    """Event model for download_cancelled"""
+
+    __event_name__ = "download_cancelled"
+
+    @classmethod
+    def build(cls, job: "DownloadJob") -> "DownloadCancelledEvent":
+        return cls(source=str(job.source))
+
+
+@payload_schema.register
+class DownloadErrorEvent(DownloadEventBase):
+    """Event model for download_error"""
+
+    __event_name__ = "download_error"
+
+    error_type: str = Field(description="The type of error")
+    error: str = Field(description="The error message")
+
+    @classmethod
+    def build(cls, job: "DownloadJob") -> "DownloadErrorEvent":
+        assert job.error_type
+        assert job.error
+        return cls(source=str(job.source), error_type=job.error_type, error=job.error)
+
+
+class ModelEventBase(EventBase):
+    """Base class for events associated with a model"""
+
+
+@payload_schema.register
+class ModelLoadStartedEvent(ModelEventBase):
+    """Event model for model_load_started"""
+
+    __event_name__ = "model_load_started"
+
+    config: AnyModelConfig = Field(description="The model's config")
+    submodel_type: Optional[SubModelType] = Field(default=None, description="The submodel type, if any")
+
+    @classmethod
+    def build(cls, config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> "ModelLoadStartedEvent":
+        return cls(config=config, submodel_type=submodel_type)
+
+
+@payload_schema.register
+class ModelLoadCompleteEvent(ModelEventBase):
+    """Event model for model_load_complete"""
+
+    __event_name__ = "model_load_complete"
+
+    config: AnyModelConfig = Field(description="The model's config")
+    submodel_type: Optional[SubModelType] = Field(default=None, description="The submodel type, if any")
+
+    @classmethod
+    def build(cls, config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> "ModelLoadCompleteEvent":
+        return cls(config=config, submodel_type=submodel_type)
+
+
+@payload_schema.register
+class ModelInstallDownloadProgressEvent(ModelEventBase):
+    """Event model for model_install_download_progress"""
+
+    __event_name__ = "model_install_download_progress"
+
+    id: int = Field(description="The ID of the install job")
+    source: str = Field(description="Source of the model; local path, repo_id or url")
+    local_path: str = Field(description="Where model is downloading to")
+    bytes: int = Field(description="Number of bytes downloaded so far")
+    total_bytes: int = Field(description="Total size of download, including all files")
+    parts: list[dict[str, int | str]] = Field(
+        description="Progress of downloading URLs that comprise the model, if any"
+    )
+
+    @classmethod
+    def build(cls, job: "ModelInstallJob") -> "ModelInstallDownloadProgressEvent":
+        parts: list[dict[str, str | int]] = [
+            {
+                "url": str(x.source),
+                "local_path": str(x.download_path),
+                "bytes": x.bytes,
+                "total_bytes": x.total_bytes,
+            }
+            for x in job.download_parts
+        ]
+        return cls(
+            id=job.id,
+            source=str(job.source),
+            local_path=job.local_path.as_posix(),
+            parts=parts,
+            bytes=job.bytes,
+            total_bytes=job.total_bytes,
+        )
+
+
+@payload_schema.register
+class ModelInstallDownloadsCompleteEvent(ModelEventBase):
+    """Emitted once when an install job becomes active."""
+
+    __event_name__ = "model_install_downloads_complete"
+
+    id: int = Field(description="The ID of the install job")
+    source: str = Field(description="Source of the model; local path, repo_id or url")
+
+    @classmethod
+    def build(cls, job: "ModelInstallJob") -> "ModelInstallDownloadsCompleteEvent":
+        return cls(id=job.id, source=str(job.source))
+
+
+@payload_schema.register
+class ModelInstallStartedEvent(ModelEventBase):
+    """Event model for model_install_started"""
+
+    __event_name__ = "model_install_started"
+
+    id: int = Field(description="The ID of the install job")
+    source: str = Field(description="Source of the model; local path, repo_id or url")
+
+    @classmethod
+    def build(cls, job: "ModelInstallJob") -> "ModelInstallStartedEvent":
+        return cls(id=job.id, source=str(job.source))
+
+
+@payload_schema.register
+class ModelInstallCompleteEvent(ModelEventBase):
+    """Event model for model_install_complete"""
+
+    __event_name__ = "model_install_complete"
+
+    id: int = Field(description="The ID of the install job")
+    source: str = Field(description="Source of the model; local path, repo_id or url")
+    key: str = Field(description="Model config record key")
+    total_bytes: Optional[int] = Field(description="Size of the model (may be None for installation of a local path)")
+
+    @classmethod
+    def build(cls, job: "ModelInstallJob") -> "ModelInstallCompleteEvent":
+        assert job.config_out is not None
+        return cls(id=job.id, source=str(job.source), key=(job.config_out.key), total_bytes=job.total_bytes)
+
+
+@payload_schema.register
+class ModelInstallCancelledEvent(ModelEventBase):
+    """Event model for model_install_cancelled"""
+
+    __event_name__ = "model_install_cancelled"
+
+    id: int = Field(description="The ID of the install job")
+    source: str = Field(description="Source of the model; local path, repo_id or url")
+
+    @classmethod
+    def build(cls, job: "ModelInstallJob") -> "ModelInstallCancelledEvent":
+        return cls(id=job.id, source=str(job.source))
+
+
+@payload_schema.register
+class ModelInstallErrorEvent(ModelEventBase):
+    """Event model for model_install_error"""
+
+    __event_name__ = "model_install_error"
+
+    id: int = Field(description="The ID of the install job")
+    source: str = Field(description="Source of the model; local path, repo_id or url")
+    error_type: str = Field(description="The name of the exception")
+    error: str = Field(description="A text description of the exception")
+
+    @classmethod
+    def build(cls, job: "ModelInstallJob") -> "ModelInstallErrorEvent":
+        assert job.error_type is not None
+        assert job.error is not None
+        return cls(id=job.id, source=str(job.source), error_type=job.error_type, error=job.error)
+
+
+class BulkDownloadEventBase(EventBase):
+    """Base class for events associated with a bulk image download"""
+
+    bulk_download_id: str = Field(description="The ID of the bulk image download")
+    bulk_download_item_id: str = Field(description="The ID of the bulk image download item")
+    bulk_download_item_name: str = Field(description="The name of the bulk image download item")
+
+
+@payload_schema.register
+class BulkDownloadStartedEvent(BulkDownloadEventBase):
+    """Event model for bulk_download_started"""
+
+    __event_name__ = "bulk_download_started"
+
+    @classmethod
+    def build(
+        cls, bulk_download_id: str, bulk_download_item_id: str, bulk_download_item_name: str
+    ) -> "BulkDownloadStartedEvent":
+        return cls(
+            bulk_download_id=bulk_download_id,
+            bulk_download_item_id=bulk_download_item_id,
+            bulk_download_item_name=bulk_download_item_name,
+        )
+
+
+@payload_schema.register
+class BulkDownloadCompleteEvent(BulkDownloadEventBase):
+    """Event model for bulk_download_complete"""
+
+    __event_name__ = "bulk_download_complete"
+
+    @classmethod
+    def build(
+        cls, bulk_download_id: str, bulk_download_item_id: str, bulk_download_item_name: str
+    ) -> "BulkDownloadCompleteEvent":
+        return cls(
+            bulk_download_id=bulk_download_id,
+            bulk_download_item_id=bulk_download_item_id,
+            bulk_download_item_name=bulk_download_item_name,
+        )
+
+
+@payload_schema.register
+class BulkDownloadErrorEvent(BulkDownloadEventBase):
+    """Event model for bulk_download_error"""
+
+    __event_name__ = "bulk_download_error"
+
+    error: str = Field(description="The error message")
+
+    @classmethod
+    def build(
+        cls, bulk_download_id: str, bulk_download_item_id: str, bulk_download_item_name: str, error: str
+    ) -> "BulkDownloadErrorEvent":
+        return cls(
+            bulk_download_id=bulk_download_id,
+            bulk_download_item_id=bulk_download_item_id,
+            bulk_download_item_name=bulk_download_item_name,
+            error=error,
+        )

invokeai/app/services/events/events_fastapievents.py

@@ -0,0 +1,47 @@
+# Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
+
+import asyncio
+import threading
+from queue import Empty, Queue
+
+from fastapi_events.dispatcher import dispatch
+
+from invokeai.app.services.events.events_common import (
+    EventBase,
+)
+
+from .events_base import EventServiceBase
+
+
+class FastAPIEventService(EventServiceBase):
+    def __init__(self, event_handler_id: int) -> None:
+        self.event_handler_id = event_handler_id
+        self._queue = Queue[EventBase | None]()
+        self._stop_event = threading.Event()
+        asyncio.create_task(self._dispatch_from_queue(stop_event=self._stop_event))
+
+        super().__init__()
+
+    def stop(self, *args, **kwargs):
+        self._stop_event.set()
+        self._queue.put(None)
+
+    def dispatch(self, event: EventBase) -> None:
+        self._queue.put(event)
+
+    async def _dispatch_from_queue(self, stop_event: threading.Event):
+        """Get events on from the queue and dispatch them, from the correct thread"""
+        while not stop_event.is_set():
+            try:
+                event = self._queue.get(block=False)
+                if not event:  # Probably stopping
+                    continue
+                # Leave the payloads as live pydantic models
+                dispatch(event, middleware_id=self.event_handler_id, payload_schema_dump=False)
+
+            except Empty:
+                await asyncio.sleep(0.1)
+                pass
+
+            except asyncio.CancelledError as e:
+                raise e  # Raise a proper error

invokeai/app/services/image_files/image_files_base.py

@@ -4,9 +4,6 @@ from typing import Optional
 
 from PIL.Image import Image as PILImageType
 
-from invokeai.app.invocations.fields import MetadataField
-from invokeai.app.services.workflow_records.workflow_records_common import WorkflowWithoutID
-
 
 class ImageFileStorageBase(ABC):
     """Low-level service responsible for storing and retrieving image files."""
@@ -33,8 +30,9 @@ class ImageFileStorageBase(ABC):
         self,
         image: PILImageType,
         image_name: str,
-        metadata: Optional[MetadataField] = None,
-        workflow: Optional[WorkflowWithoutID] = None,
+        metadata: Optional[str] = None,
+        workflow: Optional[str] = None,
+        graph: Optional[str] = None,
         thumbnail_size: int = 256,
     ) -> None:
         """Saves an image and a 256x256 WEBP thumbnail. Returns a tuple of the image name, thumbnail name, and created timestamp."""
@@ -46,6 +44,11 @@ class ImageFileStorageBase(ABC):
         pass
 
     @abstractmethod
-    def get_workflow(self, image_name: str) -> Optional[WorkflowWithoutID]:
+    def get_workflow(self, image_name: str) -> Optional[str]:
         """Gets the workflow of an image."""
         pass
+
+    @abstractmethod
+    def get_graph(self, image_name: str) -> Optional[str]:
+        """Gets the graph of an image."""
+        pass

invokeai/app/services/image_files/image_files_disk.py

@@ -7,9 +7,7 @@ from PIL import Image, PngImagePlugin
 from PIL.Image import Image as PILImageType
 from send2trash import send2trash
 
-from invokeai.app.invocations.fields import MetadataField
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.workflow_records.workflow_records_common import WorkflowWithoutID
 from invokeai.app.util.thumbnails import get_thumbnail_name, make_thumbnail
 
 from .image_files_base import ImageFileStorageBase
@@ -56,8 +54,9 @@ class DiskImageFileStorage(ImageFileStorageBase):
         self,
         image: PILImageType,
         image_name: str,
-        metadata: Optional[MetadataField] = None,
-        workflow: Optional[WorkflowWithoutID] = None,
+        metadata: Optional[str] = None,
+        workflow: Optional[str] = None,
+        graph: Optional[str] = None,
         thumbnail_size: int = 256,
     ) -> None:
         try:
@@ -68,13 +67,14 @@ class DiskImageFileStorage(ImageFileStorageBase):
             info_dict = {}
 
             if metadata is not None:
-                metadata_json = metadata.model_dump_json()
-                info_dict["invokeai_metadata"] = metadata_json
-                pnginfo.add_text("invokeai_metadata", metadata_json)
+                info_dict["invokeai_metadata"] = metadata
+                pnginfo.add_text("invokeai_metadata", metadata)
             if workflow is not None:
-                workflow_json = workflow.model_dump_json()
-                info_dict["invokeai_workflow"] = workflow_json
-                pnginfo.add_text("invokeai_workflow", workflow_json)
+                info_dict["invokeai_workflow"] = workflow
+                pnginfo.add_text("invokeai_workflow", workflow)
+            if graph is not None:
+                info_dict["invokeai_graph"] = graph
+                pnginfo.add_text("invokeai_graph", graph)
 
             # When saving the image, the image object's info field is not populated. We need to set it
             image.info = info_dict
@@ -129,11 +129,18 @@ class DiskImageFileStorage(ImageFileStorageBase):
         path = path if isinstance(path, Path) else Path(path)
         return path.exists()
 
-    def get_workflow(self, image_name: str) -> WorkflowWithoutID | None:
+    def get_workflow(self, image_name: str) -> str | None:
         image = self.get(image_name)
         workflow = image.info.get("invokeai_workflow", None)
-        if workflow is not None:
-            return WorkflowWithoutID.model_validate_json(workflow)
+        if isinstance(workflow, str):
+            return workflow
+        return None
+
+    def get_graph(self, image_name: str) -> str | None:
+        image = self.get(image_name)
+        graph = image.info.get("invokeai_graph", None)
+        if isinstance(graph, str):
+            return graph
         return None
 
     def __validate_storage_folders(self) -> None:

invokeai/app/services/image_records/image_records_base.py

@@ -80,7 +80,7 @@ class ImageRecordStorageBase(ABC):
         starred: Optional[bool] = False,
         session_id: Optional[str] = None,
         node_id: Optional[str] = None,
-        metadata: Optional[MetadataField] = None,
+        metadata: Optional[str] = None,
     ) -> datetime:
         """Saves an image record."""
         pass

invokeai/app/services/image_records/image_records_sqlite.py

@@ -328,10 +328,9 @@ class SqliteImageRecordStorage(ImageRecordStorageBase):
         starred: Optional[bool] = False,
         session_id: Optional[str] = None,
         node_id: Optional[str] = None,
-        metadata: Optional[MetadataField] = None,
+        metadata: Optional[str] = None,
     ) -> datetime:
         try:
-            metadata_json = metadata.model_dump_json() if metadata is not None else None
             self._lock.acquire()
             self._cursor.execute(
                 """--sql
@@ -358,7 +357,7 @@ class SqliteImageRecordStorage(ImageRecordStorageBase):
                     height,
                     node_id,
                     session_id,
-                    metadata_json,
+                    metadata,
                     is_intermediate,
                     starred,
                     has_workflow,

invokeai/app/services/images/images_base.py

@@ -12,7 +12,6 @@ from invokeai.app.services.image_records.image_records_common import (
 )
 from invokeai.app.services.images.images_common import ImageDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.workflow_records.workflow_records_common import WorkflowWithoutID
 
 
 class ImageServiceABC(ABC):
@@ -51,8 +50,9 @@ class ImageServiceABC(ABC):
         session_id: Optional[str] = None,
         board_id: Optional[str] = None,
         is_intermediate: Optional[bool] = False,
-        metadata: Optional[MetadataField] = None,
-        workflow: Optional[WorkflowWithoutID] = None,
+        metadata: Optional[str] = None,
+        workflow: Optional[str] = None,
+        graph: Optional[str] = None,
     ) -> ImageDTO:
         """Creates an image, storing the file and its metadata."""
         pass
@@ -87,7 +87,12 @@ class ImageServiceABC(ABC):
         pass
 
     @abstractmethod
-    def get_workflow(self, image_name: str) -> Optional[WorkflowWithoutID]:
+    def get_workflow(self, image_name: str) -> Optional[str]:
+        """Gets an image's workflow."""
+        pass
+
+    @abstractmethod
+    def get_graph(self, image_name: str) -> Optional[str]:
         """Gets an image's workflow."""
         pass
 

invokeai/app/services/images/images_default.py

@@ -5,7 +5,6 @@ from PIL.Image import Image as PILImageType
 from invokeai.app.invocations.fields import MetadataField
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.workflow_records.workflow_records_common import WorkflowWithoutID
 
 from ..image_files.image_files_common import (
     ImageFileDeleteException,
@@ -42,8 +41,9 @@ class ImageService(ImageServiceABC):
         session_id: Optional[str] = None,
         board_id: Optional[str] = None,
         is_intermediate: Optional[bool] = False,
-        metadata: Optional[MetadataField] = None,
-        workflow: Optional[WorkflowWithoutID] = None,
+        metadata: Optional[str] = None,
+        workflow: Optional[str] = None,
+        graph: Optional[str] = None,
     ) -> ImageDTO:
         if image_origin not in ResourceOrigin:
             raise InvalidOriginException
@@ -64,7 +64,7 @@ class ImageService(ImageServiceABC):
                 image_category=image_category,
                 width=width,
                 height=height,
-                has_workflow=workflow is not None,
+                has_workflow=workflow is not None or graph is not None,
                 # Meta fields
                 is_intermediate=is_intermediate,
                 # Nullable fields
@@ -75,7 +75,7 @@ class ImageService(ImageServiceABC):
             if board_id is not None:
                 self.__invoker.services.board_image_records.add_image_to_board(board_id=board_id, image_name=image_name)
             self.__invoker.services.image_files.save(
-                image_name=image_name, image=image, metadata=metadata, workflow=workflow
+                image_name=image_name, image=image, metadata=metadata, workflow=workflow, graph=graph
             )
             image_dto = self.get_dto(image_name)
 
@@ -157,7 +157,7 @@ class ImageService(ImageServiceABC):
             self.__invoker.services.logger.error("Problem getting image metadata")
             raise e
 
-    def get_workflow(self, image_name: str) -> Optional[WorkflowWithoutID]:
+    def get_workflow(self, image_name: str) -> Optional[str]:
         try:
             return self.__invoker.services.image_files.get_workflow(image_name)
         except ImageFileNotFoundException:
@@ -167,6 +167,16 @@ class ImageService(ImageServiceABC):
             self.__invoker.services.logger.error("Problem getting image workflow")
             raise
 
+    def get_graph(self, image_name: str) -> Optional[str]:
+        try:
+            return self.__invoker.services.image_files.get_graph(image_name)
+        except ImageFileNotFoundException:
+            self.__invoker.services.logger.error("Image file not found")
+            raise
+        except Exception:
+            self.__invoker.services.logger.error("Problem getting image graph")
+            raise
+
     def get_path(self, image_name: str, thumbnail: bool = False) -> str:
         try:
             return str(self.__invoker.services.image_files.get_path(image_name, thumbnail))

invokeai/app/services/model_install/__init__.py

@@ -1,11 +1,13 @@
 """Initialization file for model install service package."""
 
 from .model_install_base import (
+    ModelInstallServiceBase,
+)
+from .model_install_common import (
     HFModelSource,
     InstallStatus,
     LocalModelSource,
     ModelInstallJob,
-    ModelInstallServiceBase,
     ModelSource,
     UnknownInstallJobException,
     URLModelSource,

invokeai/app/services/model_install/model_install_base.py

@@ -1,244 +1,19 @@
 # Copyright 2023 Lincoln D. Stein and the InvokeAI development team
 """Baseclass definitions for the model installer."""
 
-import re
-import traceback
 from abc import ABC, abstractmethod
-from enum import Enum
 from pathlib import Path
-from typing import Any, Dict, List, Literal, Optional, Set, Union
+from typing import Any, Dict, List, Optional, Union
 
-from pydantic import BaseModel, Field, PrivateAttr, field_validator
 from pydantic.networks import AnyHttpUrl
-from typing_extensions import Annotated
 
 from invokeai.app.services.config import InvokeAIAppConfig
-from invokeai.app.services.download import DownloadJob, DownloadQueueServiceBase
+from invokeai.app.services.download import DownloadQueueServiceBase
 from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
+from invokeai.app.services.model_install.model_install_common import ModelInstallJob, ModelSource
 from invokeai.app.services.model_records import ModelRecordServiceBase
-from invokeai.backend.model_manager import AnyModelConfig, ModelRepoVariant
-from invokeai.backend.model_manager.config import ModelSourceType
-from invokeai.backend.model_manager.metadata import AnyModelRepoMetadata
-
-
-class InstallStatus(str, Enum):
-    """State of an install job running in the background."""
-
-    WAITING = "waiting"  # waiting to be dequeued
-    DOWNLOADING = "downloading"  # downloading of model files in process
-    DOWNLOADS_DONE = "downloads_done"  # downloading done, waiting to run
-    RUNNING = "running"  # being processed
-    COMPLETED = "completed"  # finished running
-    ERROR = "error"  # terminated with an error message
-    CANCELLED = "cancelled"  # terminated with an error message
-
-
-class ModelInstallPart(BaseModel):
-    url: AnyHttpUrl
-    path: Path
-    bytes: int = 0
-    total_bytes: int = 0
-
-
-class UnknownInstallJobException(Exception):
-    """Raised when the status of an unknown job is requested."""
-
-
-class StringLikeSource(BaseModel):
-    """
-    Base class for model sources, implements functions that lets the source be sorted and indexed.
-
-    These shenanigans let this stuff work:
-
-      source1 = LocalModelSource(path='C:/users/mort/foo.safetensors')
-      mydict = {source1: 'model 1'}
-      assert mydict['C:/users/mort/foo.safetensors'] == 'model 1'
-      assert mydict[LocalModelSource(path='C:/users/mort/foo.safetensors')] == 'model 1'
-
-      source2 = LocalModelSource(path=Path('C:/users/mort/foo.safetensors'))
-      assert source1 == source2
-      assert source1 == 'C:/users/mort/foo.safetensors'
-    """
-
-    def __hash__(self) -> int:
-        """Return hash of the path field, for indexing."""
-        return hash(str(self))
-
-    def __lt__(self, other: object) -> int:
-        """Return comparison of the stringified version, for sorting."""
-        return str(self) < str(other)
-
-    def __eq__(self, other: object) -> bool:
-        """Return equality on the stringified version."""
-        if isinstance(other, Path):
-            return str(self) == other.as_posix()
-        else:
-            return str(self) == str(other)
-
-
-class LocalModelSource(StringLikeSource):
-    """A local file or directory path."""
-
-    path: str | Path
-    inplace: Optional[bool] = False
-    type: Literal["local"] = "local"
-
-    # these methods allow the source to be used in a string-like way,
-    # for example as an index into a dict
-    def __str__(self) -> str:
-        """Return string version of path when string rep needed."""
-        return Path(self.path).as_posix()
-
-
-class HFModelSource(StringLikeSource):
-    """
-    A HuggingFace repo_id with optional variant, sub-folder and access token.
-    Note that the variant option, if not provided to the constructor, will default to fp16, which is
-    what people (almost) always want.
-    """
-
-    repo_id: str
-    variant: Optional[ModelRepoVariant] = ModelRepoVariant.FP16
-    subfolder: Optional[Path] = None
-    access_token: Optional[str] = None
-    type: Literal["hf"] = "hf"
-
-    @field_validator("repo_id")
-    @classmethod
-    def proper_repo_id(cls, v: str) -> str:  # noqa D102
-        if not re.match(r"^([.\w-]+/[.\w-]+)$", v):
-            raise ValueError(f"{v}: invalid repo_id format")
-        return v
-
-    def __str__(self) -> str:
-        """Return string version of repoid when string rep needed."""
-        base: str = self.repo_id
-        if self.variant:
-            base += f":{self.variant or ''}"
-        if self.subfolder:
-            base += f":{self.subfolder}"
-        return base
-
-
-class URLModelSource(StringLikeSource):
-    """A generic URL point to a checkpoint file."""
-
-    url: AnyHttpUrl
-    access_token: Optional[str] = None
-    type: Literal["url"] = "url"
-
-    def __str__(self) -> str:
-        """Return string version of the url when string rep needed."""
-        return str(self.url)
-
-
-ModelSource = Annotated[Union[LocalModelSource, HFModelSource, URLModelSource], Field(discriminator="type")]
-
-MODEL_SOURCE_TO_TYPE_MAP = {
-    URLModelSource: ModelSourceType.Url,
-    HFModelSource: ModelSourceType.HFRepoID,
-    LocalModelSource: ModelSourceType.Path,
-}
-
-
-class ModelInstallJob(BaseModel):
-    """Object that tracks the current status of an install request."""
-
-    id: int = Field(description="Unique ID for this job")
-    status: InstallStatus = Field(default=InstallStatus.WAITING, description="Current status of install process")
-    error_reason: Optional[str] = Field(default=None, description="Information about why the job failed")
-    config_in: Dict[str, Any] = Field(
-        default_factory=dict, description="Configuration information (e.g. 'description') to apply to model."
-    )
-    config_out: Optional[AnyModelConfig] = Field(
-        default=None, description="After successful installation, this will hold the configuration object."
-    )
-    inplace: bool = Field(
-        default=False, description="Leave model in its current location; otherwise install under models directory"
-    )
-    source: ModelSource = Field(description="Source (URL, repo_id, or local path) of model")
-    local_path: Path = Field(description="Path to locally-downloaded model; may be the same as the source")
-    bytes: int = Field(
-        default=0, description="For a remote model, the number of bytes downloaded so far (may not be available)"
-    )
-    total_bytes: int = Field(default=0, description="Total size of the model to be installed")
-    source_metadata: Optional[AnyModelRepoMetadata] = Field(
-        default=None, description="Metadata provided by the model source"
-    )
-    download_parts: Set[DownloadJob] = Field(
-        default_factory=set, description="Download jobs contributing to this install"
-    )
-    error: Optional[str] = Field(
-        default=None, description="On an error condition, this field will contain the text of the exception"
-    )
-    error_traceback: Optional[str] = Field(
-        default=None, description="On an error condition, this field will contain the exception traceback"
-    )
-    # internal flags and transitory settings
-    _install_tmpdir: Optional[Path] = PrivateAttr(default=None)
-    _exception: Optional[Exception] = PrivateAttr(default=None)
-
-    def set_error(self, e: Exception) -> None:
-        """Record the error and traceback from an exception."""
-        self._exception = e
-        self.error = str(e)
-        self.error_traceback = self._format_error(e)
-        self.status = InstallStatus.ERROR
-        self.error_reason = self._exception.__class__.__name__ if self._exception else None
-
-    def cancel(self) -> None:
-        """Call to cancel the job."""
-        self.status = InstallStatus.CANCELLED
-
-    @property
-    def error_type(self) -> Optional[str]:
-        """Class name of the exception that led to status==ERROR."""
-        return self._exception.__class__.__name__ if self._exception else None
-
-    def _format_error(self, exception: Exception) -> str:
-        """Error traceback."""
-        return "".join(traceback.format_exception(exception))
-
-    @property
-    def cancelled(self) -> bool:
-        """Set status to CANCELLED."""
-        return self.status == InstallStatus.CANCELLED
-
-    @property
-    def errored(self) -> bool:
-        """Return true if job has errored."""
-        return self.status == InstallStatus.ERROR
-
-    @property
-    def waiting(self) -> bool:
-        """Return true if job is waiting to run."""
-        return self.status == InstallStatus.WAITING
-
-    @property
-    def downloading(self) -> bool:
-        """Return true if job is downloading."""
-        return self.status == InstallStatus.DOWNLOADING
-
-    @property
-    def downloads_done(self) -> bool:
-        """Return true if job's downloads ae done."""
-        return self.status == InstallStatus.DOWNLOADS_DONE
-
-    @property
-    def running(self) -> bool:
-        """Return true if job is running."""
-        return self.status == InstallStatus.RUNNING
-
-    @property
-    def complete(self) -> bool:
-        """Return true if job completed without errors."""
-        return self.status == InstallStatus.COMPLETED
-
-    @property
-    def in_terminal_state(self) -> bool:
-        """Return true if job is in a terminal state."""
-        return self.status in [InstallStatus.COMPLETED, InstallStatus.ERROR, InstallStatus.CANCELLED]
+from invokeai.backend.model_manager.config import AnyModelConfig
 
 
 class ModelInstallServiceBase(ABC):
@@ -282,7 +57,7 @@ class ModelInstallServiceBase(ABC):
 
     @property
     @abstractmethod
-    def event_bus(self) -> Optional[EventServiceBase]:
+    def event_bus(self) -> Optional["EventServiceBase"]:
         """Return the event service base object associated with the installer."""
 
     @abstractmethod

invokeai/app/services/model_install/model_install_common.py

@@ -0,0 +1,233 @@
+import re
+import traceback
+from enum import Enum
+from pathlib import Path
+from typing import Any, Dict, Literal, Optional, Set, Union
+
+from pydantic import BaseModel, Field, PrivateAttr, field_validator
+from pydantic.networks import AnyHttpUrl
+from typing_extensions import Annotated
+
+from invokeai.app.services.download import DownloadJob
+from invokeai.backend.model_manager import AnyModelConfig, ModelRepoVariant
+from invokeai.backend.model_manager.config import ModelSourceType
+from invokeai.backend.model_manager.metadata import AnyModelRepoMetadata
+
+
+class InstallStatus(str, Enum):
+    """State of an install job running in the background."""
+
+    WAITING = "waiting"  # waiting to be dequeued
+    DOWNLOADING = "downloading"  # downloading of model files in process
+    DOWNLOADS_DONE = "downloads_done"  # downloading done, waiting to run
+    RUNNING = "running"  # being processed
+    COMPLETED = "completed"  # finished running
+    ERROR = "error"  # terminated with an error message
+    CANCELLED = "cancelled"  # terminated with an error message
+
+
+class ModelInstallPart(BaseModel):
+    url: AnyHttpUrl
+    path: Path
+    bytes: int = 0
+    total_bytes: int = 0
+
+
+class UnknownInstallJobException(Exception):
+    """Raised when the status of an unknown job is requested."""
+
+
+class StringLikeSource(BaseModel):
+    """
+    Base class for model sources, implements functions that lets the source be sorted and indexed.
+
+    These shenanigans let this stuff work:
+
+      source1 = LocalModelSource(path='C:/users/mort/foo.safetensors')
+      mydict = {source1: 'model 1'}
+      assert mydict['C:/users/mort/foo.safetensors'] == 'model 1'
+      assert mydict[LocalModelSource(path='C:/users/mort/foo.safetensors')] == 'model 1'
+
+      source2 = LocalModelSource(path=Path('C:/users/mort/foo.safetensors'))
+      assert source1 == source2
+      assert source1 == 'C:/users/mort/foo.safetensors'
+    """
+
+    def __hash__(self) -> int:
+        """Return hash of the path field, for indexing."""
+        return hash(str(self))
+
+    def __lt__(self, other: object) -> int:
+        """Return comparison of the stringified version, for sorting."""
+        return str(self) < str(other)
+
+    def __eq__(self, other: object) -> bool:
+        """Return equality on the stringified version."""
+        if isinstance(other, Path):
+            return str(self) == other.as_posix()
+        else:
+            return str(self) == str(other)
+
+
+class LocalModelSource(StringLikeSource):
+    """A local file or directory path."""
+
+    path: str | Path
+    inplace: Optional[bool] = False
+    type: Literal["local"] = "local"
+
+    # these methods allow the source to be used in a string-like way,
+    # for example as an index into a dict
+    def __str__(self) -> str:
+        """Return string version of path when string rep needed."""
+        return Path(self.path).as_posix()
+
+
+class HFModelSource(StringLikeSource):
+    """
+    A HuggingFace repo_id with optional variant, sub-folder and access token.
+    Note that the variant option, if not provided to the constructor, will default to fp16, which is
+    what people (almost) always want.
+    """
+
+    repo_id: str
+    variant: Optional[ModelRepoVariant] = ModelRepoVariant.FP16
+    subfolder: Optional[Path] = None
+    access_token: Optional[str] = None
+    type: Literal["hf"] = "hf"
+
+    @field_validator("repo_id")
+    @classmethod
+    def proper_repo_id(cls, v: str) -> str:  # noqa D102
+        if not re.match(r"^([.\w-]+/[.\w-]+)$", v):
+            raise ValueError(f"{v}: invalid repo_id format")
+        return v
+
+    def __str__(self) -> str:
+        """Return string version of repoid when string rep needed."""
+        base: str = self.repo_id
+        if self.variant:
+            base += f":{self.variant or ''}"
+        if self.subfolder:
+            base += f":{self.subfolder}"
+        return base
+
+
+class URLModelSource(StringLikeSource):
+    """A generic URL point to a checkpoint file."""
+
+    url: AnyHttpUrl
+    access_token: Optional[str] = None
+    type: Literal["url"] = "url"
+
+    def __str__(self) -> str:
+        """Return string version of the url when string rep needed."""
+        return str(self.url)
+
+
+ModelSource = Annotated[Union[LocalModelSource, HFModelSource, URLModelSource], Field(discriminator="type")]
+
+MODEL_SOURCE_TO_TYPE_MAP = {
+    URLModelSource: ModelSourceType.Url,
+    HFModelSource: ModelSourceType.HFRepoID,
+    LocalModelSource: ModelSourceType.Path,
+}
+
+
+class ModelInstallJob(BaseModel):
+    """Object that tracks the current status of an install request."""
+
+    id: int = Field(description="Unique ID for this job")
+    status: InstallStatus = Field(default=InstallStatus.WAITING, description="Current status of install process")
+    error_reason: Optional[str] = Field(default=None, description="Information about why the job failed")
+    config_in: Dict[str, Any] = Field(
+        default_factory=dict, description="Configuration information (e.g. 'description') to apply to model."
+    )
+    config_out: Optional[AnyModelConfig] = Field(
+        default=None, description="After successful installation, this will hold the configuration object."
+    )
+    inplace: bool = Field(
+        default=False, description="Leave model in its current location; otherwise install under models directory"
+    )
+    source: ModelSource = Field(description="Source (URL, repo_id, or local path) of model")
+    local_path: Path = Field(description="Path to locally-downloaded model; may be the same as the source")
+    bytes: int = Field(
+        default=0, description="For a remote model, the number of bytes downloaded so far (may not be available)"
+    )
+    total_bytes: int = Field(default=0, description="Total size of the model to be installed")
+    source_metadata: Optional[AnyModelRepoMetadata] = Field(
+        default=None, description="Metadata provided by the model source"
+    )
+    download_parts: Set[DownloadJob] = Field(
+        default_factory=set, description="Download jobs contributing to this install"
+    )
+    error: Optional[str] = Field(
+        default=None, description="On an error condition, this field will contain the text of the exception"
+    )
+    error_traceback: Optional[str] = Field(
+        default=None, description="On an error condition, this field will contain the exception traceback"
+    )
+    # internal flags and transitory settings
+    _install_tmpdir: Optional[Path] = PrivateAttr(default=None)
+    _exception: Optional[Exception] = PrivateAttr(default=None)
+
+    def set_error(self, e: Exception) -> None:
+        """Record the error and traceback from an exception."""
+        self._exception = e
+        self.error = str(e)
+        self.error_traceback = self._format_error(e)
+        self.status = InstallStatus.ERROR
+        self.error_reason = self._exception.__class__.__name__ if self._exception else None
+
+    def cancel(self) -> None:
+        """Call to cancel the job."""
+        self.status = InstallStatus.CANCELLED
+
+    @property
+    def error_type(self) -> Optional[str]:
+        """Class name of the exception that led to status==ERROR."""
+        return self._exception.__class__.__name__ if self._exception else None
+
+    def _format_error(self, exception: Exception) -> str:
+        """Error traceback."""
+        return "".join(traceback.format_exception(exception))
+
+    @property
+    def cancelled(self) -> bool:
+        """Set status to CANCELLED."""
+        return self.status == InstallStatus.CANCELLED
+
+    @property
+    def errored(self) -> bool:
+        """Return true if job has errored."""
+        return self.status == InstallStatus.ERROR
+
+    @property
+    def waiting(self) -> bool:
+        """Return true if job is waiting to run."""
+        return self.status == InstallStatus.WAITING
+
+    @property
+    def downloading(self) -> bool:
+        """Return true if job is downloading."""
+        return self.status == InstallStatus.DOWNLOADING
+
+    @property
+    def downloads_done(self) -> bool:
+        """Return true if job's downloads ae done."""
+        return self.status == InstallStatus.DOWNLOADS_DONE
+
+    @property
+    def running(self) -> bool:
+        """Return true if job is running."""
+        return self.status == InstallStatus.RUNNING
+
+    @property
+    def complete(self) -> bool:
+        """Return true if job completed without errors."""
+        return self.status == InstallStatus.COMPLETED
+
+    @property
+    def in_terminal_state(self) -> bool:
+        """Return true if job is in a terminal state."""
+        return self.status in [InstallStatus.COMPLETED, InstallStatus.ERROR, InstallStatus.CANCELLED]

invokeai/app/services/model_install/model_install_default.py

@@ -10,7 +10,7 @@ from pathlib import Path
 from queue import Empty, Queue
 from shutil import copyfile, copytree, move, rmtree
 from tempfile import mkdtemp
-from typing import Any, Dict, List, Optional, Union
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Union
 
 import torch
 import yaml
@@ -20,8 +20,8 @@ from requests import Session
 
 from invokeai.app.services.config import InvokeAIAppConfig
 from invokeai.app.services.download import DownloadJob, DownloadQueueServiceBase, TqdmProgress
-from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
+from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
 from invokeai.app.services.model_records import DuplicateModelException, ModelRecordServiceBase
 from invokeai.app.services.model_records.model_records_base import ModelRecordChanges
 from invokeai.backend.model_manager.config import (
@@ -45,13 +45,12 @@ from invokeai.backend.util import InvokeAILogger
 from invokeai.backend.util.catch_sigint import catch_sigint
 from invokeai.backend.util.devices import TorchDevice
 
-from .model_install_base import (
+from .model_install_common import (
     MODEL_SOURCE_TO_TYPE_MAP,
     HFModelSource,
     InstallStatus,
     LocalModelSource,
     ModelInstallJob,
-    ModelInstallServiceBase,
     ModelSource,
     StringLikeSource,
     URLModelSource,
@@ -59,6 +58,9 @@ from .model_install_base import (
 
 TMPDIR_PREFIX = "tmpinstall_"
 
+if TYPE_CHECKING:
+    from invokeai.app.services.events.events_base import EventServiceBase
+
 
 class ModelInstallService(ModelInstallServiceBase):
     """class for InvokeAI model installation."""
@@ -68,7 +70,7 @@ class ModelInstallService(ModelInstallServiceBase):
         app_config: InvokeAIAppConfig,
         record_store: ModelRecordServiceBase,
         download_queue: DownloadQueueServiceBase,
-        event_bus: Optional[EventServiceBase] = None,
+        event_bus: Optional["EventServiceBase"] = None,
         session: Optional[Session] = None,
     ):
         """
@@ -104,7 +106,7 @@ class ModelInstallService(ModelInstallServiceBase):
         return self._record_store
 
     @property
-    def event_bus(self) -> Optional[EventServiceBase]:  # noqa D102
+    def event_bus(self) -> Optional["EventServiceBase"]:  # noqa D102
         return self._event_bus
 
     # make the invoker optional here because we don't need it and it
@@ -855,35 +857,17 @@ class ModelInstallService(ModelInstallServiceBase):
         job.status = InstallStatus.RUNNING
         self._logger.info(f"Model install started: {job.source}")
         if self._event_bus:
-            self._event_bus.emit_model_install_running(str(job.source))
+            self._event_bus.emit_model_install_started(job)
 
     def _signal_job_downloading(self, job: ModelInstallJob) -> None:
         if self._event_bus:
-            parts: List[Dict[str, str | int]] = [
-                {
-                    "url": str(x.source),
-                    "local_path": str(x.download_path),
-                    "bytes": x.bytes,
-                    "total_bytes": x.total_bytes,
-                }
-                for x in job.download_parts
-            ]
-            assert job.bytes is not None
-            assert job.total_bytes is not None
-            self._event_bus.emit_model_install_downloading(
-                str(job.source),
-                local_path=job.local_path.as_posix(),
-                parts=parts,
-                bytes=job.bytes,
-                total_bytes=job.total_bytes,
-                id=job.id,
-            )
+            self._event_bus.emit_model_install_download_progress(job)
 
     def _signal_job_downloads_done(self, job: ModelInstallJob) -> None:
         job.status = InstallStatus.DOWNLOADS_DONE
         self._logger.info(f"Model download complete: {job.source}")
         if self._event_bus:
-            self._event_bus.emit_model_install_downloads_done(str(job.source))
+            self._event_bus.emit_model_install_downloads_complete(job)
 
     def _signal_job_completed(self, job: ModelInstallJob) -> None:
         job.status = InstallStatus.COMPLETED
@@ -891,24 +875,19 @@ class ModelInstallService(ModelInstallServiceBase):
         self._logger.info(f"Model install complete: {job.source}")
         self._logger.debug(f"{job.local_path} registered key {job.config_out.key}")
         if self._event_bus:
-            assert job.local_path is not None
-            assert job.config_out is not None
-            key = job.config_out.key
-            self._event_bus.emit_model_install_completed(str(job.source), key, id=job.id)
+            self._event_bus.emit_model_install_complete(job)
 
     def _signal_job_errored(self, job: ModelInstallJob) -> None:
         self._logger.error(f"Model install error: {job.source}\n{job.error_type}: {job.error}")
         if self._event_bus:
-            error_type = job.error_type
-            error = job.error
-            assert error_type is not None
-            assert error is not None
-            self._event_bus.emit_model_install_error(str(job.source), error_type, error, id=job.id)
+            assert job.error_type is not None
+            assert job.error is not None
+            self._event_bus.emit_model_install_error(job)
 
     def _signal_job_cancelled(self, job: ModelInstallJob) -> None:
         self._logger.info(f"Model install canceled: {job.source}")
         if self._event_bus:
-            self._event_bus.emit_model_install_cancelled(str(job.source), id=job.id)
+            self._event_bus.emit_model_install_cancelled(job)
 
     @staticmethod
     def get_fetcher_from_url(url: str) -> ModelMetadataFetchBase:

invokeai/app/services/model_load/model_load_base.py

@@ -4,7 +4,6 @@
 from abc import ABC, abstractmethod
 from typing import Optional
 
-from invokeai.app.services.shared.invocation_context import InvocationContextData
 from invokeai.backend.model_manager import AnyModel, AnyModelConfig, SubModelType
 from invokeai.backend.model_manager.load import LoadedModel
 from invokeai.backend.model_manager.load.convert_cache import ModelConvertCacheBase
@@ -15,18 +14,12 @@ class ModelLoadServiceBase(ABC):
     """Wrapper around AnyModelLoader."""
 
     @abstractmethod
-    def load_model(
-        self,
-        model_config: AnyModelConfig,
-        submodel_type: Optional[SubModelType] = None,
-        context_data: Optional[InvocationContextData] = None,
-    ) -> LoadedModel:
+    def load_model(self, model_config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> LoadedModel:
         """
         Given a model's configuration, load it and return the LoadedModel object.
 
         :param model_config: Model configuration record (as returned by ModelRecordBase.get_model())
         :param submodel: For main (pipeline models), the submodel to fetch.
-        :param context_data: Invocation context data used for event reporting
         """
 
     @property

invokeai/app/services/model_load/model_load_default.py

@@ -5,7 +5,6 @@ from typing import Optional, Type
 
 from invokeai.app.services.config import InvokeAIAppConfig
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.shared.invocation_context import InvocationContextData
 from invokeai.backend.model_manager import AnyModel, AnyModelConfig, SubModelType
 from invokeai.backend.model_manager.load import (
     LoadedModel,
@@ -51,25 +50,18 @@ class ModelLoadService(ModelLoadServiceBase):
         """Return the checkpoint convert cache used by this loader."""
         return self._convert_cache
 
-    def load_model(
-        self,
-        model_config: AnyModelConfig,
-        submodel_type: Optional[SubModelType] = None,
-        context_data: Optional[InvocationContextData] = None,
-    ) -> LoadedModel:
+    def load_model(self, model_config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> LoadedModel:
         """
         Given a model's configuration, load it and return the LoadedModel object.
 
         :param model_config: Model configuration record (as returned by ModelRecordBase.get_model())
         :param submodel: For main (pipeline models), the submodel to fetch.
-        :param context: Invocation context used for event reporting
         """
-        if context_data:
-            self._emit_load_event(
-                context_data=context_data,
-                model_config=model_config,
-                submodel_type=submodel_type,
-            )
+
+        # We don't have an invoker during testing
+        # TODO(psyche): Mock this method on the invoker in the tests
+        if hasattr(self, "_invoker"):
+            self._invoker.services.events.emit_model_load_started(model_config, submodel_type)
 
         implementation, model_config, submodel_type = self._registry.get_implementation(model_config, submodel_type)  # type: ignore
         loaded_model: LoadedModel = implementation(
@@ -79,40 +71,7 @@ class ModelLoadService(ModelLoadServiceBase):
             convert_cache=self._convert_cache,
         ).load_model(model_config, submodel_type)
 
-        if context_data:
-            self._emit_load_event(
-                context_data=context_data,
-                model_config=model_config,
-                submodel_type=submodel_type,
-                loaded=True,
-            )
+        if hasattr(self, "_invoker"):
+            self._invoker.services.events.emit_model_load_complete(model_config, submodel_type)
+
         return loaded_model
-
-    def _emit_load_event(
-        self,
-        context_data: InvocationContextData,
-        model_config: AnyModelConfig,
-        loaded: Optional[bool] = False,
-        submodel_type: Optional[SubModelType] = None,
-    ) -> None:
-        if not self._invoker:
-            return
-
-        if not loaded:
-            self._invoker.services.events.emit_model_load_started(
-                queue_id=context_data.queue_item.queue_id,
-                queue_item_id=context_data.queue_item.item_id,
-                queue_batch_id=context_data.queue_item.batch_id,
-                graph_execution_state_id=context_data.queue_item.session_id,
-                model_config=model_config,
-                submodel_type=submodel_type,
-            )
-        else:
-            self._invoker.services.events.emit_model_load_completed(
-                queue_id=context_data.queue_item.queue_id,
-                queue_item_id=context_data.queue_item.item_id,
-                queue_batch_id=context_data.queue_item.batch_id,
-                graph_execution_state_id=context_data.queue_item.session_id,
-                model_config=model_config,
-                submodel_type=submodel_type,
-            )

invokeai/app/services/session_processor/session_processor_base.py

@@ -1,6 +1,49 @@
 from abc import ABC, abstractmethod
+from threading import Event
+from typing import Optional, Protocol
 
+from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput
+from invokeai.app.services.invocation_services import InvocationServices
 from invokeai.app.services.session_processor.session_processor_common import SessionProcessorStatus
+from invokeai.app.services.session_queue.session_queue_common import SessionQueueItem
+from invokeai.app.util.profiler import Profiler
+
+
+class SessionRunnerBase(ABC):
+    """
+    Base class for session runner.
+    """
+
+    @abstractmethod
+    def start(self, services: InvocationServices, cancel_event: Event, profiler: Optional[Profiler] = None) -> None:
+        """Starts the session runner.
+
+        Args:
+            services: The invocation services.
+            cancel_event: The cancel event.
+            profiler: The profiler to use for session profiling via cProfile. Omit to disable profiling. Basic session
+                stats will be still be recorded and logged when profiling is disabled.
+        """
+        pass
+
+    @abstractmethod
+    def run(self, queue_item: SessionQueueItem) -> None:
+        """Runs a session.
+
+        Args:
+            queue_item: The session to run.
+        """
+        pass
+
+    @abstractmethod
+    def run_node(self, invocation: BaseInvocation, queue_item: SessionQueueItem) -> None:
+        """Run a single node in the graph.
+
+        Args:
+            invocation: The invocation to run.
+            queue_item: The session queue item.
+        """
+        pass
 
 
 class SessionProcessorBase(ABC):
@@ -26,3 +69,85 @@ class SessionProcessorBase(ABC):
     def get_status(self) -> SessionProcessorStatus:
         """Gets the status of the session processor"""
         pass
+
+
+class OnBeforeRunNode(Protocol):
+    def __call__(self, invocation: BaseInvocation, queue_item: SessionQueueItem) -> None:
+        """Callback to run before executing a node.
+
+        Args:
+            invocation: The invocation that will be executed.
+            queue_item: The session queue item.
+        """
+        ...
+
+
+class OnAfterRunNode(Protocol):
+    def __call__(self, invocation: BaseInvocation, queue_item: SessionQueueItem, output: BaseInvocationOutput) -> None:
+        """Callback to run before executing a node.
+
+        Args:
+            invocation: The invocation that was executed.
+            queue_item: The session queue item.
+        """
+        ...
+
+
+class OnNodeError(Protocol):
+    def __call__(
+        self,
+        invocation: BaseInvocation,
+        queue_item: SessionQueueItem,
+        error_type: str,
+        error_message: str,
+        error_traceback: str,
+    ) -> None:
+        """Callback to run when a node has an error.
+
+        Args:
+            invocation: The invocation that errored.
+            queue_item: The session queue item.
+            error_type: The type of error, e.g. "ValueError".
+            error_message: The error message, e.g. "Invalid value".
+            error_traceback: The stringified error traceback.
+        """
+        ...
+
+
+class OnBeforeRunSession(Protocol):
+    def __call__(self, queue_item: SessionQueueItem) -> None:
+        """Callback to run before executing a session.
+
+        Args:
+            queue_item: The session queue item.
+        """
+        ...
+
+
+class OnAfterRunSession(Protocol):
+    def __call__(self, queue_item: SessionQueueItem) -> None:
+        """Callback to run after executing a session.
+
+        Args:
+            queue_item: The session queue item.
+        """
+        ...
+
+
+class OnNonFatalProcessorError(Protocol):
+    def __call__(
+        self,
+        queue_item: Optional[SessionQueueItem],
+        error_type: str,
+        error_message: str,
+        error_traceback: str,
+    ) -> None:
+        """Callback to run when a non-fatal error occurs in the processor.
+
+        Args:
+            queue_item: The session queue item, if one was being executed when the error occurred.
+            error_type: The type of error, e.g. "ValueError".
+            error_message: The error message, e.g. "Invalid value".
+            error_traceback: The stringified error traceback.
+        """
+        ...

invokeai/app/services/session_processor/session_processor_default.py

@@ -4,24 +4,325 @@ from threading import BoundedSemaphore, Thread
 from threading import Event as ThreadEvent
 from typing import Optional
 
-from fastapi_events.handlers.local import local_handler
-from fastapi_events.typing import Event as FastAPIEvent
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation
-from invokeai.app.services.events.events_base import EventServiceBase
+from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput
+from invokeai.app.services.events.events_common import (
+    BatchEnqueuedEvent,
+    FastAPIEvent,
+    QueueClearedEvent,
+    QueueItemStatusChangedEvent,
+    register_events,
+)
 from invokeai.app.services.invocation_stats.invocation_stats_common import GESStatsNotFoundError
+from invokeai.app.services.session_processor.session_processor_base import (
+    OnAfterRunNode,
+    OnAfterRunSession,
+    OnBeforeRunNode,
+    OnBeforeRunSession,
+    OnNodeError,
+    OnNonFatalProcessorError,
+)
 from invokeai.app.services.session_processor.session_processor_common import CanceledException
-from invokeai.app.services.session_queue.session_queue_common import SessionQueueItem
+from invokeai.app.services.session_queue.session_queue_common import SessionQueueItem, SessionQueueItemNotFoundError
+from invokeai.app.services.shared.graph import NodeInputError
 from invokeai.app.services.shared.invocation_context import InvocationContextData, build_invocation_context
 from invokeai.app.util.profiler import Profiler
 
 from ..invoker import Invoker
-from .session_processor_base import SessionProcessorBase
+from .session_processor_base import InvocationServices, SessionProcessorBase, SessionRunnerBase
 from .session_processor_common import SessionProcessorStatus
 
 
+class DefaultSessionRunner(SessionRunnerBase):
+    """Processes a single session's invocations."""
+
+    def __init__(
+        self,
+        on_before_run_session_callbacks: Optional[list[OnBeforeRunSession]] = None,
+        on_before_run_node_callbacks: Optional[list[OnBeforeRunNode]] = None,
+        on_after_run_node_callbacks: Optional[list[OnAfterRunNode]] = None,
+        on_node_error_callbacks: Optional[list[OnNodeError]] = None,
+        on_after_run_session_callbacks: Optional[list[OnAfterRunSession]] = None,
+    ):
+        """
+        Args:
+            on_before_run_session_callbacks: Callbacks to run before the session starts.
+            on_before_run_node_callbacks: Callbacks to run before each node starts.
+            on_after_run_node_callbacks: Callbacks to run after each node completes.
+            on_node_error_callbacks: Callbacks to run when a node errors.
+            on_after_run_session_callbacks: Callbacks to run after the session completes.
+        """
+
+        self._on_before_run_session_callbacks = on_before_run_session_callbacks or []
+        self._on_before_run_node_callbacks = on_before_run_node_callbacks or []
+        self._on_after_run_node_callbacks = on_after_run_node_callbacks or []
+        self._on_node_error_callbacks = on_node_error_callbacks or []
+        self._on_after_run_session_callbacks = on_after_run_session_callbacks or []
+
+    def start(self, services: InvocationServices, cancel_event: ThreadEvent, profiler: Optional[Profiler] = None):
+        self._services = services
+        self._cancel_event = cancel_event
+        self._profiler = profiler
+
+    def _is_canceled(self) -> bool:
+        """Check if the cancel event is set. This is also passed to the invocation context builder and called during
+        denoising to check if the session has been canceled."""
+        return self._cancel_event.is_set()
+
+    def run(self, queue_item: SessionQueueItem):
+        # Exceptions raised outside `run_node` are handled by the processor. There is no need to catch them here.
+
+        self._on_before_run_session(queue_item=queue_item)
+
+        # Loop over invocations until the session is complete or canceled
+        while True:
+            try:
+                invocation = queue_item.session.next()
+            # Anything other than a `NodeInputError` is handled as a processor error
+            except NodeInputError as e:
+                error_type = e.__class__.__name__
+                error_message = str(e)
+                error_traceback = traceback.format_exc()
+                self._on_node_error(
+                    invocation=e.node,
+                    queue_item=queue_item,
+                    error_type=error_type,
+                    error_message=error_message,
+                    error_traceback=error_traceback,
+                )
+                break
+
+            if invocation is None or self._is_canceled():
+                break
+
+            self.run_node(invocation, queue_item)
+
+            # The session is complete if all invocations have been run or there is an error on the session.
+            # At this time, the queue item may be canceled, but the object itself here won't be updated yet. We must
+            # use the cancel event to check if the session is canceled.
+            if (
+                queue_item.session.is_complete()
+                or self._is_canceled()
+                or queue_item.status in ["failed", "canceled", "completed"]
+            ):
+                break
+
+        self._on_after_run_session(queue_item=queue_item)
+
+    def run_node(self, invocation: BaseInvocation, queue_item: SessionQueueItem):
+        try:
+            # Any unhandled exception in this scope is an invocation error & will fail the graph
+            with self._services.performance_statistics.collect_stats(invocation, queue_item.session_id):
+                self._on_before_run_node(invocation, queue_item)
+
+                data = InvocationContextData(
+                    invocation=invocation,
+                    source_invocation_id=queue_item.session.prepared_source_mapping[invocation.id],
+                    queue_item=queue_item,
+                )
+                context = build_invocation_context(
+                    data=data,
+                    services=self._services,
+                    is_canceled=self._is_canceled,
+                )
+
+                # Invoke the node
+                output = invocation.invoke_internal(context=context, services=self._services)
+                # Save output and history
+                queue_item.session.complete(invocation.id, output)
+
+                self._on_after_run_node(invocation, queue_item, output)
+
+        except KeyboardInterrupt:
+            # TODO(psyche): This is expected to be caught in the main thread. Do we need to catch this here?
+            pass
+        except CanceledException:
+            # A CanceledException is raised during the denoising step callback if the cancel event is set. We don't need
+            # to do any handling here, and no error should be set - just pass and the cancellation will be handled
+            # correctly in the next iteration of the session runner loop.
+            #
+            # See the comment in the processor's `_on_queue_item_status_changed()` method for more details on how we
+            # handle cancellation.
+            pass
+        except Exception as e:
+            error_type = e.__class__.__name__
+            error_message = str(e)
+            error_traceback = traceback.format_exc()
+            self._on_node_error(
+                invocation=invocation,
+                queue_item=queue_item,
+                error_type=error_type,
+                error_message=error_message,
+                error_traceback=error_traceback,
+            )
+
+    def _on_before_run_session(self, queue_item: SessionQueueItem) -> None:
+        """Called before a session is run.
+
+        - Start the profiler if profiling is enabled.
+        - Run any callbacks registered for this event.
+        """
+
+        self._services.logger.debug(
+            f"On before run session: queue item {queue_item.item_id}, session {queue_item.session_id}"
+        )
+
+        # If profiling is enabled, start the profiler
+        if self._profiler is not None:
+            self._profiler.start(profile_id=queue_item.session_id)
+
+        for callback in self._on_before_run_session_callbacks:
+            callback(queue_item=queue_item)
+
+    def _on_after_run_session(self, queue_item: SessionQueueItem) -> None:
+        """Called after a session is run.
+
+        - Stop the profiler if profiling is enabled.
+        - Update the queue item's session object in the database.
+        - If not already canceled or failed, complete the queue item.
+        - Log and reset performance statistics.
+        - Run any callbacks registered for this event.
+        """
+
+        self._services.logger.debug(
+            f"On after run session: queue item {queue_item.item_id}, session {queue_item.session_id}"
+        )
+
+        # If we are profiling, stop the profiler and dump the profile & stats
+        if self._profiler is not None:
+            profile_path = self._profiler.stop()
+            stats_path = profile_path.with_suffix(".json")
+            self._services.performance_statistics.dump_stats(
+                graph_execution_state_id=queue_item.session.id, output_path=stats_path
+            )
+
+        try:
+            # Update the queue item with the completed session. If the queue item has been removed from the queue,
+            # we'll get a SessionQueueItemNotFoundError and we can ignore it. This can happen if the queue is cleared
+            # while the session is running.
+            queue_item = self._services.session_queue.set_queue_item_session(queue_item.item_id, queue_item.session)
+
+            # The queue item may have been canceled or failed while the session was running. We should only complete it
+            # if it is not already canceled or failed.
+            if queue_item.status not in ["canceled", "failed"]:
+                queue_item = self._services.session_queue.complete_queue_item(queue_item.item_id)
+
+            # We'll get a GESStatsNotFoundError if we try to log stats for an untracked graph, but in the processor
+            # we don't care about that - suppress the error.
+            with suppress(GESStatsNotFoundError):
+                self._services.performance_statistics.log_stats(queue_item.session.id)
+                self._services.performance_statistics.reset_stats()
+
+            for callback in self._on_after_run_session_callbacks:
+                callback(queue_item=queue_item)
+        except SessionQueueItemNotFoundError:
+            pass
+
+    def _on_before_run_node(self, invocation: BaseInvocation, queue_item: SessionQueueItem):
+        """Called before a node is run.
+
+        - Emits an invocation started event.
+        - Run any callbacks registered for this event.
+        """
+
+        self._services.logger.debug(
+            f"On before run node: queue item {queue_item.item_id}, session {queue_item.session_id}, node {invocation.id} ({invocation.get_type()})"
+        )
+
+        # Send starting event
+        self._services.events.emit_invocation_started(queue_item=queue_item, invocation=invocation)
+
+        for callback in self._on_before_run_node_callbacks:
+            callback(invocation=invocation, queue_item=queue_item)
+
+    def _on_after_run_node(
+        self, invocation: BaseInvocation, queue_item: SessionQueueItem, output: BaseInvocationOutput
+    ):
+        """Called after a node is run.
+
+        - Emits an invocation complete event.
+        - Run any callbacks registered for this event.
+        """
+
+        self._services.logger.debug(
+            f"On after run node: queue item {queue_item.item_id}, session {queue_item.session_id}, node {invocation.id} ({invocation.get_type()})"
+        )
+
+        # Send complete event on successful runs
+        self._services.events.emit_invocation_complete(invocation=invocation, queue_item=queue_item, output=output)
+
+        for callback in self._on_after_run_node_callbacks:
+            callback(invocation=invocation, queue_item=queue_item, output=output)
+
+    def _on_node_error(
+        self,
+        invocation: BaseInvocation,
+        queue_item: SessionQueueItem,
+        error_type: str,
+        error_message: str,
+        error_traceback: str,
+    ):
+        """Called when a node errors. Node errors may occur when running or preparing the node..
+
+        - Set the node error on the session object.
+        - Log the error.
+        - Fail the queue item.
+        - Emits an invocation error event.
+        - Run any callbacks registered for this event.
+        """
+
+        self._services.logger.debug(
+            f"On node error: queue item {queue_item.item_id}, session {queue_item.session_id}, node {invocation.id} ({invocation.get_type()})"
+        )
+
+        # Node errors do not get the full traceback. Only the queue item gets the full traceback.
+        node_error = f"{error_type}: {error_message}"
+        queue_item.session.set_node_error(invocation.id, node_error)
+        self._services.logger.error(
+            f"Error while invoking session {queue_item.session_id}, invocation {invocation.id} ({invocation.get_type()}): {error_message}"
+        )
+        self._services.logger.error(error_traceback)
+
+        # Fail the queue item
+        queue_item = self._services.session_queue.set_queue_item_session(queue_item.item_id, queue_item.session)
+        queue_item = self._services.session_queue.fail_queue_item(
+            queue_item.item_id, error_type, error_message, error_traceback
+        )
+
+        # Send error event
+        self._services.events.emit_invocation_error(
+            queue_item=queue_item,
+            invocation=invocation,
+            error_type=error_type,
+            error_message=error_message,
+            error_traceback=error_traceback,
+        )
+
+        for callback in self._on_node_error_callbacks:
+            callback(
+                invocation=invocation,
+                queue_item=queue_item,
+                error_type=error_type,
+                error_message=error_message,
+                error_traceback=error_traceback,
+            )
+
+
 class DefaultSessionProcessor(SessionProcessorBase):
-    def start(self, invoker: Invoker, thread_limit: int = 1, polling_interval: int = 1) -> None:
+    def __init__(
+        self,
+        session_runner: Optional[SessionRunnerBase] = None,
+        on_non_fatal_processor_error_callbacks: Optional[list[OnNonFatalProcessorError]] = None,
+        thread_limit: int = 1,
+        polling_interval: int = 1,
+    ) -> None:
+        super().__init__()
+
+        self.session_runner = session_runner if session_runner else DefaultSessionRunner()
+        self._on_non_fatal_processor_error_callbacks = on_non_fatal_processor_error_callbacks or []
+        self._thread_limit = thread_limit
+        self._polling_interval = polling_interval
+
+    def start(self, invoker: Invoker) -> None:
         self._invoker: Invoker = invoker
         self._queue_item: Optional[SessionQueueItem] = None
         self._invocation: Optional[BaseInvocation] = None
@@ -31,11 +332,11 @@ class DefaultSessionProcessor(SessionProcessorBase):
         self._poll_now_event = ThreadEvent()
         self._cancel_event = ThreadEvent()
 
-        local_handler.register(event_name=EventServiceBase.queue_event, _func=self._on_queue_event)
+        register_events(QueueClearedEvent, self._on_queue_cleared)
+        register_events(BatchEnqueuedEvent, self._on_batch_enqueued)
+        register_events(QueueItemStatusChangedEvent, self._on_queue_item_status_changed)
 
-        self._thread_limit = thread_limit
-        self._thread_semaphore = BoundedSemaphore(thread_limit)
-        self._polling_interval = polling_interval
+        self._thread_semaphore = BoundedSemaphore(self._thread_limit)
 
         # If profiling is enabled, create a profiler. The same profiler will be used for all sessions. Internally,
         # the profiler will create a new profile for each session.
@@ -49,6 +350,7 @@ class DefaultSessionProcessor(SessionProcessorBase):
             else None
         )
 
+        self.session_runner.start(services=invoker.services, cancel_event=self._cancel_event, profiler=self._profiler)
         self._thread = Thread(
             name="session_processor",
             target=self._process,
@@ -67,30 +369,25 @@ class DefaultSessionProcessor(SessionProcessorBase):
     def _poll_now(self) -> None:
         self._poll_now_event.set()
 
-    async def _on_queue_event(self, event: FastAPIEvent) -> None:
-        event_name = event[1]["event"]
+    async def _on_queue_cleared(self, event: FastAPIEvent[QueueClearedEvent]) -> None:
+        if self._queue_item and self._queue_item.queue_id == event[1].queue_id:
+            self._cancel_event.set()
+            self._poll_now()
 
-        if (
-            event_name == "session_canceled"
-            and self._queue_item
-            and self._queue_item.item_id == event[1]["data"]["queue_item_id"]
-        ):
-            self._cancel_event.set()
-            self._poll_now()
-        elif (
-            event_name == "queue_cleared"
-            and self._queue_item
-            and self._queue_item.queue_id == event[1]["data"]["queue_id"]
-        ):
-            self._cancel_event.set()
-            self._poll_now()
-        elif event_name == "batch_enqueued":
-            self._poll_now()
-        elif event_name == "queue_item_status_changed" and event[1]["data"]["queue_item"]["status"] in [
-            "completed",
-            "failed",
-            "canceled",
-        ]:
+    async def _on_batch_enqueued(self, event: FastAPIEvent[BatchEnqueuedEvent]) -> None:
+        self._poll_now()
+
+    async def _on_queue_item_status_changed(self, event: FastAPIEvent[QueueItemStatusChangedEvent]) -> None:
+        if self._queue_item and event[1].status in ["completed", "failed", "canceled"]:
+            # When the queue item is canceled via HTTP, the queue item status is set to `"canceled"` and this event is
+            # emitted. We need to respond to this event and stop graph execution. This is done by setting the cancel
+            # event, which the session runner checks between invocations. If set, the session runner loop is broken.
+            #
+            # Long-running nodes that cannot be interrupted easily present a challenge. `denoise_latents` is one such
+            # node, but it gets a step callback, called on each step of denoising. This callback checks if the queue item
+            # is canceled, and if it is, raises a `CanceledException` to stop execution immediately.
+            if event[1].status == "canceled":
+                self._cancel_event.set()
             self._poll_now()
 
     def resume(self) -> SessionProcessorStatus:
@@ -116,8 +413,8 @@ class DefaultSessionProcessor(SessionProcessorBase):
         resume_event: ThreadEvent,
         cancel_event: ThreadEvent,
     ):
-        # Outermost processor try block; any unhandled exception is a fatal processor error
         try:
+            # Any unhandled exception in this block is a fatal processor error and will stop the processor.
             self._thread_semaphore.acquire()
             stop_event.clear()
             resume_event.set()
@@ -125,8 +422,8 @@ class DefaultSessionProcessor(SessionProcessorBase):
 
             while not stop_event.is_set():
                 poll_now_event.clear()
-                # Middle processor try block; any unhandled exception is a non-fatal processor error
                 try:
+                    # Any unhandled exception in this block is a nonfatal processor error and will be handled.
                     # If we are paused, wait for resume event
                     resume_event.wait()
 
@@ -142,157 +439,69 @@ class DefaultSessionProcessor(SessionProcessorBase):
                     self._invoker.services.logger.debug(f"Executing queue item {self._queue_item.item_id}")
                     cancel_event.clear()
 
-                    # If profiling is enabled, start the profiler
-                    if self._profiler is not None:
-                        self._profiler.start(profile_id=self._queue_item.session_id)
+                    # Run the graph
+                    self.session_runner.run(queue_item=self._queue_item)
 
-                    # Prepare invocations and take the first
-                    self._invocation = self._queue_item.session.next()
-
-                    # Loop over invocations until the session is complete or canceled
-                    while self._invocation is not None and not cancel_event.is_set():
-                        # get the source node id to provide to clients (the prepared node id is not as useful)
-                        source_invocation_id = self._queue_item.session.prepared_source_mapping[self._invocation.id]
-
-                        # Send starting event
-                        self._invoker.services.events.emit_invocation_started(
-                            queue_batch_id=self._queue_item.batch_id,
-                            queue_item_id=self._queue_item.item_id,
-                            queue_id=self._queue_item.queue_id,
-                            graph_execution_state_id=self._queue_item.session_id,
-                            node=self._invocation.model_dump(),
-                            source_node_id=source_invocation_id,
-                        )
-
-                        # Innermost processor try block; any unhandled exception is an invocation error & will fail the graph
-                        try:
-                            with self._invoker.services.performance_statistics.collect_stats(
-                                self._invocation, self._queue_item.session.id
-                            ):
-                                # Build invocation context (the node-facing API)
-                                data = InvocationContextData(
-                                    invocation=self._invocation,
-                                    source_invocation_id=source_invocation_id,
-                                    queue_item=self._queue_item,
-                                )
-                                context = build_invocation_context(
-                                    data=data,
-                                    services=self._invoker.services,
-                                    cancel_event=self._cancel_event,
-                                )
-
-                                # Invoke the node
-                                outputs = self._invocation.invoke_internal(
-                                    context=context, services=self._invoker.services
-                                )
-
-                                # Save outputs and history
-                                self._queue_item.session.complete(self._invocation.id, outputs)
-
-                                # Send complete event
-                                self._invoker.services.events.emit_invocation_complete(
-                                    queue_batch_id=self._queue_item.batch_id,
-                                    queue_item_id=self._queue_item.item_id,
-                                    queue_id=self._queue_item.queue_id,
-                                    graph_execution_state_id=self._queue_item.session.id,
-                                    node=self._invocation.model_dump(),
-                                    source_node_id=source_invocation_id,
-                                    result=outputs.model_dump(),
-                                )
-
-                        except KeyboardInterrupt:
-                            # TODO(MM2): Create an event for this
-                            pass
-
-                        except CanceledException:
-                            # When the user cancels the graph, we first set the cancel event. The event is checked
-                            # between invocations, in this loop. Some invocations are long-running, and we need to
-                            # be able to cancel them mid-execution.
-                            #
-                            # For example, denoising is a long-running invocation with many steps. A step callback
-                            # is executed after each step. This step callback checks if the canceled event is set,
-                            # then raises a CanceledException to stop execution immediately.
-                            #
-                            # When we get a CanceledException, we don't need to do anything - just pass and let the
-                            # loop go to its next iteration, and the cancel event will be handled correctly.
-                            pass
-
-                        except Exception as e:
-                            error = traceback.format_exc()
-
-                            # Save error
-                            self._queue_item.session.set_node_error(self._invocation.id, error)
-                            self._invoker.services.logger.error(
-                                f"Error while invoking session {self._queue_item.session_id}, invocation {self._invocation.id} ({self._invocation.get_type()}):\n{e}"
-                            )
-                            self._invoker.services.logger.error(error)
-
-                            # Send error event
-                            self._invoker.services.events.emit_invocation_error(
-                                queue_batch_id=self._queue_item.session_id,
-                                queue_item_id=self._queue_item.item_id,
-                                queue_id=self._queue_item.queue_id,
-                                graph_execution_state_id=self._queue_item.session.id,
-                                node=self._invocation.model_dump(),
-                                source_node_id=source_invocation_id,
-                                error_type=e.__class__.__name__,
-                                error=error,
-                            )
-                            pass
-
-                        # The session is complete if the all invocations are complete or there was an error
-                        if self._queue_item.session.is_complete() or cancel_event.is_set():
-                            # Send complete event
-                            self._invoker.services.events.emit_graph_execution_complete(
-                                queue_batch_id=self._queue_item.batch_id,
-                                queue_item_id=self._queue_item.item_id,
-                                queue_id=self._queue_item.queue_id,
-                                graph_execution_state_id=self._queue_item.session.id,
-                            )
-                            # If we are profiling, stop the profiler and dump the profile & stats
-                            if self._profiler:
-                                profile_path = self._profiler.stop()
-                                stats_path = profile_path.with_suffix(".json")
-                                self._invoker.services.performance_statistics.dump_stats(
-                                    graph_execution_state_id=self._queue_item.session.id, output_path=stats_path
-                                )
-                            # We'll get a GESStatsNotFoundError if we try to log stats for an untracked graph, but in the processor
-                            # we don't care about that - suppress the error.
-                            with suppress(GESStatsNotFoundError):
-                                self._invoker.services.performance_statistics.log_stats(self._queue_item.session.id)
-                                self._invoker.services.performance_statistics.reset_stats()
-
-                            # Set the invocation to None to prepare for the next session
-                            self._invocation = None
-                        else:
-                            # Prepare the next invocation
-                            self._invocation = self._queue_item.session.next()
-                    else:
-                        # The queue was empty, wait for next polling interval or event to try again
-                        self._invoker.services.logger.debug("Waiting for next polling interval or event")
-                        poll_now_event.wait(self._polling_interval)
-                        continue
-                except Exception:
-                    # Non-fatal error in processor
-                    self._invoker.services.logger.error(
-                        f"Non-fatal error in session processor:\n{traceback.format_exc()}"
+                except Exception as e:
+                    error_type = e.__class__.__name__
+                    error_message = str(e)
+                    error_traceback = traceback.format_exc()
+                    self._on_non_fatal_processor_error(
+                        queue_item=self._queue_item,
+                        error_type=error_type,
+                        error_message=error_message,
+                        error_traceback=error_traceback,
                     )
-                    # Cancel the queue item
-                    if self._queue_item is not None:
-                        self._invoker.services.session_queue.cancel_queue_item(
-                            self._queue_item.item_id, error=traceback.format_exc()
-                        )
-                    # Reset the invocation to None to prepare for the next session
-                    self._invocation = None
-                    # Immediately poll for next queue item
+                    # Wait for next polling interval or event to try again
                     poll_now_event.wait(self._polling_interval)
                     continue
-        except Exception:
+        except Exception as e:
             # Fatal error in processor, log and pass - we're done here
-            self._invoker.services.logger.error(f"Fatal Error in session processor:\n{traceback.format_exc()}")
+            error_type = e.__class__.__name__
+            error_message = str(e)
+            error_traceback = traceback.format_exc()
+            self._invoker.services.logger.error(f"Fatal Error in session processor {error_type}: {error_message}")
+            self._invoker.services.logger.error(error_traceback)
             pass
         finally:
             stop_event.clear()
             poll_now_event.clear()
             self._queue_item = None
             self._thread_semaphore.release()
+
+    def _on_non_fatal_processor_error(
+        self,
+        queue_item: Optional[SessionQueueItem],
+        error_type: str,
+        error_message: str,
+        error_traceback: str,
+    ) -> None:
+        """Called when a non-fatal error occurs in the processor.
+
+        - Log the error.
+        - If a queue item is provided, update the queue item with the completed session & fail it.
+        - Run any callbacks registered for this event.
+        """
+
+        self._invoker.services.logger.error(f"Non-fatal error in session processor {error_type}: {error_message}")
+        self._invoker.services.logger.error(error_traceback)
+
+        if queue_item is not None:
+            # Update the queue item with the completed session & fail it
+            queue_item = self._invoker.services.session_queue.set_queue_item_session(
+                queue_item.item_id, queue_item.session
+            )
+            queue_item = self._invoker.services.session_queue.fail_queue_item(
+                item_id=queue_item.item_id,
+                error_type=error_type,
+                error_message=error_message,
+                error_traceback=error_traceback,
+            )
+
+        for callback in self._on_non_fatal_processor_error_callbacks:
+            callback(
+                queue_item=queue_item,
+                error_type=error_type,
+                error_message=error_message,
+                error_traceback=error_traceback,
+            )

invokeai/app/services/session_queue/session_queue_base.py

@@ -16,6 +16,7 @@ from invokeai.app.services.session_queue.session_queue_common import (
     SessionQueueItemDTO,
     SessionQueueStatus,
 )
+from invokeai.app.services.shared.graph import GraphExecutionState
 from invokeai.app.services.shared.pagination import CursorPaginatedResults
 
 
@@ -73,10 +74,22 @@ class SessionQueueBase(ABC):
         pass
 
     @abstractmethod
-    def cancel_queue_item(self, item_id: int, error: Optional[str] = None) -> SessionQueueItem:
+    def complete_queue_item(self, item_id: int) -> SessionQueueItem:
+        """Completes a session queue item"""
+        pass
+
+    @abstractmethod
+    def cancel_queue_item(self, item_id: int) -> SessionQueueItem:
         """Cancels a session queue item"""
         pass
 
+    @abstractmethod
+    def fail_queue_item(
+        self, item_id: int, error_type: str, error_message: str, error_traceback: str
+    ) -> SessionQueueItem:
+        """Fails a session queue item"""
+        pass
+
     @abstractmethod
     def cancel_by_batch_ids(self, queue_id: str, batch_ids: list[str]) -> CancelByBatchIDsResult:
         """Cancels all queue items with matching batch IDs"""
@@ -103,3 +116,8 @@ class SessionQueueBase(ABC):
     def get_queue_item(self, item_id: int) -> SessionQueueItem:
         """Gets a session queue item by ID"""
         pass
+
+    @abstractmethod
+    def set_queue_item_session(self, item_id: int, session: GraphExecutionState) -> SessionQueueItem:
+        """Sets the session for a session queue item. Use this to update the session state."""
+        pass

invokeai/app/services/session_queue/session_queue_common.py

@@ -3,7 +3,16 @@ import json
 from itertools import chain, product
 from typing import Generator, Iterable, Literal, NamedTuple, Optional, TypeAlias, Union, cast
 
-from pydantic import BaseModel, ConfigDict, Field, StrictStr, TypeAdapter, field_validator, model_validator
+from pydantic import (
+    AliasChoices,
+    BaseModel,
+    ConfigDict,
+    Field,
+    StrictStr,
+    TypeAdapter,
+    field_validator,
+    model_validator,
+)
 from pydantic_core import to_jsonable_python
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation
@@ -189,7 +198,13 @@ class SessionQueueItemWithoutGraph(BaseModel):
     session_id: str = Field(
         description="The ID of the session associated with this queue item. The session doesn't exist in graph_executions until the queue item is executed."
     )
-    error: Optional[str] = Field(default=None, description="The error message if this queue item errored")
+    error_type: Optional[str] = Field(default=None, description="The error type if this queue item errored")
+    error_message: Optional[str] = Field(default=None, description="The error message if this queue item errored")
+    error_traceback: Optional[str] = Field(
+        default=None,
+        description="The error traceback if this queue item errored",
+        validation_alias=AliasChoices("error_traceback", "error"),
+    )
     created_at: Union[datetime.datetime, str] = Field(description="When this queue item was created")
     updated_at: Union[datetime.datetime, str] = Field(description="When this queue item was updated")
     started_at: Optional[Union[datetime.datetime, str]] = Field(description="When this queue item was started")

invokeai/app/services/session_queue/session_queue_sqlite.py

@@ -2,10 +2,6 @@ import sqlite3
 import threading
 from typing import Optional, Union, cast
 
-from fastapi_events.handlers.local import local_handler
-from fastapi_events.typing import Event as FastAPIEvent
-
-from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.session_queue.session_queue_base import SessionQueueBase
 from invokeai.app.services.session_queue.session_queue_common import (
@@ -27,6 +23,7 @@ from invokeai.app.services.session_queue.session_queue_common import (
     calc_session_count,
     prepare_values_to_insert,
 )
+from invokeai.app.services.shared.graph import GraphExecutionState
 from invokeai.app.services.shared.pagination import CursorPaginatedResults
 from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
 
@@ -41,7 +38,7 @@ class SqliteSessionQueue(SessionQueueBase):
         self.__invoker = invoker
         self._set_in_progress_to_canceled()
         prune_result = self.prune(DEFAULT_QUEUE_ID)
-        local_handler.register(event_name=EventServiceBase.queue_event, _func=self._on_session_event)
+
         if prune_result.deleted > 0:
             self.__invoker.services.logger.info(f"Pruned {prune_result.deleted} finished queue items")
 
@@ -51,52 +48,6 @@ class SqliteSessionQueue(SessionQueueBase):
         self.__conn = db.conn
         self.__cursor = self.__conn.cursor()
 
-    def _match_event_name(self, event: FastAPIEvent, match_in: list[str]) -> bool:
-        return event[1]["event"] in match_in
-
-    async def _on_session_event(self, event: FastAPIEvent) -> FastAPIEvent:
-        event_name = event[1]["event"]
-
-        # This was a match statement, but match is not supported on python 3.9
-        if event_name == "graph_execution_state_complete":
-            await self._handle_complete_event(event)
-        elif event_name == "invocation_error":
-            await self._handle_error_event(event)
-        elif event_name == "session_canceled":
-            await self._handle_cancel_event(event)
-        return event
-
-    async def _handle_complete_event(self, event: FastAPIEvent) -> None:
-        try:
-            item_id = event[1]["data"]["queue_item_id"]
-            # When a queue item has an error, we get an error event, then a completed event.
-            # Mark the queue item completed only if it isn't already marked completed, e.g.
-            # by a previously-handled error event.
-            queue_item = self.get_queue_item(item_id)
-            if queue_item.status not in ["completed", "failed", "canceled"]:
-                queue_item = self._set_queue_item_status(item_id=queue_item.item_id, status="completed")
-        except SessionQueueItemNotFoundError:
-            return
-
-    async def _handle_error_event(self, event: FastAPIEvent) -> None:
-        try:
-            item_id = event[1]["data"]["queue_item_id"]
-            error = event[1]["data"]["error"]
-            queue_item = self.get_queue_item(item_id)
-            # always set to failed if have an error, even if previously the item was marked completed or canceled
-            queue_item = self._set_queue_item_status(item_id=queue_item.item_id, status="failed", error=error)
-        except SessionQueueItemNotFoundError:
-            return
-
-    async def _handle_cancel_event(self, event: FastAPIEvent) -> None:
-        try:
-            item_id = event[1]["data"]["queue_item_id"]
-            queue_item = self.get_queue_item(item_id)
-            if queue_item.status not in ["completed", "failed", "canceled"]:
-                queue_item = self._set_queue_item_status(item_id=queue_item.item_id, status="canceled")
-        except SessionQueueItemNotFoundError:
-            return
-
     def _set_in_progress_to_canceled(self) -> None:
         """
         Sets all in_progress queue items to canceled. Run on app startup, not associated with any queue.
@@ -271,17 +222,22 @@ class SqliteSessionQueue(SessionQueueBase):
         return SessionQueueItem.queue_item_from_dict(dict(result))
 
     def _set_queue_item_status(
-        self, item_id: int, status: QUEUE_ITEM_STATUS, error: Optional[str] = None
+        self,
+        item_id: int,
+        status: QUEUE_ITEM_STATUS,
+        error_type: Optional[str] = None,
+        error_message: Optional[str] = None,
+        error_traceback: Optional[str] = None,
     ) -> SessionQueueItem:
         try:
             self.__lock.acquire()
             self.__cursor.execute(
                 """--sql
                 UPDATE session_queue
-                SET status = ?, error = ?
+                SET status = ?, error_type = ?, error_message = ?, error_traceback = ?
                 WHERE item_id = ?
                 """,
-                (status, error, item_id),
+                (status, error_type, error_message, error_traceback, item_id),
             )
             self.__conn.commit()
         except Exception:
@@ -292,11 +248,7 @@ class SqliteSessionQueue(SessionQueueBase):
         queue_item = self.get_queue_item(item_id)
         batch_status = self.get_batch_status(queue_id=queue_item.queue_id, batch_id=queue_item.batch_id)
         queue_status = self.get_queue_status(queue_id=queue_item.queue_id)
-        self.__invoker.services.events.emit_queue_item_status_changed(
-            session_queue_item=queue_item,
-            batch_status=batch_status,
-            queue_status=queue_status,
-        )
+        self.__invoker.services.events.emit_queue_item_status_changed(queue_item, batch_status, queue_status)
         return queue_item
 
     def is_empty(self, queue_id: str) -> IsEmptyResult:
@@ -338,26 +290,6 @@ class SqliteSessionQueue(SessionQueueBase):
             self.__lock.release()
         return IsFullResult(is_full=is_full)
 
-    def delete_queue_item(self, item_id: int) -> SessionQueueItem:
-        queue_item = self.get_queue_item(item_id=item_id)
-        try:
-            self.__lock.acquire()
-            self.__cursor.execute(
-                """--sql
-                DELETE FROM session_queue
-                WHERE
-                  item_id = ?
-                """,
-                (item_id,),
-            )
-            self.__conn.commit()
-        except Exception:
-            self.__conn.rollback()
-            raise
-        finally:
-            self.__lock.release()
-        return queue_item
-
     def clear(self, queue_id: str) -> ClearResult:
         try:
             self.__lock.acquire()
@@ -424,17 +356,28 @@ class SqliteSessionQueue(SessionQueueBase):
             self.__lock.release()
         return PruneResult(deleted=count)
 
-    def cancel_queue_item(self, item_id: int, error: Optional[str] = None) -> SessionQueueItem:
-        queue_item = self.get_queue_item(item_id)
-        if queue_item.status not in ["canceled", "failed", "completed"]:
-            status = "failed" if error is not None else "canceled"
-            queue_item = self._set_queue_item_status(item_id=item_id, status=status, error=error)  # type: ignore [arg-type] # mypy seems to not narrow the Literals here
-            self.__invoker.services.events.emit_session_canceled(
-                queue_item_id=queue_item.item_id,
-                queue_id=queue_item.queue_id,
-                queue_batch_id=queue_item.batch_id,
-                graph_execution_state_id=queue_item.session_id,
-            )
+    def cancel_queue_item(self, item_id: int) -> SessionQueueItem:
+        queue_item = self._set_queue_item_status(item_id=item_id, status="canceled")
+        return queue_item
+
+    def complete_queue_item(self, item_id: int) -> SessionQueueItem:
+        queue_item = self._set_queue_item_status(item_id=item_id, status="completed")
+        return queue_item
+
+    def fail_queue_item(
+        self,
+        item_id: int,
+        error_type: str,
+        error_message: str,
+        error_traceback: str,
+    ) -> SessionQueueItem:
+        queue_item = self._set_queue_item_status(
+            item_id=item_id,
+            status="failed",
+            error_type=error_type,
+            error_message=error_message,
+            error_traceback=error_traceback,
+        )
         return queue_item
 
     def cancel_by_batch_ids(self, queue_id: str, batch_ids: list[str]) -> CancelByBatchIDsResult:
@@ -470,18 +413,10 @@ class SqliteSessionQueue(SessionQueueBase):
             )
             self.__conn.commit()
             if current_queue_item is not None and current_queue_item.batch_id in batch_ids:
-                self.__invoker.services.events.emit_session_canceled(
-                    queue_item_id=current_queue_item.item_id,
-                    queue_id=current_queue_item.queue_id,
-                    queue_batch_id=current_queue_item.batch_id,
-                    graph_execution_state_id=current_queue_item.session_id,
-                )
                 batch_status = self.get_batch_status(queue_id=queue_id, batch_id=current_queue_item.batch_id)
                 queue_status = self.get_queue_status(queue_id=queue_id)
                 self.__invoker.services.events.emit_queue_item_status_changed(
-                    session_queue_item=current_queue_item,
-                    batch_status=batch_status,
-                    queue_status=queue_status,
+                    current_queue_item, batch_status, queue_status
                 )
         except Exception:
             self.__conn.rollback()
@@ -521,18 +456,10 @@ class SqliteSessionQueue(SessionQueueBase):
             )
             self.__conn.commit()
             if current_queue_item is not None and current_queue_item.queue_id == queue_id:
-                self.__invoker.services.events.emit_session_canceled(
-                    queue_item_id=current_queue_item.item_id,
-                    queue_id=current_queue_item.queue_id,
-                    queue_batch_id=current_queue_item.batch_id,
-                    graph_execution_state_id=current_queue_item.session_id,
-                )
                 batch_status = self.get_batch_status(queue_id=queue_id, batch_id=current_queue_item.batch_id)
                 queue_status = self.get_queue_status(queue_id=queue_id)
                 self.__invoker.services.events.emit_queue_item_status_changed(
-                    session_queue_item=current_queue_item,
-                    batch_status=batch_status,
-                    queue_status=queue_status,
+                    current_queue_item, batch_status, queue_status
                 )
         except Exception:
             self.__conn.rollback()
@@ -562,6 +489,29 @@ class SqliteSessionQueue(SessionQueueBase):
             raise SessionQueueItemNotFoundError(f"No queue item with id {item_id}")
         return SessionQueueItem.queue_item_from_dict(dict(result))
 
+    def set_queue_item_session(self, item_id: int, session: GraphExecutionState) -> SessionQueueItem:
+        try:
+            # Use exclude_none so we don't end up with a bunch of nulls in the graph - this can cause validation errors
+            # when the graph is loaded. Graph execution occurs purely in memory - the session saved here is not referenced
+            # during execution.
+            session_json = session.model_dump_json(warnings=False, exclude_none=True)
+            self.__lock.acquire()
+            self.__cursor.execute(
+                """--sql
+                UPDATE session_queue
+                SET session = ?
+                WHERE item_id = ?
+                """,
+                (session_json, item_id),
+            )
+            self.__conn.commit()
+        except Exception:
+            self.__conn.rollback()
+            raise
+        finally:
+            self.__lock.release()
+        return self.get_queue_item(item_id)
+
     def list_queue_items(
         self,
         queue_id: str,
@@ -578,7 +528,9 @@ class SqliteSessionQueue(SessionQueueBase):
                     status,
                     priority,
                     field_values,
-                    error,
+                    error_type,
+                    error_message,
+                    error_traceback,
                     created_at,
                     updated_at,
                     completed_at,

invokeai/app/services/shared/graph.py

@@ -2,17 +2,19 @@
 
 import copy
 import itertools
-from typing import Annotated, Any, Optional, TypeVar, Union, get_args, get_origin, get_type_hints
+from typing import Any, Optional, TypeVar, Union, get_args, get_origin, get_type_hints
 
 import networkx as nx
 from pydantic import (
     BaseModel,
+    GetCoreSchemaHandler,
     GetJsonSchemaHandler,
+    ValidationError,
     field_validator,
 )
 from pydantic.fields import Field
 from pydantic.json_schema import JsonSchemaValue
-from pydantic_core import CoreSchema
+from pydantic_core import core_schema
 
 # Importing * is bad karma but needed here for node detection
 from invokeai.app.invocations import *  # noqa: F401 F403
@@ -190,6 +192,39 @@ class UnknownGraphValidationError(ValueError):
     pass
 
 
+class NodeInputError(ValueError):
+    """Raised when a node fails preparation. This occurs when a node's inputs are being set from its incomers, but an
+    input fails validation.
+
+    Attributes:
+        node: The node that failed preparation. Note: only successfully set fields will be accurate. Review the error to
+            determine which field caused the failure.
+    """
+
+    def __init__(self, node: BaseInvocation, e: ValidationError):
+        self.original_error = e
+        self.node = node
+        # When preparing a node, we set each input one-at-a-time. We may thus safely assume that the first error
+        # represents the first input that failed.
+        self.failed_input = loc_to_dot_sep(e.errors()[0]["loc"])
+        super().__init__(f"Node {node.id} has invalid incoming input for {self.failed_input}")
+
+
+def loc_to_dot_sep(loc: tuple[Union[str, int], ...]) -> str:
+    """Helper to pretty-print pydantic error locations as dot-separated strings.
+    Taken from https://docs.pydantic.dev/latest/errors/errors/#customize-error-messages
+    """
+    path = ""
+    for i, x in enumerate(loc):
+        if isinstance(x, str):
+            if i > 0:
+                path += "."
+            path += x
+        else:
+            path += f"[{x}]"
+    return path
+
+
 @invocation_output("iterate_output")
 class IterateInvocationOutput(BaseInvocationOutput):
     """Used to connect iteration outputs. Will be expanded to a specific output."""
@@ -243,73 +278,58 @@ class CollectInvocation(BaseInvocation):
         return CollectInvocationOutput(collection=copy.copy(self.collection))
 
 
+class AnyInvocation(BaseInvocation):
+    @classmethod
+    def __get_pydantic_core_schema__(cls, source_type: Any, handler: GetCoreSchemaHandler) -> core_schema.CoreSchema:
+        def validate_invocation(v: Any) -> "AnyInvocation":
+            return BaseInvocation.get_typeadapter().validate_python(v)
+
+        return core_schema.no_info_plain_validator_function(validate_invocation)
+
+    @classmethod
+    def __get_pydantic_json_schema__(
+        cls, core_schema: core_schema.CoreSchema, handler: GetJsonSchemaHandler
+    ) -> JsonSchemaValue:
+        # Nodes are too powerful, we have to make our own OpenAPI schema manually
+        # No but really, because the schema is dynamic depending on loaded nodes, we need to generate it manually
+        oneOf: list[dict[str, str]] = []
+        names = [i.__name__ for i in BaseInvocation.get_invocations()]
+        for name in sorted(names):
+            oneOf.append({"$ref": f"#/components/schemas/{name}"})
+        return {"oneOf": oneOf}
+
+
+class AnyInvocationOutput(BaseInvocationOutput):
+    @classmethod
+    def __get_pydantic_core_schema__(cls, source_type: Any, handler: GetCoreSchemaHandler):
+        def validate_invocation_output(v: Any) -> "AnyInvocationOutput":
+            return BaseInvocationOutput.get_typeadapter().validate_python(v)
+
+        return core_schema.no_info_plain_validator_function(validate_invocation_output)
+
+    @classmethod
+    def __get_pydantic_json_schema__(
+        cls, core_schema: core_schema.CoreSchema, handler: GetJsonSchemaHandler
+    ) -> JsonSchemaValue:
+        # Nodes are too powerful, we have to make our own OpenAPI schema manually
+        # No but really, because the schema is dynamic depending on loaded nodes, we need to generate it manually
+
+        oneOf: list[dict[str, str]] = []
+        names = [i.__name__ for i in BaseInvocationOutput.get_outputs()]
+        for name in sorted(names):
+            oneOf.append({"$ref": f"#/components/schemas/{name}"})
+        return {"oneOf": oneOf}
+
+
 class Graph(BaseModel):
     id: str = Field(description="The id of this graph", default_factory=uuid_string)
     # TODO: use a list (and never use dict in a BaseModel) because pydantic/fastapi hates me
-    nodes: dict[str, BaseInvocation] = Field(description="The nodes in this graph", default_factory=dict)
+    nodes: dict[str, AnyInvocation] = Field(description="The nodes in this graph", default_factory=dict)
     edges: list[Edge] = Field(
         description="The connections between nodes and their fields in this graph",
         default_factory=list,
     )
 
-    @field_validator("nodes", mode="plain")
-    @classmethod
-    def validate_nodes(cls, v: dict[str, Any]):
-        """Validates the nodes in the graph by retrieving a union of all node types and validating each node."""
-
-        # Invocations register themselves as their python modules are executed. The union of all invocations is
-        # constructed at runtime. We use pydantic to validate `Graph.nodes` using that union.
-        #
-        # It's possible that when `graph.py` is executed, not all invocation-containing modules will have executed. If
-        # we construct the invocation union as `graph.py` is executed, we may miss some invocations. Those missing
-        # invocations will cause a graph to fail if they are used.
-        #
-        # We can get around this by validating the nodes in the graph using a "plain" validator, which overrides the
-        # pydantic validation entirely. This allows us to validate the nodes using the union of invocations at runtime.
-        #
-        # This same pattern is used in `GraphExecutionState`.
-
-        nodes: dict[str, BaseInvocation] = {}
-        typeadapter = BaseInvocation.get_typeadapter()
-        for node_id, node in v.items():
-            nodes[node_id] = typeadapter.validate_python(node)
-        return nodes
-
-    @classmethod
-    def __get_pydantic_json_schema__(cls, core_schema: CoreSchema, handler: GetJsonSchemaHandler) -> JsonSchemaValue:
-        # We use a "plain" validator to validate the nodes in the graph. Pydantic is unable to create a JSON Schema for
-        # fields that use "plain" validators, so we have to hack around this. Also, we need to add all invocations to
-        # the generated schema as options for the `nodes` field.
-        #
-        # The workaround is to create a new BaseModel that has the same fields as `Graph` but without the validator and
-        # with the invocation union as the type for the `nodes` field. Pydantic then generates the JSON Schema as
-        # expected.
-        #
-        # You might be tempted to do something like this:
-        #
-        # ```py
-        # cloned_model = create_model(cls.__name__, __base__=cls, nodes=...)
-        # delattr(cloned_model, "validate_nodes")
-        # cloned_model.model_rebuild(force=True)
-        # json_schema = handler(cloned_model.__pydantic_core_schema__)
-        # ```
-        #
-        # Unfortunately, this does not work. Calling `handler` here results in infinite recursion as pydantic attempts
-        # to build the JSON Schema for the cloned model. Instead, we have to manually clone the model.
-        #
-        # This same pattern is used in `GraphExecutionState`.
-
-        class Graph(BaseModel):
-            id: Optional[str] = Field(default=None, description="The id of this graph")
-            nodes: dict[
-                str, Annotated[Union[tuple(BaseInvocation._invocation_classes)], Field(discriminator="type")]
-            ] = Field(description="The nodes in this graph")
-            edges: list[Edge] = Field(description="The connections between nodes and their fields in this graph")
-
-        json_schema = handler(Graph.__pydantic_core_schema__)
-        json_schema = handler.resolve_ref_schema(json_schema)
-        return json_schema
-
     def add_node(self, node: BaseInvocation) -> None:
         """Adds a node to a graph
 
@@ -740,7 +760,7 @@ class GraphExecutionState(BaseModel):
     )
 
     # The results of executed nodes
-    results: dict[str, BaseInvocationOutput] = Field(description="The results of node executions", default_factory=dict)
+    results: dict[str, AnyInvocationOutput] = Field(description="The results of node executions", default_factory=dict)
 
     # Errors raised when executing nodes
     errors: dict[str, str] = Field(description="Errors raised when executing nodes", default_factory=dict)
@@ -757,52 +777,12 @@ class GraphExecutionState(BaseModel):
         default_factory=dict,
     )
 
-    @field_validator("results", mode="plain")
-    @classmethod
-    def validate_results(cls, v: dict[str, BaseInvocationOutput]):
-        """Validates the results in the GES by retrieving a union of all output types and validating each result."""
-
-        # See the comment in `Graph.validate_nodes` for an explanation of this logic.
-        results: dict[str, BaseInvocationOutput] = {}
-        typeadapter = BaseInvocationOutput.get_typeadapter()
-        for result_id, result in v.items():
-            results[result_id] = typeadapter.validate_python(result)
-        return results
-
     @field_validator("graph")
     def graph_is_valid(cls, v: Graph):
         """Validates that the graph is valid"""
         v.validate_self()
         return v
 
-    @classmethod
-    def __get_pydantic_json_schema__(cls, core_schema: CoreSchema, handler: GetJsonSchemaHandler) -> JsonSchemaValue:
-        # See the comment in `Graph.__get_pydantic_json_schema__` for an explanation of this logic.
-        class GraphExecutionState(BaseModel):
-            """Tracks the state of a graph execution"""
-
-            id: str = Field(description="The id of the execution state")
-            graph: Graph = Field(description="The graph being executed")
-            execution_graph: Graph = Field(description="The expanded graph of activated and executed nodes")
-            executed: set[str] = Field(description="The set of node ids that have been executed")
-            executed_history: list[str] = Field(
-                description="The list of node ids that have been executed, in order of execution"
-            )
-            results: dict[
-                str, Annotated[Union[tuple(BaseInvocationOutput._output_classes)], Field(discriminator="type")]
-            ] = Field(description="The results of node executions")
-            errors: dict[str, str] = Field(description="Errors raised when executing nodes")
-            prepared_source_mapping: dict[str, str] = Field(
-                description="The map of prepared nodes to original graph nodes"
-            )
-            source_prepared_mapping: dict[str, set[str]] = Field(
-                description="The map of original graph nodes to prepared nodes"
-            )
-
-        json_schema = handler(GraphExecutionState.__pydantic_core_schema__)
-        json_schema = handler.resolve_ref_schema(json_schema)
-        return json_schema
-
     def next(self) -> Optional[BaseInvocation]:
         """Gets the next node ready to execute."""
 
@@ -821,7 +801,10 @@ class GraphExecutionState(BaseModel):
 
         # Get values from edges
         if next_node is not None:
-            self._prepare_inputs(next_node)
+            try:
+                self._prepare_inputs(next_node)
+            except ValidationError as e:
+                raise NodeInputError(next_node, e)
 
         # If next is still none, there's no next node, return None
         return next_node

invokeai/app/services/shared/invocation_context.py

@@ -1,7 +1,6 @@
-import threading
 from dataclasses import dataclass
 from pathlib import Path
-from typing import TYPE_CHECKING, Optional, Union
+from typing import TYPE_CHECKING, Callable, Optional, Union
 
 from PIL.Image import Image
 from torch import Tensor
@@ -180,9 +179,9 @@ class ImagesInterface(InvocationContextInterface):
         # If `metadata` is provided directly, use that. Else, use the metadata provided by `WithMetadata`, falling back to None.
         metadata_ = None
         if metadata:
-            metadata_ = metadata
-        elif isinstance(self._data.invocation, WithMetadata):
-            metadata_ = self._data.invocation.metadata
+            metadata_ = metadata.model_dump_json()
+        elif isinstance(self._data.invocation, WithMetadata) and self._data.invocation.metadata:
+            metadata_ = self._data.invocation.metadata.model_dump_json()
 
         # If `board_id` is provided directly, use that. Else, use the board provided by `WithBoard`, falling back to None.
         board_id_ = None
@@ -191,6 +190,14 @@ class ImagesInterface(InvocationContextInterface):
         elif isinstance(self._data.invocation, WithBoard) and self._data.invocation.board:
             board_id_ = self._data.invocation.board.board_id
 
+        workflow_ = None
+        if self._data.queue_item.workflow:
+            workflow_ = self._data.queue_item.workflow.model_dump_json()
+
+        graph_ = None
+        if self._data.queue_item.session.graph:
+            graph_ = self._data.queue_item.session.graph.model_dump_json()
+
         return self._services.images.create(
             image=image,
             is_intermediate=self._data.invocation.is_intermediate,
@@ -198,7 +205,8 @@ class ImagesInterface(InvocationContextInterface):
             board_id=board_id_,
             metadata=metadata_,
             image_origin=ResourceOrigin.INTERNAL,
-            workflow=self._data.queue_item.workflow,
+            workflow=workflow_,
+            graph=graph_,
             session_id=self._data.queue_item.session_id,
             node_id=self._data.invocation.id,
         )
@@ -344,11 +352,11 @@ class ModelsInterface(InvocationContextInterface):
 
         if isinstance(identifier, str):
             model = self._services.model_manager.store.get_model(identifier)
-            return self._services.model_manager.load.load_model(model, submodel_type, self._data)
+            return self._services.model_manager.load.load_model(model, submodel_type)
         else:
             _submodel_type = submodel_type or identifier.submodel_type
             model = self._services.model_manager.store.get_model(identifier.key)
-            return self._services.model_manager.load.load_model(model, _submodel_type, self._data)
+            return self._services.model_manager.load.load_model(model, _submodel_type)
 
     def load_by_attrs(
         self, name: str, base: BaseModelType, type: ModelType, submodel_type: Optional[SubModelType] = None
@@ -373,7 +381,7 @@ class ModelsInterface(InvocationContextInterface):
         if len(configs) > 1:
             raise ValueError(f"More than one model found with name {name}, base {base}, and type {type}")
 
-        return self._services.model_manager.load.load_model(configs[0], submodel_type, self._data)
+        return self._services.model_manager.load.load_model(configs[0], submodel_type)
 
     def get_config(self, identifier: Union[str, "ModelIdentifierField"]) -> AnyModelConfig:
         """Gets a model's config.
@@ -440,10 +448,10 @@ class ConfigInterface(InvocationContextInterface):
 
 class UtilInterface(InvocationContextInterface):
     def __init__(
-        self, services: InvocationServices, data: InvocationContextData, cancel_event: threading.Event
+        self, services: InvocationServices, data: InvocationContextData, is_canceled: Callable[[], bool]
     ) -> None:
         super().__init__(services, data)
-        self._cancel_event = cancel_event
+        self._is_canceled = is_canceled
 
     def is_canceled(self) -> bool:
         """Checks if the current session has been canceled.
@@ -451,7 +459,7 @@ class UtilInterface(InvocationContextInterface):
         Returns:
             True if the current session has been canceled, False if not.
         """
-        return self._cancel_event.is_set()
+        return self._is_canceled()
 
     def sd_step_callback(self, intermediate_state: PipelineIntermediateState, base_model: BaseModelType) -> None:
         """
@@ -526,7 +534,7 @@ class InvocationContext:
 def build_invocation_context(
     services: InvocationServices,
     data: InvocationContextData,
-    cancel_event: threading.Event,
+    is_canceled: Callable[[], bool],
 ) -> InvocationContext:
     """Builds the invocation context for a specific invocation execution.
 
@@ -543,7 +551,7 @@ def build_invocation_context(
     tensors = TensorsInterface(services=services, data=data)
     models = ModelsInterface(services=services, data=data)
     config = ConfigInterface(services=services, data=data)
-    util = UtilInterface(services=services, data=data, cancel_event=cancel_event)
+    util = UtilInterface(services=services, data=data, is_canceled=is_canceled)
     conditioning = ConditioningInterface(services=services, data=data)
     boards = BoardsInterface(services=services, data=data)
 

invokeai/app/services/shared/sqlite/sqlite_util.py

@@ -12,6 +12,7 @@ from invokeai.app.services.shared.sqlite_migrator.migrations.migration_6 import
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_7 import build_migration_7
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_8 import build_migration_8
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_9 import build_migration_9
+from invokeai.app.services.shared.sqlite_migrator.migrations.migration_10 import build_migration_10
 from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_impl import SqliteMigrator
 
 
@@ -41,6 +42,7 @@ def init_db(config: InvokeAIAppConfig, logger: Logger, image_files: ImageFileSto
     migrator.register_migration(build_migration_7())
     migrator.register_migration(build_migration_8(app_config=config))
     migrator.register_migration(build_migration_9())
+    migrator.register_migration(build_migration_10())
     migrator.run_migrations()
 
     return db

invokeai/app/services/shared/sqlite_migrator/migrations/migration_10.py

@@ -0,0 +1,35 @@
+import sqlite3
+
+from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_common import Migration
+
+
+class Migration10Callback:
+    def __call__(self, cursor: sqlite3.Cursor) -> None:
+        self._update_error_cols(cursor)
+
+    def _update_error_cols(self, cursor: sqlite3.Cursor) -> None:
+        """
+        - Adds `error_type` and `error_message` columns to the session queue table.
+        - Renames the `error` column to `error_traceback`.
+        """
+
+        cursor.execute("ALTER TABLE session_queue ADD COLUMN error_type TEXT;")
+        cursor.execute("ALTER TABLE session_queue ADD COLUMN error_message TEXT;")
+        cursor.execute("ALTER TABLE session_queue RENAME COLUMN error TO error_traceback;")
+
+
+def build_migration_10() -> Migration:
+    """
+    Build the migration from database version 9 to 10.
+
+    This migration does the following:
+    - Adds `error_type` and `error_message` columns to the session queue table.
+    - Renames the `error` column to `error_traceback`.
+    """
+    migration_10 = Migration(
+        from_version=9,
+        to_version=10,
+        callback=Migration10Callback(),
+    )
+
+    return migration_10

invokeai/app/util/controlnet_utils.py

@@ -289,7 +289,7 @@ def prepare_control_image(
     width: int,
     height: int,
     num_channels: int = 3,
-    device: str = "cuda",
+    device: str | torch.device = "cuda",
     dtype: torch.dtype = torch.float16,
     control_mode: CONTROLNET_MODE_VALUES = "balanced",
     resize_mode: CONTROLNET_RESIZE_VALUES = "just_resize_simple",
@@ -304,7 +304,7 @@ def prepare_control_image(
         num_channels (int, optional): The target number of image channels. This is achieved by converting the input
             image to RGB, then naively taking the first `num_channels` channels. The primary use case is converting a
             RGB image to a single-channel grayscale image. Raises if `num_channels` cannot be achieved. Defaults to 3.
-        device (str, optional): The target device for the output image. Defaults to "cuda".
+        device (str | torch.Device, optional): The target device for the output image. Defaults to "cuda".
         dtype (_type_, optional): The dtype for the output image. Defaults to torch.float16.
         do_classifier_free_guidance (bool, optional): If True, repeat the output image along the batch dimension.
             Defaults to True.

invokeai/app/util/custom_openapi.py

@@ -0,0 +1,116 @@
+from typing import Any, Callable, Optional
+
+from fastapi import FastAPI
+from fastapi.openapi.utils import get_openapi
+from pydantic.json_schema import models_json_schema
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, UIConfigBase
+from invokeai.app.invocations.fields import InputFieldJSONSchemaExtra, OutputFieldJSONSchemaExtra
+from invokeai.app.invocations.model import ModelIdentifierField
+from invokeai.app.services.events.events_common import EventBase
+from invokeai.app.services.session_processor.session_processor_common import ProgressImage
+
+
+def move_defs_to_top_level(openapi_schema: dict[str, Any], component_schema: dict[str, Any]) -> None:
+    """Moves a component schema's $defs to the top level of the openapi schema. Useful when generating a schema
+    for a single model that needs to be added back to the top level of the schema. Mutates openapi_schema and
+    component_schema."""
+
+    defs = component_schema.pop("$defs", {})
+    for schema_key, json_schema in defs.items():
+        if schema_key in openapi_schema["components"]["schemas"]:
+            continue
+        openapi_schema["components"]["schemas"][schema_key] = json_schema
+
+
+def get_openapi_func(
+    app: FastAPI, post_transform: Optional[Callable[[dict[str, Any]], dict[str, Any]]] = None
+) -> Callable[[], dict[str, Any]]:
+    """Gets the OpenAPI schema generator function.
+
+    Args:
+        app (FastAPI): The FastAPI app to generate the schema for.
+        post_transform (Optional[Callable[[dict[str, Any]], dict[str, Any]]], optional): A function to apply to the
+            generated schema before returning it. Defaults to None.
+
+    Returns:
+        Callable[[], dict[str, Any]]: The OpenAPI schema generator function. When first called, the generated schema is
+            cached in `app.openapi_schema`. On subsequent calls, the cached schema is returned. This caching behaviour
+            matches FastAPI's default schema generation caching.
+    """
+
+    def openapi() -> dict[str, Any]:
+        if app.openapi_schema:
+            return app.openapi_schema
+
+        openapi_schema = get_openapi(
+            title=app.title,
+            description="An API for invoking AI image operations",
+            version="1.0.0",
+            routes=app.routes,
+            separate_input_output_schemas=False,  # https://fastapi.tiangolo.com/how-to/separate-openapi-schemas/
+        )
+
+        # We'll create a map of invocation type to output schema to make some types simpler on the client.
+        invocation_output_map_properties: dict[str, Any] = {}
+        invocation_output_map_required: list[str] = []
+
+        # We need to manually add all outputs to the schema - pydantic doesn't add them because they aren't used directly.
+        for output in BaseInvocationOutput.get_outputs():
+            json_schema = output.model_json_schema(mode="serialization", ref_template="#/components/schemas/{model}")
+            move_defs_to_top_level(openapi_schema, json_schema)
+            openapi_schema["components"]["schemas"][output.__name__] = json_schema
+
+        # Technically, invocations are added to the schema by pydantic, but we still need to manually set their output
+        # property, so we'll just do it all manually.
+        for invocation in BaseInvocation.get_invocations():
+            json_schema = invocation.model_json_schema(
+                mode="serialization", ref_template="#/components/schemas/{model}"
+            )
+            move_defs_to_top_level(openapi_schema, json_schema)
+            output_title = invocation.get_output_annotation().__name__
+            outputs_ref = {"$ref": f"#/components/schemas/{output_title}"}
+            json_schema["output"] = outputs_ref
+            openapi_schema["components"]["schemas"][invocation.__name__] = json_schema
+
+            # Add this invocation and its output to the output map
+            invocation_type = invocation.get_type()
+            invocation_output_map_properties[invocation_type] = json_schema["output"]
+            invocation_output_map_required.append(invocation_type)
+
+        # Add the output map to the schema
+        openapi_schema["components"]["schemas"]["InvocationOutputMap"] = {
+            "type": "object",
+            "properties": invocation_output_map_properties,
+            "required": invocation_output_map_required,
+        }
+
+        # Some models don't end up in the schemas as standalone definitions because they aren't used directly in the API.
+        # We need to add them manually here. WARNING: Pydantic can choke if you call `model.model_json_schema()` to get
+        # a schema. This has something to do with schema refs - not totally clear. For whatever reason, using
+        # `models_json_schema` seems to work fine.
+        additional_models = [
+            *EventBase.get_events(),
+            UIConfigBase,
+            InputFieldJSONSchemaExtra,
+            OutputFieldJSONSchemaExtra,
+            ModelIdentifierField,
+            ProgressImage,
+        ]
+
+        additional_schemas = models_json_schema(
+            [(m, "serialization") for m in additional_models],
+            ref_template="#/components/schemas/{model}",
+        )
+        # additional_schemas[1] is a dict of $defs that we need to add to the top level of the schema
+        move_defs_to_top_level(openapi_schema, additional_schemas[1])
+
+        if post_transform is not None:
+            openapi_schema = post_transform(openapi_schema)
+
+        openapi_schema["components"]["schemas"] = dict(sorted(openapi_schema["components"]["schemas"].items()))
+
+        app.openapi_schema = openapi_schema
+        return app.openapi_schema
+
+    return openapi

invokeai/app/util/step_callback.py

@@ -1,4 +1,4 @@
-from typing import TYPE_CHECKING, Callable
+from typing import TYPE_CHECKING, Callable, Optional
 
 import torch
 from PIL import Image
@@ -13,8 +13,36 @@ if TYPE_CHECKING:
     from invokeai.app.services.events.events_base import EventServiceBase
     from invokeai.app.services.shared.invocation_context import InvocationContextData
 
+# fast latents preview matrix for sdxl
+# generated by @StAlKeR7779
+SDXL_LATENT_RGB_FACTORS = [
+    #   R        G        B
+    [0.3816, 0.4930, 0.5320],
+    [-0.3753, 0.1631, 0.1739],
+    [0.1770, 0.3588, -0.2048],
+    [-0.4350, -0.2644, -0.4289],
+]
+SDXL_SMOOTH_MATRIX = [
+    [0.0358, 0.0964, 0.0358],
+    [0.0964, 0.4711, 0.0964],
+    [0.0358, 0.0964, 0.0358],
+]
 
-def sample_to_lowres_estimated_image(samples, latent_rgb_factors, smooth_matrix=None):
+# origingally adapted from code by @erucipe and @keturn here:
+# https://discuss.huggingface.co/t/decoding-latents-to-rgb-without-upscaling/23204/7
+# these updated numbers for v1.5 are from @torridgristle
+SD1_5_LATENT_RGB_FACTORS = [
+    #    R        G        B
+    [0.3444, 0.1385, 0.0670],  # L1
+    [0.1247, 0.4027, 0.1494],  # L2
+    [-0.3192, 0.2513, 0.2103],  # L3
+    [-0.1307, -0.1874, -0.7445],  # L4
+]
+
+
+def sample_to_lowres_estimated_image(
+    samples: torch.Tensor, latent_rgb_factors: torch.Tensor, smooth_matrix: Optional[torch.Tensor] = None
+):
     latent_image = samples[0].permute(1, 2, 0) @ latent_rgb_factors
 
     if smooth_matrix is not None:
@@ -47,64 +75,12 @@ def stable_diffusion_step_callback(
     else:
         sample = intermediate_state.latents
 
-    # TODO: This does not seem to be needed any more?
-    # # txt2img provides a Tensor in the step_callback
-    # # img2img provides a PipelineIntermediateState
-    # if isinstance(sample, PipelineIntermediateState):
-    #     # this was an img2img
-    #     print('img2img')
-    #     latents = sample.latents
-    #     step = sample.step
-    # else:
-    #     print('txt2img')
-    #     latents = sample
-    #     step = intermediate_state.step
-
-    # TODO: only output a preview image when requested
-
     if base_model in [BaseModelType.StableDiffusionXL, BaseModelType.StableDiffusionXLRefiner]:
-        # fast latents preview matrix for sdxl
-        # generated by @StAlKeR7779
-        sdxl_latent_rgb_factors = torch.tensor(
-            [
-                #   R        G        B
-                [0.3816, 0.4930, 0.5320],
-                [-0.3753, 0.1631, 0.1739],
-                [0.1770, 0.3588, -0.2048],
-                [-0.4350, -0.2644, -0.4289],
-            ],
-            dtype=sample.dtype,
-            device=sample.device,
-        )
-
-        sdxl_smooth_matrix = torch.tensor(
-            [
-                [0.0358, 0.0964, 0.0358],
-                [0.0964, 0.4711, 0.0964],
-                [0.0358, 0.0964, 0.0358],
-            ],
-            dtype=sample.dtype,
-            device=sample.device,
-        )
-
+        sdxl_latent_rgb_factors = torch.tensor(SDXL_LATENT_RGB_FACTORS, dtype=sample.dtype, device=sample.device)
+        sdxl_smooth_matrix = torch.tensor(SDXL_SMOOTH_MATRIX, dtype=sample.dtype, device=sample.device)
         image = sample_to_lowres_estimated_image(sample, sdxl_latent_rgb_factors, sdxl_smooth_matrix)
     else:
-        # origingally adapted from code by @erucipe and @keturn here:
-        # https://discuss.huggingface.co/t/decoding-latents-to-rgb-without-upscaling/23204/7
-
-        # these updated numbers for v1.5 are from @torridgristle
-        v1_5_latent_rgb_factors = torch.tensor(
-            [
-                #    R        G        B
-                [0.3444, 0.1385, 0.0670],  # L1
-                [0.1247, 0.4027, 0.1494],  # L2
-                [-0.3192, 0.2513, 0.2103],  # L3
-                [-0.1307, -0.1874, -0.7445],  # L4
-            ],
-            dtype=sample.dtype,
-            device=sample.device,
-        )
-
+        v1_5_latent_rgb_factors = torch.tensor(SD1_5_LATENT_RGB_FACTORS, dtype=sample.dtype, device=sample.device)
         image = sample_to_lowres_estimated_image(sample, v1_5_latent_rgb_factors)
 
     (width, height) = image.size
@@ -113,15 +89,9 @@ def stable_diffusion_step_callback(
 
     dataURL = image_to_dataURL(image, image_format="JPEG")
 
-    events.emit_generator_progress(
-        queue_id=context_data.queue_item.queue_id,
-        queue_item_id=context_data.queue_item.item_id,
-        queue_batch_id=context_data.queue_item.batch_id,
-        graph_execution_state_id=context_data.queue_item.session_id,
-        node_id=context_data.invocation.id,
-        source_node_id=context_data.source_invocation_id,
-        progress_image=ProgressImage(width=width, height=height, dataURL=dataURL),
-        step=intermediate_state.step,
-        order=intermediate_state.order,
-        total_steps=intermediate_state.total_steps,
+    events.emit_invocation_denoise_progress(
+        context_data.queue_item,
+        context_data.invocation,
+        intermediate_state,
+        ProgressImage(dataURL=dataURL, width=width, height=height),
     )

invokeai/backend/image_util/__init__.py

@@ -4,5 +4,4 @@ Initialization file for invokeai.backend.image_util methods.
 
 from .infill_methods.patchmatch import PatchMatch  # noqa: F401
 from .pngwriter import PngWriter, PromptFormatter, retrieve_metadata, write_metadata  # noqa: F401
-from .seamless import configure_model_padding  # noqa: F401
 from .util import InitImageResizer, make_grid  # noqa: F401

invokeai/backend/image_util/safety_checker.py

@@ -8,7 +8,7 @@ from pathlib import Path
 
 import numpy as np
 from diffusers.pipelines.stable_diffusion.safety_checker import StableDiffusionSafetyChecker
-from PIL import Image
+from PIL import Image, ImageFilter
 from transformers import AutoFeatureExtractor
 
 import invokeai.backend.util.logging as logger
@@ -16,6 +16,7 @@ from invokeai.app.services.config.config_default import get_config
 from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.silence_warnings import SilenceWarnings
 
+repo_id = "CompVis/stable-diffusion-safety-checker"
 CHECKER_PATH = "core/convert/stable-diffusion-safety-checker"
 
 
@@ -24,30 +25,30 @@ class SafetyChecker:
     Wrapper around SafetyChecker model.
     """
 
-    safety_checker = None
     feature_extractor = None
-    tried_load: bool = False
+    safety_checker = None
 
     @classmethod
     def _load_safety_checker(cls):
-        if cls.tried_load:
+        if cls.safety_checker is not None and cls.feature_extractor is not None:
             return
 
         try:
-            cls.safety_checker = StableDiffusionSafetyChecker.from_pretrained(get_config().models_path / CHECKER_PATH)
-            cls.feature_extractor = AutoFeatureExtractor.from_pretrained(get_config().models_path / CHECKER_PATH)
+            model_path = get_config().models_path / CHECKER_PATH
+            if model_path.exists():
+                cls.feature_extractor = AutoFeatureExtractor.from_pretrained(model_path)
+                cls.safety_checker = StableDiffusionSafetyChecker.from_pretrained(model_path)
+            else:
+                model_path.mkdir(parents=True, exist_ok=True)
+                cls.feature_extractor = AutoFeatureExtractor.from_pretrained(repo_id)
+                cls.feature_extractor.save_pretrained(model_path, safe_serialization=True)
+                cls.safety_checker = StableDiffusionSafetyChecker.from_pretrained(repo_id)
+                cls.safety_checker.save_pretrained(model_path, safe_serialization=True)
         except Exception as e:
             logger.warning(f"Could not load NSFW checker: {str(e)}")
-        cls.tried_load = True
-
-    @classmethod
-    def safety_checker_available(cls) -> bool:
-        return Path(get_config().models_path, CHECKER_PATH).exists()
 
     @classmethod
     def has_nsfw_concept(cls, image: Image.Image) -> bool:
-        if not cls.safety_checker_available() and cls.tried_load:
-            return False
         cls._load_safety_checker()
         if cls.safety_checker is None or cls.feature_extractor is None:
             return False
@@ -60,3 +61,24 @@ class SafetyChecker:
         with SilenceWarnings():
             checked_image, has_nsfw_concept = cls.safety_checker(images=x_image, clip_input=features.pixel_values)
         return has_nsfw_concept[0]
+
+    @classmethod
+    def blur_if_nsfw(cls, image: Image.Image) -> Image.Image:
+        if cls.has_nsfw_concept(image):
+            logger.warning("A potentially NSFW image has been detected. Image will be blurred.")
+            blurry_image = image.filter(filter=ImageFilter.GaussianBlur(radius=32))
+            caution = cls._get_caution_img()
+            # Center the caution image on the blurred image
+            x = (blurry_image.width - caution.width) // 2
+            y = (blurry_image.height - caution.height) // 2
+            blurry_image.paste(caution, (x, y), caution)
+            image = blurry_image
+
+        return image
+
+    @classmethod
+    def _get_caution_img(cls) -> Image.Image:
+        import invokeai.app.assets.images as image_assets
+
+        caution = Image.open(Path(image_assets.__path__[0]) / "caution.png")
+        return caution.resize((caution.width // 2, caution.height // 2))

invokeai/backend/image_util/seamless.py

@@ -1,52 +0,0 @@
-import torch.nn as nn
-
-
-def _conv_forward_asymmetric(self, input, weight, bias):
-    """
-    Patch for Conv2d._conv_forward that supports asymmetric padding
-    """
-    working = nn.functional.pad(input, self.asymmetric_padding["x"], mode=self.asymmetric_padding_mode["x"])
-    working = nn.functional.pad(working, self.asymmetric_padding["y"], mode=self.asymmetric_padding_mode["y"])
-    return nn.functional.conv2d(
-        working,
-        weight,
-        bias,
-        self.stride,
-        nn.modules.utils._pair(0),
-        self.dilation,
-        self.groups,
-    )
-
-
-def configure_model_padding(model, seamless, seamless_axes):
-    """
-    Modifies the 2D convolution layers to use a circular padding mode based on
-    the `seamless` and `seamless_axes` options.
-    """
-    # TODO: get an explicit interface for this in diffusers: https://github.com/huggingface/diffusers/issues/556
-    for m in model.modules():
-        if isinstance(m, (nn.Conv2d, nn.ConvTranspose2d)):
-            if seamless:
-                m.asymmetric_padding_mode = {}
-                m.asymmetric_padding = {}
-                m.asymmetric_padding_mode["x"] = "circular" if ("x" in seamless_axes) else "constant"
-                m.asymmetric_padding["x"] = (
-                    m._reversed_padding_repeated_twice[0],
-                    m._reversed_padding_repeated_twice[1],
-                    0,
-                    0,
-                )
-                m.asymmetric_padding_mode["y"] = "circular" if ("y" in seamless_axes) else "constant"
-                m.asymmetric_padding["y"] = (
-                    0,
-                    0,
-                    m._reversed_padding_repeated_twice[2],
-                    m._reversed_padding_repeated_twice[3],
-                )
-                m._conv_forward = _conv_forward_asymmetric.__get__(m, nn.Conv2d)
-            else:
-                m._conv_forward = nn.Conv2d._conv_forward.__get__(m, nn.Conv2d)
-                if hasattr(m, "asymmetric_padding_mode"):
-                    del m.asymmetric_padding_mode
-                if hasattr(m, "asymmetric_padding"):
-                    del m.asymmetric_padding

invokeai/backend/image_util/util.py

@@ -144,10 +144,8 @@ def resize_image_to_resolution(input_image: np.ndarray, resolution: int) -> np.n
     h = float(input_image.shape[0])
     w = float(input_image.shape[1])
     scaling_factor = float(resolution) / min(h, w)
-    h *= scaling_factor
-    w *= scaling_factor
-    h = int(np.round(h / 64.0)) * 64
-    w = int(np.round(w / 64.0)) * 64
+    h = int(h * scaling_factor)
+    w = int(w * scaling_factor)
     if scaling_factor > 1:
         return cv2.resize(input_image, (w, h), interpolation=cv2.INTER_LANCZOS4)
     else:

invokeai/backend/model_manager/load/model_cache/model_cache_base.py

@@ -42,10 +42,26 @@ T = TypeVar("T")
 
 @dataclass
 class CacheRecord(Generic[T]):
-    """Elements of the cache."""
+    """
+    Elements of the cache:
+
+    key: Unique key for each model, same as used in the models database.
+    model: Model in memory.
+    state_dict: A read-only copy of the model's state dict in RAM. It will be
+                used as a template for creating a copy in the VRAM.
+    size: Size of the model
+    loaded: True if the model's state dict is currently in VRAM
+
+    Before a model is executed, the state_dict template is copied into VRAM,
+    and then injected into the model. When the model is finished, the VRAM
+    copy of the state dict is deleted, and the RAM version is reinjected
+    into the model.
+    """
 
     key: str
     model: T
+    device: torch.device
+    state_dict: Optional[Dict[str, torch.Tensor]]
     size: int
     loaded: bool = False
     _locks: int = 0

invokeai/backend/model_manager/load/model_cache/model_cache_default.py

@@ -20,7 +20,6 @@ context. Use like this:
 
 import gc
 import math
-import sys
 import time
 from contextlib import suppress
 from logging import Logger
@@ -162,7 +161,9 @@ class ModelCache(ModelCacheBase[AnyModel]):
         if key in self._cached_models:
             return
         self.make_room(size)
-        cache_record = CacheRecord(key, model, size)
+
+        state_dict = model.state_dict() if isinstance(model, torch.nn.Module) else None
+        cache_record = CacheRecord(key=key, model=model, device=self.storage_device, state_dict=state_dict, size=size)
         self._cached_models[key] = cache_record
         self._cache_stack.append(key)
 
@@ -257,17 +258,37 @@ class ModelCache(ModelCacheBase[AnyModel]):
         if not (hasattr(cache_entry.model, "device") and hasattr(cache_entry.model, "to")):
             return
 
-        source_device = cache_entry.model.device
+        source_device = cache_entry.device
 
         # Note: We compare device types only so that 'cuda' == 'cuda:0'.
         # This would need to be revised to support multi-GPU.
         if torch.device(source_device).type == torch.device(target_device).type:
             return
 
+        # This roundabout method for moving the model around is done to avoid
+        # the cost of moving the model from RAM to VRAM and then back from VRAM to RAM.
+        # When moving to VRAM, we copy (not move) each element of the state dict from
+        # RAM to a new state dict in VRAM, and then inject it into the model.
+        # This operation is slightly faster than running `to()` on the whole model.
+        #
+        # When the model needs to be removed from VRAM we simply delete the copy
+        # of the state dict in VRAM, and reinject the state dict that is cached
+        # in RAM into the model. So this operation is very fast.
         start_model_to_time = time.time()
         snapshot_before = self._capture_memory_snapshot()
+
         try:
+            if cache_entry.state_dict is not None:
+                assert hasattr(cache_entry.model, "load_state_dict")
+                if target_device == self.storage_device:
+                    cache_entry.model.load_state_dict(cache_entry.state_dict, assign=True)
+                else:
+                    new_dict: Dict[str, torch.Tensor] = {}
+                    for k, v in cache_entry.state_dict.items():
+                        new_dict[k] = v.to(torch.device(target_device), copy=True)
+                    cache_entry.model.load_state_dict(new_dict, assign=True)
             cache_entry.model.to(target_device)
+            cache_entry.device = target_device
         except Exception as e:  # blow away cache entry
             self._delete_cache_entry(cache_entry)
             raise e
@@ -347,43 +368,12 @@ class ModelCache(ModelCacheBase[AnyModel]):
         while current_size + bytes_needed > maximum_size and pos < len(self._cache_stack):
             model_key = self._cache_stack[pos]
             cache_entry = self._cached_models[model_key]
-
-            refs = sys.getrefcount(cache_entry.model)
-
-            # HACK: This is a workaround for a memory-management issue that we haven't tracked down yet. We are directly
-            # going against the advice in the Python docs by using `gc.get_referrers(...)` in this way:
-            # https://docs.python.org/3/library/gc.html#gc.get_referrers
-
-            # manualy clear local variable references of just finished function calls
-            # for some reason python don't want to collect it even by gc.collect() immidiately
-            if refs > 2:
-                while True:
-                    cleared = False
-                    for referrer in gc.get_referrers(cache_entry.model):
-                        if type(referrer).__name__ == "frame":
-                            # RuntimeError: cannot clear an executing frame
-                            with suppress(RuntimeError):
-                                referrer.clear()
-                                cleared = True
-                                # break
-
-                    # repeat if referrers changes(due to frame clear), else exit loop
-                    if cleared:
-                        gc.collect()
-                    else:
-                        break
-
             device = cache_entry.model.device if hasattr(cache_entry.model, "device") else None
             self.logger.debug(
-                f"Model: {model_key}, locks: {cache_entry._locks}, device: {device}, loaded: {cache_entry.loaded},"
-                f" refs: {refs}"
+                f"Model: {model_key}, locks: {cache_entry._locks}, device: {device}, loaded: {cache_entry.loaded}"
             )
 
-            # Expected refs:
-            # 1 from cache_entry
-            # 1 from getrefcount function
-            # 1 from onnx runtime object
-            if not cache_entry.locked and refs <= (3 if "onnx" in model_key else 2):
+            if not cache_entry.locked:
                 self.logger.debug(
                     f"Removing {model_key} from RAM cache to free at least {(size/GIG):.2f} GB (-{(cache_entry.size/GIG):.2f} GB)"
                 )

invokeai/backend/model_manager/load/model_cache/model_locker.py

@@ -60,5 +60,5 @@ class ModelLocker(ModelLockerBase):
 
         self._cache_entry.unlock()
         if not self._cache.lazy_offloading:
-            self._cache.offload_unlocked_models(self._cache_entry.size)
+            self._cache.offload_unlocked_models(0)
             self._cache.print_cuda_stats()

invokeai/backend/model_manager/probe.py

@@ -10,7 +10,7 @@ from picklescan.scanner import scan_file_path
 import invokeai.backend.util.logging as logger
 from invokeai.app.util.misc import uuid_string
 from invokeai.backend.model_hash.model_hash import HASHING_ALGORITHMS, ModelHash
-from invokeai.backend.util.util import SilenceWarnings
+from invokeai.backend.util.silence_warnings import SilenceWarnings
 
 from .config import (
     AnyModelConfig,
@@ -51,6 +51,7 @@ LEGACY_CONFIGS: Dict[BaseModelType, Dict[ModelVariantType, Union[str, Dict[Sched
     },
     BaseModelType.StableDiffusionXL: {
         ModelVariantType.Normal: "sd_xl_base.yaml",
+        ModelVariantType.Inpaint: "sd_xl_inpaint.yaml",
     },
     BaseModelType.StableDiffusionXLRefiner: {
         ModelVariantType.Normal: "sd_xl_refiner.yaml",

invokeai/backend/stable_diffusion/diffusers_pipeline.py

@@ -11,7 +11,6 @@ import psutil
 import torch
 import torchvision.transforms as T
 from diffusers.models import AutoencoderKL, UNet2DConditionModel
-from diffusers.models.controlnet import ControlNetModel
 from diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion import StableDiffusionPipeline
 from diffusers.pipelines.stable_diffusion.safety_checker import StableDiffusionSafetyChecker
 from diffusers.schedulers import KarrasDiffusionSchedulers
@@ -26,6 +25,7 @@ from invokeai.backend.stable_diffusion.diffusion.shared_invokeai_diffusion impor
 from invokeai.backend.stable_diffusion.diffusion.unet_attention_patcher import UNetAttentionPatcher, UNetIPAdapterData
 from invokeai.backend.util.attention import auto_detect_slice_size
 from invokeai.backend.util.devices import TorchDevice
+from invokeai.backend.util.hotfixes import ControlNetModel
 
 
 @dataclass

invokeai/backend/stable_diffusion/schedulers/schedulers.py

@@ -13,6 +13,7 @@ from diffusers import (
     LCMScheduler,
     LMSDiscreteScheduler,
     PNDMScheduler,
+    TCDScheduler,
     UniPCMultistepScheduler,
 )
 
@@ -40,4 +41,5 @@ SCHEDULER_MAP = {
     "dpmpp_sde_k": (DPMSolverSDEScheduler, {"use_karras_sigmas": True, "noise_sampler_seed": 0}),
     "unipc": (UniPCMultistepScheduler, {"cpu_only": True}),
     "lcm": (LCMScheduler, {}),
+    "tcd": (TCDScheduler, {}),
 }

invokeai/backend/stable_diffusion/seamless.py

@@ -1,89 +1,51 @@
-from __future__ import annotations
-
 from contextlib import contextmanager
-from typing import Callable, List, Union
+from typing import Callable, List, Optional, Tuple, Union
 
+import torch
 import torch.nn as nn
 from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL
 from diffusers.models.autoencoders.autoencoder_tiny import AutoencoderTiny
+from diffusers.models.lora import LoRACompatibleConv
 from diffusers.models.unets.unet_2d_condition import UNet2DConditionModel
 
 
-def _conv_forward_asymmetric(self, input, weight, bias):
-    """
-    Patch for Conv2d._conv_forward that supports asymmetric padding
-    """
-    working = nn.functional.pad(input, self.asymmetric_padding["x"], mode=self.asymmetric_padding_mode["x"])
-    working = nn.functional.pad(working, self.asymmetric_padding["y"], mode=self.asymmetric_padding_mode["y"])
-    return nn.functional.conv2d(
-        working,
-        weight,
-        bias,
-        self.stride,
-        nn.modules.utils._pair(0),
-        self.dilation,
-        self.groups,
-    )
-
-
 @contextmanager
 def set_seamless(model: Union[UNet2DConditionModel, AutoencoderKL, AutoencoderTiny], seamless_axes: List[str]):
     if not seamless_axes:
         yield
         return
 
-    # Callable: (input: Tensor, weight: Tensor, bias: Optional[Tensor]) -> Tensor
-    to_restore: list[tuple[nn.Conv2d | nn.ConvTranspose2d, Callable]] = []
+    # override conv_forward
+    # https://github.com/huggingface/diffusers/issues/556#issuecomment-1993287019
+    def _conv_forward_asymmetric(self, input: torch.Tensor, weight: torch.Tensor, bias: Optional[torch.Tensor] = None):
+        self.paddingX = (self._reversed_padding_repeated_twice[0], self._reversed_padding_repeated_twice[1], 0, 0)
+        self.paddingY = (0, 0, self._reversed_padding_repeated_twice[2], self._reversed_padding_repeated_twice[3])
+        working = torch.nn.functional.pad(input, self.paddingX, mode=x_mode)
+        working = torch.nn.functional.pad(working, self.paddingY, mode=y_mode)
+        return torch.nn.functional.conv2d(
+            working, weight, bias, self.stride, torch.nn.modules.utils._pair(0), self.dilation, self.groups
+        )
+
+    original_layers: List[Tuple[nn.Conv2d, Callable]] = []
+
     try:
-        # Hard coded to skip down block layers, allowing for seamless tiling at the expense of prompt adherence
-        skipped_layers = 1
-        for m_name, m in model.named_modules():
-            if not isinstance(m, (nn.Conv2d, nn.ConvTranspose2d)):
-                continue
+        x_mode = "circular" if "x" in seamless_axes else "constant"
+        y_mode = "circular" if "y" in seamless_axes else "constant"
 
-            if isinstance(model, UNet2DConditionModel) and m_name.startswith("down_blocks.") and ".resnets." in m_name:
-                # down_blocks.1.resnets.1.conv1
-                _, block_num, _, resnet_num, submodule_name = m_name.split(".")
-                block_num = int(block_num)
-                resnet_num = int(resnet_num)
+        conv_layers: List[torch.nn.Conv2d] = []
 
-                if block_num >= len(model.down_blocks) - skipped_layers:
-                    continue
+        for module in model.modules():
+            if isinstance(module, torch.nn.Conv2d):
+                conv_layers.append(module)
 
-                # Skip the second resnet (could be configurable)
-                if resnet_num > 0:
-                    continue
-
-                # Skip Conv2d layers (could be configurable)
-                if submodule_name == "conv2":
-                    continue
-
-            m.asymmetric_padding_mode = {}
-            m.asymmetric_padding = {}
-            m.asymmetric_padding_mode["x"] = "circular" if ("x" in seamless_axes) else "constant"
-            m.asymmetric_padding["x"] = (
-                m._reversed_padding_repeated_twice[0],
-                m._reversed_padding_repeated_twice[1],
-                0,
-                0,
-            )
-            m.asymmetric_padding_mode["y"] = "circular" if ("y" in seamless_axes) else "constant"
-            m.asymmetric_padding["y"] = (
-                0,
-                0,
-                m._reversed_padding_repeated_twice[2],
-                m._reversed_padding_repeated_twice[3],
-            )
-
-            to_restore.append((m, m._conv_forward))
-            m._conv_forward = _conv_forward_asymmetric.__get__(m, nn.Conv2d)
+        for layer in conv_layers:
+            if isinstance(layer, LoRACompatibleConv) and layer.lora_layer is None:
+                layer.lora_layer = lambda *x: 0
+            original_layers.append((layer, layer._conv_forward))
+            layer._conv_forward = _conv_forward_asymmetric.__get__(layer, torch.nn.Conv2d)
 
         yield
 
     finally:
-        for module, orig_conv_forward in to_restore:
-            module._conv_forward = orig_conv_forward
-            if hasattr(module, "asymmetric_padding_mode"):
-                del module.asymmetric_padding_mode
-            if hasattr(module, "asymmetric_padding"):
-                del module.asymmetric_padding
+        for layer, orig_conv_forward in original_layers:
+            layer._conv_forward = orig_conv_forward

invokeai/backend/textual_inversion.py

@@ -1,7 +1,7 @@
 """Textual Inversion wrapper class."""
 
 from pathlib import Path
-from typing import Dict, List, Optional, Union
+from typing import Optional, Union
 
 import torch
 from compel.embeddings_provider import BaseTextualInversionManager
@@ -66,35 +66,52 @@ class TextualInversionModelRaw(RawModel):
         return result
 
 
-# no type hints for BaseTextualInversionManager?
-class TextualInversionManager(BaseTextualInversionManager):  # type: ignore
-    pad_tokens: Dict[int, List[int]]
-    tokenizer: CLIPTokenizer
+class TextualInversionManager(BaseTextualInversionManager):
+    """TextualInversionManager implements the BaseTextualInversionManager ABC from the compel library."""
 
     def __init__(self, tokenizer: CLIPTokenizer):
-        self.pad_tokens = {}
+        self.pad_tokens: dict[int, list[int]] = {}
         self.tokenizer = tokenizer
 
     def expand_textual_inversion_token_ids_if_necessary(self, token_ids: list[int]) -> list[int]:
+        """Given a list of tokens ids, expand any TI tokens to their corresponding pad tokens.
+
+        For example, suppose we have a `<ti_dog>` TI with 4 vectors that was added to the tokenizer with the following
+        mapping of tokens to token_ids:
+        ```
+        <ti_dog>: 49408
+        <ti_dog-!pad-1>: 49409
+        <ti_dog-!pad-2>: 49410
+        <ti_dog-!pad-3>: 49411
+        ```
+        `self.pad_tokens` would be set to `{49408: [49408, 49409, 49410, 49411]}`.
+        This function is responsible for expanding `49408` in the token_ids list to `[49408, 49409, 49410, 49411]`.
+        """
+        # Short circuit if there are no pad tokens to save a little time.
         if len(self.pad_tokens) == 0:
             return token_ids
 
+        # This function assumes that compel has not included the BOS and EOS tokens in the token_ids list. We verify
+        # this assumption here.
         if token_ids[0] == self.tokenizer.bos_token_id:
             raise ValueError("token_ids must not start with bos_token_id")
         if token_ids[-1] == self.tokenizer.eos_token_id:
             raise ValueError("token_ids must not end with eos_token_id")
 
-        new_token_ids = []
+        # Expand any TI tokens to their corresponding pad tokens.
+        new_token_ids: list[int] = []
         for token_id in token_ids:
             new_token_ids.append(token_id)
             if token_id in self.pad_tokens:
                 new_token_ids.extend(self.pad_tokens[token_id])
 
-        # Do not exceed the max model input size
-        # The -2 here is compensating for compensate compel.embeddings_provider.get_token_ids(),
-        # which first removes and then adds back the start and end tokens.
-        max_length = list(self.tokenizer.max_model_input_sizes.values())[0] - 2
+        # Do not exceed the max model input size. The -2 here is compensating for
+        # compel.embeddings_provider.get_token_ids(), which first removes and then adds back the start and end tokens.
+        max_length = self.tokenizer.model_max_length - 2
         if len(new_token_ids) > max_length:
+            # HACK: If TI token expansion causes us to exceed the max text encoder input length, we silently discard
+            # tokens. Token expansion should happen in a way that is compatible with compel's default handling of long
+            # prompts.
             new_token_ids = new_token_ids[0:max_length]
 
         return new_token_ids

invokeai/backend/util/silence_warnings.py

@@ -1,29 +1,36 @@
-"""Context class to silence transformers and diffusers warnings."""
-
 import warnings
-from typing import Any
+from contextlib import ContextDecorator
 
-from diffusers import logging as diffusers_logging
+from diffusers.utils import logging as diffusers_logging
 from transformers import logging as transformers_logging
 
 
-class SilenceWarnings(object):
-    """Use in context to temporarily turn off warnings from transformers & diffusers modules.
+# Inherit from ContextDecorator to allow using SilenceWarnings as both a context manager and a decorator.
+class SilenceWarnings(ContextDecorator):
+    """A context manager that disables warnings from transformers & diffusers modules while active.
 
+    As context manager:
+    ```
     with SilenceWarnings():
         # do something
+    ```
+
+    As decorator:
+    ```
+    @SilenceWarnings()
+    def some_function():
+        # do something
+    ```
     """
 
-    def __init__(self) -> None:
-        self.transformers_verbosity = transformers_logging.get_verbosity()
-        self.diffusers_verbosity = diffusers_logging.get_verbosity()
-
     def __enter__(self) -> None:
+        self._transformers_verbosity = transformers_logging.get_verbosity()
+        self._diffusers_verbosity = diffusers_logging.get_verbosity()
         transformers_logging.set_verbosity_error()
         diffusers_logging.set_verbosity_error()
         warnings.simplefilter("ignore")
 
-    def __exit__(self, *args: Any) -> None:
-        transformers_logging.set_verbosity(self.transformers_verbosity)
-        diffusers_logging.set_verbosity(self.diffusers_verbosity)
+    def __exit__(self, *args) -> None:
+        transformers_logging.set_verbosity(self._transformers_verbosity)
+        diffusers_logging.set_verbosity(self._diffusers_verbosity)
         warnings.simplefilter("default")

invokeai/backend/util/util.py

@@ -1,12 +1,9 @@
 import base64
 import io
 import os
-import warnings
 from pathlib import Path
 
-from diffusers import logging as diffusers_logging
 from PIL import Image
-from transformers import logging as transformers_logging
 
 # actual size of a gig
 GIG = 1073741824
@@ -51,21 +48,3 @@ class Chdir(object):
 
     def __exit__(self, *args):
         os.chdir(self.original)
-
-
-class SilenceWarnings(object):
-    """Context manager to temporarily lower verbosity of diffusers & transformers warning messages."""
-
-    def __enter__(self):
-        """Set verbosity to error."""
-        self.transformers_verbosity = transformers_logging.get_verbosity()
-        self.diffusers_verbosity = diffusers_logging.get_verbosity()
-        transformers_logging.set_verbosity_error()
-        diffusers_logging.set_verbosity_error()
-        warnings.simplefilter("ignore")
-
-    def __exit__(self, type, value, traceback):
-        """Restore logger verbosity to state before context was entered."""
-        transformers_logging.set_verbosity(self.transformers_verbosity)
-        diffusers_logging.set_verbosity(self.diffusers_verbosity)
-        warnings.simplefilter("default")

invokeai/configs/stable-diffusion/sd_xl_inpaint.yaml

@@ -0,0 +1,98 @@
+model:
+  target: sgm.models.diffusion.DiffusionEngine
+  params:
+    scale_factor: 0.13025
+    disable_first_stage_autocast: True
+
+    denoiser_config:
+      target: sgm.modules.diffusionmodules.denoiser.DiscreteDenoiser
+      params:
+        num_idx: 1000
+
+        weighting_config:
+          target: sgm.modules.diffusionmodules.denoiser_weighting.EpsWeighting
+        scaling_config:
+          target: sgm.modules.diffusionmodules.denoiser_scaling.EpsScaling
+        discretization_config:
+          target: sgm.modules.diffusionmodules.discretizer.LegacyDDPMDiscretization
+
+    network_config:
+      target: sgm.modules.diffusionmodules.openaimodel.UNetModel
+      params:
+        adm_in_channels: 2816
+        num_classes: sequential
+        use_checkpoint: True
+        in_channels: 9
+        out_channels: 4
+        model_channels: 320
+        attention_resolutions: [4, 2]
+        num_res_blocks: 2
+        channel_mult: [1, 2, 4]
+        num_head_channels: 64
+        use_spatial_transformer: True
+        use_linear_in_transformer: True
+        transformer_depth: [1, 2, 10]  # note: the first is unused (due to attn_res starting at 2) 32, 16, 8 --> 64, 32, 16
+        context_dim: 2048
+        spatial_transformer_attn_type: softmax-xformers
+        legacy: False
+
+    conditioner_config:
+      target: sgm.modules.GeneralConditioner
+      params:
+        emb_models:
+          # crossattn cond
+          - is_trainable: False
+            input_key: txt
+            target: sgm.modules.encoders.modules.FrozenCLIPEmbedder
+            params:
+              layer: hidden
+              layer_idx: 11
+          # crossattn and vector cond
+          - is_trainable: False
+            input_key: txt
+            target: sgm.modules.encoders.modules.FrozenOpenCLIPEmbedder2
+            params:
+              arch: ViT-bigG-14
+              version: laion2b_s39b_b160k
+              freeze: True
+              layer: penultimate
+              always_return_pooled: True
+              legacy: False
+          # vector cond
+          - is_trainable: False
+            input_key: original_size_as_tuple
+            target: sgm.modules.encoders.modules.ConcatTimestepEmbedderND
+            params:
+              outdim: 256  # multiplied by two
+          # vector cond
+          - is_trainable: False
+            input_key: crop_coords_top_left
+            target: sgm.modules.encoders.modules.ConcatTimestepEmbedderND
+            params:
+              outdim: 256  # multiplied by two
+          # vector cond
+          - is_trainable: False
+            input_key: target_size_as_tuple
+            target: sgm.modules.encoders.modules.ConcatTimestepEmbedderND
+            params:
+              outdim: 256  # multiplied by two
+
+    first_stage_config:
+      target: sgm.models.autoencoder.AutoencoderKLInferenceWrapper
+      params:
+        embed_dim: 4
+        monitor: val/rec_loss
+        ddconfig:
+          attn_type: vanilla-xformers
+          double_z: true
+          z_channels: 4
+          resolution: 256
+          in_channels: 3
+          out_ch: 3
+          ch: 128
+          ch_mult: [1, 2, 4, 4]
+          num_res_blocks: 2
+          attn_resolutions: []
+          dropout: 0.0
+        lossconfig:
+          target: torch.nn.Identity

invokeai/frontend/web/.eslintrc.js

@@ -10,6 +10,8 @@ module.exports = {
     'path/no-relative-imports': ['error', { maxDepth: 0 }],
     // https://github.com/edvardchen/eslint-plugin-i18next/blob/HEAD/docs/rules/no-literal-string.md
     'i18next/no-literal-string': 'error',
+    // https://eslint.org/docs/latest/rules/no-console
+    'no-console': 'error',
   },
   overrides: [
     /**

invokeai/frontend/web/.gitignore

@@ -43,4 +43,5 @@ stats.html
 yalc.lock
 
 # vitest
-tsconfig.vitest-temp.json
+tsconfig.vitest-temp.json
+coverage/

invokeai/frontend/web/package.json

@@ -25,7 +25,7 @@
     "typegen": "node scripts/typegen.js",
     "preview": "vite preview",
     "lint:knip": "knip",
-    "lint:dpdm": "dpdm --no-warning --no-tree --transform --exit-code circular:0 src/main.tsx",
+    "lint:dpdm": "dpdm --no-warning --no-tree --transform --exit-code circular:1 src/main.tsx",
     "lint:eslint": "eslint --max-warnings=0 .",
     "lint:prettier": "prettier --check .",
     "lint:tsc": "tsc --noEmit",
@@ -35,6 +35,7 @@
     "storybook": "storybook dev -p 6006",
     "build-storybook": "storybook build",
     "test": "vitest",
+    "test:ui": "vitest --coverage --ui",
     "test:no-watch": "vitest --no-watch"
   },
   "madge": {
@@ -52,47 +53,48 @@
   },
   "dependencies": {
     "@chakra-ui/react-use-size": "^2.1.0",
-    "@dagrejs/dagre": "^1.1.1",
-    "@dagrejs/graphlib": "^2.2.1",
+    "@dagrejs/dagre": "^1.1.2",
+    "@dagrejs/graphlib": "^2.2.2",
     "@dnd-kit/core": "^6.1.0",
     "@dnd-kit/sortable": "^8.0.0",
     "@dnd-kit/utilities": "^3.2.2",
-    "@fontsource-variable/inter": "^5.0.17",
-    "@invoke-ai/ui-library": "^0.0.21",
+    "@fontsource-variable/inter": "^5.0.18",
+    "@invoke-ai/ui-library": "^0.0.25",
     "@nanostores/react": "^0.7.2",
-    "@reduxjs/toolkit": "2.2.2",
+    "@reduxjs/toolkit": "2.2.3",
     "@roarr/browser-log-writer": "^1.3.0",
     "chakra-react-select": "^4.7.6",
     "compare-versions": "^6.1.0",
     "dateformat": "^5.0.3",
-    "framer-motion": "^11.0.22",
-    "i18next": "^23.10.1",
-    "i18next-http-backend": "^2.5.0",
+    "fracturedjsonjs": "^4.0.1",
+    "framer-motion": "^11.1.8",
+    "i18next": "^23.11.3",
+    "i18next-http-backend": "^2.5.1",
     "idb-keyval": "^6.2.1",
     "jsondiffpatch": "^0.6.0",
     "konva": "^9.3.6",
     "lodash-es": "^4.17.21",
-    "nanostores": "^0.10.0",
+    "nanostores": "^0.10.3",
     "new-github-issue-url": "^1.0.0",
-    "overlayscrollbars": "^2.6.1",
-    "overlayscrollbars-react": "^0.5.5",
+    "overlayscrollbars": "^2.7.3",
+    "overlayscrollbars-react": "^0.5.6",
     "query-string": "^9.0.0",
-    "react": "^18.2.0",
+    "react": "^18.3.1",
     "react-colorful": "^5.6.1",
-    "react-dom": "^18.2.0",
+    "react-dom": "^18.3.1",
     "react-dropzone": "^14.2.3",
     "react-error-boundary": "^4.0.13",
-    "react-hook-form": "^7.51.2",
+    "react-hook-form": "^7.51.4",
     "react-hotkeys-hook": "4.5.0",
-    "react-i18next": "^14.1.0",
-    "react-icons": "^5.0.1",
+    "react-i18next": "^14.1.1",
+    "react-icons": "^5.2.0",
     "react-konva": "^18.2.10",
-    "react-redux": "9.1.0",
-    "react-resizable-panels": "^2.0.16",
+    "react-redux": "9.1.2",
+    "react-resizable-panels": "^2.0.19",
     "react-select": "5.8.0",
     "react-use": "^17.5.0",
-    "react-virtuoso": "^4.7.5",
-    "reactflow": "^11.10.4",
+    "react-virtuoso": "^4.7.10",
+    "reactflow": "^11.11.3",
     "redux-dynamic-middlewares": "^2.2.0",
     "redux-remember": "^5.1.0",
     "redux-undo": "^1.1.0",
@@ -101,10 +103,11 @@
     "serialize-error": "^11.0.3",
     "socket.io-client": "^4.7.5",
     "use-debounce": "^10.0.0",
+    "use-device-pixel-ratio": "^1.1.2",
     "use-image": "^1.1.1",
     "uuid": "^9.0.1",
-    "zod": "^3.22.4",
-    "zod-validation-error": "^3.0.3"
+    "zod": "^3.23.6",
+    "zod-validation-error": "^3.2.0"
   },
   "peerDependencies": {
     "@chakra-ui/react": "^2.8.2",
@@ -115,40 +118,42 @@
   "devDependencies": {
     "@invoke-ai/eslint-config-react": "^0.0.14",
     "@invoke-ai/prettier-config-react": "^0.0.7",
-    "@storybook/addon-essentials": "^8.0.4",
-    "@storybook/addon-interactions": "^8.0.4",
-    "@storybook/addon-links": "^8.0.4",
-    "@storybook/addon-storysource": "^8.0.4",
-    "@storybook/manager-api": "^8.0.4",
-    "@storybook/react": "^8.0.4",
-    "@storybook/react-vite": "^8.0.4",
-    "@storybook/theming": "^8.0.4",
+    "@storybook/addon-essentials": "^8.0.10",
+    "@storybook/addon-interactions": "^8.0.10",
+    "@storybook/addon-links": "^8.0.10",
+    "@storybook/addon-storysource": "^8.0.10",
+    "@storybook/manager-api": "^8.0.10",
+    "@storybook/react": "^8.0.10",
+    "@storybook/react-vite": "^8.0.10",
+    "@storybook/theming": "^8.0.10",
     "@types/dateformat": "^5.0.2",
     "@types/lodash-es": "^4.17.12",
-    "@types/node": "^20.11.30",
-    "@types/react": "^18.2.73",
-    "@types/react-dom": "^18.2.22",
+    "@types/node": "^20.12.10",
+    "@types/react": "^18.3.1",
+    "@types/react-dom": "^18.3.0",
     "@types/uuid": "^9.0.8",
     "@vitejs/plugin-react-swc": "^3.6.0",
+    "@vitest/coverage-v8": "^1.5.0",
+    "@vitest/ui": "^1.5.0",
     "concurrently": "^8.2.2",
     "dpdm": "^3.14.0",
     "eslint": "^8.57.0",
     "eslint-plugin-i18next": "^6.0.3",
     "eslint-plugin-path": "^1.3.0",
-    "knip": "^5.6.1",
+    "knip": "^5.12.3",
     "openapi-types": "^12.1.3",
     "openapi-typescript": "^6.7.5",
     "prettier": "^3.2.5",
     "rollup-plugin-visualizer": "^5.12.0",
-    "storybook": "^8.0.4",
+    "storybook": "^8.0.10",
     "ts-toolbelt": "^9.6.0",
     "tsafe": "^1.6.6",
-    "typescript": "^5.4.3",
-    "vite": "^5.2.6",
-    "vite-plugin-css-injected-by-js": "^3.5.0",
-    "vite-plugin-dts": "^3.8.0",
+    "typescript": "^5.4.5",
+    "vite": "^5.2.11",
+    "vite-plugin-css-injected-by-js": "^3.5.1",
+    "vite-plugin-dts": "^3.9.1",
     "vite-plugin-eslint": "^1.8.1",
     "vite-tsconfig-paths": "^4.3.2",
-    "vitest": "^1.4.0"
+    "vitest": "^1.6.0"
   }
 }

invokeai/frontend/web/public/locales/de.json

@@ -76,7 +76,9 @@
         "aboutHeading": "Nutzen Sie Ihre kreative Energie",
         "toResolve": "Lösen",
         "add": "Hinzufügen",
-        "loglevel": "Protokoll Stufe"
+        "loglevel": "Protokoll Stufe",
+        "selected": "Ausgewählt",
+        "beta": "Beta"
     },
     "gallery": {
         "galleryImageSize": "Bildgröße",
@@ -86,7 +88,7 @@
         "noImagesInGallery": "Keine Bilder in der Galerie",
         "loading": "Lade",
         "deleteImage_one": "Lösche Bild",
-        "deleteImage_other": "",
+        "deleteImage_other": "Lösche {{count}} Bilder",
         "copy": "Kopieren",
         "download": "Runterladen",
         "setCurrentImage": "Setze aktuelle Bild",
@@ -397,7 +399,14 @@
         "cancel": "Stornieren",
         "defaultSettingsSaved": "Standardeinstellungen gespeichert",
         "addModels": "Model hinzufügen",
-        "deleteModelImage": "Lösche Model Bild"
+        "deleteModelImage": "Lösche Model Bild",
+        "hfTokenInvalidErrorMessage": "Falscher oder fehlender HuggingFace Schlüssel.",
+        "huggingFaceRepoID": "HuggingFace Repo ID",
+        "hfToken": "HuggingFace Schlüssel",
+        "hfTokenInvalid": "Falscher oder fehlender HF Schlüssel",
+        "huggingFacePlaceholder": "besitzer/model-name",
+        "hfTokenSaved": "HF Schlüssel gespeichert",
+        "hfTokenUnableToVerify": "Konnte den HF Schlüssel nicht validieren"
     },
     "parameters": {
         "images": "Bilder",
@@ -686,7 +695,11 @@
         "hands": "Hände",
         "dwOpenpose": "DW Openpose",
         "dwOpenposeDescription": "Posenschätzung mit DW Openpose",
-        "selectCLIPVisionModel": "Wähle ein CLIP Vision Model aus"
+        "selectCLIPVisionModel": "Wähle ein CLIP Vision Model aus",
+        "ipAdapterMethod": "Methode",
+        "composition": "Nur Komposition",
+        "full": "Voll",
+        "style": "Nur Style"
     },
     "queue": {
         "status": "Status",
@@ -717,7 +730,6 @@
         "resume": "Wieder aufnehmen",
         "item": "Auftrag",
         "notReady": "Warteschlange noch nicht bereit",
-        "queueCountPrediction": "{{promptsCount}} Prompts × {{iterations}} Iterationen -> {{count}} Generationen",
         "clearQueueAlertDialog": "\"Die Warteschlange leeren\" stoppt den aktuellen Prozess und leert die Warteschlange komplett.",
         "completedIn": "Fertig in",
         "cancelBatchSucceeded": "Stapel abgebrochen",
@@ -1009,7 +1021,8 @@
         "float": "Kommazahlen",
         "enum": "Aufzählung",
         "fullyContainNodes": "Vollständig ausgewählte Nodes auswählen",
-        "editMode": "Im Workflow-Editor bearbeiten"
+        "editMode": "Im Workflow-Editor bearbeiten",
+        "resetToDefaultValue": "Auf Standardwert zurücksetzen"
     },
     "hrf": {
         "enableHrf": "Korrektur für hohe Auflösungen",

invokeai/frontend/web/public/locales/en.json

@@ -2,6 +2,7 @@
     "accessibility": {
         "about": "About",
         "createIssue": "Create Issue",
+        "submitSupportTicket": "Submit Support Ticket",
         "invokeProgressBar": "Invoke progress bar",
         "menu": "Menu",
         "mode": "Mode",
@@ -88,11 +89,13 @@
         "negativePrompt": "Negative Prompt",
         "discordLabel": "Discord",
         "dontAskMeAgain": "Don't ask me again",
+        "editor": "Editor",
         "error": "Error",
         "file": "File",
         "folder": "Folder",
         "format": "format",
         "githubLabel": "Github",
+        "goTo": "Go to",
         "hotkeysLabel": "Hotkeys",
         "imageFailedToLoad": "Unable to Load Image",
         "img2img": "Image To Image",
@@ -140,7 +143,15 @@
         "blue": "Blue",
         "alpha": "Alpha",
         "selected": "Selected",
-        "viewer": "Viewer"
+        "tab": "Tab",
+        "viewing": "Viewing",
+        "viewingDesc": "Review images in a large gallery view",
+        "editing": "Editing",
+        "editingDesc": "Edit on the Control Layers canvas",
+        "comparing": "Comparing",
+        "comparingDesc": "Comparing two images",
+        "enabled": "Enabled",
+        "disabled": "Disabled"
     },
     "controlnet": {
         "controlAdapter_one": "Control Adapter",
@@ -156,6 +167,7 @@
         "balanced": "Balanced",
         "base": "Base",
         "beginEndStepPercent": "Begin / End Step Percentage",
+        "beginEndStepPercentShort": "Begin/End %",
         "bgth": "bg_th",
         "canny": "Canny",
         "cannyDescription": "Canny edge detection",
@@ -224,10 +236,11 @@
         "composition": "Composition Only",
         "safe": "Safe",
         "saveControlImage": "Save Control Image",
-        "scribble": "scribble",
+        "scribble": "Scribble",
         "selectModel": "Select a model",
         "selectCLIPVisionModel": "Select a CLIP Vision model",
-        "setControlImageDimensions": "Set Control Image Dimensions To W/H",
+        "setControlImageDimensions": "Copy size to W/H (optimize for model)",
+        "setControlImageDimensionsForce": "Copy size to W/H (ignore model)",
         "showAdvanced": "Show Advanced",
         "small": "Small",
         "toggleControlNet": "Toggle this ControlNet",
@@ -253,7 +266,6 @@
         "queue": "Queue",
         "queueFront": "Add to Front of Queue",
         "queueBack": "Add to Queue",
-        "queueCountPrediction": "{{promptsCount}} prompts \u00d7 {{iterations}} iterations -> {{count}} generations",
         "queueEmpty": "Queue Empty",
         "enqueueing": "Queueing Batch",
         "resume": "Resume",
@@ -306,7 +318,13 @@
         "batchFailedToQueue": "Failed to Queue Batch",
         "graphQueued": "Graph queued",
         "graphFailedToQueue": "Failed to queue graph",
-        "openQueue": "Open Queue"
+        "openQueue": "Open Queue",
+        "prompts_one": "Prompt",
+        "prompts_other": "Prompts",
+        "iterations_one": "Iteration",
+        "iterations_other": "Iterations",
+        "generations_one": "Generation",
+        "generations_other": "Generations"
     },
     "invocationCache": {
         "invocationCache": "Invocation Cache",
@@ -359,7 +377,23 @@
         "bulkDownloadRequestFailed": "Problem Preparing Download",
         "bulkDownloadFailed": "Download Failed",
         "problemDeletingImages": "Problem Deleting Images",
-        "problemDeletingImagesDesc": "One or more images could not be deleted"
+        "problemDeletingImagesDesc": "One or more images could not be deleted",
+        "viewerImage": "Viewer Image",
+        "compareImage": "Compare Image",
+        "openInViewer": "Open in Viewer",
+        "selectForCompare": "Select for Compare",
+        "selectAnImageToCompare": "Select an Image to Compare",
+        "slider": "Slider",
+        "sideBySide": "Side-by-Side",
+        "hover": "Hover",
+        "swapImages": "Swap Images",
+        "compareOptions": "Comparison Options",
+        "stretchToFit": "Stretch to Fit",
+        "exitCompare": "Exit Compare",
+        "compareHelp1": "Hold <Kbd>Alt</Kbd> while clicking a gallery image or using the arrow keys to change the compare image.",
+        "compareHelp2": "Press <Kbd>M</Kbd> to cycle through comparison modes.",
+        "compareHelp3": "Press <Kbd>C</Kbd> to swap the compared images.",
+        "compareHelp4": "Press <Kbd>Z</Kbd> or <Kbd>Esc</Kbd> to exit."
     },
     "hotkeys": {
         "searchHotkeys": "Search Hotkeys",
@@ -582,6 +616,10 @@
         "upscale": {
             "desc": "Upscale the current image",
             "title": "Upscale"
+        },
+        "toggleViewer": {
+            "desc": "Switches between the Image Viewer and workspace for the current tab.",
+            "title": "Toggle Image Viewer"
         }
     },
     "metadata": {
@@ -757,10 +795,15 @@
         "cannotConnectOutputToOutput": "Cannot connect output to output",
         "cannotConnectToSelf": "Cannot connect to self",
         "cannotDuplicateConnection": "Cannot create duplicate connections",
+        "cannotMixAndMatchCollectionItemTypes": "Cannot mix and match collection item types",
+        "missingNode": "Missing invocation node",
+        "missingInvocationTemplate": "Missing invocation template",
+        "missingFieldTemplate": "Missing field template",
         "nodePack": "Node pack",
         "collection": "Collection",
-        "collectionFieldType": "{{name}} Collection",
-        "collectionOrScalarFieldType": "{{name}} Collection|Scalar",
+        "singleFieldType": "{{name}} (Single)",
+        "collectionFieldType": "{{name}} (Collection)",
+        "collectionOrScalarFieldType": "{{name}} (Single or Collection)",
         "colorCodeEdges": "Color-Code Edges",
         "colorCodeEdgesHelp": "Color-code edges according to their connected fields",
         "connectionWouldCreateCycle": "Connection would create a cycle",
@@ -862,6 +905,7 @@
         "versionUnknown": " Version Unknown",
         "workflow": "Workflow",
         "graph": "Graph",
+        "noGraph": "No Graph",
         "workflowAuthor": "Author",
         "workflowContact": "Contact",
         "workflowDescription": "Short Description",
@@ -874,7 +918,10 @@
         "zoomInNodes": "Zoom In",
         "zoomOutNodes": "Zoom Out",
         "betaDesc": "This invocation is in beta. Until it is stable, it may have breaking changes during app updates. We plan to support this invocation long-term.",
-        "prototypeDesc": "This invocation is a prototype. It may have breaking changes during app updates and may be removed at any time."
+        "prototypeDesc": "This invocation is a prototype. It may have breaking changes during app updates and may be removed at any time.",
+        "imageAccessError": "Unable to find image {{image_name}}, resetting to default",
+        "boardAccessError": "Unable to find board {{board_id}}, resetting to default",
+        "modelAccessError": "Unable to find model {{key}}, resetting to default"
     },
     "parameters": {
         "aspect": "Aspect",
@@ -915,23 +962,37 @@
             "missingInputForField": "{{nodeLabel}} -> {{fieldLabel}} missing input",
             "missingNodeTemplate": "Missing node template",
             "noControlImageForControlAdapter": "Control Adapter #{{number}} has no control image",
+            "imageNotProcessedForControlAdapter": "Control Adapter #{{number}}'s image is not processed",
             "noInitialImageSelected": "No initial image selected",
             "noModelForControlAdapter": "Control Adapter #{{number}} has no model selected.",
             "incompatibleBaseModelForControlAdapter": "Control Adapter #{{number}} model is incompatible with main model.",
             "noModelSelected": "No model selected",
             "noPrompts": "No prompts generated",
             "noNodesInGraph": "No nodes in graph",
-            "systemDisconnected": "System disconnected"
+            "systemDisconnected": "System disconnected",
+            "layer": {
+                "initialImageNoImageSelected": "no initial image selected",
+                "controlAdapterNoModelSelected": "no Control Adapter model selected",
+                "controlAdapterIncompatibleBaseModel": "incompatible Control Adapter base model",
+                "controlAdapterNoImageSelected": "no Control Adapter image selected",
+                "controlAdapterImageNotProcessed": "Control Adapter image not processed",
+                "t2iAdapterIncompatibleDimensions": "T2I Adapter requires image dimension to be multiples of {{multiple}}",
+                "ipAdapterNoModelSelected": "no IP adapter selected",
+                "ipAdapterIncompatibleBaseModel": "incompatible IP Adapter base model",
+                "ipAdapterNoImageSelected": "no IP Adapter image selected",
+                "rgNoPromptsOrIPAdapters": "no text prompts or IP Adapters",
+                "rgNoRegion": "no region selected"
+            }
         },
         "maskBlur": "Mask Blur",
         "negativePromptPlaceholder": "Negative Prompt",
+        "globalNegativePromptPlaceholder": "Global Negative Prompt",
         "noiseThreshold": "Noise Threshold",
         "patchmatchDownScaleSize": "Downscale",
         "perlinNoise": "Perlin Noise",
         "positivePromptPlaceholder": "Positive Prompt",
+        "globalPositivePromptPlaceholder": "Global Positive Prompt",
         "iterations": "Iterations",
-        "iterationsWithCount_one": "{{count}} Iteration",
-        "iterationsWithCount_other": "{{count}} Iterations",
         "scale": "Scale",
         "scaleBeforeProcessing": "Scale Before Processing",
         "scaledHeight": "Scaled H",
@@ -1033,8 +1094,9 @@
     },
     "toast": {
         "addedToBoard": "Added to board",
-        "baseModelChangedCleared_one": "Base model changed, cleared or disabled {{count}} incompatible submodel",
-        "baseModelChangedCleared_other": "Base model changed, cleared or disabled {{count}} incompatible submodels",
+        "baseModelChanged": "Base Model Changed",
+        "baseModelChangedCleared_one": "Cleared or disabled {{count}} incompatible submodel",
+        "baseModelChangedCleared_other": "Cleared or disabled {{count}} incompatible submodels",
         "canceled": "Processing Canceled",
         "canvasCopiedClipboard": "Canvas Copied to Clipboard",
         "canvasDownloaded": "Canvas Downloaded",
@@ -1055,10 +1117,17 @@
         "metadataLoadFailed": "Failed to load metadata",
         "modelAddedSimple": "Model Added to Queue",
         "modelImportCanceled": "Model Import Canceled",
+        "outOfMemoryError": "Out of Memory Error",
+        "outOfMemoryErrorDesc": "Your current generation settings exceed system capacity. Please adjust your settings and try again.",
         "parameters": "Parameters",
-        "parameterNotSet": "{{parameter}} not set",
-        "parameterSet": "{{parameter}} set",
-        "parametersNotSet": "Parameters Not Set",
+        "parameterSet": "Parameter Recalled",
+        "parameterSetDesc": "Recalled {{parameter}}",
+        "parameterNotSet": "Parameter Not Recalled",
+        "parameterNotSetDesc": "Unable to recall {{parameter}}",
+        "parameterNotSetDescWithMessage": "Unable to recall {{parameter}}: {{message}}",
+        "parametersSet": "Parameters Recalled",
+        "parametersNotSet": "Parameters Not Recalled",
+        "errorCopied": "Error Copied",
         "problemCopyingCanvas": "Problem Copying Canvas",
         "problemCopyingCanvasDesc": "Unable to export base layer",
         "problemCopyingImage": "Unable to Copy Image",
@@ -1078,11 +1147,13 @@
         "sentToImageToImage": "Sent To Image To Image",
         "sentToUnifiedCanvas": "Sent to Unified Canvas",
         "serverError": "Server Error",
+        "sessionRef": "Session: {{sessionId}}",
         "setAsCanvasInitialImage": "Set as canvas initial image",
         "setCanvasInitialImage": "Set canvas initial image",
         "setControlImage": "Set as control image",
         "setInitialImage": "Set as initial image",
         "setNodeField": "Set as node field",
+        "somethingWentWrong": "Something Went Wrong",
         "uploadFailed": "Upload failed",
         "uploadFailedInvalidUploadDesc": "Must be single PNG or JPEG image",
         "uploadInitialImage": "Upload Initial Image",
@@ -1511,7 +1582,7 @@
     "app": {
         "storeNotInitialized": "Store is not initialized"
     },
-    "regionalPrompts": {
+    "controlLayers": {
         "deleteAll": "Delete All",
         "addLayer": "Add Layer",
         "moveToFront": "Move to Front",
@@ -1519,11 +1590,9 @@
         "moveForward": "Move Forward",
         "moveBackward": "Move Backward",
         "brushSize": "Brush Size",
-        "regionalControl": "Regional Control (ALPHA)",
-        "enableRegionalPrompts": "Enable $t(regionalPrompts.regionalPrompts)",
+        "controlLayers": "Control Layers",
         "globalMaskOpacity": "Global Mask Opacity",
         "autoNegative": "Auto Negative",
-        "toggleVisibility": "Toggle Layer Visibility",
         "deletePrompt": "Delete Prompt",
         "resetRegion": "Reset Region",
         "debugLayers": "Debug Layers",
@@ -1531,6 +1600,35 @@
         "maskPreviewColor": "Mask Preview Color",
         "addPositivePrompt": "Add $t(common.positivePrompt)",
         "addNegativePrompt": "Add $t(common.negativePrompt)",
-        "addIPAdapter": "Add $t(common.ipAdapter)"
+        "addIPAdapter": "Add $t(common.ipAdapter)",
+        "regionalGuidance": "Regional Guidance",
+        "regionalGuidanceLayer": "$t(controlLayers.regionalGuidance) $t(unifiedCanvas.layer)",
+        "opacity": "Opacity",
+        "globalControlAdapter": "Global $t(controlnet.controlAdapter_one)",
+        "globalControlAdapterLayer": "Global $t(controlnet.controlAdapter_one) $t(unifiedCanvas.layer)",
+        "globalIPAdapter": "Global $t(common.ipAdapter)",
+        "globalIPAdapterLayer": "Global $t(common.ipAdapter) $t(unifiedCanvas.layer)",
+        "globalInitialImage": "Global Initial Image",
+        "globalInitialImageLayer": "$t(controlLayers.globalInitialImage) $t(unifiedCanvas.layer)",
+        "opacityFilter": "Opacity Filter",
+        "clearProcessor": "Clear Processor",
+        "resetProcessor": "Reset Processor to Defaults",
+        "noLayersAdded": "No Layers Added",
+        "layers_one": "Layer",
+        "layers_other": "Layers"
+    },
+    "ui": {
+        "tabs": {
+            "generation": "Generation",
+            "generationTab": "$t(ui.tabs.generation) $t(common.tab)",
+            "canvas": "Canvas",
+            "canvasTab": "$t(ui.tabs.canvas) $t(common.tab)",
+            "workflows": "Workflows",
+            "workflowsTab": "$t(ui.tabs.workflows) $t(common.tab)",
+            "models": "Models",
+            "modelsTab": "$t(ui.tabs.models) $t(common.tab)",
+            "queue": "Queue",
+            "queueTab": "$t(ui.tabs.queue) $t(common.tab)"
+        }
     }
 }

invokeai/frontend/web/public/locales/es.json

@@ -6,7 +6,7 @@
         "settingsLabel": "Ajustes",
         "img2img": "Imagen a Imagen",
         "unifiedCanvas": "Lienzo Unificado",
-        "nodes": "Editor del flujo de trabajo",
+        "nodes": "Flujos de trabajo",
         "upload": "Subir imagen",
         "load": "Cargar",
         "statusDisconnected": "Desconectado",
@@ -14,7 +14,7 @@
         "discordLabel": "Discord",
         "back": "Atrás",
         "loading": "Cargando",
-        "postprocessing": "Tratamiento posterior",
+        "postprocessing": "Postprocesado",
         "txt2img": "De texto a imagen",
         "accept": "Aceptar",
         "cancel": "Cancelar",
@@ -25,7 +25,59 @@
         "areYouSure": "¿Estas seguro?",
         "batch": "Administrador de lotes",
         "modelManager": "Administrador de modelos",
-        "communityLabel": "Comunidad"
+        "communityLabel": "Comunidad",
+        "direction": "Dirección",
+        "ai": "Ia",
+        "add": "Añadir",
+        "auto": "Automático",
+        "copyError": "Error $t(gallery.copy)",
+        "details": "Detalles",
+        "or": "o",
+        "checkpoint": "Punto de control",
+        "controlNet": "ControlNet",
+        "aboutHeading": "Sea dueño de su poder creativo",
+        "advanced": "Avanzado",
+        "data": "Fecha",
+        "delete": "Borrar",
+        "copy": "Copiar",
+        "beta": "Beta",
+        "on": "En",
+        "aboutDesc": "¿Utilizas Invoke para trabajar? Mira aquí:",
+        "installed": "Instalado",
+        "green": "Verde",
+        "editor": "Editor",
+        "orderBy": "Ordenar por",
+        "file": "Archivo",
+        "goTo": "Ir a",
+        "imageFailedToLoad": "No se puede cargar la imagen",
+        "saveAs": "Guardar Como",
+        "somethingWentWrong": "Algo salió mal",
+        "nextPage": "Página Siguiente",
+        "selected": "Seleccionado",
+        "tab": "Tabulador",
+        "positivePrompt": "Prompt Positivo",
+        "negativePrompt": "Prompt Negativo",
+        "error": "Error",
+        "format": "formato",
+        "unknown": "Desconocido",
+        "input": "Entrada",
+        "nodeEditor": "Editor de nodos",
+        "template": "Plantilla",
+        "prevPage": "Página Anterior",
+        "red": "Rojo",
+        "alpha": "Transparencia",
+        "outputs": "Salidas",
+        "editing": "Editando",
+        "learnMore": "Aprende más",
+        "enabled": "Activado",
+        "disabled": "Desactivado",
+        "folder": "Carpeta",
+        "updated": "Actualizado",
+        "created": "Creado",
+        "save": "Guardar",
+        "unknownError": "Error Desconocido",
+        "blue": "Azul",
+        "viewingDesc": "Revisar imágenes en una vista de galería grande"
     },
     "gallery": {
         "galleryImageSize": "Tamaño de la imagen",
@@ -365,7 +417,7 @@
         "canvasMerged": "Lienzo consolidado",
         "sentToImageToImage": "Enviar hacia Imagen a Imagen",
         "sentToUnifiedCanvas": "Enviar hacia Lienzo Consolidado",
-        "parametersNotSet": "Parámetros no establecidos",
+        "parametersNotSet": "Parámetros no recuperados",
         "metadataLoadFailed": "Error al cargar metadatos",
         "serverError": "Error en el servidor",
         "canceled": "Procesando la cancelación",
@@ -373,7 +425,8 @@
         "uploadFailedInvalidUploadDesc": "Debe ser una sola imagen PNG o JPEG",
         "parameterSet": "Conjunto de parámetros",
         "parameterNotSet": "Parámetro no configurado",
-        "problemCopyingImage": "No se puede copiar la imagen"
+        "problemCopyingImage": "No se puede copiar la imagen",
+        "errorCopied": "Error al copiar"
     },
     "tooltip": {
         "feature": {
@@ -443,7 +496,14 @@
         "previousImage": "Imagen anterior",
         "nextImage": "Siguiente imagen",
         "showOptionsPanel": "Mostrar el panel lateral",
-        "menu": "Menú"
+        "menu": "Menú",
+        "showGalleryPanel": "Mostrar panel de galería",
+        "loadMore": "Cargar más",
+        "about": "Acerca de",
+        "createIssue": "Crear un problema",
+        "resetUI": "Interfaz de usuario $t(accessibility.reset)",
+        "mode": "Modo",
+        "submitSupportTicket": "Enviar Ticket de Soporte"
     },
     "nodes": {
         "zoomInNodes": "Acercar",
@@ -456,5 +516,80 @@
         "reloadNodeTemplates": "Recargar las plantillas de nodos",
         "loadWorkflow": "Cargar el flujo de trabajo",
         "downloadWorkflow": "Descargar el flujo de trabajo en un archivo JSON"
+    },
+    "boards": {
+        "autoAddBoard": "Agregar panel automáticamente",
+        "changeBoard": "Cambiar el panel",
+        "clearSearch": "Borrar la búsqueda",
+        "deleteBoard": "Borrar el panel",
+        "selectBoard": "Seleccionar un panel",
+        "uncategorized": "Sin categoría",
+        "cancel": "Cancelar",
+        "addBoard": "Agregar un panel",
+        "movingImagesToBoard_one": "Moviendo {{count}} imagen al panel:",
+        "movingImagesToBoard_many": "Moviendo {{count}} imágenes al panel:",
+        "movingImagesToBoard_other": "Moviendo {{count}} imágenes al panel:",
+        "bottomMessage": "Al eliminar este panel y las imágenes que contiene, se restablecerán las funciones que los estén utilizando actualmente.",
+        "deleteBoardAndImages": "Borrar el panel y las imágenes",
+        "loading": "Cargando...",
+        "deletedBoardsCannotbeRestored": "Los paneles eliminados no se pueden restaurar",
+        "move": "Mover",
+        "menuItemAutoAdd": "Agregar automáticamente a este panel",
+        "searchBoard": "Buscando paneles…",
+        "topMessage": "Este panel contiene imágenes utilizadas en las siguientes funciones:",
+        "downloadBoard": "Descargar panel",
+        "deleteBoardOnly": "Borrar solo el panel",
+        "myBoard": "Mi panel",
+        "noMatching": "No hay paneles que coincidan"
+    },
+    "accordions": {
+        "compositing": {
+            "title": "Composición",
+            "infillTab": "Relleno"
+        },
+        "generation": {
+            "title": "Generación"
+        },
+        "image": {
+            "title": "Imagen"
+        },
+        "control": {
+            "title": "Control"
+        },
+        "advanced": {
+            "options": "$t(accordions.advanced.title) opciones",
+            "title": "Avanzado"
+        }
+    },
+    "ui": {
+        "tabs": {
+            "generationTab": "$t(ui.tabs.generation) $t(common.tab)",
+            "canvas": "Lienzo",
+            "generation": "Generación",
+            "queue": "Cola",
+            "queueTab": "$t(ui.tabs.queue) $t(common.tab)",
+            "workflows": "Flujos de trabajo",
+            "models": "Modelos",
+            "modelsTab": "$t(ui.tabs.models) $t(common.tab)",
+            "canvasTab": "$t(ui.tabs.canvas) $t(common.tab)",
+            "workflowsTab": "$t(ui.tabs.workflows) $t(common.tab)"
+        }
+    },
+    "controlLayers": {
+        "layers_one": "Capa",
+        "layers_many": "Capas",
+        "layers_other": "Capas"
+    },
+    "controlnet": {
+        "crop": "Cortar",
+        "delete": "Eliminar",
+        "depthAnythingDescription": "Generación de mapa de profundidad usando la técnica de Depth Anything",
+        "duplicate": "Duplicar",
+        "colorMapDescription": "Genera un mapa de color desde la imagen",
+        "depthMidasDescription": "Crea un mapa de profundidad con Midas",
+        "balanced": "Equilibrado",
+        "beginEndStepPercent": "Inicio / Final Porcentaje de pasos",
+        "detectResolution": "Detectar resolución",
+        "beginEndStepPercentShort": "Inicio / Final %"
     }
 }

invokeai/frontend/web/public/locales/it.json

@@ -5,7 +5,7 @@
         "reportBugLabel": "Segnala un errore",
         "settingsLabel": "Impostazioni",
         "img2img": "Immagine a Immagine",
-        "unifiedCanvas": "Tela unificata",
+        "unifiedCanvas": "Tela",
         "nodes": "Flussi di lavoro",
         "upload": "Caricamento",
         "load": "Carica",
@@ -45,7 +45,7 @@
         "outputs": "Risultati",
         "data": "Dati",
         "somethingWentWrong": "Qualcosa è andato storto",
-        "copyError": "$t(gallery.copy) Errore",
+        "copyError": "Errore $t(gallery.copy)",
         "input": "Ingresso",
         "notInstalled": "Non $t(common.installed)",
         "unknownError": "Errore sconosciuto",
@@ -74,7 +74,22 @@
         "file": "File",
         "toResolve": "Da risolvere",
         "add": "Aggiungi",
-        "loglevel": "Livello di log"
+        "loglevel": "Livello di log",
+        "beta": "Beta",
+        "positivePrompt": "Prompt positivo",
+        "negativePrompt": "Prompt negativo",
+        "selected": "Selezionato",
+        "goTo": "Vai a",
+        "editor": "Editor",
+        "tab": "Scheda",
+        "viewing": "Visualizza",
+        "viewingDesc": "Rivedi le immagini in un'ampia vista della galleria",
+        "editing": "Modifica",
+        "editingDesc": "Modifica nell'area Livelli di controllo",
+        "enabled": "Abilitato",
+        "disabled": "Disabilitato",
+        "comparingDesc": "Confronta due immagini",
+        "comparing": "Confronta"
     },
     "gallery": {
         "galleryImageSize": "Dimensione dell'immagine",
@@ -111,14 +126,30 @@
         "bulkDownloadRequestedDesc": "La tua richiesta di download è in preparazione. L'operazione potrebbe richiedere alcuni istanti.",
         "bulkDownloadRequestFailed": "Problema durante la preparazione del download",
         "bulkDownloadFailed": "Scaricamento fallito",
-        "alwaysShowImageSizeBadge": "Mostra sempre le dimensioni dell'immagine"
+        "alwaysShowImageSizeBadge": "Mostra sempre le dimensioni dell'immagine",
+        "openInViewer": "Apri nel visualizzatore",
+        "selectForCompare": "Seleziona per il confronto",
+        "selectAnImageToCompare": "Seleziona un'immagine da confrontare",
+        "slider": "Cursore",
+        "sideBySide": "Fianco a Fianco",
+        "compareImage": "Immagine di confronto",
+        "viewerImage": "Immagine visualizzata",
+        "hover": "Al passaggio del mouse",
+        "swapImages": "Scambia le immagini",
+        "compareOptions": "Opzioni di confronto",
+        "stretchToFit": "Scala per adattare",
+        "exitCompare": "Esci dal confronto",
+        "compareHelp1": "Tieni premuto <Kbd>Alt</Kbd> mentre fai clic su un'immagine della galleria o usi i tasti freccia per cambiare l'immagine di confronto.",
+        "compareHelp2": "Premi <Kbd>M</Kbd> per scorrere le modalità di confronto.",
+        "compareHelp3": "Premi <Kbd>C</Kbd> per scambiare le immagini confrontate.",
+        "compareHelp4": "Premi <Kbd>Z</Kbd> o <Kbd>Esc</Kbd> per uscire."
     },
     "hotkeys": {
         "keyboardShortcuts": "Tasti di scelta rapida",
         "appHotkeys": "Applicazione",
         "generalHotkeys": "Generale",
         "galleryHotkeys": "Galleria",
-        "unifiedCanvasHotkeys": "Tela Unificata",
+        "unifiedCanvasHotkeys": "Tela",
         "invoke": {
             "title": "Invoke",
             "desc": "Genera un'immagine"
@@ -136,8 +167,8 @@
             "desc": "Apre e chiude il pannello delle opzioni"
         },
         "pinOptions": {
-            "title": "Appunta le opzioni",
-            "desc": "Blocca il pannello delle opzioni"
+            "title": "Fissa le opzioni",
+            "desc": "Fissa il pannello delle opzioni"
         },
         "toggleGallery": {
             "title": "Attiva/disattiva galleria",
@@ -180,8 +211,8 @@
             "desc": "Mostra le informazioni sui metadati dell'immagine corrente"
         },
         "sendToImageToImage": {
-            "title": "Invia a Immagine a Immagine",
-            "desc": "Invia l'immagine corrente a da Immagine a Immagine"
+            "title": "Invia a Generazione da immagine",
+            "desc": "Invia l'immagine corrente a Generazione da immagine"
         },
         "deleteImage": {
             "title": "Elimina immagine",
@@ -321,19 +352,23 @@
             "title": "Annulla e cancella"
         },
         "resetOptionsAndGallery": {
-            "title": "Ripristina Opzioni e Galleria",
-            "desc": "Reimposta le opzioni e i pannelli della galleria"
+            "title": "Ripristina le opzioni e la galleria",
+            "desc": "Reimposta i pannelli delle opzioni e della galleria"
         },
         "searchHotkeys": "Cerca tasti di scelta rapida",
         "noHotkeysFound": "Nessun tasto di scelta rapida trovato",
         "toggleOptionsAndGallery": {
             "desc": "Apre e chiude le opzioni e i pannelli della galleria",
-            "title": "Attiva/disattiva le Opzioni e la Galleria"
+            "title": "Attiva/disattiva le opzioni e la galleria"
         },
         "clearSearch": "Cancella ricerca",
         "remixImage": {
             "desc": "Utilizza tutti i parametri tranne il seme dell'immagine corrente",
             "title": "Remixa l'immagine"
+        },
+        "toggleViewer": {
+            "title": "Attiva/disattiva il visualizzatore di immagini",
+            "desc": "Passa dal visualizzatore immagini all'area di lavoro per la scheda corrente."
         }
     },
     "modelManager": {
@@ -363,7 +398,7 @@
         "convertToDiffusers": "Converti in Diffusori",
         "convertToDiffusersHelpText2": "Questo processo sostituirà la voce in Gestione Modelli con la versione Diffusori dello stesso modello.",
         "convertToDiffusersHelpText4": "Questo è un processo una tantum. Potrebbero essere necessari circa 30-60 secondi a seconda delle specifiche del tuo computer.",
-        "convertToDiffusersHelpText5": "Assicurati di avere spazio su disco sufficiente. I modelli generalmente variano tra 2 GB e 7 GB di dimensioni.",
+        "convertToDiffusersHelpText5": "Assicurati di avere spazio su disco sufficiente. I modelli generalmente variano tra 2 GB e 7 GB in dimensione.",
         "convertToDiffusersHelpText6": "Vuoi convertire questo modello?",
         "modelConverted": "Modello convertito",
         "alpha": "Alpha",
@@ -471,8 +506,8 @@
         "scaledHeight": "Altezza ridimensionata",
         "infillMethod": "Metodo di riempimento",
         "tileSize": "Dimensione piastrella",
-        "sendToImg2Img": "Invia a Immagine a Immagine",
-        "sendToUnifiedCanvas": "Invia a Tela Unificata",
+        "sendToImg2Img": "Invia a Generazione da immagine",
+        "sendToUnifiedCanvas": "Invia alla Tela",
         "downloadImage": "Scarica l'immagine",
         "usePrompt": "Usa Prompt",
         "useSeed": "Usa Seme",
@@ -508,13 +543,24 @@
             "incompatibleBaseModelForControlAdapter": "Il modello dell'adattatore di controllo #{{number}} non è compatibile con il modello principale.",
             "missingNodeTemplate": "Modello di nodo mancante",
             "missingInputForField": "{{nodeLabel}} -> {{fieldLabel}} ingresso mancante",
-            "missingFieldTemplate": "Modello di campo mancante"
+            "missingFieldTemplate": "Modello di campo mancante",
+            "imageNotProcessedForControlAdapter": "L'immagine dell'adattatore di controllo #{{number}} non è stata elaborata",
+            "layer": {
+                "initialImageNoImageSelected": "Nessuna immagine iniziale selezionata",
+                "t2iAdapterIncompatibleDimensions": "L'adattatore T2I richiede che la dimensione dell'immagine sia un multiplo di {{multiple}}",
+                "controlAdapterNoModelSelected": "Nessun modello di adattatore di controllo selezionato",
+                "controlAdapterIncompatibleBaseModel": "Il modello base dell'adattatore di controllo non è compatibile",
+                "controlAdapterNoImageSelected": "Nessuna immagine dell'adattatore di controllo selezionata",
+                "controlAdapterImageNotProcessed": "Immagine dell'adattatore di controllo non elaborata",
+                "ipAdapterNoModelSelected": "Nessun adattatore IP selezionato",
+                "ipAdapterIncompatibleBaseModel": "Il modello base dell'adattatore IP non è compatibile",
+                "ipAdapterNoImageSelected": "Nessuna immagine dell'adattatore IP selezionata",
+                "rgNoPromptsOrIPAdapters": "Nessun prompt o adattatore IP",
+                "rgNoRegion": "Nessuna regione selezionata"
+            }
         },
         "useCpuNoise": "Usa la CPU per generare rumore",
         "iterations": "Iterazioni",
-        "iterationsWithCount_one": "{{count}} Iterazione",
-        "iterationsWithCount_many": "{{count}} Iterazioni",
-        "iterationsWithCount_other": "{{count}} Iterazioni",
         "isAllowedToUpscale": {
             "useX2Model": "L'immagine è troppo grande per l'ampliamento con il modello x4, utilizza il modello x2",
             "tooLarge": "L'immagine è troppo grande per l'ampliamento, seleziona un'immagine più piccola"
@@ -534,7 +580,10 @@
         "infillMosaicMinColor": "Colore minimo",
         "infillMosaicMaxColor": "Colore massimo",
         "infillMosaicTileHeight": "Altezza piastrella",
-        "infillColorValue": "Colore di riempimento"
+        "infillColorValue": "Colore di riempimento",
+        "globalSettings": "Impostazioni globali",
+        "globalPositivePromptPlaceholder": "Prompt positivo globale",
+        "globalNegativePromptPlaceholder": "Prompt negativo globale"
     },
     "settings": {
         "models": "Modelli",
@@ -559,7 +608,7 @@
         "intermediatesCleared_one": "Cancellata {{count}} immagine intermedia",
         "intermediatesCleared_many": "Cancellate {{count}} immagini intermedie",
         "intermediatesCleared_other": "Cancellate {{count}} immagini intermedie",
-        "clearIntermediatesDesc1": "La cancellazione delle immagini intermedie ripristinerà lo stato di Tela Unificata e ControlNet.",
+        "clearIntermediatesDesc1": "La cancellazione delle immagini intermedie ripristinerà lo stato della Tela e degli Adattatori di Controllo.",
         "intermediatesClearedFailed": "Problema con la cancellazione delle immagini intermedie",
         "clearIntermediatesWithCount_one": "Cancella {{count}} immagine intermedia",
         "clearIntermediatesWithCount_many": "Cancella {{count}} immagini intermedie",
@@ -575,27 +624,27 @@
         "imageCopied": "Immagine copiata",
         "imageNotLoadedDesc": "Impossibile trovare l'immagine",
         "canvasMerged": "Tela unita",
-        "sentToImageToImage": "Inviato a Immagine a Immagine",
-        "sentToUnifiedCanvas": "Inviato a Tela Unificata",
-        "parametersNotSet": "Parametri non impostati",
+        "sentToImageToImage": "Inviato a Generazione da immagine",
+        "sentToUnifiedCanvas": "Inviato alla Tela",
+        "parametersNotSet": "Parametri non richiamati",
         "metadataLoadFailed": "Impossibile caricare i metadati",
         "serverError": "Errore del Server",
-        "connected": "Connesso al Server",
+        "connected": "Connesso al server",
         "canceled": "Elaborazione annullata",
         "uploadFailedInvalidUploadDesc": "Deve essere una singola immagine PNG o JPEG",
-        "parameterSet": "{{parameter}} impostato",
-        "parameterNotSet": "{{parameter}} non impostato",
+        "parameterSet": "Parametro richiamato",
+        "parameterNotSet": "Parametro non richiamato",
         "problemCopyingImage": "Impossibile copiare l'immagine",
-        "baseModelChangedCleared_one": "Il modello base è stato modificato, cancellato o disabilitato {{count}} sotto-modello incompatibile",
-        "baseModelChangedCleared_many": "Il modello base è stato modificato, cancellato o disabilitato {{count}} sotto-modelli incompatibili",
-        "baseModelChangedCleared_other": "Il modello base è stato modificato, cancellato o disabilitato {{count}} sotto-modelli incompatibili",
+        "baseModelChangedCleared_one": "Cancellato o disabilitato {{count}} sottomodello incompatibile",
+        "baseModelChangedCleared_many": "Cancellati o disabilitati {{count}} sottomodelli incompatibili",
+        "baseModelChangedCleared_other": "Cancellati o disabilitati {{count}} sottomodelli incompatibili",
         "imageSavingFailed": "Salvataggio dell'immagine non riuscito",
         "canvasSentControlnetAssets": "Tela inviata a ControlNet & Risorse",
         "problemCopyingCanvasDesc": "Impossibile copiare la tela",
         "loadedWithWarnings": "Flusso di lavoro caricato con avvisi",
         "canvasCopiedClipboard": "Tela copiata negli appunti",
         "maskSavedAssets": "Maschera salvata nelle risorse",
-        "problemDownloadingCanvas": "Problema durante il download della tela",
+        "problemDownloadingCanvas": "Problema durante lo scarico della tela",
         "problemMergingCanvas": "Problema nell'unione delle tele",
         "imageUploaded": "Immagine caricata",
         "addedToBoard": "Aggiunto alla bacheca",
@@ -629,7 +678,17 @@
         "problemDownloadingImage": "Impossibile scaricare l'immagine",
         "prunedQueue": "Coda ripulita",
         "modelImportCanceled": "Importazione del modello annullata",
-        "parameters": "Parametri"
+        "parameters": "Parametri",
+        "parameterSetDesc": "{{parameter}} richiamato",
+        "parameterNotSetDesc": "Impossibile richiamare {{parameter}}",
+        "parameterNotSetDescWithMessage": "Impossibile richiamare {{parameter}}: {{message}}",
+        "parametersSet": "Parametri richiamati",
+        "errorCopied": "Errore copiato",
+        "outOfMemoryError": "Errore di memoria esaurita",
+        "baseModelChanged": "Modello base modificato",
+        "sessionRef": "Sessione: {{sessionId}}",
+        "somethingWentWrong": "Qualcosa è andato storto",
+        "outOfMemoryErrorDesc": "Le impostazioni della generazione attuale superano la capacità del sistema. Modifica le impostazioni e riprova."
     },
     "tooltip": {
         "feature": {
@@ -645,7 +704,7 @@
         "layer": "Livello",
         "base": "Base",
         "mask": "Maschera",
-        "maskingOptions": "Opzioni di mascheramento",
+        "maskingOptions": "Opzioni maschera",
         "enableMask": "Abilita maschera",
         "preserveMaskedArea": "Mantieni area mascherata",
         "clearMask": "Cancella maschera (Shift+C)",
@@ -716,7 +775,8 @@
         "mode": "Modalità",
         "resetUI": "$t(accessibility.reset) l'Interfaccia Utente",
         "createIssue": "Segnala un problema",
-        "about": "Informazioni"
+        "about": "Informazioni",
+        "submitSupportTicket": "Invia ticket di supporto"
     },
     "nodes": {
         "zoomOutNodes": "Rimpicciolire",
@@ -761,7 +821,7 @@
         "workflowNotes": "Note",
         "versionUnknown": " Versione sconosciuta",
         "unableToValidateWorkflow": "Impossibile convalidare il flusso di lavoro",
-        "updateApp": "Aggiorna App",
+        "updateApp": "Aggiorna Applicazione",
         "unableToLoadWorkflow": "Impossibile caricare il flusso di lavoro",
         "updateNode": "Aggiorna nodo",
         "version": "Versione",
@@ -795,7 +855,7 @@
         "float": "In virgola mobile",
         "currentImageDescription": "Visualizza l'immagine corrente nell'editor dei nodi",
         "fieldTypesMustMatch": "I tipi di campo devono corrispondere",
-        "edge": "Bordo",
+        "edge": "Collegamento",
         "currentImage": "Immagine corrente",
         "integer": "Numero Intero",
         "inputMayOnlyHaveOneConnection": "L'ingresso può avere solo una connessione",
@@ -808,8 +868,8 @@
         "unableToUpdateNodes_other": "Impossibile aggiornare {{count}} nodi",
         "addLinearView": "Aggiungi alla vista Lineare",
         "unknownErrorValidatingWorkflow": "Errore sconosciuto durante la convalida del flusso di lavoro",
-        "collectionFieldType": "{{name}} Raccolta",
-        "collectionOrScalarFieldType": "{{name}} Raccolta|Scalare",
+        "collectionFieldType": "{{name}} (Raccolta)",
+        "collectionOrScalarFieldType": "{{name}} (Singola o Raccolta)",
         "nodeVersion": "Versione Nodo",
         "inputFieldTypeParseError": "Impossibile analizzare il tipo di campo di input {{node}}.{{field}} ({{message}})",
         "unsupportedArrayItemType": "Tipo di elemento dell'array non supportato \"{{type}}\"",
@@ -845,11 +905,22 @@
         "resetToDefaultValue": "Ripristina il valore predefinito",
         "noFieldsViewMode": "Questo flusso di lavoro non ha campi selezionati da visualizzare. Visualizza il flusso di lavoro completo per configurare i valori.",
         "edit": "Modifica",
-        "graph": "Grafico"
+        "graph": "Grafico",
+        "showEdgeLabelsHelp": "Mostra etichette sui collegamenti, che indicano i nodi collegati",
+        "showEdgeLabels": "Mostra le etichette del collegamento",
+        "cannotMixAndMatchCollectionItemTypes": "Impossibile combinare e abbinare i tipi di elementi della raccolta",
+        "noGraph": "Nessun grafico",
+        "missingNode": "Nodo di invocazione mancante",
+        "missingInvocationTemplate": "Modello di invocazione mancante",
+        "missingFieldTemplate": "Modello di campo mancante",
+        "singleFieldType": "{{name}} (Singola)",
+        "imageAccessError": "Impossibile trovare l'immagine {{image_name}}, ripristino delle impostazioni predefinite",
+        "boardAccessError": "Impossibile trovare la bacheca {{board_id}}, ripristino ai valori predefiniti",
+        "modelAccessError": "Impossibile trovare il modello {{key}}, ripristino ai valori predefiniti"
     },
     "boards": {
         "autoAddBoard": "Aggiungi automaticamente bacheca",
-        "menuItemAutoAdd": "Aggiungi automaticamente a questa Bacheca",
+        "menuItemAutoAdd": "Aggiungi automaticamente a questa bacheca",
         "cancel": "Annulla",
         "addBoard": "Aggiungi Bacheca",
         "bottomMessage": "L'eliminazione di questa bacheca e delle sue immagini ripristinerà tutte le funzionalità che le stanno attualmente utilizzando.",
@@ -861,7 +932,7 @@
         "myBoard": "Bacheca",
         "searchBoard": "Cerca bacheche ...",
         "noMatching": "Nessuna bacheca corrispondente",
-        "selectBoard": "Seleziona una Bacheca",
+        "selectBoard": "Seleziona una bacheca",
         "uncategorized": "Non categorizzato",
         "downloadBoard": "Scarica la bacheca",
         "deleteBoardOnly": "solo la Bacheca",
@@ -882,7 +953,7 @@
         "control": "Controllo",
         "crop": "Ritaglia",
         "depthMidas": "Profondità (Midas)",
-        "detectResolution": "Rileva risoluzione",
+        "detectResolution": "Rileva la risoluzione",
         "controlMode": "Modalità di controllo",
         "cannyDescription": "Canny rilevamento bordi",
         "depthZoe": "Profondità (Zoe)",
@@ -893,7 +964,7 @@
         "showAdvanced": "Mostra opzioni Avanzate",
         "bgth": "Soglia rimozione sfondo",
         "importImageFromCanvas": "Importa immagine dalla Tela",
-        "lineartDescription": "Converte l'immagine in lineart",
+        "lineartDescription": "Converte l'immagine in linea",
         "importMaskFromCanvas": "Importa maschera dalla Tela",
         "hideAdvanced": "Nascondi opzioni avanzate",
         "resetControlImage": "Reimposta immagine di controllo",
@@ -909,7 +980,7 @@
         "pidiDescription": "Elaborazione immagini PIDI",
         "fill": "Riempie",
         "colorMapDescription": "Genera una mappa dei colori dall'immagine",
-        "lineartAnimeDescription": "Elaborazione lineart in stile anime",
+        "lineartAnimeDescription": "Elaborazione linea in stile anime",
         "imageResolution": "Risoluzione dell'immagine",
         "colorMap": "Colore",
         "lowThreshold": "Soglia inferiore",
@@ -922,7 +993,7 @@
         "colorMapTileSize": "Dimensione piastrella",
         "mediapipeFaceDescription": "Rilevamento dei volti tramite Mediapipe",
         "hedDescription": "Rilevamento dei bordi nidificati olisticamente",
-        "setControlImageDimensions": "Imposta le dimensioni dell'immagine di controllo su L/A",
+        "setControlImageDimensions": "Copia le dimensioni in L/A (ottimizza per il modello)",
         "maxFaces": "Numero massimo di volti",
         "addT2IAdapter": "Aggiungi $t(common.t2iAdapter)",
         "addControlNet": "Aggiungi $t(common.controlNet)",
@@ -951,12 +1022,17 @@
         "mediapipeFace": "Mediapipe Volto",
         "ip_adapter": "$t(controlnet.controlAdapter_one) #{{number}} ($t(common.ipAdapter))",
         "t2i_adapter": "$t(controlnet.controlAdapter_one) #{{number}} ($t(common.t2iAdapter))",
-        "selectCLIPVisionModel": "Seleziona un modello CLIP Vision"
+        "selectCLIPVisionModel": "Seleziona un modello CLIP Vision",
+        "ipAdapterMethod": "Metodo",
+        "full": "Completo",
+        "composition": "Solo la composizione",
+        "style": "Solo lo stile",
+        "beginEndStepPercentShort": "Inizio/Fine %",
+        "setControlImageDimensionsForce": "Copia le dimensioni in L/A (ignora il modello)"
     },
     "queue": {
         "queueFront": "Aggiungi all'inizio della coda",
         "queueBack": "Aggiungi alla coda",
-        "queueCountPrediction": "{{promptsCount}} prompt × {{iterations}} iterazioni -> {{count}} generazioni",
         "queue": "Coda",
         "status": "Stato",
         "pruneSucceeded": "Rimossi {{item_count}} elementi completati dalla coda",
@@ -993,7 +1069,7 @@
         "cancelBatchSucceeded": "Lotto annullato",
         "clearTooltip": "Annulla e cancella tutti gli elementi",
         "current": "Attuale",
-        "pauseTooltip": "Sospende l'elaborazione",
+        "pauseTooltip": "Sospendi l'elaborazione",
         "failed": "Falliti",
         "cancelItem": "Annulla l'elemento",
         "next": "Prossimo",
@@ -1011,7 +1087,16 @@
         "graphFailedToQueue": "Impossibile mettere in coda il grafico",
         "batchFieldValues": "Valori Campi Lotto",
         "time": "Tempo",
-        "openQueue": "Apri coda"
+        "openQueue": "Apri coda",
+        "iterations_one": "Iterazione",
+        "iterations_many": "Iterazioni",
+        "iterations_other": "Iterazioni",
+        "prompts_one": "Prompt",
+        "prompts_many": "Prompt",
+        "prompts_other": "Prompt",
+        "generations_one": "Generazione",
+        "generations_many": "Generazioni",
+        "generations_other": "Generazioni"
     },
     "models": {
         "noMatchingModels": "Nessun modello corrispondente",
@@ -1394,6 +1479,12 @@
             "paragraphs": [
                 "La dimensione del bordo del passaggio di coerenza."
             ]
+        },
+        "ipAdapterMethod": {
+            "heading": "Metodo",
+            "paragraphs": [
+                "Metodo con cui applicare l'adattatore IP corrente."
+            ]
         }
     },
     "sdxl": {
@@ -1522,5 +1613,55 @@
         "compatibleEmbeddings": "Incorporamenti compatibili",
         "addPromptTrigger": "Aggiungi Trigger nel prompt",
         "noMatchingTriggers": "Nessun Trigger corrispondente"
+    },
+    "controlLayers": {
+        "opacityFilter": "Filtro opacità",
+        "deleteAll": "Cancella tutto",
+        "addLayer": "Aggiungi Livello",
+        "moveToFront": "Sposta in primo piano",
+        "moveToBack": "Sposta in fondo",
+        "moveForward": "Sposta avanti",
+        "moveBackward": "Sposta indietro",
+        "brushSize": "Dimensioni del pennello",
+        "globalMaskOpacity": "Opacità globale della maschera",
+        "autoNegative": "Auto Negativo",
+        "deletePrompt": "Cancella il prompt",
+        "debugLayers": "Debug dei Livelli",
+        "rectangle": "Rettangolo",
+        "maskPreviewColor": "Colore anteprima maschera",
+        "addPositivePrompt": "Aggiungi $t(common.positivePrompt)",
+        "addNegativePrompt": "Aggiungi $t(common.negativePrompt)",
+        "addIPAdapter": "Aggiungi $t(common.ipAdapter)",
+        "regionalGuidance": "Guida regionale",
+        "regionalGuidanceLayer": "$t(unifiedCanvas.layer) $t(controlLayers.regionalGuidance)",
+        "opacity": "Opacità",
+        "globalControlAdapter": "$t(controlnet.controlAdapter_one) Globale",
+        "globalControlAdapterLayer": "$t(controlnet.controlAdapter_one) - $t(unifiedCanvas.layer) Globale",
+        "globalIPAdapter": "$t(common.ipAdapter) Globale",
+        "globalIPAdapterLayer": "$t(common.ipAdapter) - $t(unifiedCanvas.layer) Globale",
+        "globalInitialImage": "Immagine iniziale",
+        "globalInitialImageLayer": "$t(controlLayers.globalInitialImage) - $t(unifiedCanvas.layer) Globale",
+        "clearProcessor": "Cancella processore",
+        "resetProcessor": "Ripristina il processore alle impostazioni predefinite",
+        "noLayersAdded": "Nessun livello aggiunto",
+        "resetRegion": "Reimposta la regione",
+        "controlLayers": "Livelli di controllo",
+        "layers_one": "Livello",
+        "layers_many": "Livelli",
+        "layers_other": "Livelli"
+    },
+    "ui": {
+        "tabs": {
+            "generation": "Generazione",
+            "generationTab": "$t(ui.tabs.generation) $t(common.tab)",
+            "canvas": "Tela",
+            "canvasTab": "$t(ui.tabs.canvas) $t(common.tab)",
+            "workflows": "Flussi di lavoro",
+            "workflowsTab": "$t(ui.tabs.workflows) $t(common.tab)",
+            "models": "Modelli",
+            "modelsTab": "$t(ui.tabs.models) $t(common.tab)",
+            "queue": "Coda",
+            "queueTab": "$t(ui.tabs.queue) $t(common.tab)"
+        }
     }
 }

invokeai/frontend/web/public/locales/ja.json

@@ -570,7 +570,6 @@
         "pauseSucceeded": "処理が一時停止されました",
         "queueFront": "キューの先頭へ追加",
         "queueBack": "キューに追加",
-        "queueCountPrediction": "{{promptsCount}} プロンプト × {{iterations}} イテレーション -> {{count}} 枚生成",
         "pause": "一時停止",
         "queue": "キュー",
         "pauseTooltip": "処理を一時停止",

invokeai/frontend/web/public/locales/ko.json

@@ -505,7 +505,6 @@
         "completed": "완성된",
         "queueBack": "Queue에 추가",
         "cancelFailed": "항목 취소 중 발생한 문제",
-        "queueCountPrediction": "Queue에 {{predicted}} 추가",
         "batchQueued": "Batch Queued",
         "pauseFailed": "프로세서 중지 중 발생한 문제",
         "clearFailed": "Queue 제거 중 발생한 문제",

invokeai/frontend/web/public/locales/nl.json

@@ -6,7 +6,7 @@
         "settingsLabel": "Instellingen",
         "img2img": "Afbeelding naar afbeelding",
         "unifiedCanvas": "Centraal canvas",
-        "nodes": "Werkstroom-editor",
+        "nodes": "Werkstromen",
         "upload": "Upload",
         "load": "Laad",
         "statusDisconnected": "Niet verbonden",
@@ -34,7 +34,60 @@
         "controlNet": "ControlNet",
         "imageFailedToLoad": "Kan afbeelding niet laden",
         "learnMore": "Meer informatie",
-        "advanced": "Uitgebreid"
+        "advanced": "Uitgebreid",
+        "file": "Bestand",
+        "installed": "Geïnstalleerd",
+        "notInstalled": "Niet $t(common.installed)",
+        "simple": "Eenvoudig",
+        "somethingWentWrong": "Er ging iets mis",
+        "add": "Voeg toe",
+        "checkpoint": "Checkpoint",
+        "details": "Details",
+        "outputs": "Uitvoeren",
+        "save": "Bewaar",
+        "nextPage": "Volgende pagina",
+        "blue": "Blauw",
+        "alpha": "Alfa",
+        "red": "Rood",
+        "editor": "Editor",
+        "folder": "Map",
+        "format": "structuur",
+        "goTo": "Ga naar",
+        "template": "Sjabloon",
+        "input": "Invoer",
+        "loglevel": "Logboekniveau",
+        "safetensors": "Safetensors",
+        "saveAs": "Bewaar als",
+        "created": "Gemaakt",
+        "green": "Groen",
+        "tab": "Tab",
+        "positivePrompt": "Positieve prompt",
+        "negativePrompt": "Negatieve prompt",
+        "selected": "Geselecteerd",
+        "orderBy": "Sorteer op",
+        "prevPage": "Vorige pagina",
+        "beta": "Bèta",
+        "copyError": "$t(gallery.copy) Fout",
+        "toResolve": "Op te lossen",
+        "aboutDesc": "Gebruik je Invoke voor het werk? Kijk dan naar:",
+        "aboutHeading": "Creatieve macht voor jou",
+        "copy": "Kopieer",
+        "data": "Gegevens",
+        "or": "of",
+        "updated": "Bijgewerkt",
+        "outpaint": "outpainten",
+        "viewing": "Bekijken",
+        "viewingDesc": "Beoordeel afbeelding in een grote galerijweergave",
+        "editing": "Bewerken",
+        "editingDesc": "Bewerk op het canvas Stuurlagen",
+        "ai": "ai",
+        "inpaint": "inpainten",
+        "unknown": "Onbekend",
+        "delete": "Verwijder",
+        "direction": "Richting",
+        "error": "Fout",
+        "localSystem": "Lokaal systeem",
+        "unknownError": "Onbekende fout"
     },
     "gallery": {
         "galleryImageSize": "Afbeeldingsgrootte",
@@ -310,10 +363,41 @@
         "modelSyncFailed": "Synchronisatie modellen mislukt",
         "modelDeleteFailed": "Model kon niet verwijderd worden",
         "convertingModelBegin": "Model aan het converteren. Even geduld.",
-        "predictionType": "Soort voorspelling (voor Stable Diffusion 2.x-modellen en incidentele Stable Diffusion 1.x-modellen)",
+        "predictionType": "Soort voorspelling",
         "advanced": "Uitgebreid",
         "modelType": "Soort model",
-        "vaePrecision": "Nauwkeurigheid VAE"
+        "vaePrecision": "Nauwkeurigheid VAE",
+        "loraTriggerPhrases": "LoRA-triggerzinnen",
+        "urlOrLocalPathHelper": "URL's zouden moeten wijzen naar een los bestand. Lokale paden kunnen wijzen naar een los bestand of map voor een individueel Diffusers-model.",
+        "modelName": "Modelnaam",
+        "path": "Pad",
+        "triggerPhrases": "Triggerzinnen",
+        "typePhraseHere": "Typ zin hier in",
+        "useDefaultSettings": "Gebruik standaardinstellingen",
+        "modelImageDeleteFailed": "Fout bij verwijderen modelafbeelding",
+        "modelImageUpdated": "Modelafbeelding bijgewerkt",
+        "modelImageUpdateFailed": "Fout bij bijwerken modelafbeelding",
+        "noMatchingModels": "Geen overeenkomende modellen",
+        "scanPlaceholder": "Pad naar een lokale map",
+        "noModelsInstalled": "Geen modellen geïnstalleerd",
+        "noModelsInstalledDesc1": "Installeer modellen met de",
+        "noModelSelected": "Geen model geselecteerd",
+        "starterModels": "Beginnermodellen",
+        "textualInversions": "Tekstuele omkeringen",
+        "upcastAttention": "Upcast-aandacht",
+        "uploadImage": "Upload afbeelding",
+        "mainModelTriggerPhrases": "Triggerzinnen hoofdmodel",
+        "urlOrLocalPath": "URL of lokaal pad",
+        "scanFolderHelper": "De map zal recursief worden ingelezen voor modellen. Dit kan enige tijd in beslag nemen voor erg grote mappen.",
+        "simpleModelPlaceholder": "URL of pad naar een lokaal pad of Diffusers-map",
+        "modelSettings": "Modelinstellingen",
+        "pathToConfig": "Pad naar configuratie",
+        "prune": "Snoei",
+        "pruneTooltip": "Snoei voltooide importeringen uit wachtrij",
+        "repoVariant": "Repovariant",
+        "scanFolder": "Lees map in",
+        "scanResults": "Resultaten inlezen",
+        "source": "Bron"
     },
     "parameters": {
         "images": "Afbeeldingen",
@@ -353,13 +437,13 @@
         "copyImage": "Kopieer afbeelding",
         "denoisingStrength": "Sterkte ontruisen",
         "scheduler": "Planner",
-        "seamlessXAxis": "X-as",
-        "seamlessYAxis": "Y-as",
+        "seamlessXAxis": "Naadloze tegels in x-as",
+        "seamlessYAxis": "Naadloze tegels in y-as",
         "clipSkip": "Overslaan CLIP",
         "negativePromptPlaceholder": "Negatieve prompt",
         "controlNetControlMode": "Aansturingsmodus",
         "positivePromptPlaceholder": "Positieve prompt",
-        "maskBlur": "Vervaag",
+        "maskBlur": "Vervaging van masker",
         "invoke": {
             "noNodesInGraph": "Geen knooppunten in graaf",
             "noModelSelected": "Geen model ingesteld",
@@ -369,11 +453,25 @@
             "missingInputForField": "{{nodeLabel}} -> {{fieldLabel}} invoer ontbreekt",
             "noControlImageForControlAdapter": "Controle-adapter #{{number}} heeft geen controle-afbeelding",
             "noModelForControlAdapter": "Control-adapter #{{number}} heeft geen model ingesteld staan.",
-            "incompatibleBaseModelForControlAdapter": "Model van controle-adapter #{{number}} is ongeldig in combinatie met het hoofdmodel.",
+            "incompatibleBaseModelForControlAdapter": "Model van controle-adapter #{{number}} is niet compatibel met het hoofdmodel.",
             "systemDisconnected": "Systeem is niet verbonden",
             "missingNodeTemplate": "Knooppuntsjabloon ontbreekt",
             "missingFieldTemplate": "Veldsjabloon ontbreekt",
-            "addingImagesTo": "Bezig met toevoegen van afbeeldingen aan"
+            "addingImagesTo": "Bezig met toevoegen van afbeeldingen aan",
+            "layer": {
+                "initialImageNoImageSelected": "geen initiële afbeelding geselecteerd",
+                "controlAdapterNoModelSelected": "geen controle-adaptermodel geselecteerd",
+                "controlAdapterIncompatibleBaseModel": "niet-compatibele basismodel voor controle-adapter",
+                "controlAdapterNoImageSelected": "geen afbeelding voor controle-adapter geselecteerd",
+                "controlAdapterImageNotProcessed": "Afbeelding voor controle-adapter niet verwerkt",
+                "ipAdapterIncompatibleBaseModel": "niet-compatibele basismodel voor IP-adapter",
+                "ipAdapterNoImageSelected": "geen afbeelding voor IP-adapter geselecteerd",
+                "rgNoRegion": "geen gebied geselecteerd",
+                "rgNoPromptsOrIPAdapters": "geen tekstprompts of IP-adapters",
+                "t2iAdapterIncompatibleDimensions": "T2I-adapter vereist een afbeelding met afmetingen met een veelvoud van 64",
+                "ipAdapterNoModelSelected": "geen IP-adapter geselecteerd"
+            },
+            "imageNotProcessedForControlAdapter": "De afbeelding van controle-adapter #{{number}} is niet verwerkt"
         },
         "isAllowedToUpscale": {
             "useX2Model": "Afbeelding is te groot om te vergroten met het x4-model. Gebruik hiervoor het x2-model",
@@ -383,9 +481,26 @@
         "useCpuNoise": "Gebruik CPU-ruis",
         "imageActions": "Afbeeldingshandeling",
         "iterations": "Iteraties",
-        "iterationsWithCount_one": "{{count}} iteratie",
-        "iterationsWithCount_other": "{{count}} iteraties",
-        "coherenceMode": "Modus"
+        "coherenceMode": "Modus",
+        "infillColorValue": "Vulkleur",
+        "remixImage": "Meng afbeelding opnieuw",
+        "setToOptimalSize": "Optimaliseer grootte voor het model",
+        "setToOptimalSizeTooSmall": "$t(parameters.setToOptimalSize) (is mogelijk te klein)",
+        "aspect": "Beeldverhouding",
+        "infillMosaicTileWidth": "Breedte tegel",
+        "setToOptimalSizeTooLarge": "$t(parameters.setToOptimalSize) (is mogelijk te groot)",
+        "lockAspectRatio": "Zet beeldverhouding vast",
+        "infillMosaicTileHeight": "Hoogte tegel",
+        "globalNegativePromptPlaceholder": "Globale negatieve prompt",
+        "globalPositivePromptPlaceholder": "Globale positieve prompt",
+        "useSize": "Gebruik grootte",
+        "swapDimensions": "Wissel afmetingen om",
+        "globalSettings": "Globale instellingen",
+        "coherenceEdgeSize": "Randgrootte",
+        "coherenceMinDenoise": "Min. ontruising",
+        "infillMosaicMinColor": "Min. kleur",
+        "infillMosaicMaxColor": "Max. kleur",
+        "cfgRescaleMultiplier": "Vermenigvuldiger voor CFG-herschaling"
     },
     "settings": {
         "models": "Modellen",
@@ -412,7 +527,12 @@
         "intermediatesCleared_one": "{{count}} tussentijdse afbeelding gewist",
         "intermediatesCleared_other": "{{count}} tussentijdse afbeeldingen gewist",
         "clearIntermediatesDesc1": "Als je tussentijdse afbeeldingen wist, dan wordt de staat hersteld van je canvas en van ControlNet.",
-        "intermediatesClearedFailed": "Fout bij wissen van tussentijdse afbeeldingen"
+        "intermediatesClearedFailed": "Fout bij wissen van tussentijdse afbeeldingen",
+        "clearIntermediatesDisabled": "Wachtrij moet leeg zijn om tussentijdse afbeeldingen te kunnen leegmaken",
+        "enableInformationalPopovers": "Schakel informatieve hulpballonnen in",
+        "enableInvisibleWatermark": "Schakel onzichtbaar watermerk in",
+        "enableNSFWChecker": "Schakel NSFW-controle in",
+        "reloadingIn": "Opnieuw laden na"
     },
     "toast": {
         "uploadFailed": "Upload mislukt",
@@ -427,8 +547,8 @@
         "connected": "Verbonden met server",
         "canceled": "Verwerking geannuleerd",
         "uploadFailedInvalidUploadDesc": "Moet een enkele PNG- of JPEG-afbeelding zijn",
-        "parameterNotSet": "Parameter niet ingesteld",
-        "parameterSet": "Instellen parameters",
+        "parameterNotSet": "{{parameter}} niet ingesteld",
+        "parameterSet": "{{parameter}} ingesteld",
         "problemCopyingImage": "Kan Afbeelding Niet Kopiëren",
         "baseModelChangedCleared_one": "Basismodel is gewijzigd: {{count}} niet-compatibel submodel weggehaald of uitgeschakeld",
         "baseModelChangedCleared_other": "Basismodel is gewijzigd: {{count}} niet-compatibele submodellen weggehaald of uitgeschakeld",
@@ -445,11 +565,11 @@
         "maskSavedAssets": "Masker bewaard in Assets",
         "problemDownloadingCanvas": "Fout bij downloaden van canvas",
         "problemMergingCanvas": "Fout bij samenvoegen canvas",
-        "setCanvasInitialImage": "Ingesteld als initiële canvasafbeelding",
+        "setCanvasInitialImage": "Initiële canvasafbeelding ingesteld",
         "imageUploaded": "Afbeelding geüpload",
         "addedToBoard": "Toegevoegd aan bord",
         "workflowLoaded": "Werkstroom geladen",
-        "modelAddedSimple": "Model toegevoegd",
+        "modelAddedSimple": "Model toegevoegd aan wachtrij",
         "problemImportingMaskDesc": "Kan masker niet exporteren",
         "problemCopyingCanvas": "Fout bij kopiëren canvas",
         "problemSavingCanvas": "Fout bij bewaren canvas",
@@ -461,7 +581,18 @@
         "maskSentControlnetAssets": "Masker gestuurd naar ControlNet en Assets",
         "canvasSavedGallery": "Canvas bewaard in galerij",
         "imageUploadFailed": "Fout bij uploaden afbeelding",
-        "problemImportingMask": "Fout bij importeren masker"
+        "problemImportingMask": "Fout bij importeren masker",
+        "workflowDeleted": "Werkstroom verwijderd",
+        "invalidUpload": "Ongeldige upload",
+        "uploadInitialImage": "Initiële afbeelding uploaden",
+        "setAsCanvasInitialImage": "Ingesteld als initiële afbeelding voor canvas",
+        "problemRetrievingWorkflow": "Fout bij ophalen van werkstroom",
+        "parameters": "Parameters",
+        "modelImportCanceled": "Importeren model geannuleerd",
+        "problemDeletingWorkflow": "Fout bij verwijderen van werkstroom",
+        "prunedQueue": "Wachtrij gesnoeid",
+        "problemDownloadingImage": "Fout bij downloaden afbeelding",
+        "resetInitialImage": "Initiële afbeelding hersteld"
     },
     "tooltip": {
         "feature": {
@@ -535,7 +666,11 @@
         "showOptionsPanel": "Toon zijscherm",
         "menu": "Menu",
         "showGalleryPanel": "Toon deelscherm Galerij",
-        "loadMore": "Laad meer"
+        "loadMore": "Laad meer",
+        "about": "Over",
+        "mode": "Modus",
+        "resetUI": "$t(accessibility.reset) UI",
+        "createIssue": "Maak probleem aan"
     },
     "nodes": {
         "zoomOutNodes": "Uitzoomen",
@@ -549,7 +684,7 @@
         "loadWorkflow": "Laad werkstroom",
         "downloadWorkflow": "Download JSON van werkstroom",
         "scheduler": "Planner",
-        "missingTemplate": "Ontbrekende sjabloon",
+        "missingTemplate": "Ongeldig knooppunt: knooppunt {{node}} van het soort {{type}} heeft een ontbrekend sjabloon (niet geïnstalleerd?)",
         "workflowDescription": "Korte beschrijving",
         "versionUnknown": " Versie onbekend",
         "noNodeSelected": "Geen knooppunt gekozen",
@@ -565,7 +700,7 @@
         "integer": "Geheel getal",
         "nodeTemplate": "Sjabloon knooppunt",
         "nodeOpacity": "Dekking knooppunt",
-        "unableToLoadWorkflow": "Kan werkstroom niet valideren",
+        "unableToLoadWorkflow": "Fout bij laden werkstroom",
         "snapToGrid": "Lijn uit op raster",
         "noFieldsLinearview": "Geen velden toegevoegd aan lineaire weergave",
         "nodeSearch": "Zoek naar knooppunten",
@@ -616,11 +751,56 @@
         "unknownField": "Onbekend veld",
         "colorCodeEdges": "Kleurgecodeerde randen",
         "unknownNode": "Onbekend knooppunt",
-        "mismatchedVersion": "Heeft niet-overeenkomende versie",
+        "mismatchedVersion": "Ongeldig knooppunt: knooppunt {{node}} van het soort {{type}} heeft een niet-overeenkomende versie (probeer het bij te werken?)",
         "addNodeToolTip": "Voeg knooppunt toe (Shift+A, spatie)",
         "loadingNodes": "Bezig met laden van knooppunten...",
         "snapToGridHelp": "Lijn knooppunten uit op raster bij verplaatsing",
-        "workflowSettings": "Instellingen werkstroomeditor"
+        "workflowSettings": "Instellingen werkstroomeditor",
+        "addLinearView": "Voeg toe aan lineaire weergave",
+        "nodePack": "Knooppuntpakket",
+        "unknownInput": "Onbekende invoer: {{name}}",
+        "sourceNodeFieldDoesNotExist": "Ongeldige rand: bron-/uitvoerveld {{node}}.{{field}} bestaat niet",
+        "collectionFieldType": "Verzameling {{name}}",
+        "deletedInvalidEdge": "Ongeldige hoek {{source}} -> {{target}} verwijderd",
+        "graph": "Grafiek",
+        "targetNodeDoesNotExist": "Ongeldige rand: doel-/invoerknooppunt {{node}} bestaat niet",
+        "resetToDefaultValue": "Herstel naar standaardwaarden",
+        "editMode": "Bewerk in Werkstroom-editor",
+        "showEdgeLabels": "Toon randlabels",
+        "showEdgeLabelsHelp": "Toon labels aan randen, waarmee de verbonden knooppunten mee worden aangegeven",
+        "clearWorkflowDesc2": "Je huidige werkstroom heeft niet-bewaarde wijzigingen.",
+        "unableToParseFieldType": "fout bij bepalen soort veld",
+        "sourceNodeDoesNotExist": "Ongeldige rand: bron-/uitvoerknooppunt {{node}} bestaat niet",
+        "unsupportedArrayItemType": "niet-ondersteunde soort van het array-onderdeel \"{{type}}\"",
+        "targetNodeFieldDoesNotExist": "Ongeldige rand: doel-/invoerveld {{node}}.{{field}} bestaat niet",
+        "reorderLinearView": "Herorden lineaire weergave",
+        "newWorkflowDesc": "Een nieuwe werkstroom aanmaken?",
+        "collectionOrScalarFieldType": "Verzameling|scalair {{name}}",
+        "newWorkflow": "Nieuwe werkstroom",
+        "unknownErrorValidatingWorkflow": "Onbekende fout bij valideren werkstroom",
+        "unsupportedAnyOfLength": "te veel union-leden ({{count}})",
+        "unknownOutput": "Onbekende uitvoer: {{name}}",
+        "viewMode": "Gebruik in lineaire weergave",
+        "unableToExtractSchemaNameFromRef": "fout bij het extraheren van de schemanaam via de ref",
+        "unsupportedMismatchedUnion": "niet-overeenkomende soort CollectionOrScalar met basissoorten {{firstType}} en {{secondType}}",
+        "unknownNodeType": "Onbekend soort knooppunt",
+        "edit": "Bewerk",
+        "updateAllNodes": "Werk knooppunten bij",
+        "allNodesUpdated": "Alle knooppunten bijgewerkt",
+        "nodeVersion": "Knooppuntversie",
+        "newWorkflowDesc2": "Je huidige werkstroom heeft niet-bewaarde wijzigingen.",
+        "clearWorkflow": "Maak werkstroom leeg",
+        "clearWorkflowDesc": "Deze werkstroom leegmaken en met een nieuwe beginnen?",
+        "inputFieldTypeParseError": "Fout bij bepalen van het soort invoerveld {{node}}.{{field}} ({{message}})",
+        "outputFieldTypeParseError": "Fout bij het bepalen van het soort uitvoerveld {{node}}.{{field}} ({{message}})",
+        "unableToExtractEnumOptions": "fout bij extraheren enumeratie-opties",
+        "unknownFieldType": "Soort $t(nodes.unknownField): {{type}}",
+        "unableToGetWorkflowVersion": "Fout bij ophalen schemaversie van werkstroom",
+        "betaDesc": "Deze uitvoering is in bèta. Totdat deze stabiel is kunnen er wijzigingen voorkomen gedurende app-updates die zaken kapotmaken. We zijn van plan om deze uitvoering op lange termijn te gaan ondersteunen.",
+        "prototypeDesc": "Deze uitvoering is een prototype. Er kunnen wijzigingen voorkomen gedurende app-updates die zaken kapotmaken. Deze kunnen op een willekeurig moment verwijderd worden.",
+        "noFieldsViewMode": "Deze werkstroom heeft geen geselecteerde velden om te tonen. Bekijk de volledige werkstroom om de waarden te configureren.",
+        "unableToUpdateNodes_one": "Fout bij bijwerken van {{count}} knooppunt",
+        "unableToUpdateNodes_other": "Fout bij bijwerken van {{count}} knooppunten"
     },
     "controlnet": {
         "amult": "a_mult",
@@ -693,9 +873,28 @@
         "canny": "Canny",
         "depthZoeDescription": "Genereer diepteblad via Zoe",
         "hedDescription": "Herkenning van holistisch-geneste randen",
-        "setControlImageDimensions": "Stel afmetingen controle-afbeelding in op B/H",
+        "setControlImageDimensions": "Kopieer grootte naar B/H (optimaliseer voor model)",
         "scribble": "Krabbel",
-        "maxFaces": "Max. gezichten"
+        "maxFaces": "Max. gezichten",
+        "dwOpenpose": "DW Openpose",
+        "depthAnything": "Depth Anything",
+        "base": "Basis",
+        "hands": "Handen",
+        "selectCLIPVisionModel": "Selecteer een CLIP Vision-model",
+        "modelSize": "Modelgrootte",
+        "small": "Klein",
+        "large": "Groot",
+        "resizeSimple": "Wijzig grootte (eenvoudig)",
+        "beginEndStepPercentShort": "Begin-/eind-%",
+        "depthAnythingDescription": "Genereren dieptekaart d.m.v. de techniek Depth Anything",
+        "face": "Gezicht",
+        "body": "Lichaam",
+        "dwOpenposeDescription": "Schatting menselijke pose d.m.v. DW Openpose",
+        "ipAdapterMethod": "Methode",
+        "full": "Volledig",
+        "style": "Alleen stijl",
+        "composition": "Alleen samenstelling",
+        "setControlImageDimensionsForce": "Kopieer grootte naar B/H (negeer model)"
     },
     "dynamicPrompts": {
         "seedBehaviour": {
@@ -708,7 +907,10 @@
         "maxPrompts": "Max. prompts",
         "promptsWithCount_one": "{{count}} prompt",
         "promptsWithCount_other": "{{count}} prompts",
-        "dynamicPrompts": "Dynamische prompts"
+        "dynamicPrompts": "Dynamische prompts",
+        "showDynamicPrompts": "Toon dynamische prompts",
+        "loading": "Genereren van dynamische prompts...",
+        "promptsPreview": "Voorvertoning prompts"
     },
     "popovers": {
         "noiseUseCPU": {
@@ -721,7 +923,7 @@
         },
         "paramScheduler": {
             "paragraphs": [
-                "De planner bepaalt hoe ruis per iteratie wordt toegevoegd aan een afbeelding of hoe een monster wordt bijgewerkt op basis van de uitvoer van een model."
+                "De planner gebruikt gedurende het genereringsproces."
             ],
             "heading": "Planner"
         },
@@ -808,8 +1010,8 @@
         },
         "clipSkip": {
             "paragraphs": [
-                "Kies hoeveel CLIP-modellagen je wilt overslaan.",
-                "Bepaalde modellen werken beter met bepaalde Overslaan CLIP-instellingen."
+                "Aantal over te slaan CLIP-modellagen.",
+                "Bepaalde modellen zijn beter geschikt met bepaalde Overslaan CLIP-instellingen."
             ],
             "heading": "Overslaan CLIP"
         },
@@ -940,7 +1142,6 @@
         "completed": "Voltooid",
         "queueBack": "Voeg toe aan wachtrij",
         "cancelFailed": "Fout bij annuleren onderdeel",
-        "queueCountPrediction": "Voeg {{predicted}} toe aan wachtrij",
         "batchQueued": "Reeks in wachtrij geplaatst",
         "pauseFailed": "Fout bij onderbreken verwerker",
         "clearFailed": "Fout bij wissen van wachtrij",
@@ -994,17 +1195,26 @@
         "denoisingStrength": "Sterkte ontruising",
         "refinermodel": "Verfijningsmodel",
         "posAestheticScore": "Positieve esthetische score",
-        "concatPromptStyle": "Plak prompt- en stijltekst aan elkaar",
+        "concatPromptStyle": "Koppelen van prompt en stijl",
         "loading": "Bezig met laden...",
         "steps": "Stappen",
-        "posStylePrompt": "Positieve-stijlprompt"
+        "posStylePrompt": "Positieve-stijlprompt",
+        "freePromptStyle": "Handmatige stijlprompt",
+        "refinerSteps": "Aantal stappen verfijner"
     },
     "models": {
         "noMatchingModels": "Geen overeenkomend modellen",
         "loading": "bezig met laden",
         "noMatchingLoRAs": "Geen overeenkomende LoRA's",
         "noModelsAvailable": "Geen modellen beschikbaar",
-        "selectModel": "Kies een model"
+        "selectModel": "Kies een model",
+        "noLoRAsInstalled": "Geen LoRA's geïnstalleerd",
+        "noRefinerModelsInstalled": "Geen SDXL-verfijningsmodellen geïnstalleerd",
+        "defaultVAE": "Standaard-VAE",
+        "lora": "LoRA",
+        "esrganModel": "ESRGAN-model",
+        "addLora": "Voeg LoRA toe",
+        "concepts": "Concepten"
     },
     "boards": {
         "autoAddBoard": "Voeg automatisch bord toe",
@@ -1022,7 +1232,13 @@
         "downloadBoard": "Download bord",
         "changeBoard": "Wijzig bord",
         "loading": "Bezig met laden...",
-        "clearSearch": "Maak zoekopdracht leeg"
+        "clearSearch": "Maak zoekopdracht leeg",
+        "deleteBoard": "Verwijder bord",
+        "deleteBoardAndImages": "Verwijder bord en afbeeldingen",
+        "deleteBoardOnly": "Verwijder alleen bord",
+        "deletedBoardsCannotbeRestored": "Verwijderde borden kunnen niet worden hersteld",
+        "movingImagesToBoard_one": "Verplaatsen van {{count}} afbeelding naar bord:",
+        "movingImagesToBoard_other": "Verplaatsen van {{count}} afbeeldingen naar bord:"
     },
     "invocationCache": {
         "disable": "Schakel uit",
@@ -1039,5 +1255,39 @@
         "clear": "Wis",
         "maxCacheSize": "Max. grootte cache",
         "cacheSize": "Grootte cache"
+    },
+    "accordions": {
+        "generation": {
+            "title": "Genereren"
+        },
+        "image": {
+            "title": "Afbeelding"
+        },
+        "advanced": {
+            "title": "Geavanceerd",
+            "options": "$t(accordions.advanced.title) Opties"
+        },
+        "control": {
+            "title": "Besturing"
+        },
+        "compositing": {
+            "title": "Samenstellen",
+            "coherenceTab": "Coherentiefase",
+            "infillTab": "Invullen"
+        }
+    },
+    "hrf": {
+        "upscaleMethod": "Opschaalmethode",
+        "metadata": {
+            "strength": "Sterkte oplossing voor hoge resolutie",
+            "method": "Methode oplossing voor hoge resolutie",
+            "enabled": "Oplossing voor hoge resolutie ingeschakeld"
+        },
+        "hrf": "Oplossing voor hoge resolutie",
+        "enableHrf": "Schakel oplossing in voor hoge resolutie"
+    },
+    "prompt": {
+        "addPromptTrigger": "Voeg prompttrigger toe",
+        "compatibleEmbeddings": "Compatibele embeddings"
     }
 }

invokeai/frontend/web/public/locales/ru.json

@@ -76,7 +76,22 @@
         "localSystem": "Локальная система",
         "aboutDesc": "Используя Invoke для работы? Проверьте это:",
         "add": "Добавить",
-        "loglevel": "Уровень логов"
+        "loglevel": "Уровень логов",
+        "beta": "Бета",
+        "selected": "Выбрано",
+        "positivePrompt": "Позитивный запрос",
+        "negativePrompt": "Негативный запрос",
+        "editor": "Редактор",
+        "goTo": "Перейти к",
+        "tab": "Вкладка",
+        "viewing": "Просмотр",
+        "editing": "Редактирование",
+        "viewingDesc": "Просмотр изображений в режиме большой галереи",
+        "editingDesc": "Редактировать на холсте слоёв управления",
+        "enabled": "Включено",
+        "disabled": "Отключено",
+        "comparingDesc": "Сравнение двух изображений",
+        "comparing": "Сравнение"
     },
     "gallery": {
         "galleryImageSize": "Размер изображений",
@@ -87,8 +102,8 @@
         "deleteImagePermanent": "Удаленные изображения невозможно восстановить.",
         "deleteImageBin": "Удаленные изображения будут отправлены в корзину вашей операционной системы.",
         "deleteImage_one": "Удалить изображение",
-        "deleteImage_few": "",
-        "deleteImage_many": "",
+        "deleteImage_few": "Удалить {{count}} изображения",
+        "deleteImage_many": "Удалить {{count}} изображений",
         "assets": "Ресурсы",
         "autoAssignBoardOnClick": "Авто-назначение доски по клику",
         "deleteSelection": "Удалить выделенное",
@@ -113,7 +128,23 @@
         "bulkDownloadRequested": "Подготовка к скачиванию",
         "bulkDownloadRequestedDesc": "Ваш запрос на скачивание готовится. Это может занять несколько минут.",
         "bulkDownloadRequestFailed": "Возникла проблема при подготовке скачивания",
-        "alwaysShowImageSizeBadge": "Всегда показывать значок размера изображения"
+        "alwaysShowImageSizeBadge": "Всегда показывать значок размера изображения",
+        "openInViewer": "Открыть в просмотрщике",
+        "selectForCompare": "Выбрать для сравнения",
+        "hover": "Наведение",
+        "swapImages": "Поменять местами",
+        "stretchToFit": "Растягивание до нужного размера",
+        "exitCompare": "Выйти из сравнения",
+        "compareHelp4": "Нажмите <Kbd>Z</Kbd> или <Kbd>Esc</Kbd> для выхода.",
+        "compareImage": "Сравнить изображение",
+        "viewerImage": "Изображение просмотрщика",
+        "selectAnImageToCompare": "Выберите изображение для сравнения",
+        "slider": "Слайдер",
+        "sideBySide": "Бок о бок",
+        "compareOptions": "Варианты сравнения",
+        "compareHelp1": "Удерживайте <Kbd>Alt</Kbd> при нажатии на изображение в галерее или при помощи клавиш со стрелками, чтобы изменить сравниваемое изображение.",
+        "compareHelp2": "Нажмите <Kbd>M</Kbd>, чтобы переключиться между режимами сравнения.",
+        "compareHelp3": "Нажмите <Kbd>C</Kbd>, чтобы поменять местами сравниваемые изображения."
     },
     "hotkeys": {
         "keyboardShortcuts": "Горячие клавиши",
@@ -336,6 +367,10 @@
         "remixImage": {
             "desc": "Используйте все параметры, кроме сида из текущего изображения",
             "title": "Ремикс изображения"
+        },
+        "toggleViewer": {
+            "title": "Переключить просмотр изображений",
+            "desc": "Переключение между средством просмотра изображений и рабочей областью для текущей вкладки."
         }
     },
     "modelManager": {
@@ -512,7 +547,21 @@
             "missingNodeTemplate": "Отсутствует шаблон узла",
             "missingFieldTemplate": "Отсутствует шаблон поля",
             "addingImagesTo": "Добавление изображений в",
-            "invoke": "Создать"
+            "invoke": "Создать",
+            "imageNotProcessedForControlAdapter": "Изображение адаптера контроля №{{number}} не обрабатывается",
+            "layer": {
+                "controlAdapterImageNotProcessed": "Изображение адаптера контроля не обработано",
+                "ipAdapterNoModelSelected": "IP адаптер не выбран",
+                "controlAdapterNoModelSelected": "не выбрана модель адаптера контроля",
+                "controlAdapterIncompatibleBaseModel": "несовместимая базовая модель адаптера контроля",
+                "controlAdapterNoImageSelected": "не выбрано изображение контрольного адаптера",
+                "initialImageNoImageSelected": "начальное изображение не выбрано",
+                "rgNoRegion": "регион не выбран",
+                "rgNoPromptsOrIPAdapters": "нет текстовых запросов или IP-адаптеров",
+                "ipAdapterIncompatibleBaseModel": "несовместимая базовая модель IP-адаптера",
+                "t2iAdapterIncompatibleDimensions": "Адаптер T2I требует, чтобы размеры изображения были кратны {{multiple}}",
+                "ipAdapterNoImageSelected": "изображение IP-адаптера не выбрано"
+            }
         },
         "isAllowedToUpscale": {
             "useX2Model": "Изображение слишком велико для увеличения с помощью модели x4. Используйте модель x2",
@@ -523,9 +572,6 @@
         "useCpuNoise": "Использовать шум CPU",
         "imageActions": "Действия с изображениями",
         "iterations": "Кол-во",
-        "iterationsWithCount_one": "{{count}} Интеграция",
-        "iterationsWithCount_few": "{{count}} Итерации",
-        "iterationsWithCount_many": "{{count}} Итераций",
         "useSize": "Использовать размер",
         "coherenceMode": "Режим",
         "aspect": "Соотношение",
@@ -541,7 +587,10 @@
         "infillMosaicTileHeight": "Высота плиток",
         "infillMosaicMinColor": "Мин цвет",
         "infillMosaicMaxColor": "Макс цвет",
-        "infillColorValue": "Цвет заливки"
+        "infillColorValue": "Цвет заливки",
+        "globalSettings": "Глобальные настройки",
+        "globalNegativePromptPlaceholder": "Глобальный негативный запрос",
+        "globalPositivePromptPlaceholder": "Глобальный запрос"
     },
     "settings": {
         "models": "Модели",
@@ -590,12 +639,12 @@
         "connected": "Подключено к серверу",
         "canceled": "Обработка отменена",
         "uploadFailedInvalidUploadDesc": "Должно быть одно изображение в формате PNG или JPEG",
-        "parameterNotSet": "Параметр {{parameter}} не задан",
-        "parameterSet": "Параметр {{parameter}} задан",
+        "parameterNotSet": "Параметр не задан",
+        "parameterSet": "Параметр задан",
         "problemCopyingImage": "Не удается скопировать изображение",
-        "baseModelChangedCleared_one": "Базовая модель изменила, очистила или отключила {{count}} несовместимую подмодель",
-        "baseModelChangedCleared_few": "Базовая модель изменила, очистила или отключила {{count}} несовместимые подмодели",
-        "baseModelChangedCleared_many": "Базовая модель изменила, очистила или отключила {{count}} несовместимых подмоделей",
+        "baseModelChangedCleared_one": "Очищена или отключена {{count}} несовместимая подмодель",
+        "baseModelChangedCleared_few": "Очищены или отключены {{count}} несовместимые подмодели",
+        "baseModelChangedCleared_many": "Очищены или отключены {{count}} несовместимых подмоделей",
         "imageSavingFailed": "Не удалось сохранить изображение",
         "canvasSentControlnetAssets": "Холст отправлен в ControlNet и ресурсы",
         "problemCopyingCanvasDesc": "Невозможно экспортировать базовый слой",
@@ -636,7 +685,17 @@
         "resetInitialImage": "Сбросить начальное изображение",
         "prunedQueue": "Урезанная очередь",
         "modelImportCanceled": "Импорт модели отменен",
-        "parameters": "Параметры"
+        "parameters": "Параметры",
+        "parameterSetDesc": "Задан {{parameter}}",
+        "parameterNotSetDesc": "Невозможно задать {{parameter}}",
+        "baseModelChanged": "Базовая модель сменена",
+        "parameterNotSetDescWithMessage": "Не удалось задать {{parameter}}: {{message}}",
+        "parametersSet": "Параметры заданы",
+        "errorCopied": "Ошибка скопирована",
+        "sessionRef": "Сессия: {{sessionId}}",
+        "outOfMemoryError": "Ошибка нехватки памяти",
+        "outOfMemoryErrorDesc": "Ваши текущие настройки генерации превышают возможности системы. Пожалуйста, измените настройки и повторите попытку.",
+        "somethingWentWrong": "Что-то пошло не так"
     },
     "tooltip": {
         "feature": {
@@ -706,7 +765,9 @@
         "coherenceModeBoxBlur": "коробчатое размытие",
         "discardCurrent": "Отбросить текущее",
         "invertBrushSizeScrollDirection": "Инвертировать прокрутку для размера кисти",
-        "initialFitImageSize": "Подогнать размер изображения при перебросе"
+        "initialFitImageSize": "Подогнать размер изображения при перебросе",
+        "hideBoundingBox": "Скрыть ограничительную рамку",
+        "showBoundingBox": "Показать ограничительную рамку"
     },
     "accessibility": {
         "uploadImage": "Загрузить изображение",
@@ -721,7 +782,8 @@
         "loadMore": "Загрузить больше",
         "resetUI": "$t(accessibility.reset) интерфейс",
         "createIssue": "Сообщить о проблеме",
-        "about": "Об этом"
+        "about": "Об этом",
+        "submitSupportTicket": "Отправить тикет в службу поддержки"
     },
     "nodes": {
         "zoomInNodes": "Увеличьте масштаб",
@@ -814,7 +876,7 @@
         "workflowName": "Название",
         "collection": "Коллекция",
         "unknownErrorValidatingWorkflow": "Неизвестная ошибка при проверке рабочего процесса",
-        "collectionFieldType": "Коллекция {{name}}",
+        "collectionFieldType": "{{name}} (Коллекция)",
         "workflowNotes": "Примечания",
         "string": "Строка",
         "unknownNodeType": "Неизвестный тип узла",
@@ -830,7 +892,7 @@
         "targetNodeDoesNotExist": "Недопустимое ребро: целевой/входной узел {{node}} не существует",
         "mismatchedVersion": "Недопустимый узел: узел {{node}} типа {{type}} имеет несоответствующую версию (попробовать обновить?)",
         "unknownFieldType": "$t(nodes.unknownField) тип: {{type}}",
-        "collectionOrScalarFieldType": "Коллекция | Скаляр {{name}}",
+        "collectionOrScalarFieldType": "{{name}} (Один или коллекция)",
         "betaDesc": "Этот вызов находится в бета-версии. Пока он не станет стабильным, в нем могут происходить изменения при обновлении приложений. Мы планируем поддерживать этот вызов в течение длительного времени.",
         "nodeVersion": "Версия узла",
         "loadingNodes": "Загрузка узлов...",
@@ -849,7 +911,19 @@
         "editMode": "Открыть в редакторе узлов",
         "resetToDefaultValue": "Сбросить к стандартному значкнию",
         "edit": "Редактировать",
-        "noFieldsViewMode": "В этом рабочем процессе нет выбранных полей для отображения. Просмотрите полный рабочий процесс для настройки значений."
+        "noFieldsViewMode": "В этом рабочем процессе нет выбранных полей для отображения. Просмотрите полный рабочий процесс для настройки значений.",
+        "graph": "График",
+        "showEdgeLabels": "Показать метки на ребрах",
+        "showEdgeLabelsHelp": "Показать метки на ребрах, указывающие на соединенные узлы",
+        "cannotMixAndMatchCollectionItemTypes": "Невозможно смешивать и сопоставлять типы элементов коллекции",
+        "missingNode": "Отсутствует узел вызова",
+        "missingInvocationTemplate": "Отсутствует шаблон вызова",
+        "missingFieldTemplate": "Отсутствующий шаблон поля",
+        "singleFieldType": "{{name}} (Один)",
+        "noGraph": "Нет графика",
+        "imageAccessError": "Невозможно найти изображение {{image_name}}, сбрасываем на значение по умолчанию",
+        "boardAccessError": "Невозможно найти доску {{board_id}}, сбрасываем на значение по умолчанию",
+        "modelAccessError": "Невозможно найти модель {{key}}, сброс на модель по умолчанию"
     },
     "controlnet": {
         "amult": "a_mult",
@@ -917,8 +991,8 @@
         "lineartAnime": "Контурный рисунок в стиле аниме",
         "mediapipeFaceDescription": "Обнаружение лиц с помощью Mediapipe",
         "hedDescription": "Целостное обнаружение границ",
-        "setControlImageDimensions": "Установите размеры контрольного изображения на Ш/В",
-        "scribble": "каракули",
+        "setControlImageDimensions": "Скопируйте размер в Ш/В (оптимизируйте для модели)",
+        "scribble": "Штрихи",
         "maxFaces": "Макс Лица",
         "mlsdDescription": "Минималистичный детектор отрезков линии",
         "resizeSimple": "Изменить размер (простой)",
@@ -933,7 +1007,18 @@
         "small": "Маленький",
         "body": "Тело",
         "hands": "Руки",
-        "selectCLIPVisionModel": "Выбрать модель CLIP Vision"
+        "selectCLIPVisionModel": "Выбрать модель CLIP Vision",
+        "ipAdapterMethod": "Метод",
+        "full": "Всё",
+        "mlsd": "M-LSD",
+        "h": "H",
+        "style": "Только стиль",
+        "dwOpenpose": "DW Openpose",
+        "pidi": "PIDI",
+        "composition": "Только композиция",
+        "hed": "HED",
+        "beginEndStepPercentShort": "Начало/конец %",
+        "setControlImageDimensionsForce": "Скопируйте размер в Ш/В (игнорируйте модель)"
     },
     "boards": {
         "autoAddBoard": "Авто добавление Доски",
@@ -1312,6 +1397,12 @@
             "paragraphs": [
                 "Плавно укладывайте изображение вдоль вертикальной оси."
             ]
+        },
+        "ipAdapterMethod": {
+            "heading": "Метод",
+            "paragraphs": [
+                "Метод, с помощью которого применяется текущий IP-адаптер."
+            ]
         }
     },
     "metadata": {
@@ -1359,7 +1450,6 @@
         "completed": "Выполнено",
         "queueBack": "Добавить в очередь",
         "cancelFailed": "Проблема с отменой элемента",
-        "queueCountPrediction": "{{promptsCount}} запросов × {{iterations}} изображений -> {{count}} генераций",
         "batchQueued": "Пакетная очередь",
         "pauseFailed": "Проблема с приостановкой рендеринга",
         "clearFailed": "Проблема с очисткой очереди",
@@ -1404,7 +1494,16 @@
         "clearQueueAlertDialog2": "Вы уверены, что хотите очистить очередь?",
         "item": "Элемент",
         "graphFailedToQueue": "Не удалось поставить график в очередь",
-        "openQueue": "Открыть очередь"
+        "openQueue": "Открыть очередь",
+        "prompts_one": "Запрос",
+        "prompts_few": "Запроса",
+        "prompts_many": "Запросов",
+        "iterations_one": "Итерация",
+        "iterations_few": "Итерации",
+        "iterations_many": "Итераций",
+        "generations_one": "Генерация",
+        "generations_few": "Генерации",
+        "generations_many": "Генераций"
     },
     "sdxl": {
         "refinerStart": "Запуск доработчика",
@@ -1475,7 +1574,11 @@
         "projectWorkflows": "Рабочие процессы проекта",
         "defaultWorkflows": "Стандартные рабочие процессы",
         "name": "Имя",
-        "noRecentWorkflows": "Нет последних рабочих процессов"
+        "noRecentWorkflows": "Нет последних рабочих процессов",
+        "loadWorkflow": "Рабочий процесс $t(common.load)",
+        "convertGraph": "Конвертировать график",
+        "loadFromGraph": "Загрузка рабочего процесса из графика",
+        "autoLayout": "Автоматическое расположение"
     },
     "hrf": {
         "enableHrf": "Включить исправление высокого разрешения",
@@ -1528,5 +1631,55 @@
         "addPromptTrigger": "Добавить триггер запроса",
         "compatibleEmbeddings": "Совместимые встраивания",
         "noMatchingTriggers": "Нет соответствующих триггеров"
+    },
+    "controlLayers": {
+        "moveToBack": "На задний план",
+        "moveForward": "Переместить вперёд",
+        "moveBackward": "Переместить назад",
+        "brushSize": "Размер кисти",
+        "controlLayers": "Слои управления",
+        "globalMaskOpacity": "Глобальная непрозрачность маски",
+        "autoNegative": "Авто негатив",
+        "deletePrompt": "Удалить запрос",
+        "resetRegion": "Сбросить регион",
+        "debugLayers": "Слои отладки",
+        "rectangle": "Прямоугольник",
+        "maskPreviewColor": "Цвет предпросмотра маски",
+        "addNegativePrompt": "Добавить $t(common.negativePrompt)",
+        "regionalGuidance": "Региональная точность",
+        "opacity": "Непрозрачность",
+        "globalControlAdapter": "Глобальный $t(controlnet.controlAdapter_one)",
+        "globalControlAdapterLayer": "Глобальный $t(controlnet.controlAdapter_one) $t(unifiedCanvas.layer)",
+        "globalIPAdapter": "Глобальный $t(common.ipAdapter)",
+        "globalIPAdapterLayer": "Глобальный $t(common.ipAdapter) $t(unifiedCanvas.layer)",
+        "opacityFilter": "Фильтр непрозрачности",
+        "deleteAll": "Удалить всё",
+        "addLayer": "Добавить слой",
+        "moveToFront": "На передний план",
+        "addPositivePrompt": "Добавить $t(common.positivePrompt)",
+        "addIPAdapter": "Добавить $t(common.ipAdapter)",
+        "regionalGuidanceLayer": "$t(controlLayers.regionalGuidance) $t(unifiedCanvas.layer)",
+        "resetProcessor": "Сброс процессора по умолчанию",
+        "clearProcessor": "Чистый процессор",
+        "globalInitialImage": "Глобальное исходное изображение",
+        "globalInitialImageLayer": "$t(controlLayers.globalInitialImage) $t(unifiedCanvas.layer)",
+        "noLayersAdded": "Без слоев",
+        "layers_one": "Слой",
+        "layers_few": "Слоя",
+        "layers_many": "Слоев"
+    },
+    "ui": {
+        "tabs": {
+            "generation": "Генерация",
+            "canvas": "Холст",
+            "workflowsTab": "$t(ui.tabs.workflows) $t(common.tab)",
+            "models": "Модели",
+            "generationTab": "$t(ui.tabs.generation) $t(common.tab)",
+            "workflows": "Рабочие процессы",
+            "canvasTab": "$t(ui.tabs.canvas) $t(common.tab)",
+            "queueTab": "$t(ui.tabs.queue) $t(common.tab)",
+            "modelsTab": "$t(ui.tabs.models) $t(common.tab)",
+            "queue": "Очередь"
+        }
     }
 }

invokeai/frontend/web/public/locales/zh_CN.json

@@ -66,7 +66,7 @@
         "saveAs": "保存为",
         "ai": "ai",
         "or": "或",
-        "aboutDesc": "使用 Invoke 工作？查看：",
+        "aboutDesc": "使用 Invoke 工作？来看看：",
         "add": "添加",
         "loglevel": "日志级别",
         "copy": "复制",
@@ -445,7 +445,6 @@
             "useX2Model": "图像太大，无法使用 x4 模型，使用 x2 模型作为替代",
             "tooLarge": "图像太大无法进行放大，请选择更小的图像"
         },
-        "iterationsWithCount_other": "{{count}} 次迭代生成",
         "cfgRescaleMultiplier": "CFG 重缩放倍数",
         "useSize": "使用尺寸",
         "setToOptimalSize": "优化模型大小",
@@ -853,7 +852,6 @@
         "pruneSucceeded": "从队列修剪 {{item_count}} 个已完成的项目",
         "notReady": "无法排队",
         "batchFailedToQueue": "批次加入队列失败",
-        "queueCountPrediction": "{{promptsCount}} 提示词 × {{iterations}} 迭代次数 -> {{count}} 次生成",
         "batchQueued": "加入队列的批次",
         "front": "前",
         "pruneTooltip": "修剪 {{item_count}} 个已完成的项目",