WIP - Started working towards MultiDiffusion batching.

.github/actions/install-frontend-deps/action.yml

@@ -9,9 +9,9 @@ runs:
         node-version: '18'
 
     - name: setup pnpm
-      uses: pnpm/action-setup@v4
+      uses: pnpm/action-setup@v2
       with:
-        version: 8.15.6
+        version: 8
         run_install: false
 
     - name: get pnpm store directory

.github/pull_request_template.md

@@ -8,7 +8,7 @@
 
 ## QA Instructions
 
-<!--WHEN APPLICABLE: Describe how you have tested the changes in this PR. Provide enough detail that a reviewer can reproduce your tests.-->
+<!--WHEN APPLICABLE: Describe how we can test the changes in this PR.-->
 
 ## Merge Plan
 

.github/workflows/python-checks.yml

@@ -62,7 +62,7 @@ jobs:
 
       - name: install ruff
         if: ${{ steps.changed-files.outputs.python_any_changed == 'true' || inputs.always_run == true }}
-        run: pip install ruff==0.6.0
+        run: pip install ruff
         shell: bash
 
       - name: ruff check

README.md

@@ -12,24 +12,12 @@
 
 Invoke is a leading creative engine built to empower professionals and enthusiasts alike. Generate and create stunning visual media using the latest AI-driven technologies. Invoke offers an industry leading web-based UI, and serves as the foundation for multiple commercial products.
 
-Invoke is available in two editions:
-
-| **Community Edition**                                                                                                      | **Professional Edition**                                                                            |
-|----------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------|
-| **For users looking for a locally installed, self-hosted and self-managed service**                                         | **For users or teams looking for a cloud-hosted, fully managed service**                            |
-| - Free to use under a commercially-friendly license                                                                         | - Monthly subscription fee with three different plan levels                                         |
-| - Download and install on compatible hardware                                                                               | - Offers additional benefits, including multi-user support, improved model training, and more                          |
-| - Includes all core studio features: generate, refine, iterate on images, and build workflows                               | - Hosted in the cloud for easy, secure model access and scalability                                               |
-| Quick Start -> [Installation and Updates][installation docs]                                                                     | More Information -> [www.invoke.com/pricing](https://www.invoke.com/pricing)                        |
+[Installation and Updates][installation docs] - [Documentation and Tutorials][docs home] - [Bug Reports][github issues] - [Contributing][contributing docs]
 
+<div align="center">
 
 ![Highlighted Features - Canvas and Workflows](https://github.com/invoke-ai/InvokeAI/assets/31807370/708f7a82-084f-4860-bfbe-e2588c53548d)
 
-# Documentation
-| **Quick Links**                                                                                                      | 
-|----------------------------------------------------------------------------------------------------------------------------|
-|  [Installation and Updates][installation docs] - [Documentation and Tutorials][docs home] - [Bug Reports][github issues] - [Contributing][contributing docs]  | 
-
 </div>
 
 ## Quick Start
@@ -49,33 +37,6 @@ Invoke is available in two editions:
 
 More detail, including hardware requirements and manual install instructions, are available in the [installation documentation][installation docs].
 
-## Docker Container
-
-We publish official container images in Github Container Registry: https://github.com/invoke-ai/InvokeAI/pkgs/container/invokeai. Both CUDA and ROCm images are available. Check the above link for relevant tags.
-
-> [!IMPORTANT]
-> Ensure that Docker is set up to use the GPU. Refer to [NVIDIA][nvidia docker docs] or [AMD][amd docker docs] documentation.
-
-### Generate!
-
-Run the container, modifying the command as necessary:
-
-```bash
-docker run --runtime=nvidia --gpus=all --publish 9090:9090 ghcr.io/invoke-ai/invokeai
-```
-
-Then open `http://localhost:9090` and install some models using the Model Manager tab to begin generating.
-
-For ROCm, add `--device /dev/kfd --device /dev/dri` to the `docker run` command.
-
-### Persist your data
-
-You will likely want to persist your workspace outside of the container. Use the `--volume /home/myuser/invokeai:/invokeai` flag to mount some local directory (using its **absolute** path) to the `/invokeai` path inside the container. Your generated images and models will reside there. You can use this directory with other InvokeAI installations, or switch between runtime directories as needed.
-
-### DIY
-
-Build your own image and customize the environment to match your needs using our `docker-compose` stack. See [README.md](./docker/README.md) in the [docker](./docker) directory.
-
 ## Troubleshooting, FAQ and Support
 
 Please review our [FAQ][faq] for solutions to common installation problems and other issues.
@@ -153,5 +114,3 @@ Original portions of the software are Copyright © 2024 by respective contributo
 [latest release link]: https://github.com/invoke-ai/InvokeAI/releases/latest
 [translation status badge]: https://hosted.weblate.org/widgets/invokeai/-/svg-badge.svg
 [translation status link]: https://hosted.weblate.org/engage/invokeai/
-[nvidia docker docs]: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
-[amd docker docs]: https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html

docker/.env.sample

@@ -19,9 +19,8 @@
 ## INVOKEAI_PORT is the port on which the InvokeAI web interface will be available
 # INVOKEAI_PORT=9090
 
-## GPU_DRIVER can be set to either `cuda` or `rocm` to enable GPU support in the container accordingly.
-# GPU_DRIVER=cuda #| rocm
+## GPU_DRIVER can be set to either `nvidia` or `rocm` to enable GPU support in the container accordingly.
+# GPU_DRIVER=nvidia #| rocm
 
 ## CONTAINER_UID can be set to the UID of the user on the host system that should own the files in the container.
-## It is usually not necessary to change this. Use `id -u` on the host system to find the UID.
 # CONTAINER_UID=1000

docker/Dockerfile

@@ -55,7 +55,6 @@ RUN --mount=type=cache,target=/root/.cache/pip \
 FROM node:20-slim AS web-builder
 ENV PNPM_HOME="/pnpm"
 ENV PATH="$PNPM_HOME:$PATH"
-RUN corepack use pnpm@8.x
 RUN corepack enable
 
 WORKDIR /build

docker/README.md

@@ -1,75 +1,41 @@
-# Invoke in Docker
+# InvokeAI Containerized
 
-- Ensure that Docker can use the GPU on your system
-- This documentation assumes Linux, but should work similarly under Windows with WSL2
-- We don't recommend running Invoke in Docker on macOS at this time. It works, but very slowly.
+All commands should be run within the `docker` directory: `cd docker`
 
-## Quickstart :lightning:
+## Quickstart :rocket:
 
-No `docker compose`, no persistence, just a simple one-liner using the official images:
+On a known working Linux+Docker+CUDA (Nvidia) system, execute `./run.sh` in this directory. It will take a few minutes - depending on your internet speed - to install the core models. Once the application starts up, open `http://localhost:9090` in your browser to Invoke!
 
-**CUDA:**
+For more configuration options (using an AMD GPU, custom root directory location, etc): read on.
 
-```bash
-docker run --runtime=nvidia --gpus=all --publish 9090:9090 ghcr.io/invoke-ai/invokeai
-```
-
-**ROCm:**
-
-```bash
-docker run --device /dev/kfd --device /dev/dri --publish 9090:9090 ghcr.io/invoke-ai/invokeai:main-rocm
-```
-
-Open `http://localhost:9090` in your browser once the container finishes booting, install some models, and generate away!
-
-> [!TIP]
-> To persist your data (including downloaded models) outside of the container, add a `--volume/-v` flag to the above command, e.g.: `docker run --volume /some/local/path:/invokeai <...the rest of the command>`
-
-## Customize the container
-
-We ship the `run.sh` script, which is a convenient wrapper around `docker compose` for cases where custom image build args are needed. Alternatively, the familiar `docker compose` commands work just as well.
-
-```bash
-cd docker
-cp .env.sample .env
-# edit .env to your liking if you need to; it is well commented.
-./run.sh
-```
-
-It will take a few minutes to build the image the first time. Once the application starts up, open `http://localhost:9090` in your browser to invoke!
-
-## Docker setup in detail
+## Detailed setup
 
 #### Linux
 
 1. Ensure builkit is enabled in the Docker daemon settings (`/etc/docker/daemon.json`)
 2. Install the `docker compose` plugin using your package manager, or follow a [tutorial](https://docs.docker.com/compose/install/linux/#install-using-the-repository).
-    - The deprecated `docker-compose` (hyphenated) CLI probably won't work. Update to a recent version.
+    - The deprecated `docker-compose` (hyphenated) CLI continues to work for now.
 3. Ensure docker daemon is able to access the GPU.
-    - [NVIDIA docs](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
-    - [AMD docs](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html)
+    - You may need to install [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
 
 #### macOS
 
-> [!TIP]
-> You'll be better off installing Invoke directly on your system, because Docker can not use the GPU on macOS.
-
-If you are still reading:
-
 1. Ensure Docker has at least 16GB RAM
 2. Enable VirtioFS for file sharing
 3. Enable `docker compose` V2 support
 
-This is done via Docker Desktop preferences.
+This is done via Docker Desktop preferences
 
-### Configure the Invoke Environment
+### Configure Invoke environment
 
-1. Make a copy of `.env.sample` and name it `.env` (`cp .env.sample .env` (Mac/Linux) or `copy example.env .env` (Windows)). Make changes as necessary. Set `INVOKEAI_ROOT` to an absolute path to the desired location of the InvokeAI runtime directory. It may be an existing directory from a previous installation (post 4.0.0).
+1. Make a copy of `.env.sample` and name it `.env` (`cp .env.sample .env` (Mac/Linux) or `copy example.env .env` (Windows)). Make changes as necessary. Set `INVOKEAI_ROOT` to an absolute path to:
+    a. the desired location of the InvokeAI runtime directory, or
+    b. an existing, v3.0.0 compatible runtime directory.
 1. Execute `run.sh`
 
 The image will be built automatically if needed.
 
-The runtime directory (holding models and outputs) will be created in the location specified by `INVOKEAI_ROOT`. The default location is `~/invokeai`. Navigate to the Model Manager tab and install some models before generating.
+The runtime directory (holding models and outputs) will be created in the location specified by `INVOKEAI_ROOT`. The default location is `~/invokeai`. The runtime directory will be populated with the base configs and models necessary to start generating.
 
 ### Use a GPU
 
@@ -77,9 +43,9 @@ The runtime directory (holding models and outputs) will be created in the locati
 - WSL2 is *required* for Windows.
 - only `x86_64` architecture is supported.
 
-The Docker daemon on the system must be already set up to use the GPU. In case of Linux, this involves installing `nvidia-docker-runtime` and configuring the `nvidia` runtime as default. Steps will be different for AMD. Please see Docker/NVIDIA/AMD documentation for the most up-to-date instructions for using your GPU with Docker.
+The Docker daemon on the system must be already set up to use the GPU. In case of Linux, this involves installing `nvidia-docker-runtime` and configuring the `nvidia` runtime as default. Steps will be different for AMD. Please see Docker documentation for the most up-to-date instructions for using your GPU with Docker.
 
-To use an AMD GPU, set `GPU_DRIVER=rocm` in your `.env` file before running `./run.sh`.
+To use an AMD GPU, set `GPU_DRIVER=rocm` in your `.env` file.
 
 ## Customize
 
@@ -93,10 +59,10 @@ Values are optional, but setting `INVOKEAI_ROOT` is highly recommended. The defa
 INVOKEAI_ROOT=/Volumes/WorkDrive/invokeai
 HUGGINGFACE_TOKEN=the_actual_token
 CONTAINER_UID=1000
-GPU_DRIVER=cuda
+GPU_DRIVER=nvidia
 ```
 
-Any environment variables supported by InvokeAI can be set here. See the [Configuration docs](https://invoke-ai.github.io/InvokeAI/features/CONFIGURATION/) for further detail.
+Any environment variables supported by InvokeAI can be set here - please see the [Configuration docs](https://invoke-ai.github.io/InvokeAI/features/CONFIGURATION/) for further detail.
 
 ## Even More Customizing!
 

docker/docker-compose.yml

@@ -1,5 +1,7 @@
 # Copyright (c) 2023 Eugene Brodsky https://github.com/ebr
 
+version: '3.8'
+
 x-invokeai: &invokeai
     image: "local/invokeai:latest"
     build:
@@ -30,7 +32,7 @@ x-invokeai: &invokeai
 
 
 services:
-  invokeai-cuda:
+  invokeai-nvidia:
     <<: *invokeai
     deploy:
       resources:

docker/docker-entrypoint.sh

@@ -23,18 +23,18 @@ usermod -u ${USER_ID} ${USER} 1>/dev/null
 # but it is useful to have the full SSH server e.g. on Runpod.
 # (use SCP to copy files to/from the image, etc)
 if [[ -v "PUBLIC_KEY" ]] && [[ ! -d "${HOME}/.ssh" ]]; then
-  apt-get update
-  apt-get install -y openssh-server
-  pushd "$HOME"
-  mkdir -p .ssh
-  echo "${PUBLIC_KEY}" >.ssh/authorized_keys
-  chmod -R 700 .ssh
-  popd
-  service ssh start
+    apt-get update
+    apt-get install -y openssh-server
+    pushd "$HOME"
+    mkdir -p .ssh
+    echo "${PUBLIC_KEY}" > .ssh/authorized_keys
+    chmod -R 700 .ssh
+    popd
+    service ssh start
 fi
 
 mkdir -p "${INVOKEAI_ROOT}"
-chown --recursive ${USER} "${INVOKEAI_ROOT}" || true
+chown --recursive ${USER} "${INVOKEAI_ROOT}"
 cd "${INVOKEAI_ROOT}"
 
 # Run the CMD as the Container User (not root).

docker/run.sh

@@ -8,15 +8,11 @@ run() {
   local build_args=""
   local profile=""
 
-  # create .env file if it doesn't exist, otherwise docker compose will fail
   touch .env
-
-  # parse .env file for build args
   build_args=$(awk '$1 ~ /=[^$]/ && $0 !~ /^#/ {print "--build-arg " $0 " "}' .env) &&
   profile="$(awk -F '=' '/GPU_DRIVER/ {print $2}' .env)"
 
-  # default to 'cuda' profile
-  [[ -z "$profile" ]] && profile="cuda"
+  [[ -z "$profile" ]] && profile="nvidia"
 
   local service_name="invokeai-$profile"
 

docs/contributing/MODEL_MANAGER.md

@@ -408,7 +408,7 @@ config = get_config()
 
 logger = InvokeAILogger.get_logger(config=config)
 db = SqliteDatabase(config.db_path, logger)
-record_store = ModelRecordServiceSQL(db, logger)
+record_store = ModelRecordServiceSQL(db)
 queue = DownloadQueueService()
 queue.start()
 

docs/installation/040_INSTALL_DOCKER.md

@@ -4,37 +4,50 @@ title: Installing with Docker
 
 # :fontawesome-brands-docker: Docker
 
-!!! warning "macOS users"
+!!! warning "macOS and AMD GPU Users"
 
-    Docker can not access the GPU on macOS, so your generation speeds will be slow. [Install InvokeAI](INSTALLATION.md) instead.
+    We highly recommend to Install InvokeAI locally using [these instructions](INSTALLATION.md),
+    because Docker containers can not access the GPU on macOS.
+
+!!! warning "AMD GPU Users"
+
+    Container support for AMD GPUs has been reported to work by the community, but has not received
+    extensive testing. Please make sure to set the `GPU_DRIVER=rocm` environment variable (see below), and
+    use the `build.sh` script to build the image for this to take effect at build time.
 
 !!! tip "Linux and Windows Users"
 
-    Configure Docker to access your machine's GPU.
+    For optimal performance, configure your Docker daemon to access your machine's GPU.
     Docker Desktop on Windows [includes GPU support](https://www.docker.com/blog/wsl-2-gpu-support-for-docker-desktop-on-nvidia-gpus/).
-    Linux users should follow the [NVIDIA](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) or [AMD](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html) documentation.
+    Linux users should install and configure the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
+
+## Why containers?
+
+They provide a flexible, reliable way to build and deploy InvokeAI.
+See [Processes](https://12factor.net/processes) under the Twelve-Factor App
+methodology for details on why running applications in such a stateless fashion is important.
+
+The container is configured for CUDA by default, but can be built to support AMD GPUs
+by setting the `GPU_DRIVER=rocm` environment variable at Docker image build time.
+
+Developers on Apple silicon (M1/M2/M3): You
+[can't access your GPU cores from Docker containers](https://github.com/pytorch/pytorch/issues/81224)
+and performance is reduced compared with running it directly on macOS but for
+development purposes it's fine. Once you're done with development tasks on your
+laptop you can build for the target platform and architecture and deploy to
+another environment with NVIDIA GPUs on-premises or in the cloud.
 
 ## TL;DR
 
-Ensure your Docker setup is able to use your GPU. Then:
-
-    ```bash
-    docker run --runtime=nvidia --gpus=all --publish 9090:9090 ghcr.io/invoke-ai/invokeai
-    ```
-
-Once the container starts up, open http://localhost:9090 in your browser, install some models, and start generating.
-
-## Build-It-Yourself
-
-All the docker materials are located inside the [docker](https://github.com/invoke-ai/InvokeAI/tree/main/docker) directory in the Git repo.
+This assumes properly configured Docker on Linux or Windows/WSL2. Read on for detailed customization options.
 
     ```bash
+    # docker compose commands should be run from the `docker` directory
     cd docker
-    cp .env.sample .env
     docker compose up
     ```
 
-We also ship the `run.sh` convenience script. See the `docker/README.md` file for detailed instructions on how to customize the docker setup to your needs.
+## Installation in a Linux container (desktop)
 
 ### Prerequisites
 
@@ -45,9 +58,18 @@ Preferences, Resources, Advanced. Increase the CPUs and Memory to avoid this
 [Issue](https://github.com/invoke-ai/InvokeAI/issues/342). You may need to
 increase Swap and Disk image size too.
 
+#### Get a Huggingface-Token
+
+Besides the Docker Agent you will need an Account on
+[huggingface.co](https://huggingface.co/join).
+
+After you succesfully registered your account, go to
+[huggingface.co/settings/tokens](https://huggingface.co/settings/tokens), create
+a token and copy it, since you will need in for the next step.
+
 ### Setup
 
-Set up your environment variables. In the `docker` directory, make a copy of `.env.sample` and name it `.env`. Make changes as necessary.
+Set up your environmnent variables. In the `docker` directory, make a copy of `.env.sample` and name it `.env`. Make changes as necessary.
 
 Any environment variables supported by InvokeAI can be set here - please see the [CONFIGURATION](../features/CONFIGURATION.md) for further detail.
 
@@ -81,9 +103,10 @@ Once the container starts up (and configures the InvokeAI root directory if this
 ## Troubleshooting / FAQ
 
 - Q: I am running on Windows under WSL2, and am seeing a "no such file or directory" error.
-- A: Your `docker-entrypoint.sh` might have has Windows (CRLF) line endings, depending how you cloned the repository.
-    To solve this, change the line endings in the `docker-entrypoint.sh` file to `LF`. You can do this in VSCode
+- A: Your `docker-entrypoint.sh` file likely has Windows (CRLF) as opposed to Unix (LF) line endings,
+    and you may have cloned this repository before the issue was fixed. To solve this, please change
+    the line endings in the `docker-entrypoint.sh` file to `LF`. You can do this in VSCode
     (`Ctrl+P` and search for "line endings"), or by using the `dos2unix` utility in WSL.
     Finally, you may delete `docker-entrypoint.sh` followed by  `git pull; git checkout docker/docker-entrypoint.sh`
     to reset the file to its most recent version.
-    For more information on this issue, see [Docker Desktop documentation](https://docs.docker.com/desktop/troubleshoot/topics/#avoid-unexpected-syntax-errors-use-unix-style-line-endings-for-files-in-containers)
+    For more information on this issue, please see the [Docker Desktop documentation](https://docs.docker.com/desktop/troubleshoot/topics/#avoid-unexpected-syntax-errors-use-unix-style-line-endings-for-files-in-containers)

installer/templates/invoke.bat.in

@@ -13,7 +13,7 @@ echo 2. Open the developer console
 echo 3. Command-line help
 echo Q - Quit
 echo.
-echo To update, download and run the installer from https://github.com/invoke-ai/InvokeAI/releases/latest
+echo To update, download and run the installer from https://github.com/invoke-ai/InvokeAI/releases/latest.
 echo.
 set /P choice="Please enter 1-4, Q: [1] "
 if not defined choice set choice=1

installer/templates/invoke.sh.in

@@ -17,7 +17,7 @@
 set -eu
 
 # Ensure we're in the correct folder in case user's CWD is somewhere else
-scriptdir=$(dirname $(readlink -f "$0"))
+scriptdir=$(dirname "$0")
 cd "$scriptdir"
 
 . .venv/bin/activate

invokeai/app/api/dependencies.py

@@ -1,45 +1,40 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
 
-import asyncio
 from logging import Logger
 
 import torch
 
-from invokeai.app.services.board_image_records.board_image_records_sqlite import SqliteBoardImageRecordStorage
-from invokeai.app.services.board_images.board_images_default import BoardImagesService
-from invokeai.app.services.board_records.board_records_sqlite import SqliteBoardRecordStorage
-from invokeai.app.services.boards.boards_default import BoardService
-from invokeai.app.services.bulk_download.bulk_download_default import BulkDownloadService
-from invokeai.app.services.config.config_default import InvokeAIAppConfig
-from invokeai.app.services.download.download_default import DownloadQueueService
-from invokeai.app.services.events.events_fastapievents import FastAPIEventService
-from invokeai.app.services.image_files.image_files_disk import DiskImageFileStorage
-from invokeai.app.services.image_records.image_records_sqlite import SqliteImageRecordStorage
-from invokeai.app.services.images.images_default import ImageService
-from invokeai.app.services.invocation_cache.invocation_cache_memory import MemoryInvocationCache
-from invokeai.app.services.invocation_services import InvocationServices
-from invokeai.app.services.invocation_stats.invocation_stats_default import InvocationStatsService
-from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_images.model_images_default import ModelImageFileStorageDisk
-from invokeai.app.services.model_manager.model_manager_default import ModelManagerService
-from invokeai.app.services.model_records.model_records_sql import ModelRecordServiceSQL
-from invokeai.app.services.names.names_default import SimpleNameService
 from invokeai.app.services.object_serializer.object_serializer_disk import ObjectSerializerDisk
 from invokeai.app.services.object_serializer.object_serializer_forward_cache import ObjectSerializerForwardCache
-from invokeai.app.services.session_processor.session_processor_default import (
-    DefaultSessionProcessor,
-    DefaultSessionRunner,
-)
-from invokeai.app.services.session_queue.session_queue_sqlite import SqliteSessionQueue
 from invokeai.app.services.shared.sqlite.sqlite_util import init_db
-from invokeai.app.services.style_preset_images.style_preset_images_disk import StylePresetImageFileStorageDisk
-from invokeai.app.services.style_preset_records.style_preset_records_sqlite import SqliteStylePresetRecordsStorage
-from invokeai.app.services.urls.urls_default import LocalUrlService
-from invokeai.app.services.workflow_records.workflow_records_sqlite import SqliteWorkflowRecordsStorage
 from invokeai.backend.stable_diffusion.diffusion.conditioning_data import ConditioningFieldData
 from invokeai.backend.util.logging import InvokeAILogger
 from invokeai.version.invokeai_version import __version__
 
+from ..services.board_image_records.board_image_records_sqlite import SqliteBoardImageRecordStorage
+from ..services.board_images.board_images_default import BoardImagesService
+from ..services.board_records.board_records_sqlite import SqliteBoardRecordStorage
+from ..services.boards.boards_default import BoardService
+from ..services.bulk_download.bulk_download_default import BulkDownloadService
+from ..services.config import InvokeAIAppConfig
+from ..services.download import DownloadQueueService
+from ..services.events.events_fastapievents import FastAPIEventService
+from ..services.image_files.image_files_disk import DiskImageFileStorage
+from ..services.image_records.image_records_sqlite import SqliteImageRecordStorage
+from ..services.images.images_default import ImageService
+from ..services.invocation_cache.invocation_cache_memory import MemoryInvocationCache
+from ..services.invocation_services import InvocationServices
+from ..services.invocation_stats.invocation_stats_default import InvocationStatsService
+from ..services.invoker import Invoker
+from ..services.model_images.model_images_default import ModelImageFileStorageDisk
+from ..services.model_manager.model_manager_default import ModelManagerService
+from ..services.model_records import ModelRecordServiceSQL
+from ..services.names.names_default import SimpleNameService
+from ..services.session_processor.session_processor_default import DefaultSessionProcessor, DefaultSessionRunner
+from ..services.session_queue.session_queue_sqlite import SqliteSessionQueue
+from ..services.urls.urls_default import LocalUrlService
+from ..services.workflow_records.workflow_records_sqlite import SqliteWorkflowRecordsStorage
+
 
 # TODO: is there a better way to achieve this?
 def check_internet() -> bool:
@@ -66,12 +61,7 @@ class ApiDependencies:
     invoker: Invoker
 
     @staticmethod
-    def initialize(
-        config: InvokeAIAppConfig,
-        event_handler_id: int,
-        loop: asyncio.AbstractEventLoop,
-        logger: Logger = logger,
-    ) -> None:
+    def initialize(config: InvokeAIAppConfig, event_handler_id: int, logger: Logger = logger) -> None:
         logger.info(f"InvokeAI version {__version__}")
         logger.info(f"Root directory = {str(config.root_path)}")
 
@@ -82,7 +72,6 @@ class ApiDependencies:
         image_files = DiskImageFileStorage(f"{output_folder}/images")
 
         model_images_folder = config.models_path
-        style_presets_folder = config.style_presets_path
 
         db = init_db(config=config, logger=logger, image_files=image_files)
 
@@ -93,7 +82,7 @@ class ApiDependencies:
         board_images = BoardImagesService()
         board_records = SqliteBoardRecordStorage(db=db)
         boards = BoardService()
-        events = FastAPIEventService(event_handler_id, loop=loop)
+        events = FastAPIEventService(event_handler_id)
         bulk_download = BulkDownloadService()
         image_records = SqliteImageRecordStorage(db=db)
         images = ImageService()
@@ -108,7 +97,7 @@ class ApiDependencies:
         model_images_service = ModelImageFileStorageDisk(model_images_folder / "model_images")
         model_manager = ModelManagerService.build_model_manager(
             app_config=configuration,
-            model_record_service=ModelRecordServiceSQL(db=db, logger=logger),
+            model_record_service=ModelRecordServiceSQL(db=db),
             download_queue=download_queue_service,
             events=events,
         )
@@ -118,8 +107,6 @@ class ApiDependencies:
         session_queue = SqliteSessionQueue(db=db)
         urls = LocalUrlService()
         workflow_records = SqliteWorkflowRecordsStorage(db=db)
-        style_preset_records = SqliteStylePresetRecordsStorage(db=db)
-        style_preset_image_files = StylePresetImageFileStorageDisk(style_presets_folder / "images")
 
         services = InvocationServices(
             board_image_records=board_image_records,
@@ -145,8 +132,6 @@ class ApiDependencies:
             workflow_records=workflow_records,
             tensors=tensors,
             conditioning=conditioning,
-            style_preset_records=style_preset_records,
-            style_preset_image_files=style_preset_image_files,
         )
 
         ApiDependencies.invoker = Invoker(services)

invokeai/app/api/routers/app_info.py

@@ -10,13 +10,14 @@ from fastapi import Body
 from fastapi.routing import APIRouter
 from pydantic import BaseModel, Field
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.invocations.upscale import ESRGAN_MODELS
 from invokeai.app.services.invocation_cache.invocation_cache_common import InvocationCacheStatus
 from invokeai.backend.image_util.infill_methods.patchmatch import PatchMatch
 from invokeai.backend.util.logging import logging
 from invokeai.version import __version__
 
+from ..dependencies import ApiDependencies
+
 
 class LogLevel(int, Enum):
     NotSet = logging.NOTSET

invokeai/app/api/routers/board_images.py

@@ -2,7 +2,7 @@ from fastapi import Body, HTTPException
 from fastapi.routing import APIRouter
 from pydantic import BaseModel, Field
 
-from invokeai.app.api.dependencies import ApiDependencies
+from ..dependencies import ApiDependencies
 
 board_images_router = APIRouter(prefix="/v1/board_images", tags=["boards"])
 

invokeai/app/api/routers/boards.py

@@ -4,11 +4,12 @@ from fastapi import Body, HTTPException, Path, Query
 from fastapi.routing import APIRouter
 from pydantic import BaseModel, Field
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.services.board_records.board_records_common import BoardChanges
 from invokeai.app.services.boards.boards_common import BoardDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 
+from ..dependencies import ApiDependencies
+
 boards_router = APIRouter(prefix="/v1/boards", tags=["boards"])
 
 
@@ -31,7 +32,6 @@ class DeleteBoardResult(BaseModel):
 )
 async def create_board(
     board_name: str = Query(description="The name of the board to create"),
-    is_private: bool = Query(default=False, description="Whether the board is private"),
 ) -> BoardDTO:
     """Creates a board"""
     try:
@@ -118,13 +118,15 @@ async def list_boards(
     all: Optional[bool] = Query(default=None, description="Whether to list all boards"),
     offset: Optional[int] = Query(default=None, description="The page offset"),
     limit: Optional[int] = Query(default=None, description="The number of boards per page"),
-    include_archived: bool = Query(default=False, description="Whether or not to include archived boards in list"),
 ) -> Union[OffsetPaginatedResults[BoardDTO], list[BoardDTO]]:
     """Gets a list of boards"""
     if all:
-        return ApiDependencies.invoker.services.boards.get_all(include_archived)
+        return ApiDependencies.invoker.services.boards.get_all()
     elif offset is not None and limit is not None:
-        return ApiDependencies.invoker.services.boards.get_many(offset, limit, include_archived)
+        return ApiDependencies.invoker.services.boards.get_many(
+            offset,
+            limit,
+        )
     else:
         raise HTTPException(
             status_code=400,

invokeai/app/api/routers/download_queue.py

@@ -8,12 +8,13 @@ from fastapi.routing import APIRouter
 from pydantic.networks import AnyHttpUrl
 from starlette.exceptions import HTTPException
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.services.download import (
     DownloadJob,
     UnknownJobIDException,
 )
 
+from ..dependencies import ApiDependencies
+
 download_queue_router = APIRouter(prefix="/v1/download_queue", tags=["download_queue"])
 
 

invokeai/app/api/routers/images.py

@@ -8,16 +8,12 @@ from fastapi.routing import APIRouter
 from PIL import Image
 from pydantic import BaseModel, Field, JsonValue
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.invocations.fields import MetadataField
-from invokeai.app.services.image_records.image_records_common import (
-    ImageCategory,
-    ImageRecordChanges,
-    ResourceOrigin,
-)
+from invokeai.app.services.image_records.image_records_common import ImageCategory, ImageRecordChanges, ResourceOrigin
 from invokeai.app.services.images.images_common import ImageDTO, ImageUrlsDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
+
+from ..dependencies import ApiDependencies
 
 images_router = APIRouter(prefix="/v1/images", tags=["images"])
 
@@ -218,8 +214,9 @@ async def get_image_workflow(
         raise HTTPException(status_code=404)
 
 
-@images_router.get(
+@images_router.api_route(
     "/i/{image_name}/full",
+    methods=["GET", "HEAD"],
     operation_id="get_image_full",
     response_class=Response,
     responses={
@@ -230,30 +227,24 @@ async def get_image_workflow(
         404: {"description": "Image not found"},
     },
 )
-@images_router.head(
-    "/i/{image_name}/full",
-    operation_id="get_image_full_head",
-    response_class=Response,
-    responses={
-        200: {
-            "description": "Return the full-resolution image",
-            "content": {"image/png": {}},
-        },
-        404: {"description": "Image not found"},
-    },
-)
 async def get_image_full(
     image_name: str = Path(description="The name of full-resolution image file to get"),
-) -> Response:
+) -> FileResponse:
     """Gets a full-resolution image file"""
 
     try:
         path = ApiDependencies.invoker.services.images.get_path(image_name)
-        with open(path, "rb") as f:
-            content = f.read()
-        response = Response(content, media_type="image/png")
+
+        if not ApiDependencies.invoker.services.images.validate_path(path):
+            raise HTTPException(status_code=404)
+
+        response = FileResponse(
+            path,
+            media_type="image/png",
+            filename=image_name,
+            content_disposition_type="inline",
+        )
         response.headers["Cache-Control"] = f"max-age={IMAGE_MAX_AGE}"
-        response.headers["Content-Disposition"] = f'inline; filename="{image_name}"'
         return response
     except Exception:
         raise HTTPException(status_code=404)
@@ -273,14 +264,15 @@ async def get_image_full(
 )
 async def get_image_thumbnail(
     image_name: str = Path(description="The name of thumbnail image file to get"),
-) -> Response:
+) -> FileResponse:
     """Gets a thumbnail image file"""
 
     try:
         path = ApiDependencies.invoker.services.images.get_path(image_name, thumbnail=True)
-        with open(path, "rb") as f:
-            content = f.read()
-        response = Response(content, media_type="image/webp")
+        if not ApiDependencies.invoker.services.images.validate_path(path):
+            raise HTTPException(status_code=404)
+
+        response = FileResponse(path, media_type="image/webp", content_disposition_type="inline")
         response.headers["Cache-Control"] = f"max-age={IMAGE_MAX_AGE}"
         return response
     except Exception:
@@ -324,14 +316,16 @@ async def list_image_dtos(
     ),
     offset: int = Query(default=0, description="The page offset"),
     limit: int = Query(default=10, description="The number of images per page"),
-    order_dir: SQLiteDirection = Query(default=SQLiteDirection.Descending, description="The order of sort"),
-    starred_first: bool = Query(default=True, description="Whether to sort by starred images first"),
-    search_term: Optional[str] = Query(default=None, description="The term to search for"),
 ) -> OffsetPaginatedResults[ImageDTO]:
     """Gets a list of image DTOs"""
 
     image_dtos = ApiDependencies.invoker.services.images.get_many(
-        offset, limit, starred_first, order_dir, image_origin, categories, is_intermediate, board_id, search_term
+        offset,
+        limit,
+        image_origin,
+        categories,
+        is_intermediate,
+        board_id,
     )
 
     return image_dtos

invokeai/app/api/routers/model_manager.py

@@ -3,10 +3,10 @@
 
 import io
 import pathlib
+import shutil
 import traceback
 from copy import deepcopy
-from tempfile import TemporaryDirectory
-from typing import List, Optional, Type
+from typing import Any, Dict, List, Optional, Type
 
 from fastapi import Body, Path, Query, Response, UploadFile
 from fastapi.responses import FileResponse, HTMLResponse
@@ -16,10 +16,10 @@ from pydantic import AnyHttpUrl, BaseModel, ConfigDict, Field
 from starlette.exceptions import HTTPException
 from typing_extensions import Annotated
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.services.model_images.model_images_common import ModelImageFileNotFoundException
 from invokeai.app.services.model_install.model_install_common import ModelInstallJob
 from invokeai.app.services.model_records import (
+    DuplicateModelException,
     InvalidModelException,
     ModelRecordChanges,
     UnknownModelException,
@@ -30,12 +30,15 @@ from invokeai.backend.model_manager.config import (
     MainCheckpointConfig,
     ModelFormat,
     ModelType,
+    SubModelType,
 )
 from invokeai.backend.model_manager.metadata.fetch.huggingface import HuggingFaceMetadataFetch
 from invokeai.backend.model_manager.metadata.metadata_base import ModelMetadataWithFiles, UnknownMetadataException
 from invokeai.backend.model_manager.search import ModelSearch
 from invokeai.backend.model_manager.starter_models import STARTER_MODELS, StarterModel, StarterModelWithoutDependencies
 
+from ..dependencies import ApiDependencies
+
 model_manager_router = APIRouter(prefix="/v2/models", tags=["model_manager"])
 
 # images are immutable; set a high max-age
@@ -171,6 +174,18 @@ async def get_model_record(
         raise HTTPException(status_code=404, detail=str(e))
 
 
+# @model_manager_router.get("/summary", operation_id="list_model_summary")
+# async def list_model_summary(
+#     page: int = Query(default=0, description="The page to get"),
+#     per_page: int = Query(default=10, description="The number of models per page"),
+#     order_by: ModelRecordOrderBy = Query(default=ModelRecordOrderBy.Default, description="The attribute to order by"),
+# ) -> PaginatedResults[ModelSummary]:
+#     """Gets a page of model summary data."""
+#     record_store = ApiDependencies.invoker.services.model_manager.store
+#     results: PaginatedResults[ModelSummary] = record_store.list_models(page=page, per_page=per_page, order_by=order_by)
+#     return results
+
+
 class FoundModel(BaseModel):
     path: str = Field(description="Path to the model")
     is_installed: bool = Field(description="Whether or not the model is already installed")
@@ -430,11 +445,13 @@ async def delete_model_image(
 async def install_model(
     source: str = Query(description="Model source to install, can be a local path, repo_id, or remote URL"),
     inplace: Optional[bool] = Query(description="Whether or not to install a local model in place", default=False),
-    access_token: Optional[str] = Query(description="access token for the remote resource", default=None),
-    config: ModelRecordChanges = Body(
-        description="Object containing fields that override auto-probed values in the model config record, such as name, description and prediction_type ",
+    # TODO(MM2): Can we type this?
+    config: Optional[Dict[str, Any]] = Body(
+        description="Dict of fields that override auto-probed values in the model config record, such as name, description and prediction_type ",
+        default=None,
         example={"name": "string", "description": "string"},
     ),
+    access_token: Optional[str] = None,
 ) -> ModelInstallJob:
     """Install a model using a string identifier.
 
@@ -449,9 +466,8 @@ async def install_model(
        - model/name:fp16:path/to/model.safetensors
        - model/name::path/to/model.safetensors
 
-    `config` is a ModelRecordChanges object. Fields in this object will override
-    the ones that are probed automatically. Pass an empty object to accept
-    all the defaults.
+    `config` is an optional dict containing model configuration values that will override
+    the ones that are probed automatically.
 
     `access_token` is an optional access token for use with Urls that require
     authentication.
@@ -730,36 +746,39 @@ async def convert_model(
         logger.error(f"The model with key {key} is not a main checkpoint model.")
         raise HTTPException(400, f"The model with key {key} is not a main checkpoint model.")
 
-    with TemporaryDirectory(dir=ApiDependencies.invoker.services.configuration.models_path) as tmpdir:
-        convert_path = pathlib.Path(tmpdir) / pathlib.Path(model_config.path).stem
-        converted_model = loader.load_model(model_config)
-        # write the converted file to the convert path
-        raw_model = converted_model.model
-        assert hasattr(raw_model, "save_pretrained")
-        raw_model.save_pretrained(convert_path)  # type: ignore
-        assert convert_path.exists()
+    # loading the model will convert it into a cached diffusers file
+    try:
+        cc_size = loader.convert_cache.max_size
+        if cc_size == 0:  # temporary set the convert cache to a positive number so that cached model is written
+            loader._convert_cache.max_size = 1.0
+        loader.load_model(model_config, submodel_type=SubModelType.Scheduler)
+    finally:
+        loader._convert_cache.max_size = cc_size
 
-        # temporarily rename the original safetensors file so that there is no naming conflict
-        original_name = model_config.name
-        model_config.name = f"{original_name}.DELETE"
-        changes = ModelRecordChanges(name=model_config.name)
-        store.update_model(key, changes=changes)
+    # Get the path of the converted model from the loader
+    cache_path = loader.convert_cache.cache_path(key)
+    assert cache_path.exists()
 
-        # install the diffusers
-        try:
-            new_key = installer.install_path(
-                convert_path,
-                config=ModelRecordChanges(
-                    name=original_name,
-                    description=model_config.description,
-                    hash=model_config.hash,
-                    source=model_config.source,
-                ),
-            )
-        except Exception as e:
-            logger.error(str(e))
-            store.update_model(key, changes=ModelRecordChanges(name=original_name))
-            raise HTTPException(status_code=409, detail=str(e))
+    # temporarily rename the original safetensors file so that there is no naming conflict
+    original_name = model_config.name
+    model_config.name = f"{original_name}.DELETE"
+    changes = ModelRecordChanges(name=model_config.name)
+    store.update_model(key, changes=changes)
+
+    # install the diffusers
+    try:
+        new_key = installer.install_path(
+            cache_path,
+            config={
+                "name": original_name,
+                "description": model_config.description,
+                "hash": model_config.hash,
+                "source": model_config.source,
+            },
+        )
+    except DuplicateModelException as e:
+        logger.error(str(e))
+        raise HTTPException(status_code=409, detail=str(e))
 
     # Update the model image if the model had one
     try:
@@ -772,8 +791,8 @@ async def convert_model(
     # delete the original safetensors file
     installer.delete(key)
 
-    # delete the temporary directory
-    # shutil.rmtree(cache_path)
+    # delete the cached version
+    shutil.rmtree(cache_path)
 
     # return the config record for the new diffusers directory
     new_config = store.get_model(new_key)

invokeai/app/api/routers/session_queue.py

@@ -4,7 +4,6 @@ from fastapi import Body, Path, Query
 from fastapi.routing import APIRouter
 from pydantic import BaseModel
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.services.session_processor.session_processor_common import SessionProcessorStatus
 from invokeai.app.services.session_queue.session_queue_common import (
     QUEUE_ITEM_STATUS,
@@ -20,6 +19,8 @@ from invokeai.app.services.session_queue.session_queue_common import (
 )
 from invokeai.app.services.shared.pagination import CursorPaginatedResults
 
+from ..dependencies import ApiDependencies
+
 session_queue_router = APIRouter(prefix="/v1/queue", tags=["queue"])
 
 

invokeai/app/api/routers/style_presets.py

@@ -1,276 +0,0 @@
-import csv
-import io
-import json
-import traceback
-from typing import Optional
-
-import pydantic
-from fastapi import APIRouter, File, Form, HTTPException, Path, Response, UploadFile
-from fastapi.responses import FileResponse
-from PIL import Image
-from pydantic import BaseModel, Field
-
-from invokeai.app.api.dependencies import ApiDependencies
-from invokeai.app.api.routers.model_manager import IMAGE_MAX_AGE
-from invokeai.app.services.style_preset_images.style_preset_images_common import StylePresetImageFileNotFoundException
-from invokeai.app.services.style_preset_records.style_preset_records_common import (
-    InvalidPresetImportDataError,
-    PresetData,
-    PresetType,
-    StylePresetChanges,
-    StylePresetNotFoundError,
-    StylePresetRecordWithImage,
-    StylePresetWithoutId,
-    UnsupportedFileTypeError,
-    parse_presets_from_file,
-)
-
-
-class StylePresetUpdateFormData(BaseModel):
-    name: str = Field(description="Preset name")
-    positive_prompt: str = Field(description="Positive prompt")
-    negative_prompt: str = Field(description="Negative prompt")
-
-
-class StylePresetCreateFormData(StylePresetUpdateFormData):
-    type: PresetType = Field(description="Preset type")
-
-
-style_presets_router = APIRouter(prefix="/v1/style_presets", tags=["style_presets"])
-
-
-@style_presets_router.get(
-    "/i/{style_preset_id}",
-    operation_id="get_style_preset",
-    responses={
-        200: {"model": StylePresetRecordWithImage},
-    },
-)
-async def get_style_preset(
-    style_preset_id: str = Path(description="The style preset to get"),
-) -> StylePresetRecordWithImage:
-    """Gets a style preset"""
-    try:
-        image = ApiDependencies.invoker.services.style_preset_image_files.get_url(style_preset_id)
-        style_preset = ApiDependencies.invoker.services.style_preset_records.get(style_preset_id)
-        return StylePresetRecordWithImage(image=image, **style_preset.model_dump())
-    except StylePresetNotFoundError:
-        raise HTTPException(status_code=404, detail="Style preset not found")
-
-
-@style_presets_router.patch(
-    "/i/{style_preset_id}",
-    operation_id="update_style_preset",
-    responses={
-        200: {"model": StylePresetRecordWithImage},
-    },
-)
-async def update_style_preset(
-    image: Optional[UploadFile] = File(description="The image file to upload", default=None),
-    style_preset_id: str = Path(description="The id of the style preset to update"),
-    data: str = Form(description="The data of the style preset to update"),
-) -> StylePresetRecordWithImage:
-    """Updates a style preset"""
-    if image is not None:
-        if not image.content_type or not image.content_type.startswith("image"):
-            raise HTTPException(status_code=415, detail="Not an image")
-
-        contents = await image.read()
-        try:
-            pil_image = Image.open(io.BytesIO(contents))
-
-        except Exception:
-            ApiDependencies.invoker.services.logger.error(traceback.format_exc())
-            raise HTTPException(status_code=415, detail="Failed to read image")
-
-        try:
-            ApiDependencies.invoker.services.style_preset_image_files.save(style_preset_id, pil_image)
-        except ValueError as e:
-            raise HTTPException(status_code=409, detail=str(e))
-    else:
-        try:
-            ApiDependencies.invoker.services.style_preset_image_files.delete(style_preset_id)
-        except StylePresetImageFileNotFoundException:
-            pass
-
-    try:
-        parsed_data = json.loads(data)
-        validated_data = StylePresetUpdateFormData(**parsed_data)
-
-        name = validated_data.name
-        positive_prompt = validated_data.positive_prompt
-        negative_prompt = validated_data.negative_prompt
-
-    except pydantic.ValidationError:
-        raise HTTPException(status_code=400, detail="Invalid preset data")
-
-    preset_data = PresetData(positive_prompt=positive_prompt, negative_prompt=negative_prompt)
-    changes = StylePresetChanges(name=name, preset_data=preset_data)
-
-    style_preset_image = ApiDependencies.invoker.services.style_preset_image_files.get_url(style_preset_id)
-    style_preset = ApiDependencies.invoker.services.style_preset_records.update(
-        style_preset_id=style_preset_id, changes=changes
-    )
-    return StylePresetRecordWithImage(image=style_preset_image, **style_preset.model_dump())
-
-
-@style_presets_router.delete(
-    "/i/{style_preset_id}",
-    operation_id="delete_style_preset",
-)
-async def delete_style_preset(
-    style_preset_id: str = Path(description="The style preset to delete"),
-) -> None:
-    """Deletes a style preset"""
-    try:
-        ApiDependencies.invoker.services.style_preset_image_files.delete(style_preset_id)
-    except StylePresetImageFileNotFoundException:
-        pass
-
-    ApiDependencies.invoker.services.style_preset_records.delete(style_preset_id)
-
-
-@style_presets_router.post(
-    "/",
-    operation_id="create_style_preset",
-    responses={
-        200: {"model": StylePresetRecordWithImage},
-    },
-)
-async def create_style_preset(
-    image: Optional[UploadFile] = File(description="The image file to upload", default=None),
-    data: str = Form(description="The data of the style preset to create"),
-) -> StylePresetRecordWithImage:
-    """Creates a style preset"""
-
-    try:
-        parsed_data = json.loads(data)
-        validated_data = StylePresetCreateFormData(**parsed_data)
-
-        name = validated_data.name
-        type = validated_data.type
-        positive_prompt = validated_data.positive_prompt
-        negative_prompt = validated_data.negative_prompt
-
-    except pydantic.ValidationError:
-        raise HTTPException(status_code=400, detail="Invalid preset data")
-
-    preset_data = PresetData(positive_prompt=positive_prompt, negative_prompt=negative_prompt)
-    style_preset = StylePresetWithoutId(name=name, preset_data=preset_data, type=type)
-    new_style_preset = ApiDependencies.invoker.services.style_preset_records.create(style_preset=style_preset)
-
-    if image is not None:
-        if not image.content_type or not image.content_type.startswith("image"):
-            raise HTTPException(status_code=415, detail="Not an image")
-
-        contents = await image.read()
-        try:
-            pil_image = Image.open(io.BytesIO(contents))
-
-        except Exception:
-            ApiDependencies.invoker.services.logger.error(traceback.format_exc())
-            raise HTTPException(status_code=415, detail="Failed to read image")
-
-        try:
-            ApiDependencies.invoker.services.style_preset_image_files.save(new_style_preset.id, pil_image)
-        except ValueError as e:
-            raise HTTPException(status_code=409, detail=str(e))
-
-    preset_image = ApiDependencies.invoker.services.style_preset_image_files.get_url(new_style_preset.id)
-    return StylePresetRecordWithImage(image=preset_image, **new_style_preset.model_dump())
-
-
-@style_presets_router.get(
-    "/",
-    operation_id="list_style_presets",
-    responses={
-        200: {"model": list[StylePresetRecordWithImage]},
-    },
-)
-async def list_style_presets() -> list[StylePresetRecordWithImage]:
-    """Gets a page of style presets"""
-    style_presets_with_image: list[StylePresetRecordWithImage] = []
-    style_presets = ApiDependencies.invoker.services.style_preset_records.get_many()
-    for preset in style_presets:
-        image = ApiDependencies.invoker.services.style_preset_image_files.get_url(preset.id)
-        style_preset_with_image = StylePresetRecordWithImage(image=image, **preset.model_dump())
-        style_presets_with_image.append(style_preset_with_image)
-
-    return style_presets_with_image
-
-
-@style_presets_router.get(
-    "/i/{style_preset_id}/image",
-    operation_id="get_style_preset_image",
-    responses={
-        200: {
-            "description": "The style preset image was fetched successfully",
-        },
-        400: {"description": "Bad request"},
-        404: {"description": "The style preset image could not be found"},
-    },
-    status_code=200,
-)
-async def get_style_preset_image(
-    style_preset_id: str = Path(description="The id of the style preset image to get"),
-) -> FileResponse:
-    """Gets an image file that previews the model"""
-
-    try:
-        path = ApiDependencies.invoker.services.style_preset_image_files.get_path(style_preset_id)
-
-        response = FileResponse(
-            path,
-            media_type="image/png",
-            filename=style_preset_id + ".png",
-            content_disposition_type="inline",
-        )
-        response.headers["Cache-Control"] = f"max-age={IMAGE_MAX_AGE}"
-        return response
-    except Exception:
-        raise HTTPException(status_code=404)
-
-
-@style_presets_router.get(
-    "/export",
-    operation_id="export_style_presets",
-    responses={200: {"content": {"text/csv": {}}, "description": "A CSV file with the requested data."}},
-    status_code=200,
-)
-async def export_style_presets():
-    # Create an in-memory stream to store the CSV data
-    output = io.StringIO()
-    writer = csv.writer(output)
-
-    # Write the header
-    writer.writerow(["name", "prompt", "negative_prompt"])
-
-    style_presets = ApiDependencies.invoker.services.style_preset_records.get_many(type=PresetType.User)
-
-    for preset in style_presets:
-        writer.writerow([preset.name, preset.preset_data.positive_prompt, preset.preset_data.negative_prompt])
-
-    csv_data = output.getvalue()
-    output.close()
-
-    return Response(
-        content=csv_data,
-        media_type="text/csv",
-        headers={"Content-Disposition": "attachment; filename=prompt_templates.csv"},
-    )
-
-
-@style_presets_router.post(
-    "/import",
-    operation_id="import_style_presets",
-)
-async def import_style_presets(file: UploadFile = File(description="The file to import")):
-    try:
-        style_presets = await parse_presets_from_file(file)
-        ApiDependencies.invoker.services.style_preset_records.create_many(style_presets)
-    except InvalidPresetImportDataError as e:
-        ApiDependencies.invoker.services.logger.error(traceback.format_exc())
-        raise HTTPException(status_code=400, detail=str(e))
-    except UnsupportedFileTypeError as e:
-        ApiDependencies.invoker.services.logger.error(traceback.format_exc())
-        raise HTTPException(status_code=415, detail=str(e))

invokeai/app/api_app.py

@@ -20,9 +20,14 @@ from torch.backends.mps import is_available as is_mps_available
 # noinspection PyUnresolvedReferences
 import invokeai.backend.util.hotfixes  # noqa: F401 (monkeypatching on import)
 import invokeai.frontend.web as web_dir
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.api.no_cache_staticfiles import NoCacheStaticFiles
-from invokeai.app.api.routers import (
+from invokeai.app.services.config.config_default import get_config
+from invokeai.app.util.custom_openapi import get_openapi_func
+from invokeai.backend.util.devices import TorchDevice
+
+from ..backend.util.logging import InvokeAILogger
+from .api.dependencies import ApiDependencies
+from .api.routers import (
     app_info,
     board_images,
     boards,
@@ -30,15 +35,10 @@ from invokeai.app.api.routers import (
     images,
     model_manager,
     session_queue,
-    style_presets,
     utilities,
     workflows,
 )
-from invokeai.app.api.sockets import SocketIO
-from invokeai.app.services.config.config_default import get_config
-from invokeai.app.util.custom_openapi import get_openapi_func
-from invokeai.backend.util.devices import TorchDevice
-from invokeai.backend.util.logging import InvokeAILogger
+from .api.sockets import SocketIO
 
 app_config = get_config()
 
@@ -56,13 +56,11 @@ mimetypes.add_type("text/css", ".css")
 torch_device_name = TorchDevice.get_torch_device_name()
 logger.info(f"Using torch device: {torch_device_name}")
 
-loop = asyncio.new_event_loop()
-
 
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     # Add startup event to load dependencies
-    ApiDependencies.initialize(config=app_config, event_handler_id=event_handler_id, loop=loop, logger=logger)
+    ApiDependencies.initialize(config=app_config, event_handler_id=event_handler_id, logger=logger)
     yield
     # Shut down threads
     ApiDependencies.shutdown()
@@ -109,7 +107,6 @@ app.include_router(board_images.board_images_router, prefix="/api")
 app.include_router(app_info.app_router, prefix="/api")
 app.include_router(session_queue.session_queue_router, prefix="/api")
 app.include_router(workflows.workflows_router, prefix="/api")
-app.include_router(style_presets.style_presets_router, prefix="/api")
 
 app.openapi = get_openapi_func(app)
 
@@ -165,7 +162,6 @@ def invoke_api() -> None:
         # Taken from https://waylonwalker.com/python-find-available-port/, thanks Waylon!
         # https://github.com/WaylonWalker
         with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
-            s.settimeout(1)
             if s.connect_ex(("localhost", port)) == 0:
                 return find_port(port=port + 1)
             else:
@@ -188,6 +184,8 @@ def invoke_api() -> None:
 
     check_cudnn(logger)
 
+    # Start our own event loop for eventing usage
+    loop = asyncio.new_event_loop()
     config = uvicorn.Config(
         app=app,
         host=app_config.host,

invokeai/app/invocations/baseinvocation.py

@@ -40,7 +40,7 @@ from invokeai.app.util.misc import uuid_string
 from invokeai.backend.util.logging import InvokeAILogger
 
 if TYPE_CHECKING:
-    from invokeai.app.services.invocation_services import InvocationServices
+    from ..services.invocation_services import InvocationServices
 
 logger = InvokeAILogger.get_logger()
 

invokeai/app/invocations/collections.py

@@ -4,12 +4,13 @@
 import numpy as np
 from pydantic import ValidationInfo, field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import InputField
 from invokeai.app.invocations.primitives import IntegerCollectionOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.misc import SEED_MAX
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField
+
 
 @invocation(
     "range", title="Integer Range", tags=["collection", "integer", "range"], category="collections", version="1.0.0"

invokeai/app/invocations/compel.py

@@ -5,7 +5,6 @@ from compel import Compel, ReturnedEmbeddingsType
 from compel.prompt_parser import Blend, Conjunction, CrossAttentionControlSubstitute, FlattenedPrompt, Fragment
 from transformers import CLIPTextModel, CLIPTextModelWithProjection, CLIPTokenizer
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
 from invokeai.app.invocations.fields import (
     ConditioningField,
     FieldDescriptions,
@@ -15,7 +14,6 @@ from invokeai.app.invocations.fields import (
     TensorField,
     UIComponent,
 )
-from invokeai.app.invocations.model import CLIPField
 from invokeai.app.invocations.primitives import ConditioningOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.ti_utils import generate_ti_list
@@ -28,6 +26,9 @@ from invokeai.backend.stable_diffusion.diffusion.conditioning_data import (
 )
 from invokeai.backend.util.devices import TorchDevice
 
+from .baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
+from .model import CLIPField
+
 # unconditioned: Optional[torch.Tensor]
 
 
@@ -80,12 +81,12 @@ class CompelInvocation(BaseInvocation):
 
         with (
             # apply all patches while the model is on the target device
-            text_encoder_info.model_on_device() as (cached_weights, text_encoder),
+            text_encoder_info.model_on_device() as (model_state_dict, text_encoder),
             tokenizer_info as tokenizer,
             ModelPatcher.apply_lora_text_encoder(
                 text_encoder,
                 loras=_lora_loader(),
-                cached_weights=cached_weights,
+                model_state_dict=model_state_dict,
             ),
             # Apply CLIP Skip after LoRA to prevent LoRA application from failing on skipped layers.
             ModelPatcher.apply_clip_skip(text_encoder, self.clip.skipped_layers),
@@ -175,13 +176,13 @@ class SDXLPromptInvocationBase:
 
         with (
             # apply all patches while the model is on the target device
-            text_encoder_info.model_on_device() as (cached_weights, text_encoder),
+            text_encoder_info.model_on_device() as (state_dict, text_encoder),
             tokenizer_info as tokenizer,
             ModelPatcher.apply_lora(
                 text_encoder,
                 loras=_lora_loader(),
                 prefix=lora_prefix,
-                cached_weights=cached_weights,
+                model_state_dict=state_dict,
             ),
             # Apply CLIP Skip after LoRA to prevent LoRA application from failing on skipped layers.
             ModelPatcher.apply_clip_skip(text_encoder, clip_field.skipped_layers),

invokeai/app/invocations/constants.py

@@ -1,5 +1,6 @@
 from typing import Literal
 
+from invokeai.backend.stable_diffusion.schedulers import SCHEDULER_MAP
 from invokeai.backend.util.devices import TorchDevice
 
 LATENT_SCALE_FACTOR = 8
@@ -10,6 +11,9 @@ factor is hard-coded to a literal '8' rather than using this constant.
 The ratio of image:latent dimensions is LATENT_SCALE_FACTOR:1, or 8:1.
 """
 
+SCHEDULER_NAME_VALUES = Literal[tuple(SCHEDULER_MAP.keys())]
+"""A literal type representing the valid scheduler names."""
+
 IMAGE_MODES = Literal["L", "RGB", "RGBA", "CMYK", "YCbCr", "LAB", "HSV", "I", "F"]
 """A literal type for PIL image modes supported by Invoke"""
 

invokeai/app/invocations/controlnet_image_processors.py

@@ -21,16 +21,7 @@ from controlnet_aux import (
 from controlnet_aux.util import HWC3, ade_palette
 from PIL import Image
 from pydantic import BaseModel, Field, field_validator, model_validator
-from transformers import pipeline
-from transformers.pipelines import DepthEstimationPipeline
 
-from invokeai.app.invocations.baseinvocation import (
-    BaseInvocation,
-    BaseInvocationOutput,
-    Classification,
-    invocation,
-    invocation_output,
-)
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     ImageField,
@@ -46,12 +37,15 @@ from invokeai.app.invocations.util import validate_begin_end_step, validate_weig
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES, heuristic_resize
 from invokeai.backend.image_util.canny import get_canny_edges
-from invokeai.backend.image_util.depth_anything.depth_anything_pipeline import DepthAnythingPipeline
+from invokeai.backend.image_util.depth_anything import DEPTH_ANYTHING_MODELS, DepthAnythingDetector
 from invokeai.backend.image_util.dw_openpose import DWPOSE_MODELS, DWOpenposeDetector
 from invokeai.backend.image_util.hed import HEDProcessor
 from invokeai.backend.image_util.lineart import LineartProcessor
 from invokeai.backend.image_util.lineart_anime import LineartAnimeProcessor
 from invokeai.backend.image_util.util import np_to_pil, pil_to_np
+from invokeai.backend.util.devices import TorchDevice
+
+from .baseinvocation import BaseInvocation, BaseInvocationOutput, Classification, invocation, invocation_output
 
 
 class ControlField(BaseModel):
@@ -593,14 +587,7 @@ class ColorMapImageProcessorInvocation(ImageProcessorInvocation):
         return color_map
 
 
-DEPTH_ANYTHING_MODEL_SIZES = Literal["large", "base", "small", "small_v2"]
-# DepthAnything V2 Small model is licensed under Apache 2.0 but not the base and large models.
-DEPTH_ANYTHING_MODELS = {
-    "large": "LiheYoung/depth-anything-large-hf",
-    "base": "LiheYoung/depth-anything-base-hf",
-    "small": "LiheYoung/depth-anything-small-hf",
-    "small_v2": "depth-anything/Depth-Anything-V2-Small-hf",
-}
+DEPTH_ANYTHING_MODEL_SIZES = Literal["large", "base", "small"]
 
 
 @invocation(
@@ -608,33 +595,28 @@ DEPTH_ANYTHING_MODELS = {
     title="Depth Anything Processor",
     tags=["controlnet", "depth", "depth anything"],
     category="controlnet",
-    version="1.1.3",
+    version="1.1.2",
 )
 class DepthAnythingImageProcessorInvocation(ImageProcessorInvocation):
     """Generates a depth map based on the Depth Anything algorithm"""
 
     model_size: DEPTH_ANYTHING_MODEL_SIZES = InputField(
-        default="small_v2", description="The size of the depth model to use"
+        default="small", description="The size of the depth model to use"
     )
     resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
     def run_processor(self, image: Image.Image) -> Image.Image:
-        def load_depth_anything(model_path: Path):
-            depth_anything_pipeline = pipeline(model=str(model_path), task="depth-estimation", local_files_only=True)
-            assert isinstance(depth_anything_pipeline, DepthEstimationPipeline)
-            return DepthAnythingPipeline(depth_anything_pipeline)
+        def loader(model_path: Path):
+            return DepthAnythingDetector.load_model(
+                model_path, model_size=self.model_size, device=TorchDevice.choose_torch_device()
+            )
 
         with self._context.models.load_remote_model(
-            source=DEPTH_ANYTHING_MODELS[self.model_size], loader=load_depth_anything
-        ) as depth_anything_detector:
-            assert isinstance(depth_anything_detector, DepthAnythingPipeline)
-            depth_map = depth_anything_detector.generate_depth(image)
-
-            # Resizing to user target specified size
-            new_height = int(image.size[1] * (self.resolution / image.size[0]))
-            depth_map = depth_map.resize((self.resolution, new_height))
-
-            return depth_map
+            source=DEPTH_ANYTHING_MODELS[self.model_size], loader=loader
+        ) as model:
+            depth_anything_detector = DepthAnythingDetector(model, TorchDevice.choose_torch_device())
+            processed_image = depth_anything_detector(image=image, resolution=self.resolution)
+            return processed_image
 
 
 @invocation(

invokeai/app/invocations/create_gradient_mask.py

@@ -39,7 +39,7 @@ class GradientMaskOutput(BaseInvocationOutput):
     title="Create Gradient Mask",
     tags=["mask", "denoise"],
     category="latents",
-    version="1.2.0",
+    version="1.1.0",
 )
 class CreateGradientMaskInvocation(BaseInvocation):
     """Creates mask for denoising model run."""
@@ -93,7 +93,6 @@ class CreateGradientMaskInvocation(BaseInvocation):
 
             # redistribute blur so that the original edges are 0 and blur outwards to 1
             blur_tensor = (blur_tensor - 0.5) * 2
-            blur_tensor[blur_tensor < 0] = 0.0
 
             threshold = 1 - self.minimum_denoise
 

invokeai/app/invocations/cv.py

@@ -5,11 +5,13 @@ import cv2 as cv
 import numpy
 from PIL import Image, ImageOps
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import ImageField, InputField, WithBoard, WithMetadata
+from invokeai.app.invocations.fields import ImageField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField, WithBoard, WithMetadata
+
 
 @invocation("cv_inpaint", title="OpenCV Inpaint", tags=["opencv", "inpaint"], category="inpaint", version="1.3.1")
 class CvInpaintInvocation(BaseInvocation, WithMetadata, WithBoard):

invokeai/app/invocations/denoise_latents.py

@@ -1,6 +1,5 @@
 # Copyright (c) 2023 Kyle Schouviller (https://github.com/kyle0654)
 import inspect
-import os
 from contextlib import ExitStack
 from typing import Any, Dict, Iterator, List, Optional, Tuple, Union
 
@@ -18,7 +17,7 @@ from torchvision.transforms.functional import resize as tv_resize
 from transformers import CLIPVisionModelWithProjection
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
+from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR, SCHEDULER_NAME_VALUES
 from invokeai.app.invocations.controlnet_image_processors import ControlField
 from invokeai.app.invocations.fields import (
     ConditioningField,
@@ -37,10 +36,9 @@ from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.controlnet_utils import prepare_control_image
 from invokeai.backend.ip_adapter.ip_adapter import IPAdapter
 from invokeai.backend.lora import LoRAModelRaw
-from invokeai.backend.model_manager import BaseModelType, ModelVariantType
+from invokeai.backend.model_manager import BaseModelType
 from invokeai.backend.model_patcher import ModelPatcher
-from invokeai.backend.stable_diffusion import PipelineIntermediateState
-from invokeai.backend.stable_diffusion.denoise_context import DenoiseContext, DenoiseInputs
+from invokeai.backend.stable_diffusion import PipelineIntermediateState, set_seamless
 from invokeai.backend.stable_diffusion.diffusers_pipeline import (
     ControlNetData,
     StableDiffusionGeneratorPipeline,
@@ -55,21 +53,7 @@ from invokeai.backend.stable_diffusion.diffusion.conditioning_data import (
     TextConditioningData,
     TextConditioningRegions,
 )
-from invokeai.backend.stable_diffusion.diffusion.custom_atttention import CustomAttnProcessor2_0
-from invokeai.backend.stable_diffusion.diffusion_backend import StableDiffusionBackend
-from invokeai.backend.stable_diffusion.extension_callback_type import ExtensionCallbackType
-from invokeai.backend.stable_diffusion.extensions.controlnet import ControlNetExt
-from invokeai.backend.stable_diffusion.extensions.freeu import FreeUExt
-from invokeai.backend.stable_diffusion.extensions.inpaint import InpaintExt
-from invokeai.backend.stable_diffusion.extensions.inpaint_model import InpaintModelExt
-from invokeai.backend.stable_diffusion.extensions.lora import LoRAExt
-from invokeai.backend.stable_diffusion.extensions.preview import PreviewExt
-from invokeai.backend.stable_diffusion.extensions.rescale_cfg import RescaleCFGExt
-from invokeai.backend.stable_diffusion.extensions.seamless import SeamlessExt
-from invokeai.backend.stable_diffusion.extensions.t2i_adapter import T2IAdapterExt
-from invokeai.backend.stable_diffusion.extensions_manager import ExtensionsManager
 from invokeai.backend.stable_diffusion.schedulers import SCHEDULER_MAP
-from invokeai.backend.stable_diffusion.schedulers.schedulers import SCHEDULER_NAME_VALUES
 from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.hotfixes import ControlNetModel
 from invokeai.backend.util.mask import to_standard_float_mask
@@ -329,10 +313,9 @@ class DenoiseLatentsInvocation(BaseInvocation):
         context: InvocationContext,
         positive_conditioning_field: Union[ConditioningField, list[ConditioningField]],
         negative_conditioning_field: Union[ConditioningField, list[ConditioningField]],
+        unet: UNet2DConditionModel,
         latent_height: int,
         latent_width: int,
-        device: torch.device,
-        dtype: torch.dtype,
         cfg_scale: float | list[float],
         steps: int,
         cfg_rescale_multiplier: float,
@@ -346,10 +329,10 @@ class DenoiseLatentsInvocation(BaseInvocation):
             uncond_list = [uncond_list]
 
         cond_text_embeddings, cond_text_embedding_masks = DenoiseLatentsInvocation._get_text_embeddings_and_masks(
-            cond_list, context, device, dtype
+            cond_list, context, unet.device, unet.dtype
         )
         uncond_text_embeddings, uncond_text_embedding_masks = DenoiseLatentsInvocation._get_text_embeddings_and_masks(
-            uncond_list, context, device, dtype
+            uncond_list, context, unet.device, unet.dtype
         )
 
         cond_text_embedding, cond_regions = DenoiseLatentsInvocation._concat_regional_text_embeddings(
@@ -357,14 +340,14 @@ class DenoiseLatentsInvocation(BaseInvocation):
             masks=cond_text_embedding_masks,
             latent_height=latent_height,
             latent_width=latent_width,
-            dtype=dtype,
+            dtype=unet.dtype,
         )
         uncond_text_embedding, uncond_regions = DenoiseLatentsInvocation._concat_regional_text_embeddings(
             text_conditionings=uncond_text_embeddings,
             masks=uncond_text_embedding_masks,
             latent_height=latent_height,
             latent_width=latent_width,
-            dtype=dtype,
+            dtype=unet.dtype,
         )
 
         if isinstance(cfg_scale, list):
@@ -471,65 +454,6 @@ class DenoiseLatentsInvocation(BaseInvocation):
 
         return controlnet_data
 
-    @staticmethod
-    def parse_controlnet_field(
-        exit_stack: ExitStack,
-        context: InvocationContext,
-        control_input: ControlField | list[ControlField] | None,
-        ext_manager: ExtensionsManager,
-    ) -> None:
-        # Normalize control_input to a list.
-        control_list: list[ControlField]
-        if isinstance(control_input, ControlField):
-            control_list = [control_input]
-        elif isinstance(control_input, list):
-            control_list = control_input
-        elif control_input is None:
-            control_list = []
-        else:
-            raise ValueError(f"Unexpected control_input type: {type(control_input)}")
-
-        for control_info in control_list:
-            model = exit_stack.enter_context(context.models.load(control_info.control_model))
-            ext_manager.add_extension(
-                ControlNetExt(
-                    model=model,
-                    image=context.images.get_pil(control_info.image.image_name),
-                    weight=control_info.control_weight,
-                    begin_step_percent=control_info.begin_step_percent,
-                    end_step_percent=control_info.end_step_percent,
-                    control_mode=control_info.control_mode,
-                    resize_mode=control_info.resize_mode,
-                )
-            )
-
-    @staticmethod
-    def parse_t2i_adapter_field(
-        exit_stack: ExitStack,
-        context: InvocationContext,
-        t2i_adapters: Optional[Union[T2IAdapterField, list[T2IAdapterField]]],
-        ext_manager: ExtensionsManager,
-    ) -> None:
-        if t2i_adapters is None:
-            return
-
-        # Handle the possibility that t2i_adapters could be a list or a single T2IAdapterField.
-        if isinstance(t2i_adapters, T2IAdapterField):
-            t2i_adapters = [t2i_adapters]
-
-        for t2i_adapter_field in t2i_adapters:
-            ext_manager.add_extension(
-                T2IAdapterExt(
-                    node_context=context,
-                    model_id=t2i_adapter_field.t2i_adapter_model,
-                    image=context.images.get_pil(t2i_adapter_field.image.image_name),
-                    weight=t2i_adapter_field.weight,
-                    begin_step_percent=t2i_adapter_field.begin_step_percent,
-                    end_step_percent=t2i_adapter_field.end_step_percent,
-                    resize_mode=t2i_adapter_field.resize_mode,
-                )
-            )
-
     def prep_ip_adapter_image_prompts(
         self,
         context: InvocationContext,
@@ -739,7 +663,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
         else:
             masked_latents = torch.where(mask < 0.5, 0.0, latents)
 
-        return mask, masked_latents, self.denoise_mask.gradient
+        return 1 - mask, masked_latents, self.denoise_mask.gradient
 
     @staticmethod
     def prepare_noise_and_latents(
@@ -769,7 +693,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
             raise ValueError("'latents' or 'noise' must be provided!")
 
         if noise is not None and noise.shape[1:] != latents.shape[1:]:
-            raise ValueError(f"Incompatible 'noise' and 'latents' shapes: {latents.shape=} {noise.shape=}")
+            raise ValueError(f"Incompatable 'noise' and 'latents' shapes: {latents.shape=} {noise.shape=}")
 
         # The seed comes from (in order of priority): the noise field, the latents field, or 0.
         seed = 0
@@ -782,157 +706,12 @@ class DenoiseLatentsInvocation(BaseInvocation):
 
         return seed, noise, latents
 
+    @torch.no_grad()
+    @SilenceWarnings()  # This quenches the NSFW nag from diffusers.
     def invoke(self, context: InvocationContext) -> LatentsOutput:
-        if os.environ.get("USE_MODULAR_DENOISE", False):
-            return self._new_invoke(context)
-        else:
-            return self._old_invoke(context)
-
-    @torch.no_grad()
-    @SilenceWarnings()  # This quenches the NSFW nag from diffusers.
-    def _new_invoke(self, context: InvocationContext) -> LatentsOutput:
-        ext_manager = ExtensionsManager(is_canceled=context.util.is_canceled)
-
-        device = TorchDevice.choose_torch_device()
-        dtype = TorchDevice.choose_torch_dtype()
-
-        seed, noise, latents = self.prepare_noise_and_latents(context, self.noise, self.latents)
-        _, _, latent_height, latent_width = latents.shape
-
-        conditioning_data = self.get_conditioning_data(
-            context=context,
-            positive_conditioning_field=self.positive_conditioning,
-            negative_conditioning_field=self.negative_conditioning,
-            cfg_scale=self.cfg_scale,
-            steps=self.steps,
-            latent_height=latent_height,
-            latent_width=latent_width,
-            device=device,
-            dtype=dtype,
-            # TODO: old backend, remove
-            cfg_rescale_multiplier=self.cfg_rescale_multiplier,
-        )
-
-        scheduler = get_scheduler(
-            context=context,
-            scheduler_info=self.unet.scheduler,
-            scheduler_name=self.scheduler,
-            seed=seed,
-        )
-
-        timesteps, init_timestep, scheduler_step_kwargs = self.init_scheduler(
-            scheduler,
-            seed=seed,
-            device=device,
-            steps=self.steps,
-            denoising_start=self.denoising_start,
-            denoising_end=self.denoising_end,
-        )
-
-        # get the unet's config so that we can pass the base to sd_step_callback()
-        unet_config = context.models.get_config(self.unet.unet.key)
-
-        ### preview
-        def step_callback(state: PipelineIntermediateState) -> None:
-            context.util.sd_step_callback(state, unet_config.base)
-
-        ext_manager.add_extension(PreviewExt(step_callback))
-
-        ### cfg rescale
-        if self.cfg_rescale_multiplier > 0:
-            ext_manager.add_extension(RescaleCFGExt(self.cfg_rescale_multiplier))
-
-        ### freeu
-        if self.unet.freeu_config:
-            ext_manager.add_extension(FreeUExt(self.unet.freeu_config))
-
-        ### lora
-        if self.unet.loras:
-            for lora_field in self.unet.loras:
-                ext_manager.add_extension(
-                    LoRAExt(
-                        node_context=context,
-                        model_id=lora_field.lora,
-                        weight=lora_field.weight,
-                    )
-                )
-        ### seamless
-        if self.unet.seamless_axes:
-            ext_manager.add_extension(SeamlessExt(self.unet.seamless_axes))
-
-        ### inpaint
-        mask, masked_latents, is_gradient_mask = self.prep_inpaint_mask(context, latents)
-        # NOTE: We used to identify inpainting models by inpecting the shape of the loaded UNet model weights. Now we
-        # use the ModelVariantType config. During testing, there was a report of a user with models that had an
-        # incorrect ModelVariantType value. Re-installing the model fixed the issue. If this issue turns out to be
-        # prevalent, we will have to revisit how we initialize the inpainting extensions.
-        if unet_config.variant == ModelVariantType.Inpaint:
-            ext_manager.add_extension(InpaintModelExt(mask, masked_latents, is_gradient_mask))
-        elif mask is not None:
-            ext_manager.add_extension(InpaintExt(mask, is_gradient_mask))
-
-        # Initialize context for modular denoise
-        latents = latents.to(device=device, dtype=dtype)
-        if noise is not None:
-            noise = noise.to(device=device, dtype=dtype)
-        denoise_ctx = DenoiseContext(
-            inputs=DenoiseInputs(
-                orig_latents=latents,
-                timesteps=timesteps,
-                init_timestep=init_timestep,
-                noise=noise,
-                seed=seed,
-                scheduler_step_kwargs=scheduler_step_kwargs,
-                conditioning_data=conditioning_data,
-                attention_processor_cls=CustomAttnProcessor2_0,
-            ),
-            unet=None,
-            scheduler=scheduler,
-        )
-
-        # context for loading additional models
-        with ExitStack() as exit_stack:
-            # later should be smth like:
-            # for extension_field in self.extensions:
-            #    ext = extension_field.to_extension(exit_stack, context, ext_manager)
-            #    ext_manager.add_extension(ext)
-            self.parse_controlnet_field(exit_stack, context, self.control, ext_manager)
-            self.parse_t2i_adapter_field(exit_stack, context, self.t2i_adapter, ext_manager)
-
-            # ext: t2i/ip adapter
-            ext_manager.run_callback(ExtensionCallbackType.SETUP, denoise_ctx)
-
-            unet_info = context.models.load(self.unet.unet)
-            assert isinstance(unet_info.model, UNet2DConditionModel)
-            with (
-                unet_info.model_on_device() as (cached_weights, unet),
-                ModelPatcher.patch_unet_attention_processor(unet, denoise_ctx.inputs.attention_processor_cls),
-                # ext: controlnet
-                ext_manager.patch_extensions(denoise_ctx),
-                # ext: freeu, seamless, ip adapter, lora
-                ext_manager.patch_unet(unet, cached_weights),
-            ):
-                sd_backend = StableDiffusionBackend(unet, scheduler)
-                denoise_ctx.unet = unet
-                result_latents = sd_backend.latents_from_embeddings(denoise_ctx, ext_manager)
-
-        # https://discuss.huggingface.co/t/memory-usage-by-later-pipeline-stages/23699
-        result_latents = result_latents.detach().to("cpu")
-        TorchDevice.empty_cache()
-
-        name = context.tensors.save(tensor=result_latents)
-        return LatentsOutput.build(latents_name=name, latents=result_latents, seed=None)
-
-    @torch.no_grad()
-    @SilenceWarnings()  # This quenches the NSFW nag from diffusers.
-    def _old_invoke(self, context: InvocationContext) -> LatentsOutput:
         seed, noise, latents = self.prepare_noise_and_latents(context, self.noise, self.latents)
 
         mask, masked_latents, gradient_mask = self.prep_inpaint_mask(context, latents)
-        # At this point, the mask ranges from 0 (leave unchanged) to 1 (inpaint).
-        # We invert the mask here for compatibility with the old backend implementation.
-        if mask is not None:
-            mask = 1 - mask
 
         # TODO(ryand): I have hard-coded `do_classifier_free_guidance=True` to mirror the behaviour of ControlNets,
         # below. Investigate whether this is appropriate.
@@ -957,7 +736,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
         # The image prompts are then passed to prep_ip_adapter_data().
         image_prompts = self.prep_ip_adapter_image_prompts(context=context, ip_adapters=ip_adapters)
 
-        # get the unet's config so that we can pass the base to sd_step_callback()
+        # get the unet's config so that we can pass the base to dispatch_progress()
         unet_config = context.models.get_config(self.unet.unet.key)
 
         def step_callback(state: PipelineIntermediateState) -> None:
@@ -975,14 +754,14 @@ class DenoiseLatentsInvocation(BaseInvocation):
         assert isinstance(unet_info.model, UNet2DConditionModel)
         with (
             ExitStack() as exit_stack,
-            unet_info.model_on_device() as (cached_weights, unet),
+            unet_info.model_on_device() as (model_state_dict, unet),
             ModelPatcher.apply_freeu(unet, self.unet.freeu_config),
-            SeamlessExt.static_patch_model(unet, self.unet.seamless_axes),  # FIXME
+            set_seamless(unet, self.unet.seamless_axes),  # FIXME
             # Apply the LoRA after unet has been moved to its target device for faster patching.
             ModelPatcher.apply_lora_unet(
                 unet,
                 loras=_lora_loader(),
-                cached_weights=cached_weights,
+                model_state_dict=model_state_dict,
             ),
         ):
             assert isinstance(unet, UNet2DConditionModel)
@@ -1008,8 +787,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
                 context=context,
                 positive_conditioning_field=self.positive_conditioning,
                 negative_conditioning_field=self.negative_conditioning,
-                device=unet.device,
-                dtype=unet.dtype,
+                unet=unet,
                 latent_height=latent_height,
                 latent_width=latent_width,
                 cfg_scale=self.cfg_scale,

invokeai/app/invocations/fields.py

@@ -1,7 +1,7 @@
 from enum import Enum
 from typing import Any, Callable, Optional, Tuple
 
-from pydantic import BaseModel, ConfigDict, Field, RootModel, TypeAdapter, model_validator
+from pydantic import BaseModel, ConfigDict, Field, RootModel, TypeAdapter
 from pydantic.fields import _Unset
 from pydantic_core import PydanticUndefined
 
@@ -48,7 +48,6 @@ class UIType(str, Enum, metaclass=MetaEnum):
     ControlNetModel = "ControlNetModelField"
     IPAdapterModel = "IPAdapterModelField"
     T2IAdapterModel = "T2IAdapterModelField"
-    SpandrelImageToImageModel = "SpandrelImageToImageModelField"
     # endregion
 
     # region Misc Field Types
@@ -135,7 +134,6 @@ class FieldDescriptions:
     sdxl_main_model = "SDXL Main model (UNet, VAE, CLIP1, CLIP2) to load"
     sdxl_refiner_model = "SDXL Refiner Main Modde (UNet, VAE, CLIP2) to load"
     onnx_main_model = "ONNX Main model (UNet, VAE, CLIP) to load"
-    spandrel_image_to_image_model = "Image-to-Image model"
     lora_weight = "The weight at which the LoRA is applied to each model"
     compel_prompt = "Prompt to be parsed by Compel to create a conditioning tensor"
     raw_prompt = "Raw prompt text (no parsing)"
@@ -162,7 +160,6 @@ class FieldDescriptions:
     fp32 = "Whether or not to use full float32 precision"
     precision = "Precision to use"
     tiled = "Processing using overlapping tiles (reduce memory consumption)"
-    vae_tile_size = "The tile size for VAE tiling in pixels (image space). If set to 0, the default tile size for the model will be used. Larger tile sizes generally produce better results at the cost of higher memory usage."
     detect_res = "Pixel resolution for detection"
     image_res = "Pixel resolution for output image"
     safe_mode = "Whether or not to use safe mode"
@@ -242,31 +239,6 @@ class ConditioningField(BaseModel):
     )
 
 
-class BoundingBoxField(BaseModel):
-    """A bounding box primitive value."""
-
-    x_min: int = Field(ge=0, description="The minimum x-coordinate of the bounding box (inclusive).")
-    x_max: int = Field(ge=0, description="The maximum x-coordinate of the bounding box (exclusive).")
-    y_min: int = Field(ge=0, description="The minimum y-coordinate of the bounding box (inclusive).")
-    y_max: int = Field(ge=0, description="The maximum y-coordinate of the bounding box (exclusive).")
-
-    score: Optional[float] = Field(
-        default=None,
-        ge=0.0,
-        le=1.0,
-        description="The score associated with the bounding box. In the range [0, 1]. This value is typically set "
-        "when the bounding box was produced by a detector and has an associated confidence score.",
-    )
-
-    @model_validator(mode="after")
-    def check_coords(self):
-        if self.x_min > self.x_max:
-            raise ValueError(f"x_min ({self.x_min}) is greater than x_max ({self.x_max}).")
-        if self.y_min > self.y_max:
-            raise ValueError(f"y_min ({self.y_min}) is greater than y_max ({self.y_max}).")
-        return self
-
-
 class MetadataField(RootModel[dict[str, Any]]):
     """
     Pydantic model for metadata with custom root of type dict[str, Any].

invokeai/app/invocations/grounding_dino.py

@@ -1,100 +0,0 @@
-from pathlib import Path
-from typing import Literal
-
-import torch
-from PIL import Image
-from transformers import pipeline
-from transformers.pipelines import ZeroShotObjectDetectionPipeline
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import BoundingBoxField, ImageField, InputField
-from invokeai.app.invocations.primitives import BoundingBoxCollectionOutput
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.image_util.grounding_dino.detection_result import DetectionResult
-from invokeai.backend.image_util.grounding_dino.grounding_dino_pipeline import GroundingDinoPipeline
-
-GroundingDinoModelKey = Literal["grounding-dino-tiny", "grounding-dino-base"]
-GROUNDING_DINO_MODEL_IDS: dict[GroundingDinoModelKey, str] = {
-    "grounding-dino-tiny": "IDEA-Research/grounding-dino-tiny",
-    "grounding-dino-base": "IDEA-Research/grounding-dino-base",
-}
-
-
-@invocation(
-    "grounding_dino",
-    title="Grounding DINO (Text Prompt Object Detection)",
-    tags=["prompt", "object detection"],
-    category="image",
-    version="1.0.0",
-)
-class GroundingDinoInvocation(BaseInvocation):
-    """Runs a Grounding DINO model. Performs zero-shot bounding-box object detection from a text prompt."""
-
-    # Reference:
-    # - https://arxiv.org/pdf/2303.05499
-    # - https://huggingface.co/docs/transformers/v4.43.3/en/model_doc/grounding-dino#grounded-sam
-    # - https://github.com/NielsRogge/Transformers-Tutorials/blob/a39f33ac1557b02ebfb191ea7753e332b5ca933f/Grounding%20DINO/GroundingDINO_with_Segment_Anything.ipynb
-
-    model: GroundingDinoModelKey = InputField(description="The Grounding DINO model to use.")
-    prompt: str = InputField(description="The prompt describing the object to segment.")
-    image: ImageField = InputField(description="The image to segment.")
-    detection_threshold: float = InputField(
-        description="The detection threshold for the Grounding DINO model. All detected bounding boxes with scores above this threshold will be returned.",
-        ge=0.0,
-        le=1.0,
-        default=0.3,
-    )
-
-    @torch.no_grad()
-    def invoke(self, context: InvocationContext) -> BoundingBoxCollectionOutput:
-        # The model expects a 3-channel RGB image.
-        image_pil = context.images.get_pil(self.image.image_name, mode="RGB")
-
-        detections = self._detect(
-            context=context, image=image_pil, labels=[self.prompt], threshold=self.detection_threshold
-        )
-
-        # Convert detections to BoundingBoxCollectionOutput.
-        bounding_boxes: list[BoundingBoxField] = []
-        for detection in detections:
-            bounding_boxes.append(
-                BoundingBoxField(
-                    x_min=detection.box.xmin,
-                    x_max=detection.box.xmax,
-                    y_min=detection.box.ymin,
-                    y_max=detection.box.ymax,
-                    score=detection.score,
-                )
-            )
-        return BoundingBoxCollectionOutput(collection=bounding_boxes)
-
-    @staticmethod
-    def _load_grounding_dino(model_path: Path):
-        grounding_dino_pipeline = pipeline(
-            model=str(model_path),
-            task="zero-shot-object-detection",
-            local_files_only=True,
-            # TODO(ryand): Setting the torch_dtype here doesn't work. Investigate whether fp16 is supported by the
-            # model, and figure out how to make it work in the pipeline.
-            # torch_dtype=TorchDevice.choose_torch_dtype(),
-        )
-        assert isinstance(grounding_dino_pipeline, ZeroShotObjectDetectionPipeline)
-        return GroundingDinoPipeline(grounding_dino_pipeline)
-
-    def _detect(
-        self,
-        context: InvocationContext,
-        image: Image.Image,
-        labels: list[str],
-        threshold: float = 0.3,
-    ) -> list[DetectionResult]:
-        """Use Grounding DINO to detect bounding boxes for a set of labels in an image."""
-        # TODO(ryand): I copied this "."-handling logic from the transformers example code. Test it and see if it
-        # actually makes a difference.
-        labels = [label if label.endswith(".") else label + "." for label in labels]
-
-        with context.models.load_remote_model(
-            source=GROUNDING_DINO_MODEL_IDS[self.model], loader=GroundingDinoInvocation._load_grounding_dino
-        ) as detector:
-            assert isinstance(detector, GroundingDinoPipeline)
-            return detector.detect(image=image, candidate_labels=labels, threshold=threshold)

invokeai/app/invocations/image.py

@@ -6,7 +6,6 @@ import cv2
 import numpy
 from PIL import Image, ImageChops, ImageFilter, ImageOps
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, invocation
 from invokeai.app.invocations.constants import IMAGE_MODES
 from invokeai.app.invocations.fields import (
     ColorField,
@@ -22,6 +21,8 @@ from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.image_util.invisible_watermark import InvisibleWatermark
 from invokeai.backend.image_util.safety_checker import SafetyChecker
 
+from .baseinvocation import BaseInvocation, Classification, invocation
+
 
 @invocation("show_image", title="Show Image", tags=["image"], category="image", version="1.0.1")
 class ShowImageInvocation(BaseInvocation):

invokeai/app/invocations/image_to_latents.py

@@ -1,4 +1,3 @@
-from contextlib import nullcontext
 from functools import singledispatchmethod
 
 import einops
@@ -13,7 +12,7 @@ from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL
 from diffusers.models.autoencoders.autoencoder_tiny import AutoencoderTiny
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.constants import DEFAULT_PRECISION, LATENT_SCALE_FACTOR
+from invokeai.app.invocations.constants import DEFAULT_PRECISION
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     ImageField,
@@ -25,7 +24,6 @@ from invokeai.app.invocations.primitives import LatentsOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.model_manager import LoadedModel
 from invokeai.backend.stable_diffusion.diffusers_pipeline import image_resized_to_grid_as_tensor
-from invokeai.backend.stable_diffusion.vae_tiling import patch_vae_tiling_params
 
 
 @invocation(
@@ -33,7 +31,7 @@ from invokeai.backend.stable_diffusion.vae_tiling import patch_vae_tiling_params
     title="Image to Latents",
     tags=["latents", "image", "vae", "i2l"],
     category="latents",
-    version="1.1.0",
+    version="1.0.2",
 )
 class ImageToLatentsInvocation(BaseInvocation):
     """Encodes an image into latents."""
@@ -46,17 +44,12 @@ class ImageToLatentsInvocation(BaseInvocation):
         input=Input.Connection,
     )
     tiled: bool = InputField(default=False, description=FieldDescriptions.tiled)
-    # NOTE: tile_size = 0 is a special value. We use this rather than `int | None`, because the workflow UI does not
-    # offer a way to directly set None values.
-    tile_size: int = InputField(default=0, multiple_of=8, description=FieldDescriptions.vae_tile_size)
     fp32: bool = InputField(default=DEFAULT_PRECISION == torch.float32, description=FieldDescriptions.fp32)
 
     @staticmethod
-    def vae_encode(
-        vae_info: LoadedModel, upcast: bool, tiled: bool, image_tensor: torch.Tensor, tile_size: int = 0
-    ) -> torch.Tensor:
+    def vae_encode(vae_info: LoadedModel, upcast: bool, tiled: bool, image_tensor: torch.Tensor) -> torch.Tensor:
         with vae_info as vae:
-            assert isinstance(vae, (AutoencoderKL, AutoencoderTiny))
+            assert isinstance(vae, torch.nn.Module)
             orig_dtype = vae.dtype
             if upcast:
                 vae.to(dtype=torch.float32)
@@ -88,18 +81,9 @@ class ImageToLatentsInvocation(BaseInvocation):
             else:
                 vae.disable_tiling()
 
-            tiling_context = nullcontext()
-            if tile_size > 0:
-                tiling_context = patch_vae_tiling_params(
-                    vae,
-                    tile_sample_min_size=tile_size,
-                    tile_latent_min_size=tile_size // LATENT_SCALE_FACTOR,
-                    tile_overlap_factor=0.25,
-                )
-
             # non_noised_latents_from_image
             image_tensor = image_tensor.to(device=vae.device, dtype=vae.dtype)
-            with torch.inference_mode(), tiling_context:
+            with torch.inference_mode():
                 latents = ImageToLatentsInvocation._encode_to_tensor(vae, image_tensor)
 
             latents = vae.config.scaling_factor * latents
@@ -117,9 +101,7 @@ class ImageToLatentsInvocation(BaseInvocation):
         if image_tensor.dim() == 3:
             image_tensor = einops.rearrange(image_tensor, "c h w -> 1 c h w")
 
-        latents = self.vae_encode(
-            vae_info=vae_info, upcast=self.fp32, tiled=self.tiled, image_tensor=image_tensor, tile_size=self.tile_size
-        )
+        latents = self.vae_encode(vae_info, self.fp32, self.tiled, image_tensor)
 
         latents = latents.to("cpu")
         name = context.tensors.save(tensor=latents)

invokeai/app/invocations/infill.py

@@ -3,9 +3,7 @@ from typing import Literal, get_args
 
 from PIL import Image
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import ColorField, ImageField, InputField, WithBoard, WithMetadata
-from invokeai.app.invocations.image import PIL_RESAMPLING_MAP, PIL_RESAMPLING_MODES
+from invokeai.app.invocations.fields import ColorField, ImageField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.misc import SEED_MAX
@@ -16,6 +14,10 @@ from invokeai.backend.image_util.infill_methods.patchmatch import PatchMatch, in
 from invokeai.backend.image_util.infill_methods.tile import infill_tile
 from invokeai.backend.util.logging import InvokeAILogger
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField, WithBoard, WithMetadata
+from .image import PIL_RESAMPLING_MAP, PIL_RESAMPLING_MODES
+
 logger = InvokeAILogger.get_logger()
 
 

invokeai/app/invocations/latents_to_image.py

@@ -1,5 +1,3 @@
-from contextlib import nullcontext
-
 import torch
 from diffusers.image_processor import VaeImageProcessor
 from diffusers.models.attention_processor import (
@@ -10,9 +8,10 @@ from diffusers.models.attention_processor import (
 )
 from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL
 from diffusers.models.autoencoders.autoencoder_tiny import AutoencoderTiny
+from PIL import Image
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.constants import DEFAULT_PRECISION, LATENT_SCALE_FACTOR
+from invokeai.app.invocations.constants import DEFAULT_PRECISION
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     Input,
@@ -24,8 +23,8 @@ from invokeai.app.invocations.fields import (
 from invokeai.app.invocations.model import VAEField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.stable_diffusion.extensions.seamless import SeamlessExt
-from invokeai.backend.stable_diffusion.vae_tiling import patch_vae_tiling_params
+from invokeai.backend.model_manager.load.load_base import LoadedModel
+from invokeai.backend.stable_diffusion import set_seamless
 from invokeai.backend.util.devices import TorchDevice
 
 
@@ -34,7 +33,7 @@ from invokeai.backend.util.devices import TorchDevice
     title="Latents to Image",
     tags=["latents", "image", "vae", "l2i"],
     category="latents",
-    version="1.3.0",
+    version="1.2.2",
 )
 class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
     """Generates an image from latents."""
@@ -48,21 +47,22 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
         input=Input.Connection,
     )
     tiled: bool = InputField(default=False, description=FieldDescriptions.tiled)
-    # NOTE: tile_size = 0 is a special value. We use this rather than `int | None`, because the workflow UI does not
-    # offer a way to directly set None values.
-    tile_size: int = InputField(default=0, multiple_of=8, description=FieldDescriptions.vae_tile_size)
     fp32: bool = InputField(default=DEFAULT_PRECISION == torch.float32, description=FieldDescriptions.fp32)
 
-    @torch.no_grad()
-    def invoke(self, context: InvocationContext) -> ImageOutput:
-        latents = context.tensors.load(self.latents.latents_name)
-
-        vae_info = context.models.load(self.vae.vae)
+    @staticmethod
+    def vae_decode(
+        context: InvocationContext,
+        vae_info: LoadedModel,
+        seamless_axes: list[str],
+        latents: torch.Tensor,
+        use_fp32: bool,
+        use_tiling: bool,
+    ) -> Image.Image:
         assert isinstance(vae_info.model, (AutoencoderKL, AutoencoderTiny))
-        with SeamlessExt.static_patch_model(vae_info.model, self.vae.seamless_axes), vae_info as vae:
+        with set_seamless(vae_info.model, seamless_axes), vae_info as vae:
             assert isinstance(vae, (AutoencoderKL, AutoencoderTiny))
             latents = latents.to(vae.device)
-            if self.fp32:
+            if use_fp32:
                 vae.to(dtype=torch.float32)
 
                 use_torch_2_0_or_xformers = hasattr(vae.decoder, "mid_block") and isinstance(
@@ -87,24 +87,15 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
                 vae.to(dtype=torch.float16)
                 latents = latents.half()
 
-            if self.tiled or context.config.get().force_tiled_decode:
+            if use_tiling or context.config.get().force_tiled_decode:
                 vae.enable_tiling()
             else:
                 vae.disable_tiling()
 
-            tiling_context = nullcontext()
-            if self.tile_size > 0:
-                tiling_context = patch_vae_tiling_params(
-                    vae,
-                    tile_sample_min_size=self.tile_size,
-                    tile_latent_min_size=self.tile_size // LATENT_SCALE_FACTOR,
-                    tile_overlap_factor=0.25,
-                )
-
             # clear memory as vae decode can request a lot
             TorchDevice.empty_cache()
 
-            with torch.inference_mode(), tiling_context:
+            with torch.inference_mode():
                 # copied from diffusers pipeline
                 latents = latents / vae.config.scaling_factor
                 image = vae.decode(latents, return_dict=False)[0]
@@ -116,6 +107,21 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
 
         TorchDevice.empty_cache()
 
+        return image
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> ImageOutput:
+        latents = context.tensors.load(self.latents.latents_name)
+        vae_info = context.models.load(self.vae.vae)
+
+        image = self.vae_decode(
+            context=context,
+            vae_info=vae_info,
+            seamless_axes=self.vae.seamless_axes,
+            latents=latents,
+            use_fp32=self.fp32,
+            use_tiling=self.tiled,
+        )
         image_dto = context.images.save(image=image)
 
         return ImageOutput.build(image_dto)

invokeai/app/invocations/mask.py

@@ -1,10 +1,9 @@
 import numpy as np
 import torch
-from PIL import Image
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, InvocationContext, invocation
-from invokeai.app.invocations.fields import ImageField, InputField, TensorField, WithBoard, WithMetadata
-from invokeai.app.invocations.primitives import ImageOutput, MaskOutput
+from invokeai.app.invocations.fields import ImageField, InputField, TensorField, WithMetadata
+from invokeai.app.invocations.primitives import MaskOutput
 
 
 @invocation(
@@ -119,27 +118,3 @@ class ImageMaskToTensorInvocation(BaseInvocation, WithMetadata):
             height=mask.shape[1],
             width=mask.shape[2],
         )
-
-
-@invocation(
-    "tensor_mask_to_image",
-    title="Tensor Mask to Image",
-    tags=["mask"],
-    category="mask",
-    version="1.0.0",
-)
-class MaskTensorToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
-    """Convert a mask tensor to an image."""
-
-    mask: TensorField = InputField(description="The mask tensor to convert.")
-
-    def invoke(self, context: InvocationContext) -> ImageOutput:
-        mask = context.tensors.load(self.mask.tensor_name)
-        # Ensure that the mask is binary.
-        if mask.dtype != torch.bool:
-            mask = mask > 0.5
-        mask_np = (mask.float() * 255).byte().cpu().numpy()
-
-        mask_pil = Image.fromarray(mask_np, mode="L")
-        image_dto = context.images.save(image=mask_pil)
-        return ImageOutput.build(image_dto)

invokeai/app/invocations/math.py

@@ -5,11 +5,12 @@ from typing import Literal
 import numpy as np
 from pydantic import ValidationInfo, field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
 from invokeai.app.invocations.fields import FieldDescriptions, InputField
 from invokeai.app.invocations.primitives import FloatOutput, IntegerOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import BaseInvocation, invocation
+
 
 @invocation("add", title="Add Integers", tags=["math", "add"], category="math", version="1.0.1")
 class AddInvocation(BaseInvocation):

invokeai/app/invocations/metadata.py

@@ -14,7 +14,8 @@ from invokeai.app.invocations.fields import (
 from invokeai.app.invocations.model import ModelIdentifierField
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES
-from invokeai.version.invokeai_version import __version__
+
+from ...version import __version__
 
 
 class MetadataItemField(BaseModel):

invokeai/app/invocations/model.py

@@ -3,17 +3,18 @@ from typing import List, Optional
 
 from pydantic import BaseModel, Field
 
-from invokeai.app.invocations.baseinvocation import (
+from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, UIType
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.app.shared.models import FreeUConfig
+from invokeai.backend.model_manager.config import AnyModelConfig, BaseModelType, ModelType, SubModelType
+
+from .baseinvocation import (
     BaseInvocation,
     BaseInvocationOutput,
     Classification,
     invocation,
     invocation_output,
 )
-from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, UIType
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.app.shared.models import FreeUConfig
-from invokeai.backend.model_manager.config import AnyModelConfig, BaseModelType, ModelType, SubModelType
 
 
 class ModelIdentifierField(BaseModel):

invokeai/app/invocations/noise.py

@@ -4,12 +4,18 @@
 import torch
 from pydantic import field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
 from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
 from invokeai.app.invocations.fields import FieldDescriptions, InputField, LatentsField, OutputField
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.misc import SEED_MAX
-from invokeai.backend.util.devices import TorchDevice
+
+from ...backend.util.devices import TorchDevice
+from .baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    invocation,
+    invocation_output,
+)
 
 """
 Utilities

invokeai/app/invocations/param_easing.py

@@ -39,11 +39,12 @@ from easing_functions import (
 )
 from matplotlib.ticker import MaxNLocator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import InputField
 from invokeai.app.invocations.primitives import FloatCollectionOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField
+
 
 @invocation(
     "float_range",

invokeai/app/invocations/primitives.py

@@ -4,10 +4,8 @@ from typing import Optional
 
 import torch
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
 from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
 from invokeai.app.invocations.fields import (
-    BoundingBoxField,
     ColorField,
     ConditioningField,
     DenoiseMaskField,
@@ -23,6 +21,13 @@ from invokeai.app.invocations.fields import (
 from invokeai.app.services.images.images_common import ImageDTO
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    invocation,
+    invocation_output,
+)
+
 """
 Primitives: Boolean, Integer, Float, String, Image, Latents, Conditioning, Color
 - primitive nodes
@@ -470,42 +475,3 @@ class ConditioningCollectionInvocation(BaseInvocation):
 
 
 # endregion
-
-# region BoundingBox
-
-
-@invocation_output("bounding_box_output")
-class BoundingBoxOutput(BaseInvocationOutput):
-    """Base class for nodes that output a single bounding box"""
-
-    bounding_box: BoundingBoxField = OutputField(description="The output bounding box.")
-
-
-@invocation_output("bounding_box_collection_output")
-class BoundingBoxCollectionOutput(BaseInvocationOutput):
-    """Base class for nodes that output a collection of bounding boxes"""
-
-    collection: list[BoundingBoxField] = OutputField(description="The output bounding boxes.", title="Bounding Boxes")
-
-
-@invocation(
-    "bounding_box",
-    title="Bounding Box",
-    tags=["primitives", "segmentation", "collection", "bounding box"],
-    category="primitives",
-    version="1.0.0",
-)
-class BoundingBoxInvocation(BaseInvocation):
-    """Create a bounding box manually by supplying box coordinates"""
-
-    x_min: int = InputField(default=0, description="x-coordinate of the bounding box's top left vertex")
-    y_min: int = InputField(default=0, description="y-coordinate of the bounding box's top left vertex")
-    x_max: int = InputField(default=0, description="x-coordinate of the bounding box's bottom right vertex")
-    y_max: int = InputField(default=0, description="y-coordinate of the bounding box's bottom right vertex")
-
-    def invoke(self, context: InvocationContext) -> BoundingBoxOutput:
-        bounding_box = BoundingBoxField(x_min=self.x_min, y_min=self.y_min, x_max=self.x_max, y_max=self.y_max)
-        return BoundingBoxOutput(bounding_box=bounding_box)
-
-
-# endregion

invokeai/app/invocations/prompt.py

@@ -5,11 +5,12 @@ import numpy as np
 from dynamicprompts.generators import CombinatorialPromptGenerator, RandomPromptGenerator
 from pydantic import field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import InputField, UIComponent
 from invokeai.app.invocations.primitives import StringCollectionOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField, UIComponent
+
 
 @invocation(
     "dynamic_prompt",

invokeai/app/invocations/scheduler.py

@@ -1,4 +1,5 @@
 from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
+from invokeai.app.invocations.constants import SCHEDULER_NAME_VALUES
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     InputField,
@@ -6,7 +7,6 @@ from invokeai.app.invocations.fields import (
     UIType,
 )
 from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.stable_diffusion.schedulers.schedulers import SCHEDULER_NAME_VALUES
 
 
 @invocation_output("scheduler_output")

invokeai/app/invocations/sdxl.py

@@ -1,9 +1,15 @@
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
 from invokeai.app.invocations.fields import FieldDescriptions, InputField, OutputField, UIType
-from invokeai.app.invocations.model import CLIPField, ModelIdentifierField, UNetField, VAEField
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.model_manager import SubModelType
 
+from .baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    invocation,
+    invocation_output,
+)
+from .model import CLIPField, ModelIdentifierField, UNetField, VAEField
+
 
 @invocation_output("sdxl_model_loader_output")
 class SDXLModelLoaderOutput(BaseInvocationOutput):

invokeai/app/invocations/segment_anything.py

@@ -1,161 +0,0 @@
-from pathlib import Path
-from typing import Literal
-
-import numpy as np
-import torch
-from PIL import Image
-from transformers import AutoModelForMaskGeneration, AutoProcessor
-from transformers.models.sam import SamModel
-from transformers.models.sam.processing_sam import SamProcessor
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import BoundingBoxField, ImageField, InputField, TensorField
-from invokeai.app.invocations.primitives import MaskOutput
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.image_util.segment_anything.mask_refinement import mask_to_polygon, polygon_to_mask
-from invokeai.backend.image_util.segment_anything.segment_anything_pipeline import SegmentAnythingPipeline
-
-SegmentAnythingModelKey = Literal["segment-anything-base", "segment-anything-large", "segment-anything-huge"]
-SEGMENT_ANYTHING_MODEL_IDS: dict[SegmentAnythingModelKey, str] = {
-    "segment-anything-base": "facebook/sam-vit-base",
-    "segment-anything-large": "facebook/sam-vit-large",
-    "segment-anything-huge": "facebook/sam-vit-huge",
-}
-
-
-@invocation(
-    "segment_anything",
-    title="Segment Anything",
-    tags=["prompt", "segmentation"],
-    category="segmentation",
-    version="1.0.0",
-)
-class SegmentAnythingInvocation(BaseInvocation):
-    """Runs a Segment Anything Model."""
-
-    # Reference:
-    # - https://arxiv.org/pdf/2304.02643
-    # - https://huggingface.co/docs/transformers/v4.43.3/en/model_doc/grounding-dino#grounded-sam
-    # - https://github.com/NielsRogge/Transformers-Tutorials/blob/a39f33ac1557b02ebfb191ea7753e332b5ca933f/Grounding%20DINO/GroundingDINO_with_Segment_Anything.ipynb
-
-    model: SegmentAnythingModelKey = InputField(description="The Segment Anything model to use.")
-    image: ImageField = InputField(description="The image to segment.")
-    bounding_boxes: list[BoundingBoxField] = InputField(description="The bounding boxes to prompt the SAM model with.")
-    apply_polygon_refinement: bool = InputField(
-        description="Whether to apply polygon refinement to the masks. This will smooth the edges of the masks slightly and ensure that each mask consists of a single closed polygon (before merging).",
-        default=True,
-    )
-    mask_filter: Literal["all", "largest", "highest_box_score"] = InputField(
-        description="The filtering to apply to the detected masks before merging them into a final output.",
-        default="all",
-    )
-
-    @torch.no_grad()
-    def invoke(self, context: InvocationContext) -> MaskOutput:
-        # The models expect a 3-channel RGB image.
-        image_pil = context.images.get_pil(self.image.image_name, mode="RGB")
-
-        if len(self.bounding_boxes) == 0:
-            combined_mask = torch.zeros(image_pil.size[::-1], dtype=torch.bool)
-        else:
-            masks = self._segment(context=context, image=image_pil)
-            masks = self._filter_masks(masks=masks, bounding_boxes=self.bounding_boxes)
-
-            # masks contains bool values, so we merge them via max-reduce.
-            combined_mask, _ = torch.stack(masks).max(dim=0)
-
-        mask_tensor_name = context.tensors.save(combined_mask)
-        height, width = combined_mask.shape
-        return MaskOutput(mask=TensorField(tensor_name=mask_tensor_name), width=width, height=height)
-
-    @staticmethod
-    def _load_sam_model(model_path: Path):
-        sam_model = AutoModelForMaskGeneration.from_pretrained(
-            model_path,
-            local_files_only=True,
-            # TODO(ryand): Setting the torch_dtype here doesn't work. Investigate whether fp16 is supported by the
-            # model, and figure out how to make it work in the pipeline.
-            # torch_dtype=TorchDevice.choose_torch_dtype(),
-        )
-        assert isinstance(sam_model, SamModel)
-
-        sam_processor = AutoProcessor.from_pretrained(model_path, local_files_only=True)
-        assert isinstance(sam_processor, SamProcessor)
-        return SegmentAnythingPipeline(sam_model=sam_model, sam_processor=sam_processor)
-
-    def _segment(
-        self,
-        context: InvocationContext,
-        image: Image.Image,
-    ) -> list[torch.Tensor]:
-        """Use Segment Anything (SAM) to generate masks given an image + a set of bounding boxes."""
-        # Convert the bounding boxes to the SAM input format.
-        sam_bounding_boxes = [[bb.x_min, bb.y_min, bb.x_max, bb.y_max] for bb in self.bounding_boxes]
-
-        with (
-            context.models.load_remote_model(
-                source=SEGMENT_ANYTHING_MODEL_IDS[self.model], loader=SegmentAnythingInvocation._load_sam_model
-            ) as sam_pipeline,
-        ):
-            assert isinstance(sam_pipeline, SegmentAnythingPipeline)
-            masks = sam_pipeline.segment(image=image, bounding_boxes=sam_bounding_boxes)
-
-        masks = self._process_masks(masks)
-        if self.apply_polygon_refinement:
-            masks = self._apply_polygon_refinement(masks)
-
-        return masks
-
-    def _process_masks(self, masks: torch.Tensor) -> list[torch.Tensor]:
-        """Convert the tensor output from the Segment Anything model from a tensor of shape
-        [num_masks, channels, height, width] to a list of tensors of shape [height, width].
-        """
-        assert masks.dtype == torch.bool
-        # [num_masks, channels, height, width] -> [num_masks, height, width]
-        masks, _ = masks.max(dim=1)
-        # Split the first dimension into a list of masks.
-        return list(masks.cpu().unbind(dim=0))
-
-    def _apply_polygon_refinement(self, masks: list[torch.Tensor]) -> list[torch.Tensor]:
-        """Apply polygon refinement to the masks.
-
-        Convert each mask to a polygon, then back to a mask. This has the following effect:
-        - Smooth the edges of the mask slightly.
-        - Ensure that each mask consists of a single closed polygon
-            - Removes small mask pieces.
-            - Removes holes from the mask.
-        """
-        # Convert tensor masks to np masks.
-        np_masks = [mask.cpu().numpy().astype(np.uint8) for mask in masks]
-
-        # Apply polygon refinement.
-        for idx, mask in enumerate(np_masks):
-            shape = mask.shape
-            assert len(shape) == 2  # Assert length to satisfy type checker.
-            polygon = mask_to_polygon(mask)
-            mask = polygon_to_mask(polygon, shape)
-            np_masks[idx] = mask
-
-        # Convert np masks back to tensor masks.
-        masks = [torch.tensor(mask, dtype=torch.bool) for mask in np_masks]
-
-        return masks
-
-    def _filter_masks(self, masks: list[torch.Tensor], bounding_boxes: list[BoundingBoxField]) -> list[torch.Tensor]:
-        """Filter the detected masks based on the specified mask filter."""
-        assert len(masks) == len(bounding_boxes)
-
-        if self.mask_filter == "all":
-            return masks
-        elif self.mask_filter == "largest":
-            # Find the largest mask.
-            return [max(masks, key=lambda x: float(x.sum()))]
-        elif self.mask_filter == "highest_box_score":
-            # Find the index of the bounding box with the highest score.
-            # Note that we fallback to -1.0 if the score is None. This is mainly to satisfy the type checker. In most
-            # cases the scores should all be non-None when using this filtering mode. That being said, -1.0 is a
-            # reasonable fallback since the expected score range is [0.0, 1.0].
-            max_score_idx = max(range(len(bounding_boxes)), key=lambda i: bounding_boxes[i].score or -1.0)
-            return [masks[max_score_idx]]
-        else:
-            raise ValueError(f"Invalid mask filter: {self.mask_filter}")

invokeai/app/invocations/spandrel_image_to_image.py

@@ -1,253 +0,0 @@
-from typing import Callable
-
-import numpy as np
-import torch
-from PIL import Image
-from tqdm import tqdm
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import (
-    FieldDescriptions,
-    ImageField,
-    InputField,
-    UIType,
-    WithBoard,
-    WithMetadata,
-)
-from invokeai.app.invocations.model import ModelIdentifierField
-from invokeai.app.invocations.primitives import ImageOutput
-from invokeai.app.services.session_processor.session_processor_common import CanceledException
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.spandrel_image_to_image_model import SpandrelImageToImageModel
-from invokeai.backend.tiles.tiles import calc_tiles_min_overlap
-from invokeai.backend.tiles.utils import TBLR, Tile
-
-
-@invocation("spandrel_image_to_image", title="Image-to-Image", tags=["upscale"], category="upscale", version="1.3.0")
-class SpandrelImageToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
-    """Run any spandrel image-to-image model (https://github.com/chaiNNer-org/spandrel)."""
-
-    image: ImageField = InputField(description="The input image")
-    image_to_image_model: ModelIdentifierField = InputField(
-        title="Image-to-Image Model",
-        description=FieldDescriptions.spandrel_image_to_image_model,
-        ui_type=UIType.SpandrelImageToImageModel,
-    )
-    tile_size: int = InputField(
-        default=512, description="The tile size for tiled image-to-image. Set to 0 to disable tiling."
-    )
-
-    @classmethod
-    def scale_tile(cls, tile: Tile, scale: int) -> Tile:
-        return Tile(
-            coords=TBLR(
-                top=tile.coords.top * scale,
-                bottom=tile.coords.bottom * scale,
-                left=tile.coords.left * scale,
-                right=tile.coords.right * scale,
-            ),
-            overlap=TBLR(
-                top=tile.overlap.top * scale,
-                bottom=tile.overlap.bottom * scale,
-                left=tile.overlap.left * scale,
-                right=tile.overlap.right * scale,
-            ),
-        )
-
-    @classmethod
-    def upscale_image(
-        cls,
-        image: Image.Image,
-        tile_size: int,
-        spandrel_model: SpandrelImageToImageModel,
-        is_canceled: Callable[[], bool],
-    ) -> Image.Image:
-        # Compute the image tiles.
-        if tile_size > 0:
-            min_overlap = 20
-            tiles = calc_tiles_min_overlap(
-                image_height=image.height,
-                image_width=image.width,
-                tile_height=tile_size,
-                tile_width=tile_size,
-                min_overlap=min_overlap,
-            )
-        else:
-            # No tiling. Generate a single tile that covers the entire image.
-            min_overlap = 0
-            tiles = [
-                Tile(
-                    coords=TBLR(top=0, bottom=image.height, left=0, right=image.width),
-                    overlap=TBLR(top=0, bottom=0, left=0, right=0),
-                )
-            ]
-
-        # Sort tiles first by left x coordinate, then by top y coordinate. During tile processing, we want to iterate
-        # over tiles left-to-right, top-to-bottom.
-        tiles = sorted(tiles, key=lambda x: x.coords.left)
-        tiles = sorted(tiles, key=lambda x: x.coords.top)
-
-        # Prepare input image for inference.
-        image_tensor = SpandrelImageToImageModel.pil_to_tensor(image)
-
-        # Scale the tiles for re-assembling the final image.
-        scale = spandrel_model.scale
-        scaled_tiles = [cls.scale_tile(tile, scale=scale) for tile in tiles]
-
-        # Prepare the output tensor.
-        _, channels, height, width = image_tensor.shape
-        output_tensor = torch.zeros(
-            (height * scale, width * scale, channels), dtype=torch.uint8, device=torch.device("cpu")
-        )
-
-        image_tensor = image_tensor.to(device=spandrel_model.device, dtype=spandrel_model.dtype)
-
-        # Run the model on each tile.
-        for tile, scaled_tile in tqdm(list(zip(tiles, scaled_tiles, strict=True)), desc="Upscaling Tiles"):
-            # Exit early if the invocation has been canceled.
-            if is_canceled():
-                raise CanceledException
-
-            # Extract the current tile from the input tensor.
-            input_tile = image_tensor[
-                :, :, tile.coords.top : tile.coords.bottom, tile.coords.left : tile.coords.right
-            ].to(device=spandrel_model.device, dtype=spandrel_model.dtype)
-
-            # Run the model on the tile.
-            output_tile = spandrel_model.run(input_tile)
-
-            # Convert the output tile into the output tensor's format.
-            # (N, C, H, W) -> (C, H, W)
-            output_tile = output_tile.squeeze(0)
-            # (C, H, W) -> (H, W, C)
-            output_tile = output_tile.permute(1, 2, 0)
-            output_tile = output_tile.clamp(0, 1)
-            output_tile = (output_tile * 255).to(dtype=torch.uint8, device=torch.device("cpu"))
-
-            # Merge the output tile into the output tensor.
-            # We only keep half of the overlap on the top and left side of the tile. We do this in case there are
-            # edge artifacts. We don't bother with any 'blending' in the current implementation - for most upscalers
-            # it seems unnecessary, but we may find a need in the future.
-            top_overlap = scaled_tile.overlap.top // 2
-            left_overlap = scaled_tile.overlap.left // 2
-            output_tensor[
-                scaled_tile.coords.top + top_overlap : scaled_tile.coords.bottom,
-                scaled_tile.coords.left + left_overlap : scaled_tile.coords.right,
-                :,
-            ] = output_tile[top_overlap:, left_overlap:, :]
-
-        # Convert the output tensor to a PIL image.
-        np_image = output_tensor.detach().numpy().astype(np.uint8)
-        pil_image = Image.fromarray(np_image)
-
-        return pil_image
-
-    @torch.inference_mode()
-    def invoke(self, context: InvocationContext) -> ImageOutput:
-        # Images are converted to RGB, because most models don't support an alpha channel. In the future, we may want to
-        # revisit this.
-        image = context.images.get_pil(self.image.image_name, mode="RGB")
-
-        # Load the model.
-        spandrel_model_info = context.models.load(self.image_to_image_model)
-
-        # Do the upscaling.
-        with spandrel_model_info as spandrel_model:
-            assert isinstance(spandrel_model, SpandrelImageToImageModel)
-
-            # Upscale the image
-            pil_image = self.upscale_image(image, self.tile_size, spandrel_model, context.util.is_canceled)
-
-        image_dto = context.images.save(image=pil_image)
-        return ImageOutput.build(image_dto)
-
-
-@invocation(
-    "spandrel_image_to_image_autoscale",
-    title="Image-to-Image (Autoscale)",
-    tags=["upscale"],
-    category="upscale",
-    version="1.0.0",
-)
-class SpandrelImageToImageAutoscaleInvocation(SpandrelImageToImageInvocation):
-    """Run any spandrel image-to-image model (https://github.com/chaiNNer-org/spandrel) until the target scale is reached."""
-
-    scale: float = InputField(
-        default=4.0,
-        gt=0.0,
-        le=16.0,
-        description="The final scale of the output image. If the model does not upscale the image, this will be ignored.",
-    )
-    fit_to_multiple_of_8: bool = InputField(
-        default=False,
-        description="If true, the output image will be resized to the nearest multiple of 8 in both dimensions.",
-    )
-
-    @torch.inference_mode()
-    def invoke(self, context: InvocationContext) -> ImageOutput:
-        # Images are converted to RGB, because most models don't support an alpha channel. In the future, we may want to
-        # revisit this.
-        image = context.images.get_pil(self.image.image_name, mode="RGB")
-
-        # Load the model.
-        spandrel_model_info = context.models.load(self.image_to_image_model)
-
-        # The target size of the image, determined by the provided scale. We'll run the upscaler until we hit this size.
-        # Later, we may mutate this value if the model doesn't upscale the image or if the user requested a multiple of 8.
-        target_width = int(image.width * self.scale)
-        target_height = int(image.height * self.scale)
-
-        # Do the upscaling.
-        with spandrel_model_info as spandrel_model:
-            assert isinstance(spandrel_model, SpandrelImageToImageModel)
-
-            # First pass of upscaling. Note: `pil_image` will be mutated.
-            pil_image = self.upscale_image(image, self.tile_size, spandrel_model, context.util.is_canceled)
-
-            # Some models don't upscale the image, but we have no way to know this in advance. We'll check if the model
-            # upscaled the image and run the loop below if it did. We'll require the model to upscale both dimensions
-            # to be considered an upscale model.
-            is_upscale_model = pil_image.width > image.width and pil_image.height > image.height
-
-            if is_upscale_model:
-                # This is an upscale model, so we should keep upscaling until we reach the target size.
-                iterations = 1
-                while pil_image.width < target_width or pil_image.height < target_height:
-                    pil_image = self.upscale_image(pil_image, self.tile_size, spandrel_model, context.util.is_canceled)
-                    iterations += 1
-
-                    # Sanity check to prevent excessive or infinite loops. All known upscaling models are at least 2x.
-                    # Our max scale is 16x, so with a 2x model, we should never exceed 16x == 2^4 -> 4 iterations.
-                    # We'll allow one extra iteration "just in case" and bail at 5 upscaling iterations. In practice,
-                    # we should never reach this limit.
-                    if iterations >= 5:
-                        context.logger.warning(
-                            "Upscale loop reached maximum iteration count of 5, stopping upscaling early."
-                        )
-                        break
-            else:
-                # This model doesn't upscale the image. We should ignore the scale parameter, modifying the output size
-                # to be the same as the processed image size.
-
-                # The output size is now the size of the processed image.
-                target_width = pil_image.width
-                target_height = pil_image.height
-
-                # Warn the user if they requested a scale greater than 1.
-                if self.scale > 1:
-                    context.logger.warning(
-                        "Model does not increase the size of the image, but a greater scale than 1 was requested. Image will not be scaled."
-                    )
-
-        # We may need to resize the image to a multiple of 8. Use floor division to ensure we don't scale the image up
-        # in the final resize
-        if self.fit_to_multiple_of_8:
-            target_width = int(target_width // 8 * 8)
-            target_height = int(target_height // 8 * 8)
-
-        # Final resize. Per PIL documentation, Lanczos provides the best quality for both upscale and downscale.
-        # See: https://pillow.readthedocs.io/en/stable/handbook/concepts.html#filters-comparison-table
-        pil_image = pil_image.resize((target_width, target_height), resample=Image.Resampling.LANCZOS)
-
-        image_dto = context.images.save(image=pil_image)
-        return ImageOutput.build(image_dto)

invokeai/app/invocations/strings.py

@@ -2,11 +2,17 @@
 
 import re
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
-from invokeai.app.invocations.fields import InputField, OutputField, UIComponent
-from invokeai.app.invocations.primitives import StringOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    invocation,
+    invocation_output,
+)
+from .fields import InputField, OutputField, UIComponent
+from .primitives import StringOutput
+
 
 @invocation_output("string_pos_neg_output")
 class StringPosNegOutput(BaseInvocationOutput):

invokeai/app/invocations/tiled_multi_diffusion_denoise_latents.py

@@ -7,8 +7,8 @@ from diffusers.models.unets.unet_2d_condition import UNet2DConditionModel
 from diffusers.schedulers.scheduling_utils import SchedulerMixin
 from pydantic import field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, invocation
-from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR, SCHEDULER_NAME_VALUES
 from invokeai.app.invocations.controlnet_image_processors import ControlField
 from invokeai.app.invocations.denoise_latents import DenoiseLatentsInvocation, get_scheduler
 from invokeai.app.invocations.fields import (
@@ -24,12 +24,11 @@ from invokeai.app.invocations.primitives import LatentsOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.lora import LoRAModelRaw
 from invokeai.backend.model_patcher import ModelPatcher
-from invokeai.backend.stable_diffusion.diffusers_pipeline import ControlNetData, PipelineIntermediateState
+from invokeai.backend.stable_diffusion.diffusers_pipeline import ControlNetData
 from invokeai.backend.stable_diffusion.multi_diffusion_pipeline import (
     MultiDiffusionPipeline,
     MultiDiffusionRegionConditioning,
 )
-from invokeai.backend.stable_diffusion.schedulers.schedulers import SCHEDULER_NAME_VALUES
 from invokeai.backend.tiles.tiles import (
     calc_tiles_min_overlap,
 )
@@ -56,15 +55,15 @@ def crop_controlnet_data(control_data: ControlNetData, latent_region: TBLR) -> C
     title="Tiled Multi-Diffusion Denoise Latents",
     tags=["upscale", "denoise"],
     category="latents",
-    classification=Classification.Beta,
+    # TODO(ryand): Reset to 1.0.0 right before release.
     version="1.0.0",
 )
 class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
     """Tiled Multi-Diffusion denoising.
 
-    This node handles automatically tiling the input image, and is primarily intended for global refinement of images
-    in tiled upscaling workflows. Future Multi-Diffusion nodes should allow the user to specify custom regions with
-    different parameters for each region to harness the full power of Multi-Diffusion.
+    This node handles automatically tiling the input image. Future iterations of
+    this node should allow the user to specify custom regions with different parameters for each region to harness the
+    full power of Multi-Diffusion.
 
     This node has a similar interface to the `DenoiseLatents` node, but it has a reduced feature set (no IP-Adapter,
     T2I-Adapter, masking, etc.).
@@ -86,24 +85,21 @@ class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
         description=FieldDescriptions.latents,
         input=Input.Connection,
     )
-    tile_height: int = InputField(
-        default=1024, gt=0, multiple_of=LATENT_SCALE_FACTOR, description="Height of the tiles in image space."
-    )
-    tile_width: int = InputField(
-        default=1024, gt=0, multiple_of=LATENT_SCALE_FACTOR, description="Width of the tiles in image space."
-    )
-    tile_overlap: int = InputField(
-        default=32,
-        multiple_of=LATENT_SCALE_FACTOR,
+    # TODO(ryand): Add multiple-of validation.
+    # TODO(ryand): Smaller defaults might make more sense.
+    tile_height: int = InputField(default=112, gt=0, description="Height of the tiles in latent space.")
+    tile_width: int = InputField(default=112, gt=0, description="Width of the tiles in latent space.")
+    tile_min_overlap: int = InputField(
+        default=16,
         gt=0,
-        description="The overlap between adjacent tiles in pixel space. (Of course, tile merging is applied in latent "
-        "space.) Tiles will be cropped during merging (if necessary) to ensure that they overlap by exactly this "
-        "amount.",
+        description="The minimum overlap between adjacent tiles in latent space. The actual overlap may be larger than "
+        "this to evenly cover the entire image.",
     )
     steps: int = InputField(default=18, gt=0, description=FieldDescriptions.steps)
     cfg_scale: float | list[float] = InputField(default=6.0, description=FieldDescriptions.cfg_scale, title="CFG Scale")
+    # TODO(ryand): The default here should probably be 0.0.
     denoising_start: float = InputField(
-        default=0.0,
+        default=0.65,
         ge=0,
         le=1,
         description=FieldDescriptions.denoising_start,
@@ -154,7 +150,7 @@ class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
                 self.config = FakeVae.FakeVaeConfig()
 
         return MultiDiffusionPipeline(
-            vae=FakeVae(),
+            vae=FakeVae(),  # TODO: oh...
             text_encoder=None,
             tokenizer=None,
             unet=unet,
@@ -166,33 +162,19 @@ class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
 
     @torch.no_grad()
     def invoke(self, context: InvocationContext) -> LatentsOutput:
-        # Convert tile image-space dimensions to latent-space dimensions.
-        latent_tile_height = self.tile_height // LATENT_SCALE_FACTOR
-        latent_tile_width = self.tile_width // LATENT_SCALE_FACTOR
-        latent_tile_overlap = self.tile_overlap // LATENT_SCALE_FACTOR
-
         seed, noise, latents = DenoiseLatentsInvocation.prepare_noise_and_latents(context, self.noise, self.latents)
         _, _, latent_height, latent_width = latents.shape
 
         # Calculate the tile locations to cover the latent-space image.
-        # TODO(ryand): In the future, we may want to revisit the tile overlap strategy. Things to consider:
-        # - How much overlap 'context' to provide for each denoising step.
-        # - How much overlap to use during merging/blending.
-        # - Should we 'jitter' the tile locations in each step so that the seams are in different places?
+        # TODO(ryand): Add constraints on the tile params. Is there a multiple-of constraint?
         tiles = calc_tiles_min_overlap(
             image_height=latent_height,
             image_width=latent_width,
-            tile_height=latent_tile_height,
-            tile_width=latent_tile_width,
-            min_overlap=latent_tile_overlap,
+            tile_height=self.tile_height,
+            tile_width=self.tile_width,
+            min_overlap=self.tile_min_overlap,
         )
 
-        # Get the unet's config so that we can pass the base to sd_step_callback().
-        unet_config = context.models.get_config(self.unet.unet.key)
-
-        def step_callback(state: PipelineIntermediateState) -> None:
-            context.util.sd_step_callback(state, unet_config.base)
-
         # Prepare an iterator that yields the UNet's LoRA models and their weights.
         def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
             for lora in self.unet.loras:
@@ -222,10 +204,9 @@ class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
                 context=context,
                 positive_conditioning_field=self.positive_conditioning,
                 negative_conditioning_field=self.negative_conditioning,
-                device=unet.device,
-                dtype=unet.dtype,
-                latent_height=latent_tile_height,
-                latent_width=latent_tile_width,
+                unet=unet,
+                latent_height=self.tile_height,
+                latent_width=self.tile_width,
                 cfg_scale=self.cfg_scale,
                 steps=self.steps,
                 cfg_rescale_multiplier=self.cfg_rescale_multiplier,
@@ -252,7 +233,7 @@ class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
             for tile, tile_controlnet_data in zip(tiles, controlnet_data_tiles, strict=True):
                 multi_diffusion_conditioning.append(
                     MultiDiffusionRegionConditioning(
-                        region=tile,
+                        region=tile.coords,
                         text_conditioning_data=conditioning_data,
                         control_data=tile_controlnet_data,
                     )
@@ -270,17 +251,17 @@ class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
             # Run Multi-Diffusion denoising.
             result_latents = pipeline.multi_diffusion_denoise(
                 multi_diffusion_conditioning=multi_diffusion_conditioning,
-                target_overlap=latent_tile_overlap,
                 latents=latents,
                 scheduler_step_kwargs=scheduler_step_kwargs,
                 noise=noise,
                 timesteps=timesteps,
                 init_timestep=init_timestep,
-                callback=step_callback,
+                # TODO(ryand): Add proper callback.
+                callback=lambda x: None,
             )
 
-        result_latents = result_latents.to("cpu")
         # TODO(ryand): I copied this from DenoiseLatentsInvocation. I'm not sure if it's actually important.
+        result_latents = result_latents.to("cpu")
         TorchDevice.empty_cache()
 
         name = context.tensors.save(tensor=result_latents)

invokeai/app/invocations/tiled_stable_diffusion_refine.py

@@ -0,0 +1,380 @@
+from contextlib import ExitStack
+from typing import Iterator, Tuple
+
+import numpy as np
+import numpy.typing as npt
+import torch
+from diffusers.models.unets.unet_2d_condition import UNet2DConditionModel
+from PIL import Image
+from pydantic import field_validator
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import DEFAULT_PRECISION, LATENT_SCALE_FACTOR, SCHEDULER_NAME_VALUES
+from invokeai.app.invocations.denoise_latents import DenoiseLatentsInvocation, get_scheduler
+from invokeai.app.invocations.fields import (
+    ConditioningField,
+    FieldDescriptions,
+    ImageField,
+    Input,
+    InputField,
+    UIType,
+)
+from invokeai.app.invocations.image_to_latents import ImageToLatentsInvocation
+from invokeai.app.invocations.latents_to_image import LatentsToImageInvocation
+from invokeai.app.invocations.model import ModelIdentifierField, UNetField, VAEField
+from invokeai.app.invocations.noise import get_noise
+from invokeai.app.invocations.primitives import ImageOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES, prepare_control_image
+from invokeai.backend.lora import LoRAModelRaw
+from invokeai.backend.model_patcher import ModelPatcher
+from invokeai.backend.stable_diffusion.diffusers_pipeline import ControlNetData, image_resized_to_grid_as_tensor
+from invokeai.backend.tiles.tiles import calc_tiles_with_overlap, merge_tiles_with_linear_blending
+from invokeai.backend.tiles.utils import Tile
+from invokeai.backend.util.devices import TorchDevice
+from invokeai.backend.util.hotfixes import ControlNetModel
+
+
+@invocation(
+    "tiled_stable_diffusion_refine",
+    title="Tiled Stable Diffusion Refine",
+    tags=["upscale", "denoise"],
+    category="latents",
+    version="1.0.0",
+)
+class TiledStableDiffusionRefineInvocation(BaseInvocation):
+    """A tiled Stable Diffusion pipeline for refining high resolution images. This invocation is intended to be used to
+    refine an image after upscaling i.e. it is the second step in a typical "tiled upscaling" workflow.
+    """
+
+    image: ImageField = InputField(description="Image to be refined.")
+
+    positive_conditioning: ConditioningField = InputField(
+        description=FieldDescriptions.positive_cond, input=Input.Connection
+    )
+    negative_conditioning: ConditioningField = InputField(
+        description=FieldDescriptions.negative_cond, input=Input.Connection
+    )
+    # TODO(ryand): Add multiple-of validation.
+    tile_height: int = InputField(default=512, gt=0, description="Height of the tiles.")
+    tile_width: int = InputField(default=512, gt=0, description="Width of the tiles.")
+    tile_overlap: int = InputField(
+        default=16,
+        gt=0,
+        description="Target overlap between adjacent tiles (the last row/column may overlap more than this).",
+    )
+    steps: int = InputField(default=18, gt=0, description=FieldDescriptions.steps)
+    cfg_scale: float | list[float] = InputField(default=6.0, description=FieldDescriptions.cfg_scale, title="CFG Scale")
+    denoising_start: float = InputField(
+        default=0.65,
+        ge=0,
+        le=1,
+        description=FieldDescriptions.denoising_start,
+    )
+    denoising_end: float = InputField(default=1.0, ge=0, le=1, description=FieldDescriptions.denoising_end)
+    scheduler: SCHEDULER_NAME_VALUES = InputField(
+        default="euler",
+        description=FieldDescriptions.scheduler,
+        ui_type=UIType.Scheduler,
+    )
+    unet: UNetField = InputField(
+        description=FieldDescriptions.unet,
+        input=Input.Connection,
+        title="UNet",
+    )
+    cfg_rescale_multiplier: float = InputField(
+        title="CFG Rescale Multiplier", default=0, ge=0, lt=1, description=FieldDescriptions.cfg_rescale_multiplier
+    )
+    vae: VAEField = InputField(
+        description=FieldDescriptions.vae,
+        input=Input.Connection,
+    )
+    vae_fp32: bool = InputField(
+        default=DEFAULT_PRECISION == torch.float32, description="Whether to use float32 precision when running the VAE."
+    )
+    # HACK(ryand): We probably want to allow the user to control all of the parameters in ControlField. But, we akwardly
+    # don't want to use the image field. Figure out how best to handle this.
+    # TODO(ryand): Currently, there is no ControlNet preprocessor applied to the tile images. In other words, we pretty
+    # much assume that it is a tile ControlNet. We need to decide how we want to handle this. E.g. find a way to support
+    # CN preprocessors, raise a clear warning when a non-tile CN model is selected, hardcode the supported CN models,
+    # etc.
+    control_model: ModelIdentifierField = InputField(
+        description=FieldDescriptions.controlnet_model, ui_type=UIType.ControlNetModel
+    )
+    control_weight: float = InputField(default=0.6)
+
+    @field_validator("cfg_scale")
+    def ge_one(cls, v: list[float] | float) -> list[float] | float:
+        """Validate that all cfg_scale values are >= 1"""
+        if isinstance(v, list):
+            for i in v:
+                if i < 1:
+                    raise ValueError("cfg_scale must be greater than 1")
+        else:
+            if v < 1:
+                raise ValueError("cfg_scale must be greater than 1")
+        return v
+
+    @staticmethod
+    def crop_latents_to_tile(latents: torch.Tensor, image_tile: Tile) -> torch.Tensor:
+        """Crop the latent-space tensor to the area corresponding to the image-space tile.
+        The tile coordinates must be divisible by the LATENT_SCALE_FACTOR.
+        """
+        for coord in [image_tile.coords.top, image_tile.coords.left, image_tile.coords.right, image_tile.coords.bottom]:
+            if coord % LATENT_SCALE_FACTOR != 0:
+                raise ValueError(
+                    f"The tile coordinates must all be divisible by the latent scale factor"
+                    f" ({LATENT_SCALE_FACTOR}). {image_tile.coords=}."
+                )
+        assert latents.dim() == 4  # We expect: (batch_size, channels, height, width).
+
+        top = image_tile.coords.top // LATENT_SCALE_FACTOR
+        left = image_tile.coords.left // LATENT_SCALE_FACTOR
+        bottom = image_tile.coords.bottom // LATENT_SCALE_FACTOR
+        right = image_tile.coords.right // LATENT_SCALE_FACTOR
+        return latents[..., top:bottom, left:right]
+
+    def run_controlnet(
+        self,
+        image: Image.Image,
+        controlnet_model: ControlNetModel,
+        weight: float,
+        do_classifier_free_guidance: bool,
+        width: int,
+        height: int,
+        device: torch.device,
+        dtype: torch.dtype,
+        control_mode: CONTROLNET_MODE_VALUES = "balanced",
+        resize_mode: CONTROLNET_RESIZE_VALUES = "just_resize_simple",
+    ) -> ControlNetData:
+        control_image = prepare_control_image(
+            image=image,
+            do_classifier_free_guidance=do_classifier_free_guidance,
+            width=width,
+            height=height,
+            device=device,
+            dtype=dtype,
+            control_mode=control_mode,
+            resize_mode=resize_mode,
+        )
+        return ControlNetData(
+            model=controlnet_model,
+            image_tensor=control_image,
+            weight=weight,
+            begin_step_percent=0.0,
+            end_step_percent=1.0,
+            control_mode=control_mode,
+            # Any resizing needed should currently be happening in prepare_control_image(), but adding resize_mode to
+            # ControlNetData in case needed in the future.
+            resize_mode=resize_mode,
+        )
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> ImageOutput:
+        # TODO(ryand): Expose the seed parameter.
+        seed = 0
+
+        # Load the input image.
+        input_image = context.images.get_pil(self.image.image_name)
+
+        # Calculate the tile locations to cover the image.
+        # We have selected this tiling strategy to make it easy to achieve tile coords that are multiples of 8. This
+        # facilitates conversions between image space and latent space.
+        # TODO(ryand): Expose these tiling parameters. (Keep in mind the multiple-of constraints on these params.)
+        tiles = calc_tiles_with_overlap(
+            image_height=input_image.height,
+            image_width=input_image.width,
+            tile_height=self.tile_height,
+            tile_width=self.tile_width,
+            overlap=self.tile_overlap,
+        )
+
+        # Convert the input image to a torch.Tensor.
+        input_image_torch = image_resized_to_grid_as_tensor(input_image.convert("RGB"), multiple_of=LATENT_SCALE_FACTOR)
+        input_image_torch = input_image_torch.unsqueeze(0)  # Add a batch dimension.
+        # Validate our assumptions about the shape of input_image_torch.
+        assert input_image_torch.dim() == 4  # We expect: (batch_size, channels, height, width).
+        assert input_image_torch.shape[:2] == (1, 3)
+
+        # Split the input image into tiles in torch.Tensor format.
+        image_tiles_torch: list[torch.Tensor] = []
+        for tile in tiles:
+            image_tile = input_image_torch[
+                :,
+                :,
+                tile.coords.top : tile.coords.bottom,
+                tile.coords.left : tile.coords.right,
+            ]
+            image_tiles_torch.append(image_tile)
+
+        # Split the input image into tiles in numpy format.
+        # TODO(ryand): We currently maintain both np.ndarray and torch.Tensor tiles. Ideally, all operations should work
+        # with torch.Tensor tiles.
+        input_image_np = np.array(input_image)
+        image_tiles_np: list[npt.NDArray[np.uint8]] = []
+        for tile in tiles:
+            image_tile_np = input_image_np[
+                tile.coords.top : tile.coords.bottom,
+                tile.coords.left : tile.coords.right,
+                :,
+            ]
+            image_tiles_np.append(image_tile_np)
+
+        # VAE-encode each image tile independently.
+        # TODO(ryand): Is there any advantage to VAE-encoding the entire image before splitting it into tiles? What
+        # about for decoding?
+        vae_info = context.models.load(self.vae.vae)
+        latent_tiles: list[torch.Tensor] = []
+        for image_tile_torch in image_tiles_torch:
+            latent_tiles.append(
+                ImageToLatentsInvocation.vae_encode(
+                    vae_info=vae_info, upcast=self.vae_fp32, tiled=False, image_tensor=image_tile_torch
+                )
+            )
+
+        # Generate noise with dimensions corresponding to the full image in latent space.
+        # It is important that the noise tensor is generated at the full image dimension and then tiled, rather than
+        # generating for each tile independently. This ensures that overlapping regions between tiles use the same
+        # noise.
+        assert input_image_torch.shape[2] % LATENT_SCALE_FACTOR == 0
+        assert input_image_torch.shape[3] % LATENT_SCALE_FACTOR == 0
+        global_noise = get_noise(
+            width=input_image_torch.shape[3],
+            height=input_image_torch.shape[2],
+            device=TorchDevice.choose_torch_device(),
+            seed=seed,
+            downsampling_factor=LATENT_SCALE_FACTOR,
+            use_cpu=True,
+        )
+
+        # Crop the global noise into tiles.
+        noise_tiles = [self.crop_latents_to_tile(latents=global_noise, image_tile=t) for t in tiles]
+
+        # Prepare an iterator that yields the UNet's LoRA models and their weights.
+        def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
+            for lora in self.unet.loras:
+                lora_info = context.models.load(lora.lora)
+                assert isinstance(lora_info.model, LoRAModelRaw)
+                yield (lora_info.model, lora.weight)
+                del lora_info
+
+        # Load the UNet model.
+        unet_info = context.models.load(self.unet.unet)
+
+        refined_latent_tiles: list[torch.Tensor] = []
+        with ExitStack() as exit_stack, unet_info as unet, ModelPatcher.apply_lora_unet(unet, _lora_loader()):
+            assert isinstance(unet, UNet2DConditionModel)
+            scheduler = get_scheduler(
+                context=context,
+                scheduler_info=self.unet.scheduler,
+                scheduler_name=self.scheduler,
+                seed=seed,
+            )
+            pipeline = DenoiseLatentsInvocation.create_pipeline(unet=unet, scheduler=scheduler)
+
+            # Prepare the prompt conditioning data. The same prompt conditioning is applied to all tiles.
+            # Assume that all tiles have the same shape.
+            _, _, latent_height, latent_width = latent_tiles[0].shape
+            conditioning_data = DenoiseLatentsInvocation.get_conditioning_data(
+                context=context,
+                positive_conditioning_field=self.positive_conditioning,
+                negative_conditioning_field=self.negative_conditioning,
+                unet=unet,
+                latent_height=latent_height,
+                latent_width=latent_width,
+                cfg_scale=self.cfg_scale,
+                steps=self.steps,
+                cfg_rescale_multiplier=self.cfg_rescale_multiplier,
+            )
+
+            # Load the ControlNet model.
+            # TODO(ryand): Support multiple ControlNet models.
+            controlnet_model = exit_stack.enter_context(context.models.load(self.control_model))
+            assert isinstance(controlnet_model, ControlNetModel)
+
+            # Denoise (i.e. "refine") each tile independently.
+            for image_tile_np, latent_tile, noise_tile in zip(image_tiles_np, latent_tiles, noise_tiles, strict=True):
+                assert latent_tile.shape == noise_tile.shape
+
+                # Prepare a PIL Image for ControlNet processing.
+                # TODO(ryand): This is a bit awkward that we have to prepare both torch.Tensor and PIL.Image versions of
+                # the tiles. Ideally, the ControlNet code should be able to work with Tensors.
+                image_tile_pil = Image.fromarray(image_tile_np)
+
+                # Run the ControlNet on the image tile.
+                height, width, _ = image_tile_np.shape
+                # The height and width must be evenly divisible by LATENT_SCALE_FACTOR. This is enforced earlier, but we
+                # validate this assumption here.
+                assert height % LATENT_SCALE_FACTOR == 0
+                assert width % LATENT_SCALE_FACTOR == 0
+                controlnet_data = self.run_controlnet(
+                    image=image_tile_pil,
+                    controlnet_model=controlnet_model,
+                    weight=self.control_weight,
+                    do_classifier_free_guidance=True,
+                    width=width,
+                    height=height,
+                    device=controlnet_model.device,
+                    dtype=controlnet_model.dtype,
+                    control_mode="balanced",
+                    resize_mode="just_resize_simple",
+                )
+
+                timesteps, init_timestep, scheduler_step_kwargs = DenoiseLatentsInvocation.init_scheduler(
+                    scheduler,
+                    device=unet.device,
+                    steps=self.steps,
+                    denoising_start=self.denoising_start,
+                    denoising_end=self.denoising_end,
+                    seed=seed,
+                )
+
+                # TODO(ryand): Think about when/if latents/noise should be moved off of the device to save VRAM.
+                latent_tile = latent_tile.to(device=unet.device, dtype=unet.dtype)
+                noise_tile = noise_tile.to(device=unet.device, dtype=unet.dtype)
+                refined_latent_tile = pipeline.latents_from_embeddings(
+                    latents=latent_tile,
+                    timesteps=timesteps,
+                    init_timestep=init_timestep,
+                    noise=noise_tile,
+                    seed=seed,
+                    mask=None,
+                    masked_latents=None,
+                    scheduler_step_kwargs=scheduler_step_kwargs,
+                    conditioning_data=conditioning_data,
+                    control_data=[controlnet_data],
+                    ip_adapter_data=None,
+                    t2i_adapter_data=None,
+                    callback=lambda x: None,
+                )
+                refined_latent_tiles.append(refined_latent_tile)
+
+        # VAE-decode each refined latent tile independently.
+        refined_image_tiles: list[Image.Image] = []
+        for refined_latent_tile in refined_latent_tiles:
+            refined_image_tile = LatentsToImageInvocation.vae_decode(
+                context=context,
+                vae_info=vae_info,
+                seamless_axes=self.vae.seamless_axes,
+                latents=refined_latent_tile,
+                use_fp32=self.vae_fp32,
+                use_tiling=False,
+            )
+            refined_image_tiles.append(refined_image_tile)
+
+        # TODO(ryand): I copied this from DenoiseLatentsInvocation. I'm not sure if it's actually important.
+        TorchDevice.empty_cache()
+
+        # Merge the refined image tiles back into a single image.
+        refined_image_tiles_np = [np.array(t) for t in refined_image_tiles]
+        merged_image_np = np.zeros(shape=(input_image.height, input_image.width, 3), dtype=np.uint8)
+        # TODO(ryand): Tune the blend_amount. Should this be exposed as a parameter?
+        merge_tiles_with_linear_blending(
+            dst_image=merged_image_np, tiles=tiles, tile_images=refined_image_tiles_np, blend_amount=self.tile_overlap
+        )
+
+        # Save the refined image and return its reference.
+        merged_image_pil = Image.fromarray(merged_image_np)
+        image_dto = context.images.save(image=merged_image_pil)
+
+        return ImageOutput.build(image_dto)

invokeai/app/invocations/upscale.py

@@ -6,13 +6,15 @@ import numpy as np
 from PIL import Image
 from pydantic import ConfigDict
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import ImageField, InputField, WithBoard, WithMetadata
+from invokeai.app.invocations.fields import ImageField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.image_util.basicsr.rrdbnet_arch import RRDBNet
 from invokeai.backend.image_util.realesrgan.realesrgan import RealESRGAN
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField, WithBoard, WithMetadata
+
 # TODO: Populate this from disk?
 # TODO: Use model manager to load?
 ESRGAN_MODELS = Literal[

invokeai/app/services/board_image_records/board_image_records_sqlite.py

@@ -2,11 +2,12 @@ import sqlite3
 import threading
 from typing import Optional, cast
 
-from invokeai.app.services.board_image_records.board_image_records_base import BoardImageRecordStorageBase
 from invokeai.app.services.image_records.image_records_common import ImageRecord, deserialize_image_record
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
 
+from .board_image_records_base import BoardImageRecordStorageBase
+
 
 class SqliteBoardImageRecordStorage(BoardImageRecordStorageBase):
     _conn: sqlite3.Connection

invokeai/app/services/board_images/board_images_default.py

@@ -1,8 +1,9 @@
 from typing import Optional
 
-from invokeai.app.services.board_images.board_images_base import BoardImagesServiceABC
 from invokeai.app.services.invoker import Invoker
 
+from .board_images_base import BoardImagesServiceABC
+
 
 class BoardImagesService(BoardImagesServiceABC):
     __invoker: Invoker

invokeai/app/services/board_records/board_records_base.py

@@ -1,8 +1,9 @@
 from abc import ABC, abstractmethod
 
-from invokeai.app.services.board_records.board_records_common import BoardChanges, BoardRecord
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 
+from .board_records_common import BoardChanges, BoardRecord
+
 
 class BoardRecordStorageBase(ABC):
     """Low-level service responsible for interfacing with the board record store."""
@@ -39,12 +40,16 @@ class BoardRecordStorageBase(ABC):
 
     @abstractmethod
     def get_many(
-        self, offset: int = 0, limit: int = 10, include_archived: bool = False
+        self,
+        offset: int = 0,
+        limit: int = 10,
     ) -> OffsetPaginatedResults[BoardRecord]:
         """Gets many board records."""
         pass
 
     @abstractmethod
-    def get_all(self, include_archived: bool = False) -> list[BoardRecord]:
+    def get_all(
+        self,
+    ) -> list[BoardRecord]:
         """Gets all board records."""
         pass

invokeai/app/services/board_records/board_records_common.py

@@ -22,10 +22,6 @@ class BoardRecord(BaseModelExcludeNull):
     """The updated timestamp of the image."""
     cover_image_name: Optional[str] = Field(default=None, description="The name of the cover image of the board.")
     """The name of the cover image of the board."""
-    archived: bool = Field(description="Whether or not the board is archived.")
-    """Whether or not the board is archived."""
-    is_private: Optional[bool] = Field(default=None, description="Whether the board is private.")
-    """Whether the board is private."""
 
 
 def deserialize_board_record(board_dict: dict) -> BoardRecord:
@@ -39,8 +35,6 @@ def deserialize_board_record(board_dict: dict) -> BoardRecord:
     created_at = board_dict.get("created_at", get_iso_timestamp())
     updated_at = board_dict.get("updated_at", get_iso_timestamp())
     deleted_at = board_dict.get("deleted_at", get_iso_timestamp())
-    archived = board_dict.get("archived", False)
-    is_private = board_dict.get("is_private", False)
 
     return BoardRecord(
         board_id=board_id,
@@ -49,15 +43,12 @@ def deserialize_board_record(board_dict: dict) -> BoardRecord:
         created_at=created_at,
         updated_at=updated_at,
         deleted_at=deleted_at,
-        archived=archived,
-        is_private=is_private,
     )
 
 
 class BoardChanges(BaseModel, extra="forbid"):
     board_name: Optional[str] = Field(default=None, description="The board's new name.")
     cover_image_name: Optional[str] = Field(default=None, description="The name of the board's new cover image.")
-    archived: Optional[bool] = Field(default=None, description="Whether or not the board is archived")
 
 
 class BoardRecordNotFoundException(Exception):

invokeai/app/services/board_records/board_records_sqlite.py

@@ -2,8 +2,12 @@ import sqlite3
 import threading
 from typing import Union, cast
 
-from invokeai.app.services.board_records.board_records_base import BoardRecordStorageBase
-from invokeai.app.services.board_records.board_records_common import (
+from invokeai.app.services.shared.pagination import OffsetPaginatedResults
+from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
+from invokeai.app.util.misc import uuid_string
+
+from .board_records_base import BoardRecordStorageBase
+from .board_records_common import (
     BoardChanges,
     BoardRecord,
     BoardRecordDeleteException,
@@ -11,9 +15,6 @@ from invokeai.app.services.board_records.board_records_common import (
     BoardRecordSaveException,
     deserialize_board_record,
 )
-from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
-from invokeai.app.util.misc import uuid_string
 
 
 class SqliteBoardRecordStorage(BoardRecordStorageBase):
@@ -124,17 +125,6 @@ class SqliteBoardRecordStorage(BoardRecordStorageBase):
                     (changes.cover_image_name, board_id),
                 )
 
-            # Change the archived status of a board
-            if changes.archived is not None:
-                self._cursor.execute(
-                    """--sql
-                    UPDATE boards
-                    SET archived = ?
-                    WHERE board_id = ?;
-                    """,
-                    (changes.archived, board_id),
-                )
-
             self._conn.commit()
         except sqlite3.Error as e:
             self._conn.rollback()
@@ -144,49 +134,35 @@ class SqliteBoardRecordStorage(BoardRecordStorageBase):
         return self.get(board_id)
 
     def get_many(
-        self, offset: int = 0, limit: int = 10, include_archived: bool = False
+        self,
+        offset: int = 0,
+        limit: int = 10,
     ) -> OffsetPaginatedResults[BoardRecord]:
         try:
             self._lock.acquire()
 
-            # Build base query
-            base_query = """
+            # Get all the boards
+            self._cursor.execute(
+                """--sql
                 SELECT *
                 FROM boards
-                {archived_filter}
                 ORDER BY created_at DESC
                 LIMIT ? OFFSET ?;
-            """
-
-            # Determine archived filter condition
-            if include_archived:
-                archived_filter = ""
-            else:
-                archived_filter = "WHERE archived = 0"
-
-            final_query = base_query.format(archived_filter=archived_filter)
-
-            # Execute query to fetch boards
-            self._cursor.execute(final_query, (limit, offset))
+                """,
+                (limit, offset),
+            )
 
             result = cast(list[sqlite3.Row], self._cursor.fetchall())
             boards = [deserialize_board_record(dict(r)) for r in result]
 
-            # Determine count query
-            if include_archived:
-                count_query = """
-                    SELECT COUNT(*)
-                    FROM boards;
+            # Get the total number of boards
+            self._cursor.execute(
+                """--sql
+                SELECT COUNT(*)
+                FROM boards
+                WHERE 1=1;
                 """
-            else:
-                count_query = """
-                    SELECT COUNT(*)
-                    FROM boards
-                    WHERE archived = 0;
-                """
-
-            # Execute count query
-            self._cursor.execute(count_query)
+            )
 
             count = cast(int, self._cursor.fetchone()[0])
 
@@ -198,25 +174,20 @@ class SqliteBoardRecordStorage(BoardRecordStorageBase):
         finally:
             self._lock.release()
 
-    def get_all(self, include_archived: bool = False) -> list[BoardRecord]:
+    def get_all(
+        self,
+    ) -> list[BoardRecord]:
         try:
             self._lock.acquire()
 
-            base_query = """
+            # Get all the boards
+            self._cursor.execute(
+                """--sql
                 SELECT *
                 FROM boards
-                {archived_filter}
                 ORDER BY created_at DESC
-            """
-
-            if include_archived:
-                archived_filter = ""
-            else:
-                archived_filter = "WHERE archived = 0"
-
-            final_query = base_query.format(archived_filter=archived_filter)
-
-            self._cursor.execute(final_query)
+                """
+            )
 
             result = cast(list[sqlite3.Row], self._cursor.fetchall())
             boards = [deserialize_board_record(dict(r)) for r in result]

invokeai/app/services/boards/boards_base.py

@@ -1,9 +1,10 @@
 from abc import ABC, abstractmethod
 
 from invokeai.app.services.board_records.board_records_common import BoardChanges
-from invokeai.app.services.boards.boards_common import BoardDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 
+from .boards_common import BoardDTO
+
 
 class BoardServiceABC(ABC):
     """High-level service for board management."""
@@ -43,12 +44,16 @@ class BoardServiceABC(ABC):
 
     @abstractmethod
     def get_many(
-        self, offset: int = 0, limit: int = 10, include_archived: bool = False
+        self,
+        offset: int = 0,
+        limit: int = 10,
     ) -> OffsetPaginatedResults[BoardDTO]:
         """Gets many boards."""
         pass
 
     @abstractmethod
-    def get_all(self, include_archived: bool = False) -> list[BoardDTO]:
+    def get_all(
+        self,
+    ) -> list[BoardDTO]:
         """Gets all boards."""
         pass

invokeai/app/services/boards/boards_common.py

@@ -2,7 +2,7 @@ from typing import Optional
 
 from pydantic import Field
 
-from invokeai.app.services.board_records.board_records_common import BoardRecord
+from ..board_records.board_records_common import BoardRecord
 
 
 class BoardDTO(BoardRecord):

invokeai/app/services/boards/boards_default.py

@@ -1,9 +1,11 @@
 from invokeai.app.services.board_records.board_records_common import BoardChanges
-from invokeai.app.services.boards.boards_base import BoardServiceABC
-from invokeai.app.services.boards.boards_common import BoardDTO, board_record_to_dto
+from invokeai.app.services.boards.boards_common import BoardDTO
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 
+from .boards_base import BoardServiceABC
+from .boards_common import board_record_to_dto
+
 
 class BoardService(BoardServiceABC):
     __invoker: Invoker
@@ -46,10 +48,8 @@ class BoardService(BoardServiceABC):
     def delete(self, board_id: str) -> None:
         self.__invoker.services.board_records.delete(board_id)
 
-    def get_many(
-        self, offset: int = 0, limit: int = 10, include_archived: bool = False
-    ) -> OffsetPaginatedResults[BoardDTO]:
-        board_records = self.__invoker.services.board_records.get_many(offset, limit, include_archived)
+    def get_many(self, offset: int = 0, limit: int = 10) -> OffsetPaginatedResults[BoardDTO]:
+        board_records = self.__invoker.services.board_records.get_many(offset, limit)
         board_dtos = []
         for r in board_records.items:
             cover_image = self.__invoker.services.image_records.get_most_recent_image_for_board(r.board_id)
@@ -63,8 +63,8 @@ class BoardService(BoardServiceABC):
 
         return OffsetPaginatedResults[BoardDTO](items=board_dtos, offset=offset, limit=limit, total=len(board_dtos))
 
-    def get_all(self, include_archived: bool = False) -> list[BoardDTO]:
-        board_records = self.__invoker.services.board_records.get_all(include_archived)
+    def get_all(self) -> list[BoardDTO]:
+        board_records = self.__invoker.services.board_records.get_all()
         board_dtos = []
         for r in board_records:
             cover_image = self.__invoker.services.image_records.get_most_recent_image_for_board(r.board_id)

invokeai/app/services/bulk_download/bulk_download_default.py

@@ -4,7 +4,6 @@ from typing import Optional, Union
 from zipfile import ZipFile
 
 from invokeai.app.services.board_records.board_records_common import BoardRecordNotFoundException
-from invokeai.app.services.bulk_download.bulk_download_base import BulkDownloadBase
 from invokeai.app.services.bulk_download.bulk_download_common import (
     DEFAULT_BULK_DOWNLOAD_ID,
     BulkDownloadException,
@@ -16,6 +15,8 @@ from invokeai.app.services.images.images_common import ImageDTO
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.util.misc import uuid_string
 
+from .bulk_download_base import BulkDownloadBase
+
 
 class BulkDownloadService(BulkDownloadBase):
     def start(self, invoker: Invoker) -> None:

invokeai/app/services/config/__init__.py

@@ -1,6 +1,7 @@
 """Init file for InvokeAI configure package."""
 
 from invokeai.app.services.config.config_common import PagingArgumentParser
-from invokeai.app.services.config.config_default import InvokeAIAppConfig, get_config
+
+from .config_default import InvokeAIAppConfig, get_config
 
 __all__ = ["InvokeAIAppConfig", "get_config", "PagingArgumentParser"]

invokeai/app/services/config/config_default.py

@@ -3,7 +3,6 @@
 
 from __future__ import annotations
 
-import copy
 import locale
 import os
 import re
@@ -26,13 +25,14 @@ DB_FILE = Path("invokeai.db")
 LEGACY_INIT_FILE = Path("invokeai.init")
 DEFAULT_RAM_CACHE = 10.0
 DEFAULT_VRAM_CACHE = 0.25
+DEFAULT_CONVERT_CACHE = 20.0
 DEVICE = Literal["auto", "cpu", "cuda", "cuda:1", "mps"]
 PRECISION = Literal["auto", "float16", "bfloat16", "float32"]
 ATTENTION_TYPE = Literal["auto", "normal", "xformers", "sliced", "torch-sdp"]
 ATTENTION_SLICE_SIZE = Literal["auto", "balanced", "max", 1, 2, 3, 4, 5, 6, 7, 8]
 LOG_FORMAT = Literal["plain", "color", "syslog", "legacy"]
 LOG_LEVEL = Literal["debug", "info", "warning", "error", "critical"]
-CONFIG_SCHEMA_VERSION = "4.0.2"
+CONFIG_SCHEMA_VERSION = "4.0.1"
 
 
 def get_default_ram_cache_size() -> float:
@@ -85,13 +85,12 @@ class InvokeAIAppConfig(BaseSettings):
         log_tokenization: Enable logging of parsed prompt tokens.
         patchmatch: Enable patchmatch inpaint code.
         models_dir: Path to the models directory.
-        convert_cache_dir: Path to the converted models cache directory (DEPRECATED, but do not delete because it is needed for migration from previous versions).
+        convert_cache_dir: Path to the converted models cache directory. When loading a non-diffusers model, it will be converted and store on disk at this location.
         download_cache_dir: Path to the directory that contains dynamically downloaded models.
         legacy_conf_dir: Path to directory of legacy checkpoint config files.
         db_dir: Path to InvokeAI databases directory.
         outputs_dir: Path to directory for outputs.
         custom_nodes_dir: Path to directory for custom nodes.
-        style_presets_dir: Path to directory for style presets.
         log_handlers: Log handler. Valid options are "console", "file=<path>", "syslog=path|address:host:port", "http=<url>".
         log_format: Log format. Use "plain" for text-only, "color" for colorized output, "legacy" for 2.3-style logging and "syslog" for syslog-style.<br>Valid values: `plain`, `color`, `syslog`, `legacy`
         log_level: Emit logging messages at this level or higher.<br>Valid values: `debug`, `info`, `warning`, `error`, `critical`
@@ -103,6 +102,7 @@ class InvokeAIAppConfig(BaseSettings):
         profiles_dir: Path to profiles output directory.
         ram: Maximum memory amount used by memory model cache for rapid switching (GB).
         vram: Amount of VRAM reserved for model storage (GB).
+        convert_cache: Maximum size of on-disk converted models cache (GB).
         lazy_offload: Keep models in VRAM until their space is needed.
         log_memory_usage: If True, a memory snapshot will be captured before and after every model cache operation, and the result will be logged (at debug level). There is a time cost to capturing the memory snapshots, so it is recommended to only enable this feature if you are actively inspecting the model cache's behaviour.
         device: Preferred execution device. `auto` will choose the device depending on the hardware platform and the installed torch capabilities.<br>Valid values: `auto`, `cpu`, `cuda`, `cuda:1`, `mps`
@@ -113,7 +113,6 @@ class InvokeAIAppConfig(BaseSettings):
         force_tiled_decode: Whether to enable tiled VAE decode (reduces memory consumption with some performance penalty).
         pil_compress_level: The compress_level setting of PIL.Image.save(), used for PNG encoding. All settings are lossless. 0 = no compression, 1 = fastest with slightly larger filesize, 9 = slowest with smallest filesize. 1 is typically the best setting.
         max_queue_size: Maximum number of items in the session queue.
-        clear_queue_on_startup: Empties session queue on startup.
         allow_nodes: List of nodes to allow. Omit to allow all.
         deny_nodes: List of nodes to deny. Omit to deny none.
         node_cache_size: How many cached nodes to keep in memory.
@@ -148,13 +147,12 @@ class InvokeAIAppConfig(BaseSettings):
 
     # PATHS
     models_dir:                    Path = Field(default=Path("models"),     description="Path to the models directory.")
-    convert_cache_dir:             Path = Field(default=Path("models/.convert_cache"), description="Path to the converted models cache directory (DEPRECATED, but do not delete because it is needed for migration from previous versions).")
+    convert_cache_dir:             Path = Field(default=Path("models/.convert_cache"), description="Path to the converted models cache directory. When loading a non-diffusers model, it will be converted and store on disk at this location.")
     download_cache_dir:            Path = Field(default=Path("models/.download_cache"), description="Path to the directory that contains dynamically downloaded models.")
     legacy_conf_dir:               Path = Field(default=Path("configs"), description="Path to directory of legacy checkpoint config files.")
     db_dir:                        Path = Field(default=Path("databases"),  description="Path to InvokeAI databases directory.")
     outputs_dir:                   Path = Field(default=Path("outputs"),    description="Path to directory for outputs.")
     custom_nodes_dir:              Path = Field(default=Path("nodes"),      description="Path to directory for custom nodes.")
-    style_presets_dir:      Path = Field(default=Path("style_presets"),      description="Path to directory for style presets.")
 
     # LOGGING
     log_handlers:             list[str] = Field(default=["console"],        description='Log handler. Valid options are "console", "file=<path>", "syslog=path|address:host:port", "http=<url>".')
@@ -171,8 +169,9 @@ class InvokeAIAppConfig(BaseSettings):
     profiles_dir:                  Path = Field(default=Path("profiles"),   description="Path to profiles output directory.")
 
     # CACHE
-    ram:                           float = Field(default_factory=get_default_ram_cache_size, gt=0, description="Maximum memory amount used by memory model cache for rapid switching (GB).")
-    vram:                          float = Field(default=DEFAULT_VRAM_CACHE, ge=0, description="Amount of VRAM reserved for model storage (GB).")
+    ram:                          float = Field(default_factory=get_default_ram_cache_size, gt=0, description="Maximum memory amount used by memory model cache for rapid switching (GB).")
+    vram:                         float = Field(default=DEFAULT_VRAM_CACHE, ge=0, description="Amount of VRAM reserved for model storage (GB).")
+    convert_cache:                float = Field(default=DEFAULT_CONVERT_CACHE, ge=0, description="Maximum size of on-disk converted models cache (GB).")
     lazy_offload:                  bool = Field(default=True,               description="Keep models in VRAM until their space is needed.")
     log_memory_usage:              bool = Field(default=False,              description="If True, a memory snapshot will be captured before and after every model cache operation, and the result will be logged (at debug level). There is a time cost to capturing the memory snapshots, so it is recommended to only enable this feature if you are actively inspecting the model cache's behaviour.")
 
@@ -187,7 +186,6 @@ class InvokeAIAppConfig(BaseSettings):
     force_tiled_decode:            bool = Field(default=False,              description="Whether to enable tiled VAE decode (reduces memory consumption with some performance penalty).")
     pil_compress_level:             int = Field(default=1,                  description="The compress_level setting of PIL.Image.save(), used for PNG encoding. All settings are lossless. 0 = no compression, 1 = fastest with slightly larger filesize, 9 = slowest with smallest filesize. 1 is typically the best setting.")
     max_queue_size:                 int = Field(default=10000, gt=0,        description="Maximum number of items in the session queue.")
-    clear_queue_on_startup:        bool = Field(default=False,              description="Empties session queue on startup.")
 
     # NODES
     allow_nodes:    Optional[list[str]] = Field(default=None,               description="List of nodes to allow. Omit to allow all.")
@@ -302,11 +300,6 @@ class InvokeAIAppConfig(BaseSettings):
         """Path to the models directory, resolved to an absolute path.."""
         return self._resolve(self.models_dir)
 
-    @property
-    def style_presets_path(self) -> Path:
-        """Path to the style presets directory, resolved to an absolute path.."""
-        return self._resolve(self.style_presets_dir)
-
     @property
     def convert_cache_path(self) -> Path:
         """Path to the converted cache models directory, resolved to an absolute path.."""
@@ -362,14 +355,14 @@ class DefaultInvokeAIAppConfig(InvokeAIAppConfig):
         return (init_settings,)
 
 
-def migrate_v3_config_dict(config_dict: dict[str, Any]) -> dict[str, Any]:
-    """Migrate a v3 config dictionary to a v4.0.0.
+def migrate_v3_config_dict(config_dict: dict[str, Any]) -> InvokeAIAppConfig:
+    """Migrate a v3 config dictionary to a current config object.
 
     Args:
         config_dict: A dictionary of settings from a v3 config file.
 
     Returns:
-        An `InvokeAIAppConfig` config dict.
+        An instance of `InvokeAIAppConfig` with the migrated settings.
 
     """
     parsed_config_dict: dict[str, Any] = {}
@@ -403,41 +396,32 @@ def migrate_v3_config_dict(config_dict: dict[str, Any]) -> dict[str, Any]:
             elif k in InvokeAIAppConfig.model_fields:
                 # skip unknown fields
                 parsed_config_dict[k] = v
-    parsed_config_dict["schema_version"] = "4.0.0"
-    return parsed_config_dict
+    # When migrating the config file, we should not include currently-set environment variables.
+    config = DefaultInvokeAIAppConfig.model_validate(parsed_config_dict)
+
+    return config
 
 
-def migrate_v4_0_0_to_4_0_1_config_dict(config_dict: dict[str, Any]) -> dict[str, Any]:
-    """Migrate v4.0.0 config dictionary to a v4.0.1 config dictionary
+def migrate_v4_0_0_config_dict(config_dict: dict[str, Any]) -> InvokeAIAppConfig:
+    """Migrate v4.0.0 config dictionary to a current config object.
 
     Args:
         config_dict: A dictionary of settings from a v4.0.0 config file.
 
     Returns:
-        A config dict with the settings migrated to v4.0.1.
+        An instance of `InvokeAIAppConfig` with the migrated settings.
     """
-    parsed_config_dict: dict[str, Any] = copy.deepcopy(config_dict)
-    # precision "autocast" was replaced by "auto" in v4.0.1
-    if parsed_config_dict.get("precision") == "autocast":
-        parsed_config_dict["precision"] = "auto"
-    parsed_config_dict["schema_version"] = "4.0.1"
-    return parsed_config_dict
-
-
-def migrate_v4_0_1_to_4_0_2_config_dict(config_dict: dict[str, Any]) -> dict[str, Any]:
-    """Migrate v4.0.1 config dictionary to a v4.0.2 config dictionary.
-
-    Args:
-        config_dict: A dictionary of settings from a v4.0.1 config file.
-
-    Returns:
-        An config dict with the settings migrated to v4.0.2.
-    """
-    parsed_config_dict: dict[str, Any] = copy.deepcopy(config_dict)
-    # convert_cache was removed in 4.0.2
-    parsed_config_dict.pop("convert_cache", None)
-    parsed_config_dict["schema_version"] = "4.0.2"
-    return parsed_config_dict
+    parsed_config_dict: dict[str, Any] = {}
+    for k, v in config_dict.items():
+        # autocast was removed from precision in v4.0.1
+        if k == "precision" and v == "autocast":
+            parsed_config_dict["precision"] = "auto"
+        else:
+            parsed_config_dict[k] = v
+        if k == "schema_version":
+            parsed_config_dict[k] = CONFIG_SCHEMA_VERSION
+    config = DefaultInvokeAIAppConfig.model_validate(parsed_config_dict)
+    return config
 
 
 def load_and_migrate_config(config_path: Path) -> InvokeAIAppConfig:
@@ -451,31 +435,27 @@ def load_and_migrate_config(config_path: Path) -> InvokeAIAppConfig:
     """
     assert config_path.suffix == ".yaml"
     with open(config_path, "rt", encoding=locale.getpreferredencoding()) as file:
-        loaded_config_dict: dict[str, Any] = yaml.safe_load(file)
+        loaded_config_dict = yaml.safe_load(file)
 
     assert isinstance(loaded_config_dict, dict)
 
-    migrated = False
     if "InvokeAI" in loaded_config_dict:
-        migrated = True
-        loaded_config_dict = migrate_v3_config_dict(loaded_config_dict)  # pyright: ignore [reportUnknownArgumentType]
-    if loaded_config_dict["schema_version"] == "4.0.0":
-        migrated = True
-        loaded_config_dict = migrate_v4_0_0_to_4_0_1_config_dict(loaded_config_dict)
-    if loaded_config_dict["schema_version"] == "4.0.1":
-        migrated = True
-        loaded_config_dict = migrate_v4_0_1_to_4_0_2_config_dict(loaded_config_dict)
-
-    if migrated:
+        # This is a v3 config file, attempt to migrate it
         shutil.copy(config_path, config_path.with_suffix(".yaml.bak"))
         try:
-            # load and write without environment variables
-            migrated_config = DefaultInvokeAIAppConfig.model_validate(loaded_config_dict)
-            migrated_config.write_file(config_path)
+            # loaded_config_dict could be the wrong shape, but we will catch all exceptions below
+            migrated_config = migrate_v3_config_dict(loaded_config_dict)  # pyright: ignore [reportUnknownArgumentType]
         except Exception as e:
             shutil.copy(config_path.with_suffix(".yaml.bak"), config_path)
             raise RuntimeError(f"Failed to load and migrate v3 config file {config_path}: {e}") from e
+        migrated_config.write_file(config_path)
+        return migrated_config
 
+    if loaded_config_dict["schema_version"] == "4.0.0":
+        loaded_config_dict = migrate_v4_0_0_config_dict(loaded_config_dict)
+        loaded_config_dict.write_file(config_path)
+
+    # Attempt to load as a v4 config file
     try:
         # Meta is not included in the model fields, so we need to validate it separately
         config = InvokeAIAppConfig.model_validate(loaded_config_dict)

invokeai/app/services/download/__init__.py

@@ -1,13 +1,13 @@
 """Init file for download queue."""
 
-from invokeai.app.services.download.download_base import (
+from .download_base import (
     DownloadJob,
     DownloadJobStatus,
     DownloadQueueServiceBase,
     MultiFileDownloadJob,
     UnknownJobIDException,
 )
-from invokeai.app.services.download.download_default import DownloadQueueService, TqdmProgress
+from .download_default import DownloadQueueService, TqdmProgress
 
 __all__ = [
     "DownloadJob",

invokeai/app/services/download/download_default.py

@@ -16,7 +16,12 @@ from requests import HTTPError
 from tqdm import tqdm
 
 from invokeai.app.services.config import InvokeAIAppConfig, get_config
-from invokeai.app.services.download.download_base import (
+from invokeai.app.services.events.events_base import EventServiceBase
+from invokeai.app.util.misc import get_iso_timestamp
+from invokeai.backend.model_manager.metadata import RemoteModelFile
+from invokeai.backend.util.logging import InvokeAILogger
+
+from .download_base import (
     DownloadEventHandler,
     DownloadExceptionHandler,
     DownloadJob,
@@ -28,10 +33,6 @@ from invokeai.app.services.download.download_base import (
     ServiceInactiveException,
     UnknownJobIDException,
 )
-from invokeai.app.services.events.events_base import EventServiceBase
-from invokeai.app.util.misc import get_iso_timestamp
-from invokeai.backend.model_manager.metadata import RemoteModelFile
-from invokeai.backend.util.logging import InvokeAILogger
 
 # Maximum number of bytes to download during each call to requests.iter_content()
 DOWNLOAD_CHUNK_SIZE = 100000
@@ -184,7 +185,7 @@ class DownloadQueueService(DownloadQueueServiceBase):
             job = DownloadJob(
                 source=url,
                 dest=path,
-                access_token=access_token or self._lookup_access_token(url),
+                access_token=access_token,
             )
             mfdj.download_parts.add(job)
             self._download_part2parent[job.source] = mfdj

invokeai/app/services/events/events_fastapievents.py

@@ -1,44 +1,47 @@
+# Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
+
 import asyncio
 import threading
+from queue import Empty, Queue
 
 from fastapi_events.dispatcher import dispatch
 
-from invokeai.app.services.events.events_base import EventServiceBase
-from invokeai.app.services.events.events_common import EventBase
+from invokeai.app.services.events.events_common import (
+    EventBase,
+)
+
+from .events_base import EventServiceBase
 
 
 class FastAPIEventService(EventServiceBase):
-    def __init__(self, event_handler_id: int, loop: asyncio.AbstractEventLoop) -> None:
+    def __init__(self, event_handler_id: int) -> None:
         self.event_handler_id = event_handler_id
-        self._queue = asyncio.Queue[EventBase | None]()
+        self._queue = Queue[EventBase | None]()
         self._stop_event = threading.Event()
-        self._loop = loop
-
-        # We need to store a reference to the task so it doesn't get GC'd
-        # See: https://docs.python.org/3/library/asyncio-task.html#creating-tasks
-        self._background_tasks: set[asyncio.Task[None]] = set()
-        task = self._loop.create_task(self._dispatch_from_queue(stop_event=self._stop_event))
-        self._background_tasks.add(task)
-        task.add_done_callback(self._background_tasks.remove)
+        asyncio.create_task(self._dispatch_from_queue(stop_event=self._stop_event))
 
         super().__init__()
 
     def stop(self, *args, **kwargs):
         self._stop_event.set()
-        self._loop.call_soon_threadsafe(self._queue.put_nowait, None)
+        self._queue.put(None)
 
     def dispatch(self, event: EventBase) -> None:
-        self._loop.call_soon_threadsafe(self._queue.put_nowait, event)
+        self._queue.put(event)
 
     async def _dispatch_from_queue(self, stop_event: threading.Event):
         """Get events on from the queue and dispatch them, from the correct thread"""
         while not stop_event.is_set():
             try:
-                event = await self._queue.get()
+                event = self._queue.get(block=False)
                 if not event:  # Probably stopping
                     continue
                 # Leave the payloads as live pydantic models
                 dispatch(event, middleware_id=self.event_handler_id, payload_schema_dump=False)
 
+            except Empty:
+                await asyncio.sleep(0.1)
+                pass
+
             except asyncio.CancelledError as e:
                 raise e  # Raise a proper error

invokeai/app/services/image_files/image_files_disk.py

@@ -1,30 +1,34 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654) and the InvokeAI Team
 from pathlib import Path
 from queue import Queue
-from typing import Optional, Union
+from typing import Dict, Optional, Union
 
 from PIL import Image, PngImagePlugin
 from PIL.Image import Image as PILImageType
+from send2trash import send2trash
 
-from invokeai.app.services.image_files.image_files_base import ImageFileStorageBase
-from invokeai.app.services.image_files.image_files_common import (
-    ImageFileDeleteException,
-    ImageFileNotFoundException,
-    ImageFileSaveException,
-)
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.util.thumbnails import get_thumbnail_name, make_thumbnail
 
+from .image_files_base import ImageFileStorageBase
+from .image_files_common import ImageFileDeleteException, ImageFileNotFoundException, ImageFileSaveException
+
 
 class DiskImageFileStorage(ImageFileStorageBase):
     """Stores images on disk"""
 
+    __output_folder: Path
+    __cache_ids: Queue  # TODO: this is an incredibly naive cache
+    __cache: Dict[Path, PILImageType]
+    __max_cache_size: int
+    __invoker: Invoker
+
     def __init__(self, output_folder: Union[str, Path]):
-        self.__cache: dict[Path, PILImageType] = {}
-        self.__cache_ids = Queue[Path]()
+        self.__cache = {}
+        self.__cache_ids = Queue()
         self.__max_cache_size = 10  # TODO: get this from config
 
-        self.__output_folder = output_folder if isinstance(output_folder, Path) else Path(output_folder)
+        self.__output_folder: Path = output_folder if isinstance(output_folder, Path) else Path(output_folder)
         self.__thumbnails_folder = self.__output_folder / "thumbnails"
         # Validate required output folders at launch
         self.__validate_storage_folders()
@@ -96,7 +100,7 @@ class DiskImageFileStorage(ImageFileStorageBase):
             image_path = self.get_path(image_name)
 
             if image_path.exists():
-                image_path.unlink()
+                send2trash(image_path)
             if image_path in self.__cache:
                 del self.__cache[image_path]
 
@@ -104,7 +108,7 @@ class DiskImageFileStorage(ImageFileStorageBase):
             thumbnail_path = self.get_path(thumbnail_name, True)
 
             if thumbnail_path.exists():
-                thumbnail_path.unlink()
+                send2trash(thumbnail_path)
             if thumbnail_path in self.__cache:
                 del self.__cache[thumbnail_path]
         except Exception as e:

invokeai/app/services/image_records/image_records_base.py

@@ -3,14 +3,9 @@ from datetime import datetime
 from typing import Optional
 
 from invokeai.app.invocations.fields import MetadataField
-from invokeai.app.services.image_records.image_records_common import (
-    ImageCategory,
-    ImageRecord,
-    ImageRecordChanges,
-    ResourceOrigin,
-)
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
+
+from .image_records_common import ImageCategory, ImageRecord, ImageRecordChanges, ResourceOrigin
 
 
 class ImageRecordStorageBase(ABC):
@@ -42,13 +37,10 @@ class ImageRecordStorageBase(ABC):
         self,
         offset: int = 0,
         limit: int = 10,
-        starred_first: bool = True,
-        order_dir: SQLiteDirection = SQLiteDirection.Descending,
         image_origin: Optional[ResourceOrigin] = None,
         categories: Optional[list[ImageCategory]] = None,
         is_intermediate: Optional[bool] = None,
         board_id: Optional[str] = None,
-        search_term: Optional[str] = None,
     ) -> OffsetPaginatedResults[ImageRecord]:
         """Gets a page of image records."""
         pass

invokeai/app/services/image_records/image_records_sqlite.py

@@ -4,8 +4,11 @@ from datetime import datetime
 from typing import Optional, Union, cast
 
 from invokeai.app.invocations.fields import MetadataField, MetadataFieldValidator
-from invokeai.app.services.image_records.image_records_base import ImageRecordStorageBase
-from invokeai.app.services.image_records.image_records_common import (
+from invokeai.app.services.shared.pagination import OffsetPaginatedResults
+from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
+
+from .image_records_base import ImageRecordStorageBase
+from .image_records_common import (
     IMAGE_DTO_COLS,
     ImageCategory,
     ImageRecord,
@@ -16,9 +19,6 @@ from invokeai.app.services.image_records.image_records_common import (
     ResourceOrigin,
     deserialize_image_record,
 )
-from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
-from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
 
 
 class SqliteImageRecordStorage(ImageRecordStorageBase):
@@ -144,13 +144,10 @@ class SqliteImageRecordStorage(ImageRecordStorageBase):
         self,
         offset: int = 0,
         limit: int = 10,
-        starred_first: bool = True,
-        order_dir: SQLiteDirection = SQLiteDirection.Descending,
         image_origin: Optional[ResourceOrigin] = None,
         categories: Optional[list[ImageCategory]] = None,
         is_intermediate: Optional[bool] = None,
         board_id: Optional[str] = None,
-        search_term: Optional[str] = None,
     ) -> OffsetPaginatedResults[ImageRecord]:
         try:
             self._lock.acquire()
@@ -211,21 +208,9 @@ class SqliteImageRecordStorage(ImageRecordStorageBase):
                 """
                 query_params.append(board_id)
 
-            # Search term condition
-            if search_term:
-                query_conditions += """--sql
-                AND images.metadata LIKE ?
-                """
-                query_params.append(f"%{search_term.lower()}%")
-
-            if starred_first:
-                query_pagination = f"""--sql
-                ORDER BY images.starred DESC, images.created_at {order_dir.value} LIMIT ? OFFSET ?
-                """
-            else:
-                query_pagination = f"""--sql
-                ORDER BY images.created_at {order_dir.value} LIMIT ? OFFSET ?
-                """
+            query_pagination = """--sql
+            ORDER BY images.starred DESC, images.created_at DESC LIMIT ? OFFSET ?
+            """
 
             # Final images query with pagination
             images_query += query_conditions + query_pagination + ";"

invokeai/app/services/images/images_base.py

@@ -12,7 +12,6 @@ from invokeai.app.services.image_records.image_records_common import (
 )
 from invokeai.app.services.images.images_common import ImageDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
 
 
 class ImageServiceABC(ABC):
@@ -117,13 +116,10 @@ class ImageServiceABC(ABC):
         self,
         offset: int = 0,
         limit: int = 10,
-        starred_first: bool = True,
-        order_dir: SQLiteDirection = SQLiteDirection.Descending,
         image_origin: Optional[ResourceOrigin] = None,
         categories: Optional[list[ImageCategory]] = None,
         is_intermediate: Optional[bool] = None,
         board_id: Optional[str] = None,
-        search_term: Optional[str] = None,
     ) -> OffsetPaginatedResults[ImageDTO]:
         """Gets a paginated list of image DTOs."""
         pass

invokeai/app/services/images/images_default.py

@@ -3,12 +3,15 @@ from typing import Optional
 from PIL.Image import Image as PILImageType
 
 from invokeai.app.invocations.fields import MetadataField
-from invokeai.app.services.image_files.image_files_common import (
+from invokeai.app.services.invoker import Invoker
+from invokeai.app.services.shared.pagination import OffsetPaginatedResults
+
+from ..image_files.image_files_common import (
     ImageFileDeleteException,
     ImageFileNotFoundException,
     ImageFileSaveException,
 )
-from invokeai.app.services.image_records.image_records_common import (
+from ..image_records.image_records_common import (
     ImageCategory,
     ImageRecord,
     ImageRecordChanges,
@@ -19,11 +22,8 @@ from invokeai.app.services.image_records.image_records_common import (
     InvalidOriginException,
     ResourceOrigin,
 )
-from invokeai.app.services.images.images_base import ImageServiceABC
-from invokeai.app.services.images.images_common import ImageDTO, image_record_to_dto
-from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
+from .images_base import ImageServiceABC
+from .images_common import ImageDTO, image_record_to_dto
 
 
 class ImageService(ImageServiceABC):
@@ -73,12 +73,7 @@ class ImageService(ImageServiceABC):
                 session_id=session_id,
             )
             if board_id is not None:
-                try:
-                    self.__invoker.services.board_image_records.add_image_to_board(
-                        board_id=board_id, image_name=image_name
-                    )
-                except Exception as e:
-                    self.__invoker.services.logger.warn(f"Failed to add image to board {board_id}: {str(e)}")
+                self.__invoker.services.board_image_records.add_image_to_board(board_id=board_id, image_name=image_name)
             self.__invoker.services.image_files.save(
                 image_name=image_name, image=image, metadata=metadata, workflow=workflow, graph=graph
             )
@@ -207,25 +202,19 @@ class ImageService(ImageServiceABC):
         self,
         offset: int = 0,
         limit: int = 10,
-        starred_first: bool = True,
-        order_dir: SQLiteDirection = SQLiteDirection.Descending,
         image_origin: Optional[ResourceOrigin] = None,
         categories: Optional[list[ImageCategory]] = None,
         is_intermediate: Optional[bool] = None,
         board_id: Optional[str] = None,
-        search_term: Optional[str] = None,
     ) -> OffsetPaginatedResults[ImageDTO]:
         try:
             results = self.__invoker.services.image_records.get_many(
                 offset,
                 limit,
-                starred_first,
-                order_dir,
                 image_origin,
                 categories,
                 is_intermediate,
                 board_id,
-                search_term,
             )
 
             image_dtos = [

invokeai/app/services/invocation_services.py

@@ -4,36 +4,35 @@ from __future__ import annotations
 from typing import TYPE_CHECKING
 
 from invokeai.app.services.object_serializer.object_serializer_base import ObjectSerializerBase
-from invokeai.app.services.style_preset_images.style_preset_images_base import StylePresetImageFileStorageBase
-from invokeai.app.services.style_preset_records.style_preset_records_base import StylePresetRecordsStorageBase
 
 if TYPE_CHECKING:
     from logging import Logger
 
     import torch
 
-    from invokeai.app.services.board_image_records.board_image_records_base import BoardImageRecordStorageBase
-    from invokeai.app.services.board_images.board_images_base import BoardImagesServiceABC
-    from invokeai.app.services.board_records.board_records_base import BoardRecordStorageBase
-    from invokeai.app.services.boards.boards_base import BoardServiceABC
-    from invokeai.app.services.bulk_download.bulk_download_base import BulkDownloadBase
-    from invokeai.app.services.config import InvokeAIAppConfig
-    from invokeai.app.services.download import DownloadQueueServiceBase
-    from invokeai.app.services.events.events_base import EventServiceBase
-    from invokeai.app.services.image_files.image_files_base import ImageFileStorageBase
-    from invokeai.app.services.image_records.image_records_base import ImageRecordStorageBase
-    from invokeai.app.services.images.images_base import ImageServiceABC
-    from invokeai.app.services.invocation_cache.invocation_cache_base import InvocationCacheBase
-    from invokeai.app.services.invocation_stats.invocation_stats_base import InvocationStatsServiceBase
-    from invokeai.app.services.model_images.model_images_base import ModelImageFileStorageBase
-    from invokeai.app.services.model_manager.model_manager_base import ModelManagerServiceBase
-    from invokeai.app.services.names.names_base import NameServiceBase
-    from invokeai.app.services.session_processor.session_processor_base import SessionProcessorBase
-    from invokeai.app.services.session_queue.session_queue_base import SessionQueueBase
-    from invokeai.app.services.urls.urls_base import UrlServiceBase
-    from invokeai.app.services.workflow_records.workflow_records_base import WorkflowRecordsStorageBase
     from invokeai.backend.stable_diffusion.diffusion.conditioning_data import ConditioningFieldData
 
+    from .board_image_records.board_image_records_base import BoardImageRecordStorageBase
+    from .board_images.board_images_base import BoardImagesServiceABC
+    from .board_records.board_records_base import BoardRecordStorageBase
+    from .boards.boards_base import BoardServiceABC
+    from .bulk_download.bulk_download_base import BulkDownloadBase
+    from .config import InvokeAIAppConfig
+    from .download import DownloadQueueServiceBase
+    from .events.events_base import EventServiceBase
+    from .image_files.image_files_base import ImageFileStorageBase
+    from .image_records.image_records_base import ImageRecordStorageBase
+    from .images.images_base import ImageServiceABC
+    from .invocation_cache.invocation_cache_base import InvocationCacheBase
+    from .invocation_stats.invocation_stats_base import InvocationStatsServiceBase
+    from .model_images.model_images_base import ModelImageFileStorageBase
+    from .model_manager.model_manager_base import ModelManagerServiceBase
+    from .names.names_base import NameServiceBase
+    from .session_processor.session_processor_base import SessionProcessorBase
+    from .session_queue.session_queue_base import SessionQueueBase
+    from .urls.urls_base import UrlServiceBase
+    from .workflow_records.workflow_records_base import WorkflowRecordsStorageBase
+
 
 class InvocationServices:
     """Services that can be used by invocations"""
@@ -63,8 +62,6 @@ class InvocationServices:
         workflow_records: "WorkflowRecordsStorageBase",
         tensors: "ObjectSerializerBase[torch.Tensor]",
         conditioning: "ObjectSerializerBase[ConditioningFieldData]",
-        style_preset_records: "StylePresetRecordsStorageBase",
-        style_preset_image_files: "StylePresetImageFileStorageBase",
     ):
         self.board_images = board_images
         self.board_image_records = board_image_records
@@ -89,5 +86,3 @@ class InvocationServices:
         self.workflow_records = workflow_records
         self.tensors = tensors
         self.conditioning = conditioning
-        self.style_preset_records = style_preset_records
-        self.style_preset_image_files = style_preset_image_files

invokeai/app/services/invocation_stats/invocation_stats_default.py

@@ -9,8 +9,11 @@ import torch
 
 import invokeai.backend.util.logging as logger
 from invokeai.app.invocations.baseinvocation import BaseInvocation
-from invokeai.app.services.invocation_stats.invocation_stats_base import InvocationStatsServiceBase
-from invokeai.app.services.invocation_stats.invocation_stats_common import (
+from invokeai.app.services.invoker import Invoker
+from invokeai.backend.model_manager.load.model_cache import CacheStats
+
+from .invocation_stats_base import InvocationStatsServiceBase
+from .invocation_stats_common import (
     GESStatsNotFoundError,
     GraphExecutionStats,
     GraphExecutionStatsSummary,
@@ -19,8 +22,6 @@ from invokeai.app.services.invocation_stats.invocation_stats_common import (
     NodeExecutionStats,
     NodeExecutionStatsSummary,
 )
-from invokeai.app.services.invoker import Invoker
-from invokeai.backend.model_manager.load.model_cache import CacheStats
 
 # Size of 1GB in bytes.
 GB = 2**30

invokeai/app/services/invoker.py

@@ -1,7 +1,7 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
 
 
-from invokeai.app.services.invocation_services import InvocationServices
+from .invocation_services import InvocationServices
 
 
 class Invoker:

invokeai/app/services/model_images/model_images_default.py

@@ -2,16 +2,18 @@ from pathlib import Path
 
 from PIL import Image
 from PIL.Image import Image as PILImageType
+from send2trash import send2trash
 
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_images.model_images_base import ModelImageFileStorageBase
-from invokeai.app.services.model_images.model_images_common import (
+from invokeai.app.util.misc import uuid_string
+from invokeai.app.util.thumbnails import make_thumbnail
+
+from .model_images_base import ModelImageFileStorageBase
+from .model_images_common import (
     ModelImageFileDeleteException,
     ModelImageFileNotFoundException,
     ModelImageFileSaveException,
 )
-from invokeai.app.util.misc import uuid_string
-from invokeai.app.util.thumbnails import make_thumbnail
 
 
 class ModelImageFileStorageDisk(ModelImageFileStorageBase):
@@ -69,7 +71,7 @@ class ModelImageFileStorageDisk(ModelImageFileStorageBase):
             if not self._validate_path(path):
                 raise ModelImageFileNotFoundException
 
-            path.unlink()
+            send2trash(path)
 
         except Exception as e:
             raise ModelImageFileDeleteException from e

invokeai/app/services/model_install/__init__.py

@@ -1,7 +1,9 @@
 """Initialization file for model install service package."""
 
-from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
-from invokeai.app.services.model_install.model_install_common import (
+from .model_install_base import (
+    ModelInstallServiceBase,
+)
+from .model_install_common import (
     HFModelSource,
     InstallStatus,
     LocalModelSource,
@@ -10,7 +12,7 @@ from invokeai.app.services.model_install.model_install_common import (
     UnknownInstallJobException,
     URLModelSource,
 )
-from invokeai.app.services.model_install.model_install_default import ModelInstallService
+from .model_install_default import ModelInstallService
 
 __all__ = [
     "ModelInstallServiceBase",

invokeai/app/services/model_install/model_install_base.py

@@ -3,7 +3,7 @@
 
 from abc import ABC, abstractmethod
 from pathlib import Path
-from typing import List, Optional, Union
+from typing import Any, Dict, List, Optional, Union
 
 from pydantic.networks import AnyHttpUrl
 
@@ -12,7 +12,7 @@ from invokeai.app.services.download import DownloadQueueServiceBase
 from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.model_install.model_install_common import ModelInstallJob, ModelSource
-from invokeai.app.services.model_records import ModelRecordChanges, ModelRecordServiceBase
+from invokeai.app.services.model_records import ModelRecordServiceBase
 from invokeai.backend.model_manager import AnyModelConfig
 
 
@@ -64,7 +64,7 @@ class ModelInstallServiceBase(ABC):
     def register_path(
         self,
         model_path: Union[Path, str],
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]] = None,
     ) -> str:
         """
         Probe and register the model at model_path.
@@ -72,7 +72,7 @@ class ModelInstallServiceBase(ABC):
         This keeps the model in its current location.
 
         :param model_path: Filesystem Path to the model.
-        :param config: ModelRecordChanges object that will override autoassigned model record values.
+        :param config: Dict of attributes that will override autoassigned values.
         :returns id: The string ID of the registered model.
         """
 
@@ -92,7 +92,7 @@ class ModelInstallServiceBase(ABC):
     def install_path(
         self,
         model_path: Union[Path, str],
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]] = None,
     ) -> str:
         """
         Probe, register and install the model in the models directory.
@@ -101,7 +101,7 @@ class ModelInstallServiceBase(ABC):
         the models directory handled by InvokeAI.
 
         :param model_path: Filesystem Path to the model.
-        :param config: ModelRecordChanges object that will override autoassigned model record values.
+        :param config: Dict of attributes that will override autoassigned values.
         :returns id: The string ID of the registered model.
         """
 
@@ -109,14 +109,14 @@ class ModelInstallServiceBase(ABC):
     def heuristic_import(
         self,
         source: str,
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]] = None,
         access_token: Optional[str] = None,
         inplace: Optional[bool] = False,
     ) -> ModelInstallJob:
         r"""Install the indicated model using heuristics to interpret user intentions.
 
         :param source: String source
-        :param config: Optional ModelRecordChanges object. Any fields in this object
+        :param config: Optional dict. Any fields in this dict
          will override corresponding autoassigned probe fields in the
          model's config record as described in `import_model()`.
         :param access_token: Optional access token for remote sources.
@@ -147,7 +147,7 @@ class ModelInstallServiceBase(ABC):
     def import_model(
         self,
         source: ModelSource,
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]] = None,
     ) -> ModelInstallJob:
         """Install the indicated model.
 

invokeai/app/services/model_install/model_install_common.py

@@ -2,14 +2,13 @@ import re
 import traceback
 from enum import Enum
 from pathlib import Path
-from typing import Literal, Optional, Set, Union
+from typing import Any, Dict, Literal, Optional, Set, Union
 
 from pydantic import BaseModel, Field, PrivateAttr, field_validator
 from pydantic.networks import AnyHttpUrl
 from typing_extensions import Annotated
 
 from invokeai.app.services.download import DownloadJob, MultiFileDownloadJob
-from invokeai.app.services.model_records import ModelRecordChanges
 from invokeai.backend.model_manager import AnyModelConfig, ModelRepoVariant
 from invokeai.backend.model_manager.config import ModelSourceType
 from invokeai.backend.model_manager.metadata import AnyModelRepoMetadata
@@ -134,9 +133,8 @@ class ModelInstallJob(BaseModel):
     id: int = Field(description="Unique ID for this job")
     status: InstallStatus = Field(default=InstallStatus.WAITING, description="Current status of install process")
     error_reason: Optional[str] = Field(default=None, description="Information about why the job failed")
-    config_in: ModelRecordChanges = Field(
-        default_factory=ModelRecordChanges,
-        description="Configuration information (e.g. 'description') to apply to model.",
+    config_in: Dict[str, Any] = Field(
+        default_factory=dict, description="Configuration information (e.g. 'description') to apply to model."
     )
     config_out: Optional[AnyModelConfig] = Field(
         default=None, description="After successful installation, this will hold the configuration object."

invokeai/app/services/model_install/model_install_default.py

@@ -23,16 +23,6 @@ from invokeai.app.services.download import DownloadQueueServiceBase, MultiFileDo
 from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
-from invokeai.app.services.model_install.model_install_common import (
-    MODEL_SOURCE_TO_TYPE_MAP,
-    HFModelSource,
-    InstallStatus,
-    LocalModelSource,
-    ModelInstallJob,
-    ModelSource,
-    StringLikeSource,
-    URLModelSource,
-)
 from invokeai.app.services.model_records import DuplicateModelException, ModelRecordServiceBase
 from invokeai.app.services.model_records.model_records_base import ModelRecordChanges
 from invokeai.backend.model_manager.config import (
@@ -57,6 +47,17 @@ from invokeai.backend.util.catch_sigint import catch_sigint
 from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.util import slugify
 
+from .model_install_common import (
+    MODEL_SOURCE_TO_TYPE_MAP,
+    HFModelSource,
+    InstallStatus,
+    LocalModelSource,
+    ModelInstallJob,
+    ModelSource,
+    StringLikeSource,
+    URLModelSource,
+)
+
 TMPDIR_PREFIX = "tmpinstall_"
 
 
@@ -163,27 +164,26 @@ class ModelInstallService(ModelInstallServiceBase):
     def register_path(
         self,
         model_path: Union[Path, str],
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]] = None,
     ) -> str:  # noqa D102
         model_path = Path(model_path)
-        config = config or ModelRecordChanges()
-        if not config.source:
-            config.source = model_path.resolve().as_posix()
-        config.source_type = ModelSourceType.Path
+        config = config or {}
+        if not config.get("source"):
+            config["source"] = model_path.resolve().as_posix()
+        config["source_type"] = ModelSourceType.Path
         return self._register(model_path, config)
 
     def install_path(
         self,
         model_path: Union[Path, str],
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]] = None,
     ) -> str:  # noqa D102
         model_path = Path(model_path)
-        config = config or ModelRecordChanges()
-        info: AnyModelConfig = ModelProbe.probe(
-            Path(model_path), config.model_dump(), hash_algo=self._app_config.hashing_algorithm
-        )  # type: ignore
+        config = config or {}
 
-        if preferred_name := config.name:
+        info: AnyModelConfig = ModelProbe.probe(Path(model_path), config, hash_algo=self._app_config.hashing_algorithm)
+
+        if preferred_name := config.get("name"):
             preferred_name = Path(preferred_name).with_suffix(model_path.suffix)
 
         dest_path = (
@@ -205,7 +205,7 @@ class ModelInstallService(ModelInstallServiceBase):
     def heuristic_import(
         self,
         source: str,
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]] = None,
         access_token: Optional[str] = None,
         inplace: Optional[bool] = False,
     ) -> ModelInstallJob:
@@ -217,7 +217,7 @@ class ModelInstallService(ModelInstallServiceBase):
             source_obj.access_token = access_token
         return self.import_model(source_obj, config)
 
-    def import_model(self, source: ModelSource, config: Optional[ModelRecordChanges] = None) -> ModelInstallJob:  # noqa D102
+    def import_model(self, source: ModelSource, config: Optional[Dict[str, Any]] = None) -> ModelInstallJob:  # noqa D102
         similar_jobs = [x for x in self.list_jobs() if x.source == source and not x.in_terminal_state]
         if similar_jobs:
             self._logger.warning(f"There is already an active install job for {source}. Not enqueuing.")
@@ -319,17 +319,16 @@ class ModelInstallService(ModelInstallServiceBase):
                         model_path = self._app_config.models_path / model_path
                     model_path = model_path.resolve()
 
-                    config = ModelRecordChanges(
-                        name=model_name,
-                        description=stanza.get("description"),
-                    )
+                    config: dict[str, Any] = {}
+                    config["name"] = model_name
+                    config["description"] = stanza.get("description")
                     legacy_config_path = stanza.get("config")
                     if legacy_config_path:
                         # In v3, these paths were relative to the root. Migrate them to be relative to the legacy_conf_dir.
                         legacy_config_path = self._app_config.root_path / legacy_config_path
                         if legacy_config_path.is_relative_to(self._app_config.legacy_conf_path):
                             legacy_config_path = legacy_config_path.relative_to(self._app_config.legacy_conf_path)
-                        config.config_path = str(legacy_config_path)
+                        config["config_path"] = str(legacy_config_path)
                     try:
                         id = self.register_path(model_path=model_path, config=config)
                         self._logger.info(f"Migrated {model_name} with id {id}")
@@ -502,11 +501,11 @@ class ModelInstallService(ModelInstallServiceBase):
         job.total_bytes = self._stat_size(job.local_path)
         job.bytes = job.total_bytes
         self._signal_job_running(job)
-        job.config_in.source = str(job.source)
-        job.config_in.source_type = MODEL_SOURCE_TO_TYPE_MAP[job.source.__class__]
+        job.config_in["source"] = str(job.source)
+        job.config_in["source_type"] = MODEL_SOURCE_TO_TYPE_MAP[job.source.__class__]
         # enter the metadata, if there is any
         if isinstance(job.source_metadata, (HuggingFaceMetadata)):
-            job.config_in.source_api_response = job.source_metadata.api_response
+            job.config_in["source_api_response"] = job.source_metadata.api_response
 
         if job.inplace:
             key = self.register_path(job.local_path, job.config_in)
@@ -641,11 +640,11 @@ class ModelInstallService(ModelInstallServiceBase):
         return new_path
 
     def _register(
-        self, model_path: Path, config: Optional[ModelRecordChanges] = None, info: Optional[AnyModelConfig] = None
+        self, model_path: Path, config: Optional[Dict[str, Any]] = None, info: Optional[AnyModelConfig] = None
     ) -> str:
-        config = config or ModelRecordChanges()
+        config = config or {}
 
-        info = info or ModelProbe.probe(model_path, config.model_dump(), hash_algo=self._app_config.hashing_algorithm)  # type: ignore
+        info = info or ModelProbe.probe(model_path, config, hash_algo=self._app_config.hashing_algorithm)
 
         model_path = model_path.resolve()
 
@@ -676,13 +675,11 @@ class ModelInstallService(ModelInstallServiceBase):
         precision = TorchDevice.choose_torch_dtype()
         return ModelRepoVariant.FP16 if precision == torch.float16 else None
 
-    def _import_local_model(
-        self, source: LocalModelSource, config: Optional[ModelRecordChanges] = None
-    ) -> ModelInstallJob:
+    def _import_local_model(self, source: LocalModelSource, config: Optional[Dict[str, Any]]) -> ModelInstallJob:
         return ModelInstallJob(
             id=self._next_id(),
             source=source,
-            config_in=config or ModelRecordChanges(),
+            config_in=config or {},
             local_path=Path(source.path),
             inplace=source.inplace or False,
         )
@@ -690,7 +687,7 @@ class ModelInstallService(ModelInstallServiceBase):
     def _import_from_hf(
         self,
         source: HFModelSource,
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]] = None,
     ) -> ModelInstallJob:
         # Add user's cached access token to HuggingFace requests
         if source.access_token is None:
@@ -706,7 +703,7 @@ class ModelInstallService(ModelInstallServiceBase):
     def _import_from_url(
         self,
         source: URLModelSource,
-        config: Optional[ModelRecordChanges] = None,
+        config: Optional[Dict[str, Any]],
     ) -> ModelInstallJob:
         remote_files, metadata = self._remote_files_from_source(source)
         return self._import_remote_model(
@@ -721,7 +718,7 @@ class ModelInstallService(ModelInstallServiceBase):
         source: HFModelSource | URLModelSource,
         remote_files: List[RemoteModelFile],
         metadata: Optional[AnyModelRepoMetadata],
-        config: Optional[ModelRecordChanges],
+        config: Optional[Dict[str, Any]],
     ) -> ModelInstallJob:
         if len(remote_files) == 0:
             raise ValueError(f"{source}: No downloadable files found")
@@ -734,7 +731,7 @@ class ModelInstallService(ModelInstallServiceBase):
         install_job = ModelInstallJob(
             id=self._next_id(),
             source=source,
-            config_in=config or ModelRecordChanges(),
+            config_in=config or {},
             source_metadata=metadata,
             local_path=destdir,  # local path may change once the download has started due to content-disposition handling
             bytes=0,
@@ -851,7 +848,7 @@ class ModelInstallService(ModelInstallServiceBase):
         with self._lock:
             if install_job := self._download_cache.pop(download_job.id, None):
                 assert excp is not None
-                self._set_error(install_job, excp)
+                install_job.set_error(excp)
                 self._download_queue.cancel_job(download_job)
 
                 # Let other threads know that the number of downloads has changed

invokeai/app/services/model_load/__init__.py

@@ -1,6 +1,6 @@
 """Initialization file for model load service module."""
 
-from invokeai.app.services.model_load.model_load_base import ModelLoadServiceBase
-from invokeai.app.services.model_load.model_load_default import ModelLoadService
+from .model_load_base import ModelLoadServiceBase
+from .model_load_default import ModelLoadService
 
 __all__ = ["ModelLoadServiceBase", "ModelLoadService"]

invokeai/app/services/model_load/model_load_base.py

@@ -7,6 +7,7 @@ from typing import Callable, Optional
 
 from invokeai.backend.model_manager import AnyModel, AnyModelConfig, SubModelType
 from invokeai.backend.model_manager.load import LoadedModel, LoadedModelWithoutConfig
+from invokeai.backend.model_manager.load.convert_cache import ModelConvertCacheBase
 from invokeai.backend.model_manager.load.model_cache.model_cache_base import ModelCacheBase
 
 
@@ -27,6 +28,11 @@ class ModelLoadServiceBase(ABC):
     def ram_cache(self) -> ModelCacheBase[AnyModel]:
         """Return the RAM cache used by this loader."""
 
+    @property
+    @abstractmethod
+    def convert_cache(self) -> ModelConvertCacheBase:
+        """Return the checkpoint convert cache used by this loader."""
+
     @abstractmethod
     def load_model_from_path(
         self, model_path: Path, loader: Optional[Callable[[Path], AnyModel]] = None

invokeai/app/services/model_load/model_load_default.py

@@ -10,7 +10,6 @@ from torch import load as torch_load
 
 from invokeai.app.services.config import InvokeAIAppConfig
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_load.model_load_base import ModelLoadServiceBase
 from invokeai.backend.model_manager import AnyModel, AnyModelConfig, SubModelType
 from invokeai.backend.model_manager.load import (
     LoadedModel,
@@ -18,11 +17,14 @@ from invokeai.backend.model_manager.load import (
     ModelLoaderRegistry,
     ModelLoaderRegistryBase,
 )
+from invokeai.backend.model_manager.load.convert_cache import ModelConvertCacheBase
 from invokeai.backend.model_manager.load.model_cache.model_cache_base import ModelCacheBase
 from invokeai.backend.model_manager.load.model_loaders.generic_diffusers import GenericDiffusersLoader
 from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.logging import InvokeAILogger
 
+from .model_load_base import ModelLoadServiceBase
+
 
 class ModelLoadService(ModelLoadServiceBase):
     """Wrapper around ModelLoaderRegistry."""
@@ -31,6 +33,7 @@ class ModelLoadService(ModelLoadServiceBase):
         self,
         app_config: InvokeAIAppConfig,
         ram_cache: ModelCacheBase[AnyModel],
+        convert_cache: ModelConvertCacheBase,
         registry: Optional[Type[ModelLoaderRegistryBase]] = ModelLoaderRegistry,
     ):
         """Initialize the model load service."""
@@ -39,6 +42,7 @@ class ModelLoadService(ModelLoadServiceBase):
         self._logger = logger
         self._app_config = app_config
         self._ram_cache = ram_cache
+        self._convert_cache = convert_cache
         self._registry = registry
 
     def start(self, invoker: Invoker) -> None:
@@ -49,6 +53,11 @@ class ModelLoadService(ModelLoadServiceBase):
         """Return the RAM cache used by this loader."""
         return self._ram_cache
 
+    @property
+    def convert_cache(self) -> ModelConvertCacheBase:
+        """Return the checkpoint convert cache used by this loader."""
+        return self._convert_cache
+
     def load_model(self, model_config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> LoadedModel:
         """
         Given a model's configuration, load it and return the LoadedModel object.
@@ -67,6 +76,7 @@ class ModelLoadService(ModelLoadServiceBase):
             app_config=self._app_config,
             logger=self._logger,
             ram_cache=self._ram_cache,
+            convert_cache=self._convert_cache,
         ).load_model(model_config, submodel_type)
 
         if hasattr(self, "_invoker"):

invokeai/app/services/model_manager/__init__.py

@@ -1,9 +1,10 @@
 """Initialization file for model manager service."""
 
-from invokeai.app.services.model_manager.model_manager_default import ModelManagerService, ModelManagerServiceBase
 from invokeai.backend.model_manager import AnyModel, AnyModelConfig, BaseModelType, ModelType, SubModelType
 from invokeai.backend.model_manager.load import LoadedModel
 
+from .model_manager_default import ModelManagerService, ModelManagerServiceBase
+
 __all__ = [
     "ModelManagerServiceBase",
     "ModelManagerService",

invokeai/app/services/model_manager/model_manager_base.py

@@ -5,13 +5,14 @@ from abc import ABC, abstractmethod
 import torch
 from typing_extensions import Self
 
-from invokeai.app.services.config.config_default import InvokeAIAppConfig
-from invokeai.app.services.download.download_base import DownloadQueueServiceBase
-from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
-from invokeai.app.services.model_load.model_load_base import ModelLoadServiceBase
-from invokeai.app.services.model_records.model_records_base import ModelRecordServiceBase
+
+from ..config import InvokeAIAppConfig
+from ..download import DownloadQueueServiceBase
+from ..events.events_base import EventServiceBase
+from ..model_install import ModelInstallServiceBase
+from ..model_load import ModelLoadServiceBase
+from ..model_records import ModelRecordServiceBase
 
 
 class ModelManagerServiceBase(ABC):

invokeai/app/services/model_manager/model_manager_default.py

@@ -6,20 +6,19 @@ from typing import Optional
 import torch
 from typing_extensions import Self
 
-from invokeai.app.services.config.config_default import InvokeAIAppConfig
-from invokeai.app.services.download.download_base import DownloadQueueServiceBase
-from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
-from invokeai.app.services.model_install.model_install_default import ModelInstallService
-from invokeai.app.services.model_load.model_load_base import ModelLoadServiceBase
-from invokeai.app.services.model_load.model_load_default import ModelLoadService
-from invokeai.app.services.model_manager.model_manager_base import ModelManagerServiceBase
-from invokeai.app.services.model_records.model_records_base import ModelRecordServiceBase
-from invokeai.backend.model_manager.load import ModelCache, ModelLoaderRegistry
+from invokeai.backend.model_manager.load import ModelCache, ModelConvertCache, ModelLoaderRegistry
 from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.logging import InvokeAILogger
 
+from ..config import InvokeAIAppConfig
+from ..download import DownloadQueueServiceBase
+from ..events.events_base import EventServiceBase
+from ..model_install import ModelInstallService, ModelInstallServiceBase
+from ..model_load import ModelLoadService, ModelLoadServiceBase
+from ..model_records import ModelRecordServiceBase
+from .model_manager_base import ModelManagerServiceBase
+
 
 class ModelManagerService(ModelManagerServiceBase):
     """
@@ -87,9 +86,11 @@ class ModelManagerService(ModelManagerServiceBase):
             logger=logger,
             execution_device=execution_device or TorchDevice.choose_torch_device(),
         )
+        convert_cache = ModelConvertCache(cache_path=app_config.convert_cache_path, max_size=app_config.convert_cache)
         loader = ModelLoadService(
             app_config=app_config,
             ram_cache=ram_cache,
+            convert_cache=convert_cache,
             registry=ModelLoaderRegistry,
         )
         installer = ModelInstallService(

invokeai/app/services/model_records/model_records_base.py

@@ -18,7 +18,6 @@ from invokeai.backend.model_manager.config import (
     ControlAdapterDefaultSettings,
     MainModelDefaultSettings,
     ModelFormat,
-    ModelSourceType,
     ModelType,
     ModelVariantType,
     SchedulerPredictionType,
@@ -67,16 +66,10 @@ class ModelRecordChanges(BaseModelExcludeNull):
     """A set of changes to apply to a model."""
 
     # Changes applicable to all models
-    source: Optional[str] = Field(description="original source of the model", default=None)
-    source_type: Optional[ModelSourceType] = Field(description="type of model source", default=None)
-    source_api_response: Optional[str] = Field(description="metadata from remote source", default=None)
     name: Optional[str] = Field(description="Name of the model.", default=None)
     path: Optional[str] = Field(description="Path to the model.", default=None)
     description: Optional[str] = Field(description="Model description", default=None)
     base: Optional[BaseModelType] = Field(description="The base model.", default=None)
-    type: Optional[ModelType] = Field(description="Type of model", default=None)
-    key: Optional[str] = Field(description="Database ID for this model", default=None)
-    hash: Optional[str] = Field(description="hash of model file", default=None)
     trigger_phrases: Optional[set[str]] = Field(description="Set of trigger phrases for this model", default=None)
     default_settings: Optional[MainModelDefaultSettings | ControlAdapterDefaultSettings] = Field(
         description="Default settings for this model", default=None

invokeai/app/services/model_records/model_records_sql.py

@@ -40,24 +40,12 @@ Typical usage:
 """
 
 import json
-import logging
 import sqlite3
 from math import ceil
 from pathlib import Path
 from typing import List, Optional, Union
 
-import pydantic
-
-from invokeai.app.services.model_records.model_records_base import (
-    DuplicateModelException,
-    ModelRecordChanges,
-    ModelRecordOrderBy,
-    ModelRecordServiceBase,
-    ModelSummary,
-    UnknownModelException,
-)
 from invokeai.app.services.shared.pagination import PaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
 from invokeai.backend.model_manager.config import (
     AnyModelConfig,
     BaseModelType,
@@ -66,11 +54,21 @@ from invokeai.backend.model_manager.config import (
     ModelType,
 )
 
+from ..shared.sqlite.sqlite_database import SqliteDatabase
+from .model_records_base import (
+    DuplicateModelException,
+    ModelRecordChanges,
+    ModelRecordOrderBy,
+    ModelRecordServiceBase,
+    ModelSummary,
+    UnknownModelException,
+)
+
 
 class ModelRecordServiceSQL(ModelRecordServiceBase):
     """Implementation of the ModelConfigStore ABC using a SQL database."""
 
-    def __init__(self, db: SqliteDatabase, logger: logging.Logger):
+    def __init__(self, db: SqliteDatabase):
         """
         Initialize a new object from preexisting sqlite3 connection and threading lock objects.
 
@@ -79,7 +77,6 @@ class ModelRecordServiceSQL(ModelRecordServiceBase):
         super().__init__()
         self._db = db
         self._cursor = db.conn.cursor()
-        self._logger = logger
 
     @property
     def db(self) -> SqliteDatabase:
@@ -295,20 +292,7 @@ class ModelRecordServiceSQL(ModelRecordServiceBase):
                 tuple(bindings),
             )
             result = self._cursor.fetchall()
-
-        # Parse the model configs.
-        results: list[AnyModelConfig] = []
-        for row in result:
-            try:
-                model_config = ModelConfigFactory.make_config(json.loads(row[0]), timestamp=row[1])
-            except pydantic.ValidationError:
-                # We catch this error so that the app can still run if there are invalid model configs in the database.
-                # One reason that an invalid model config might be in the database is if someone had to rollback from a
-                # newer version of the app that added a new model type.
-                self._logger.warning(f"Found an invalid model config in the database. Ignoring this model. ({row[0]})")
-            else:
-                results.append(model_config)
-
+            results = [ModelConfigFactory.make_config(json.loads(x[0]), timestamp=x[1]) for x in result]
         return results
 
     def search_by_path(self, path: Union[str, Path]) -> List[AnyModelConfig]:

invokeai/app/services/names/names_default.py

@@ -1,6 +1,7 @@
-from invokeai.app.services.names.names_base import NameServiceBase
 from invokeai.app.util.misc import uuid_string
 
+from .names_base import NameServiceBase
+
 
 class SimpleNameService(NameServiceBase):
     """Creates image names from UUIDs."""

invokeai/app/services/session_processor/session_processor_default.py

@@ -13,24 +13,24 @@ from invokeai.app.services.events.events_common import (
     register_events,
 )
 from invokeai.app.services.invocation_stats.invocation_stats_common import GESStatsNotFoundError
-from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.session_processor.session_processor_base import (
-    InvocationServices,
     OnAfterRunNode,
     OnAfterRunSession,
     OnBeforeRunNode,
     OnBeforeRunSession,
     OnNodeError,
     OnNonFatalProcessorError,
-    SessionProcessorBase,
-    SessionRunnerBase,
 )
-from invokeai.app.services.session_processor.session_processor_common import CanceledException, SessionProcessorStatus
+from invokeai.app.services.session_processor.session_processor_common import CanceledException
 from invokeai.app.services.session_queue.session_queue_common import SessionQueueItem, SessionQueueItemNotFoundError
 from invokeai.app.services.shared.graph import NodeInputError
 from invokeai.app.services.shared.invocation_context import InvocationContextData, build_invocation_context
 from invokeai.app.util.profiler import Profiler
 
+from ..invoker import Invoker
+from .session_processor_base import InvocationServices, SessionProcessorBase, SessionRunnerBase
+from .session_processor_common import SessionProcessorStatus
+
 
 class DefaultSessionRunner(SessionRunnerBase):
     """Processes a single session's invocations."""

invokeai/app/services/session_queue/session_queue_sqlite.py

@@ -37,14 +37,10 @@ class SqliteSessionQueue(SessionQueueBase):
     def start(self, invoker: Invoker) -> None:
         self.__invoker = invoker
         self._set_in_progress_to_canceled()
-        if self.__invoker.services.configuration.clear_queue_on_startup:
-            clear_result = self.clear(DEFAULT_QUEUE_ID)
-            if clear_result.deleted > 0:
-                self.__invoker.services.logger.info(f"Cleared all {clear_result.deleted} queue items")
-        else:
-            prune_result = self.prune(DEFAULT_QUEUE_ID)
-            if prune_result.deleted > 0:
-                self.__invoker.services.logger.info(f"Pruned {prune_result.deleted} finished queue items")
+        prune_result = self.prune(DEFAULT_QUEUE_ID)
+
+        if prune_result.deleted > 0:
+            self.__invoker.services.logger.info(f"Pruned {prune_result.deleted} finished queue items")
 
     def __init__(self, db: SqliteDatabase) -> None:
         super().__init__()

invokeai/app/services/shared/graph.py

@@ -652,7 +652,7 @@ class Graph(BaseModel):
         output_fields = [get_input_field(self.get_node(e.node_id), e.field) for e in outputs]
 
         # Input type must be a list
-        if get_origin(input_field) is not list:
+        if get_origin(input_field) != list:
             return False
 
         # Validate that all outputs match the input type

invokeai/app/services/shared/sqlite/sqlite_util.py

@@ -14,9 +14,6 @@ from invokeai.app.services.shared.sqlite_migrator.migrations.migration_8 import
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_9 import build_migration_9
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_10 import build_migration_10
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_11 import build_migration_11
-from invokeai.app.services.shared.sqlite_migrator.migrations.migration_12 import build_migration_12
-from invokeai.app.services.shared.sqlite_migrator.migrations.migration_13 import build_migration_13
-from invokeai.app.services.shared.sqlite_migrator.migrations.migration_14 import build_migration_14
 from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_impl import SqliteMigrator
 
 
@@ -48,9 +45,6 @@ def init_db(config: InvokeAIAppConfig, logger: Logger, image_files: ImageFileSto
     migrator.register_migration(build_migration_9())
     migrator.register_migration(build_migration_10())
     migrator.register_migration(build_migration_11(app_config=config, logger=logger))
-    migrator.register_migration(build_migration_12(app_config=config))
-    migrator.register_migration(build_migration_13())
-    migrator.register_migration(build_migration_14())
     migrator.run_migrations()
 
     return db

invokeai/app/services/shared/sqlite_migrator/migrations/migration_12.py

@@ -1,35 +0,0 @@
-import shutil
-import sqlite3
-
-from invokeai.app.services.config import InvokeAIAppConfig
-from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_common import Migration
-
-
-class Migration12Callback:
-    def __init__(self, app_config: InvokeAIAppConfig) -> None:
-        self._app_config = app_config
-
-    def __call__(self, cursor: sqlite3.Cursor) -> None:
-        self._remove_model_convert_cache_dir()
-
-    def _remove_model_convert_cache_dir(self) -> None:
-        """
-        Removes unused model convert cache directory
-        """
-        convert_cache = self._app_config.convert_cache_path
-        shutil.rmtree(convert_cache, ignore_errors=True)
-
-
-def build_migration_12(app_config: InvokeAIAppConfig) -> Migration:
-    """
-    Build the migration from database version 11 to 12.
-
-    This migration removes the now-unused model convert cache directory.
-    """
-    migration_12 = Migration(
-        from_version=11,
-        to_version=12,
-        callback=Migration12Callback(app_config),
-    )
-
-    return migration_12

invokeai/app/services/shared/sqlite_migrator/migrations/migration_13.py

@@ -1,31 +0,0 @@
-import sqlite3
-
-from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_common import Migration
-
-
-class Migration13Callback:
-    def __call__(self, cursor: sqlite3.Cursor) -> None:
-        self._add_archived_col(cursor)
-
-    def _add_archived_col(self, cursor: sqlite3.Cursor) -> None:
-        """
-        - Adds `archived` columns to the board table.
-        """
-
-        cursor.execute("ALTER TABLE boards ADD COLUMN archived BOOLEAN DEFAULT FALSE;")
-
-
-def build_migration_13() -> Migration:
-    """
-    Build the migration from database version 12 to 13..
-
-    This migration does the following:
-    - Adds `archived` columns to the board table.
-    """
-    migration_13 = Migration(
-        from_version=12,
-        to_version=13,
-        callback=Migration13Callback(),
-    )
-
-    return migration_13

invokeai/app/services/shared/sqlite_migrator/migrations/migration_14.py

@@ -1,61 +0,0 @@
-import sqlite3
-
-from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_common import Migration
-
-
-class Migration14Callback:
-    def __call__(self, cursor: sqlite3.Cursor) -> None:
-        self._create_style_presets(cursor)
-
-    def _create_style_presets(self, cursor: sqlite3.Cursor) -> None:
-        """Create the table used to store style presets."""
-        tables = [
-            """--sql
-            CREATE TABLE IF NOT EXISTS style_presets (
-                id TEXT NOT NULL PRIMARY KEY,
-                name TEXT NOT NULL,
-                preset_data TEXT NOT NULL,
-                type TEXT NOT NULL DEFAULT "user",
-                created_at DATETIME NOT NULL DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')),
-                -- Updated via trigger
-                updated_at DATETIME NOT NULL DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))
-            );
-            """
-        ]
-
-        # Add trigger for `updated_at`.
-        triggers = [
-            """--sql
-            CREATE TRIGGER IF NOT EXISTS style_presets
-            AFTER UPDATE
-            ON style_presets FOR EACH ROW
-            BEGIN
-                UPDATE style_presets SET updated_at = STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')
-                    WHERE id = old.id;
-            END;
-            """
-        ]
-
-        # Add indexes for searchable fields
-        indices = [
-            "CREATE INDEX IF NOT EXISTS idx_style_presets_name ON style_presets(name);",
-        ]
-
-        for stmt in tables + indices + triggers:
-            cursor.execute(stmt)
-
-
-def build_migration_14() -> Migration:
-    """
-    Build the migration from database version 13 to 14..
-
-    This migration does the following:
-    - Create the table used to store style presets.
-    """
-    migration_14 = Migration(
-        from_version=13,
-        to_version=14,
-        callback=Migration14Callback(),
-    )
-
-    return migration_14