Merge branch 'main' into lstein/feat/multi-gpu

README.md

@@ -12,7 +12,7 @@
 
 Invoke is a leading creative engine built to empower professionals and enthusiasts alike. Generate and create stunning visual media using the latest AI-driven technologies. Invoke offers an industry leading web-based UI, and serves as the foundation for multiple commercial products.
 
-[Installation][installation docs] - [Documentation and Tutorials][docs home] - [Bug Reports][github issues] - [Contributing][contributing docs]
+[Installation and Updates][installation docs] - [Documentation and Tutorials][docs home] - [Bug Reports][github issues] - [Contributing][contributing docs]
 
 <div align="center">
 

docs/features/TRAINING.md

@@ -4,278 +4,6 @@ title: Training
 
 # :material-file-document: Training
 
-# Textual Inversion Training
+Invoke Training has moved to its own repository, with a dedicated UI for accessing common scripts like Textual Inversion and LoRA training.
-## **Personalizing Text-to-Image Generation**
 
-You may personalize the generated images to provide your own styles or objects
+You can find more by visiting the repo at https://github.com/invoke-ai/invoke-training
-by training a new LDM checkpoint and introducing a new vocabulary to the fixed
-model as a (.pt) embeddings file. Alternatively, you may use or train
-HuggingFace Concepts embeddings files (.bin) from
-<https://huggingface.co/sd-concepts-library> and its associated
-notebooks.
-
-## **Hardware and Software Requirements**
-
-You will need a GPU to perform training in a reasonable length of
-time, and at least 12 GB of VRAM. We recommend using the [`xformers`
-library](../installation/070_INSTALL_XFORMERS.md) to accelerate the
-training process further. During training, about ~8 GB is temporarily
-needed in order to store intermediate models, checkpoints and logs.
-
-## **Preparing for Training**
-
-To train, prepare a folder that contains 3-5 images that illustrate
-the object or concept. It is good to provide a variety of examples or
-poses to avoid overtraining the system. Format these images as PNG
-(preferred) or JPG. You do not need to resize or crop the images in
-advance, but for more control you may wish to do so.
-
-Place the training images in a directory on the machine InvokeAI runs
-on. We recommend placing them in a subdirectory of the
-`text-inversion-training-data` folder located in the InvokeAI root
-directory, ordinarily `~/invokeai` (Linux/Mac), or
-`C:\Users\your_name\invokeai` (Windows). For example, to create an
-embedding for the "psychedelic" style, you'd place the training images
-into the directory
-`~invokeai/text-inversion-training-data/psychedelic`.
-
-## **Launching Training Using the Console Front End**
-
-InvokeAI 2.3 and higher comes with a text console-based training front
-end. From within the `invoke.sh`/`invoke.bat` Invoke launcher script,
-start training tool selecting choice (3):
-
-```sh
-1 "Generate images with a browser-based interface"
-2 "Explore InvokeAI nodes using a command-line interface"
-3 "Textual inversion training"
-4 "Merge models (diffusers type only)"
-5 "Download and install models"
-6 "Change InvokeAI startup options"
-7 "Re-run the configure script to fix a broken install or to complete a major upgrade"
-8 "Open the developer console"
-9 "Update InvokeAI"
-```
-
-Alternatively, you can select option (8) or from the command line, with the InvokeAI virtual environment active,
-you can then launch the front end with the command `invokeai-ti --gui`.
-
-This will launch a text-based front end that will look like this:
-
-<figure markdown>
-![ti-frontend](../assets/textual-inversion/ti-frontend.png)
-</figure>
-
-The interface is keyboard-based. Move from field to field using
-control-N (^N) to move to the next field and control-P (^P) to the
-previous one. <Tab> and <shift-TAB> work as well. Once a field is
-active, use the cursor keys. In a checkbox group, use the up and down
-cursor keys to move from choice to choice, and <space> to select a
-choice. In a scrollbar, use the left and right cursor keys to increase
-and decrease the value of the scroll. In textfields, type the desired
-values.
-
-The number of parameters may look intimidating, but in most cases the
-predefined defaults work fine. The red circled fields in the above
-illustration are the ones you will adjust most frequently.
-
-### Model Name
-
-This will list all the diffusers models that are currently
-installed. Select the one you wish to use as the basis for your
-embedding. Be aware that if you use a SD-1.X-based model for your
-training, you will only be able to use this embedding with other
-SD-1.X-based models. Similarly, if you train on SD-2.X, you will only
-be able to use the embeddings with models based on SD-2.X.
-
-### Trigger Term
-
-This is the prompt term you will use to trigger the embedding. Type a
-single word or phrase you wish to use as the trigger, example
-"psychedelic" (without angle brackets). Within InvokeAI, you will then
-be able to activate the trigger using the syntax `<psychedelic>`.
-
-### Initializer
-
-This is a single character that is used internally during the training
-process as a placeholder for the trigger term. It defaults to "*" and
-can usually be left alone.
-
-### Resume from last saved checkpoint
-
-As training proceeds, textual inversion will write a series of
-intermediate files that can be used to resume training from where it
-was left off in the case of an interruption. This checkbox will be
-automatically selected if you provide a previously used trigger term
-and at least one checkpoint file is found on disk.
-
-Note that as of 20 January 2023, resume does not seem to be working
-properly due to an issue with the upstream code.
-
-### Data Training Directory
-
-This is the location of the images to be used for training. When you
-select a trigger term like "my-trigger", the frontend will prepopulate
-this field with `~/invokeai/text-inversion-training-data/my-trigger`,
-but you can change the path to wherever you want.
-
-### Output Destination Directory
-
-This is the location of the logs, checkpoint files, and embedding
-files created during training. When you select a trigger term like
-"my-trigger", the frontend will prepopulate this field with
-`~/invokeai/text-inversion-output/my-trigger`, but you can change the
-path to wherever you want.
-
-### Image resolution
-
-The images in the training directory will be automatically scaled to
-the value you use here. For best results, you will want to use the
-same default resolution of the underlying model (512 pixels for
-SD-1.5, 768 for the larger version of SD-2.1).
-
-### Center crop images
-
-If this is selected, your images will be center cropped to make them
-square before resizing them to the desired resolution. Center cropping
-can indiscriminately cut off the top of subjects' heads for portrait
-aspect images, so if you have images like this, you may wish to use a
-photoeditor to manually crop them to a square aspect ratio.
-
-### Mixed precision
-
-Select the floating point precision for the embedding. "no" will
-result in a full 32-bit precision, "fp16" will provide 16-bit
-precision, and "bf16" will provide mixed precision (only available
-when XFormers is used).
-
-### Max training steps
-
-How many steps the training will take before the model converges. Most
-training sets will converge with 2000-3000 steps.
-
-### Batch size
-
-This adjusts how many training images are processed simultaneously in
-each step. Higher values will cause the training process to run more
-quickly, but use more memory. The default size will run with GPUs with
-as little as 12 GB.
-
-### Learning rate
-
-The rate at which the system adjusts its internal weights during
-training. Higher values risk overtraining (getting the same image each
-time), and lower values will take more steps to train a good
-model. The default of 0.0005 is conservative; you may wish to increase
-it to 0.005 to speed up training.
-
-### Scale learning rate by number of GPUs, steps and batch size
-
-If this is selected (the default) the system will adjust the provided
-learning rate to improve performance.
-
-### Use xformers acceleration
-
-This will activate XFormers memory-efficient attention. You need to
-have XFormers installed for this to have an effect.
-
-### Learning rate scheduler
-
-This adjusts how the learning rate changes over the course of
-training. The default "constant" means to use a constant learning rate
-for the entire training session. The other values scale the learning
-rate according to various formulas.
-
-Only "constant" is supported by the XFormers library.
-
-### Gradient accumulation steps
-
-This is a parameter that allows you to use bigger batch sizes than
-your GPU's VRAM would ordinarily accommodate, at the cost of some
-performance.
-
-### Warmup steps
-
-If "constant_with_warmup" is selected in the learning rate scheduler,
-then this provides the number of warmup steps. Warmup steps have a
-very low learning rate, and are one way of preventing early
-overtraining.
-
-## The training run
-
-Start the training run by advancing to the OK button (bottom right)
-and pressing <enter>. A series of progress messages will be displayed
-as the training process proceeds. This may take an hour or two,
-depending on settings and the speed of your system. Various log and
-checkpoint files will be written into the output directory (ordinarily
-`~/invokeai/text-inversion-output/my-model/`)
-
-At the end of successful training, the system will copy the file
-`learned_embeds.bin` into the InvokeAI root directory's `embeddings`
-directory, using a subdirectory named after the trigger token. For
-example, if the trigger token was `psychedelic`, then look for the
-embeddings file in
-`~/invokeai/embeddings/psychedelic/learned_embeds.bin`
-
-You may now launch InvokeAI and try out a prompt that uses the trigger
-term. For example `a plate of banana sushi in <psychedelic> style`.
-
-## **Training with the Command-Line Script**
-
-Training can also be done using a traditional command-line script. It
-can be launched from within the "developer's console", or from the
-command line after activating InvokeAI's virtual environment.
-
-It accepts a large number of arguments, which can be summarized by
-passing the `--help` argument:
-
-```sh
-invokeai-ti --help
-```
-
-Typical usage is shown here:
-```sh
-invokeai-ti \
-       --model=stable-diffusion-1.5 \
-       --resolution=512 \
-       --learnable_property=style \
-       --initializer_token='*' \
-       --placeholder_token='<psychedelic>' \
-       --train_data_dir=/home/lstein/invokeai/training-data/psychedelic \
-       --output_dir=/home/lstein/invokeai/text-inversion-training/psychedelic \
-       --scale_lr \
-       --train_batch_size=8 \
-       --gradient_accumulation_steps=4 \
-       --max_train_steps=3000 \
-       --learning_rate=0.0005 \
-       --resume_from_checkpoint=latest \
-       --lr_scheduler=constant \
-       --mixed_precision=fp16 \
-       --only_save_embeds
-```
-
-## Troubleshooting
-
-### `Cannot load embedding for <trigger>. It was trained on a model with token dimension 1024, but the current model has token dimension 768`
-
-Messages like this indicate you trained the embedding on a different base model than the currently selected one.
-
-For example, in the error above, the training was done on SD2.1 (768x768) but it was used on SD1.5 (512x512).
-
-## Reading
-
-For more information on textual inversion, please see the following
-resources:
-
-* The [textual inversion repository](https://github.com/rinongal/textual_inversion) and
-  associated paper for details and limitations.
-* [HuggingFace's textual inversion training
-  page](https://huggingface.co/docs/diffusers/training/text_inversion)
-* [HuggingFace example script
-  documentation](https://github.com/huggingface/diffusers/tree/main/examples/textual_inversion)
-  (Note that this script is similar to, but not identical, to
-  `textual_inversion`, but produces embed files that are completely compatible.
-
----
-
-copyright (c) 2023, Lincoln Stein and the InvokeAI Development Team

docs/installation/010_INSTALL_AUTOMATED.md

@@ -1,8 +1,10 @@
-# Automatic Install
+# Automatic Install & Updates
 
-The installer is used for both new installs and updates.
+**The same packaged installer file can be used for both new installs and updates.**
+Using the installer for updates will leave everything you've added since installation, and just update the core libraries used to run Invoke.
+Simply use the same path you installed to originally.
 
-Both release and pre-release versions can be installed using it. It also supports install a wheel if needed.
+Both release and pre-release versions can be installed using the installer. It also supports install through a wheel if needed.
 
 Be sure to review the [installation requirements] and ensure your system has everything it needs to install Invoke.
 

docs/installation/INSTALLATION.md

@@ -1,4 +1,4 @@
-# Installation Overview
+# Installation and Updating Overview
 
 Before installing, review the [installation requirements] to ensure your system is set up properly.
 
@@ -6,14 +6,21 @@ See the [FAQ] for frequently-encountered installation issues.
 
 If you need more help, join our [discord] or [create an issue].
 
-<h2>Automatic Install</h2>
+<h2>Automatic Install & Updates </h2>
 
 ✅ The automatic install is the best way to run InvokeAI. Check out the [installation guide] to get started.
 
+⬆️ The same installer is also the best way to update InvokeAI - Simply rerun it for the same folder you installed to.
+
+The installation process simply manages installation for the core libraries & application dependencies that run Invoke.
+Any models, images, or other assets in the Invoke root folder won't be affected by the installation process.
+
 <h2>Manual Install</h2>
 
 If you are familiar with python and want more control over the packages that are installed, you can [install InvokeAI manually via PyPI].
 
+Updates are managed by reinstalling the latest version through PyPi.
+
 <h2>Developer Install</h2>
 
 If you want to contribute to InvokeAI, consult the [developer install guide].

invokeai/frontend/web/package.json

@@ -89,6 +89,7 @@
     "react-konva": "^18.2.10",
     "react-redux": "9.1.0",
     "react-resizable-panels": "^2.0.16",
+    "react-rnd": "^10.4.10",
     "react-select": "5.8.0",
     "react-use": "^17.5.0",
     "react-virtuoso": "^4.7.5",

invokeai/frontend/web/pnpm-lock.yaml

@@ -122,6 +122,9 @@ dependencies:
   react-resizable-panels:
     specifier: ^2.0.16
     version: 2.0.16(react-dom@18.2.0)(react@18.2.0)
+  react-rnd:
+    specifier: ^10.4.10
+    version: 10.4.10(react-dom@18.2.0)(react@18.2.0)
   react-select:
     specifier: 5.8.0
     version: 5.8.0(@types/react@18.2.73)(react-dom@18.2.0)(react@18.2.0)
@@ -7385,6 +7388,11 @@ packages:
     requiresBuild: true
     dev: true
 
+  /clsx@1.2.1:
+    resolution: {integrity: sha512-EcR6r5a8bj6pu3ycsa/E/cKVGuTgZJZdsyUYHOksG/UHIiKfjxzRxYJpyVBwYaQeOvghal9fcc4PidlgzugAQg==}
+    engines: {node: '>=6'}
+    dev: false
+
   /color-convert@1.9.3:
     resolution: {integrity: sha512-QfAUtd+vFdAtFQcC8CCyYt1fYWxSqAiK2cSD6zDB8N3cpsEBAvRxp9zOGg6G/SHHJYAT88/az/IuDGALsNVbGg==}
     dependencies:
@@ -11200,6 +11208,16 @@ packages:
       unpipe: 1.0.0
     dev: true
 
+  /re-resizable@6.9.14(react-dom@18.2.0)(react@18.2.0):
+    resolution: {integrity: sha512-2UbPrpezMr6gkHKNCRA/N6QGGU237SKOZ78yMHId204A/oXWSAREAIuGZNQ9qlrJosewzcsv2CphZH3u7hC6ng==}
+    peerDependencies:
+      react: ^16.13.1 || ^17.0.0 || ^18.0.0
+      react-dom: ^16.13.1 || ^17.0.0 || ^18.0.0
+    dependencies:
+      react: 18.2.0
+      react-dom: 18.2.0(react@18.2.0)
+    dev: false
+
   /react-clientside-effect@1.2.6(react@18.2.0):
     resolution: {integrity: sha512-XGGGRQAKY+q25Lz9a/4EPqom7WRjz3z9R2k4jhVKA/puQFH/5Nt27vFZYql4m4NVNdUvX8PS3O7r/Zzm7cjUlg==}
     peerDependencies:
@@ -11253,6 +11271,18 @@ packages:
       react: 18.2.0
       scheduler: 0.23.0
 
+  /react-draggable@4.4.6(react-dom@18.2.0)(react@18.2.0):
+    resolution: {integrity: sha512-LtY5Xw1zTPqHkVmtM3X8MUOxNDOUhv/khTgBgrUvwaS064bwVvxT+q5El0uUFNx5IEPKXuRejr7UqLwBIg5pdw==}
+    peerDependencies:
+      react: '>= 16.3.0'
+      react-dom: '>= 16.3.0'
+    dependencies:
+      clsx: 1.2.1
+      prop-types: 15.8.1
+      react: 18.2.0
+      react-dom: 18.2.0(react@18.2.0)
+    dev: false
+
   /react-dropzone@14.2.3(react@18.2.0):
     resolution: {integrity: sha512-O3om8I+PkFKbxCukfIR3QAGftYXDZfOE2N1mr/7qebQJHs7U+/RSL/9xomJNpRg9kM5h9soQSdf0Gc7OHF5Fug==}
     engines: {node: '>= 10.13'}
@@ -11466,6 +11496,19 @@ packages:
       react-dom: 18.2.0(react@18.2.0)
     dev: false
 
+  /react-rnd@10.4.10(react-dom@18.2.0)(react@18.2.0):
+    resolution: {integrity: sha512-YjQAgEeSbNUoOXSD9ZBvIiLVizFb+bNhpDk8DbIRHA557NW02CXbwsAeOTpJQnsdhEL+NP2I+Ssrwejqcodtjg==}
+    peerDependencies:
+      react: '>=16.3.0'
+      react-dom: '>=16.3.0'
+    dependencies:
+      re-resizable: 6.9.14(react-dom@18.2.0)(react@18.2.0)
+      react: 18.2.0
+      react-dom: 18.2.0(react@18.2.0)
+      react-draggable: 4.4.6(react-dom@18.2.0)(react@18.2.0)
+      tslib: 2.6.2
+    dev: false
+
   /react-select@5.7.7(@types/react@18.2.73)(react-dom@18.2.0)(react@18.2.0):
     resolution: {integrity: sha512-HhashZZJDRlfF/AKj0a0Lnfs3sRdw/46VJIRd8IbB9/Ovr74+ZIwkAdSBjSPXsFMG+u72c5xShqwLSKIJllzqw==}
     peerDependencies:

invokeai/frontend/web/public/locales/en.json

@@ -143,7 +143,8 @@
         "alpha": "Alpha",
         "selected": "Selected",
         "viewer": "Viewer",
-        "tab": "Tab"
+        "tab": "Tab",
+        "close": "Close"
     },
     "controlnet": {
         "controlAdapter_one": "Control Adapter",
@@ -365,7 +366,9 @@
         "bulkDownloadFailed": "Download Failed",
         "problemDeletingImages": "Problem Deleting Images",
         "problemDeletingImagesDesc": "One or more images could not be deleted",
-        "switchTo": "Switch to {{ tab }} (Z)"
+        "switchTo": "Switch to {{ tab }} (Z)",
+        "openFloatingViewer": "Open Floating Viewer",
+        "closeFloatingViewer": "Close Floating Viewer"
     },
     "hotkeys": {
         "searchHotkeys": "Search Hotkeys",
@@ -589,13 +592,9 @@
             "desc": "Upscale the current image",
             "title": "Upscale"
         },
-        "backToEditor": {
+        "toggleViewer": {
-            "desc": "Closes the Image Viewer and shows the Editor View (Text to Image tab only)",
+            "desc": "Switches between the Image Viewer and workspace for the current tab.",
-            "title": "Back to Editor"
+            "title": "Toggle Image Viewer"
-        },
-        "openImageViewer": {
-            "desc": "Opens the Image Viewer (Text to Image tab only)",
-            "title": "Open Image Viewer"
         }
     },
     "metadata": {
@@ -1534,7 +1533,7 @@
         "moveForward": "Move Forward",
         "moveBackward": "Move Backward",
         "brushSize": "Brush Size",
-        "controlLayers": "Control Layers (BETA)",
+        "controlLayers": "Control Layers",
         "globalMaskOpacity": "Global Mask Opacity",
         "autoNegative": "Auto Negative",
         "toggleVisibility": "Toggle Layer Visibility",

invokeai/frontend/web/src/app/components/App.tsx

@@ -12,6 +12,7 @@ import { useGlobalHotkeys } from 'common/hooks/useGlobalHotkeys';
 import ChangeBoardModal from 'features/changeBoardModal/components/ChangeBoardModal';
 import DeleteImageModal from 'features/deleteImageModal/components/DeleteImageModal';
 import { DynamicPromptsModal } from 'features/dynamicPrompts/components/DynamicPromptsPreviewModal';
+import { FloatingImageViewer } from 'features/gallery/components/ImageViewer/FloatingImageViewer';
 import { useStarterModelsToast } from 'features/modelManagerV2/hooks/useStarterModelsToast';
 import { configChanged } from 'features/system/store/configSlice';
 import { languageSelector } from 'features/system/store/systemSelectors';
@@ -96,6 +97,7 @@ const App = ({ config = DEFAULT_CONFIG, selectedImage }: Props) => {
       <DynamicPromptsModal />
       <Toaster />
       <PreselectedImage selectedImage={selectedImage} />
+      <FloatingImageViewer />
     </ErrorBoundary>
   );
 };

invokeai/frontend/web/src/app/store/middleware/listenerMiddleware/listeners/galleryImageClicked.ts

@@ -1,7 +1,7 @@
 import { createAction } from '@reduxjs/toolkit';
 import type { AppStartListening } from 'app/store/middleware/listenerMiddleware';
 import { selectListImagesQueryArgs } from 'features/gallery/store/gallerySelectors';
-import { selectionChanged } from 'features/gallery/store/gallerySlice';
+import { isImageViewerOpenChanged, selectionChanged } from 'features/gallery/store/gallerySlice';
 import { imagesApi } from 'services/api/endpoints/images';
 import type { ImageDTO } from 'services/api/types';
 import { imagesSelectors } from 'services/api/util';
@@ -62,6 +62,7 @@ export const addGalleryImageClickedListener = (startAppListening: AppStartListen
       } else {
         dispatch(selectionChanged([imageDTO]));
       }
+      dispatch(isImageViewerOpenChanged(true));
     },
   });
 };

invokeai/frontend/web/src/features/controlLayers/components/LayerCommon/LayerWrapper.tsx

@@ -10,7 +10,16 @@ type Props = PropsWithChildren<{
 
 export const LayerWrapper = memo(({ onClick, borderColor, children }: Props) => {
   return (
-    <Flex gap={2} onClick={onClick} bg={borderColor} px={2} borderRadius="base" py="1px">
+    <Flex
+      gap={2}
+      onClick={onClick}
+      bg={borderColor}
+      px={2}
+      borderRadius="base"
+      py="1px"
+      transitionProperty="all"
+      transitionDuration="0.2s"
+    >
       <Flex flexDir="column" w="full" bg="base.850" borderRadius="base">
         {children}
       </Flex>

invokeai/frontend/web/src/features/controlLayers/components/StageComponent.tsx

@@ -6,7 +6,7 @@ import { createMemoizedSelector } from 'app/store/createMemoizedSelector';
 import { useAppDispatch, useAppSelector } from 'app/store/storeHooks';
 import { useMouseEvents } from 'features/controlLayers/hooks/mouseEventHooks';
 import {
-  $cursorPosition,
+  $lastCursorPos,
   $lastMouseDownPos,
   $tool,
   isRegionalGuidanceLayer,
@@ -48,7 +48,7 @@ const useStageRenderer = (
   const state = useAppSelector((s) => s.controlLayers.present);
   const tool = useStore($tool);
   const mouseEventHandlers = useMouseEvents();
-  const cursorPosition = useStore($cursorPosition);
+  const lastCursorPos = useStore($lastCursorPos);
   const lastMouseDownPos = useStore($lastMouseDownPos);
   const selectedLayerIdColor = useAppSelector(selectSelectedLayerColor);
   const selectedLayerType = useAppSelector(selectSelectedLayerType);
@@ -141,7 +141,7 @@ const useStageRenderer = (
       selectedLayerIdColor,
       selectedLayerType,
       state.globalMaskLayerOpacity,
-      cursorPosition,
+      lastCursorPos,
       lastMouseDownPos,
       state.brushSize
     );
@@ -152,7 +152,7 @@ const useStageRenderer = (
     selectedLayerIdColor,
     selectedLayerType,
     state.globalMaskLayerOpacity,
-    cursorPosition,
+    lastCursorPos,
     lastMouseDownPos,
     state.brushSize,
     renderers,

invokeai/frontend/web/src/features/controlLayers/components/ToolChooser.tsx

@@ -4,9 +4,9 @@ import { createSelector } from '@reduxjs/toolkit';
 import { useAppDispatch, useAppSelector } from 'app/store/storeHooks';
 import {
   $tool,
+  layerReset,
   selectControlLayersSlice,
   selectedLayerDeleted,
-  selectedLayerReset,
 } from 'features/controlLayers/store/controlLayersSlice';
 import { useCallback } from 'react';
 import { useHotkeys } from 'react-hotkeys-hook';
@@ -22,6 +22,7 @@ export const ToolChooser: React.FC = () => {
   const { t } = useTranslation();
   const dispatch = useAppDispatch();
   const isDisabled = useAppSelector(selectIsDisabled);
+  const selectedLayerId = useAppSelector((s) => s.controlLayers.present.selectedLayerId);
   const tool = useStore($tool);
 
   const setToolToBrush = useCallback(() => {
@@ -42,8 +43,11 @@ export const ToolChooser: React.FC = () => {
   useHotkeys('v', setToolToMove, { enabled: !isDisabled }, [isDisabled]);
 
   const resetSelectedLayer = useCallback(() => {
-    dispatch(selectedLayerReset());
+    if (selectedLayerId === null) {
-  }, [dispatch]);
+      return;
+    }
+    dispatch(layerReset(selectedLayerId));
+  }, [dispatch, selectedLayerId]);
   useHotkeys('shift+c', resetSelectedLayer);
 
   const deleteSelectedLayer = useCallback(() => {

invokeai/frontend/web/src/features/controlLayers/hooks/layerStateHooks.ts

@@ -1,4 +1,5 @@
 import { createSelector } from '@reduxjs/toolkit';
+import { createMemoizedSelector } from 'app/store/createMemoizedSelector';
 import { useAppSelector } from 'app/store/storeHooks';
 import {
   isControlAdapterLayer,
@@ -69,7 +70,7 @@ export const useLayerType = (layerId: string) => {
 export const useLayerOpacity = (layerId: string) => {
   const selectLayer = useMemo(
     () =>
-      createSelector(selectControlLayersSlice, (controlLayers) => {
+      createMemoizedSelector(selectControlLayersSlice, (controlLayers) => {
         const layer = controlLayers.present.layers.filter(isControlAdapterLayer).find((l) => l.id === layerId);
         assert(layer, `Layer ${layerId} not found`);
         return { opacity: Math.round(layer.opacity * 100), isFilterEnabled: layer.isFilterEnabled };

invokeai/frontend/web/src/features/controlLayers/hooks/mouseEventHooks.ts

@@ -3,8 +3,8 @@ import { useStore } from '@nanostores/react';
 import { useAppDispatch, useAppSelector } from 'app/store/storeHooks';
 import { calculateNewBrushSize } from 'features/canvas/hooks/useCanvasZoom';
 import {
-  $cursorPosition,
   $isDrawing,
+  $lastCursorPos,
   $lastMouseDownPos,
   $tool,
   brushSizeChanged,
@@ -15,17 +15,41 @@ import {
 import type Konva from 'konva';
 import type { KonvaEventObject } from 'konva/lib/Node';
 import type { Vector2d } from 'konva/lib/types';
-import { useCallback, useRef } from 'react';
+import { clamp } from 'lodash-es';
+import { useCallback, useMemo, useRef } from 'react';
 
 const getIsFocused = (stage: Konva.Stage) => {
   return stage.container().contains(document.activeElement);
 };
 const getIsMouseDown = (e: KonvaEventObject<MouseEvent>) => e.evt.buttons === 1;
 
+const SNAP_PX = 10;
+
+export const snapPosToStage = (pos: Vector2d, stage: Konva.Stage) => {
+  const snappedPos = { ...pos };
+  // Get the normalized threshold for snapping to the edge of the stage
+  const thresholdX = SNAP_PX / stage.scaleX();
+  const thresholdY = SNAP_PX / stage.scaleY();
+  const stageWidth = stage.width() / stage.scaleX();
+  const stageHeight = stage.height() / stage.scaleY();
+  // Snap to the edge of the stage if within threshold
+  if (pos.x - thresholdX < 0) {
+    snappedPos.x = 0;
+  } else if (pos.x + thresholdX > stageWidth) {
+    snappedPos.x = Math.floor(stageWidth);
+  }
+  if (pos.y - thresholdY < 0) {
+    snappedPos.y = 0;
+  } else if (pos.y + thresholdY > stageHeight) {
+    snappedPos.y = Math.floor(stageHeight);
+  }
+  return snappedPos;
+};
+
 export const getScaledFlooredCursorPosition = (stage: Konva.Stage) => {
   const pointerPosition = stage.getPointerPosition();
   const stageTransform = stage.getAbsoluteTransform().copy();
-  if (!pointerPosition || !stageTransform) {
+  if (!pointerPosition) {
     return;
   }
   const scaledCursorPosition = stageTransform.invert().point(pointerPosition);
@@ -40,11 +64,13 @@ const syncCursorPos = (stage: Konva.Stage): Vector2d | null => {
   if (!pos) {
     return null;
   }
-  $cursorPosition.set(pos);
+  $lastCursorPos.set(pos);
   return pos;
 };
 
-const BRUSH_SPACING = 20;
+const BRUSH_SPACING_PCT = 10;
+const MIN_BRUSH_SPACING_PX = 5;
+const MAX_BRUSH_SPACING_PX = 15;
 
 export const useMouseEvents = () => {
   const dispatch = useAppDispatch();
@@ -60,6 +86,10 @@ export const useMouseEvents = () => {
   const lastCursorPosRef = useRef<[number, number] | null>(null);
   const shouldInvertBrushSizeScrollDirection = useAppSelector((s) => s.canvas.shouldInvertBrushSizeScrollDirection);
   const brushSize = useAppSelector((s) => s.controlLayers.present.brushSize);
+  const brushSpacingPx = useMemo(
+    () => clamp(brushSize / BRUSH_SPACING_PCT, MIN_BRUSH_SPACING_PX, MAX_BRUSH_SPACING_PX),
+    [brushSize]
+  );
 
   const onMouseDown = useCallback(
     (e: KonvaEventObject<MouseEvent>) => {
@@ -71,7 +101,6 @@ export const useMouseEvents = () => {
       if (!pos || !selectedLayerId || selectedLayerType !== 'regional_guidance_layer') {
         return;
       }
-      $lastMouseDownPos.set(pos);
       if (tool === 'brush' || tool === 'eraser') {
         dispatch(
           rgLayerLineAdded({
@@ -81,6 +110,9 @@ export const useMouseEvents = () => {
           })
         );
         $isDrawing.set(true);
+        $lastMouseDownPos.set(pos);
+      } else if (tool === 'rect') {
+        $lastMouseDownPos.set(snapPosToStage(pos, stage));
       }
     },
     [dispatch, selectedLayerId, selectedLayerType, tool]
@@ -92,21 +124,22 @@ export const useMouseEvents = () => {
       if (!stage) {
         return;
       }
-      const pos = $cursorPosition.get();
+      const pos = $lastCursorPos.get();
       if (!pos || !selectedLayerId || selectedLayerType !== 'regional_guidance_layer') {
         return;
       }
       const lastPos = $lastMouseDownPos.get();
       const tool = $tool.get();
       if (lastPos && selectedLayerId && tool === 'rect') {
+        const snappedPos = snapPosToStage(pos, stage);
         dispatch(
           rgLayerRectAdded({
             layerId: selectedLayerId,
             rect: {
-              x: Math.min(pos.x, lastPos.x),
+              x: Math.min(snappedPos.x, lastPos.x),
-              y: Math.min(pos.y, lastPos.y),
+              y: Math.min(snappedPos.y, lastPos.y),
-              width: Math.abs(pos.x - lastPos.x),
+              width: Math.abs(snappedPos.x - lastPos.x),
-              height: Math.abs(pos.y - lastPos.y),
+              height: Math.abs(snappedPos.y - lastPos.y),
             },
           })
         );
@@ -132,7 +165,7 @@ export const useMouseEvents = () => {
           // Continue the last line
           if (lastCursorPosRef.current) {
             // Dispatching redux events impacts perf substantially - using brush spacing keeps dispatches to a reasonable number
-            if (Math.hypot(lastCursorPosRef.current[0] - pos.x, lastCursorPosRef.current[1] - pos.y) < BRUSH_SPACING) {
+            if (Math.hypot(lastCursorPosRef.current[0] - pos.x, lastCursorPosRef.current[1] - pos.y) < brushSpacingPx) {
               return;
             }
           }
@@ -145,7 +178,7 @@ export const useMouseEvents = () => {
         $isDrawing.set(true);
       }
     },
-    [dispatch, selectedLayerId, selectedLayerType, tool]
+    [brushSpacingPx, dispatch, selectedLayerId, selectedLayerType, tool]
   );
 
   const onMouseLeave = useCallback(
@@ -155,14 +188,15 @@ export const useMouseEvents = () => {
         return;
       }
       const pos = syncCursorPos(stage);
+      $isDrawing.set(false);
+      $lastCursorPos.set(null);
+      $lastMouseDownPos.set(null);
       if (!pos || !selectedLayerId || selectedLayerType !== 'regional_guidance_layer') {
         return;
       }
       if (getIsFocused(stage) && getIsMouseDown(e) && (tool === 'brush' || tool === 'eraser')) {
         dispatch(rgLayerPointsAdded({ layerId: selectedLayerId, point: [pos.x, pos.y] }));
       }
-      $isDrawing.set(false);
-      $cursorPosition.set(null);
     },
     [selectedLayerId, selectedLayerType, tool, dispatch]
   );

invokeai/frontend/web/src/features/controlLayers/store/controlLayersSlice.ts

@@ -79,16 +79,6 @@ export const isRenderableLayer = (
   layer?.type === 'regional_guidance_layer' ||
   layer?.type === 'control_adapter_layer' ||
   layer?.type === 'initial_image_layer';
-const resetLayer = (layer: Layer) => {
-  if (layer.type === 'regional_guidance_layer') {
-    layer.maskObjects = [];
-    layer.bbox = null;
-    layer.isEnabled = true;
-    layer.needsPixelBbox = false;
-    layer.bboxNeedsUpdate = false;
-    return;
-  }
-};
 
 export const selectCALayerOrThrow = (state: ControlLayersState, layerId: string): ControlAdapterLayer => {
   const layer = state.layers.find((l) => l.id === layerId);
@@ -163,6 +153,9 @@ export const controlLayersSlice = createSlice({
         layer.x = x;
         layer.y = y;
       }
+      if (isRegionalGuidanceLayer(layer)) {
+        layer.uploadedMaskImage = null;
+      }
     },
     layerBboxChanged: (state, action: PayloadAction<{ layerId: string; bbox: IRect | null }>) => {
       const { layerId, bbox } = action.payload;
@@ -173,14 +166,21 @@ export const controlLayersSlice = createSlice({
         if (bbox === null && layer.type === 'regional_guidance_layer') {
           // The layer was fully erased, empty its objects to prevent accumulation of invisible objects
           layer.maskObjects = [];
+          layer.uploadedMaskImage = null;
           layer.needsPixelBbox = false;
         }
       }
     },
     layerReset: (state, action: PayloadAction<string>) => {
       const layer = state.layers.find((l) => l.id === action.payload);
-      if (layer) {
+      // TODO(psyche): Should other layer types also have reset functionality?
-        resetLayer(layer);
+      if (isRegionalGuidanceLayer(layer)) {
+        layer.maskObjects = [];
+        layer.bbox = null;
+        layer.isEnabled = true;
+        layer.needsPixelBbox = false;
+        layer.bboxNeedsUpdate = false;
+        layer.uploadedMaskImage = null;
       }
     },
     layerDeleted: (state, action: PayloadAction<string>) => {
@@ -213,12 +213,6 @@ export const controlLayersSlice = createSlice({
       moveToFront(renderableLayers, cb);
       state.layers = [...ipAdapterLayers, ...renderableLayers];
     },
-    selectedLayerReset: (state) => {
-      const layer = state.layers.find((l) => l.id === state.selectedLayerId);
-      if (layer) {
-        resetLayer(layer);
-      }
-    },
     selectedLayerDeleted: (state) => {
       state.layers = state.layers.filter((l) => l.id !== state.selectedLayerId);
       state.selectedLayerId = state.layers[0]?.id ?? null;
@@ -456,6 +450,7 @@ export const controlLayersSlice = createSlice({
           negativePrompt: null,
           ipAdapters: [],
           isSelected: true,
+          uploadedMaskImage: null,
         };
         state.layers.push(layer);
         state.selectedLayerId = layer.id;
@@ -505,6 +500,7 @@ export const controlLayersSlice = createSlice({
           strokeWidth: state.brushSize,
         });
         layer.bboxNeedsUpdate = true;
+        layer.uploadedMaskImage = null;
         if (!layer.needsPixelBbox && tool === 'eraser') {
           layer.needsPixelBbox = true;
         }
@@ -524,6 +520,7 @@ export const controlLayersSlice = createSlice({
       // TODO: Handle this in the event listener
       lastLine.points.push(point[0] - layer.x, point[1] - layer.y);
       layer.bboxNeedsUpdate = true;
+      layer.uploadedMaskImage = null;
     },
     rgLayerRectAdded: {
       reducer: (state, action: PayloadAction<{ layerId: string; rect: IRect; rectUuid: string }>) => {
@@ -543,9 +540,15 @@ export const controlLayersSlice = createSlice({
           height: rect.height,
         });
         layer.bboxNeedsUpdate = true;
+        layer.uploadedMaskImage = null;
       },
       prepare: (payload: { layerId: string; rect: IRect }) => ({ payload: { ...payload, rectUuid: uuidv4() } }),
     },
+    rgLayerMaskImageUploaded: (state, action: PayloadAction<{ layerId: string; imageDTO: ImageDTO }>) => {
+      const { layerId, imageDTO } = action.payload;
+      const layer = selectRGLayerOrThrow(state, layerId);
+      layer.uploadedMaskImage = imageDTOToImageWithDims(imageDTO);
+    },
     rgLayerAutoNegativeChanged: (
       state,
       action: PayloadAction<{ layerId: string; autoNegative: ParameterAutoNegative }>
@@ -792,7 +795,6 @@ export const {
   layerMovedToFront,
   layerMovedBackward,
   layerMovedToBack,
-  selectedLayerReset,
   selectedLayerDeleted,
   allLayersDeleted,
   // CA Layers
@@ -825,6 +827,7 @@ export const {
   rgLayerLineAdded,
   rgLayerPointsAdded,
   rgLayerRectAdded,
+  rgLayerMaskImageUploaded,
   rgLayerAutoNegativeChanged,
   rgLayerIPAdapterAdded,
   rgLayerIPAdapterDeleted,
@@ -863,7 +866,7 @@ const migrateControlLayersState = (state: any): any => {
 export const $isDrawing = atom(false);
 export const $lastMouseDownPos = atom<Vector2d | null>(null);
 export const $tool = atom<Tool>('brush');
-export const $cursorPosition = atom<Vector2d | null>(null);
+export const $lastCursorPos = atom<Vector2d | null>(null);
 
 // IDs for singleton Konva layers and objects
 export const TOOL_PREVIEW_LAYER_ID = 'tool_preview_layer';
@@ -886,6 +889,7 @@ export const RG_LAYER_RECT_NAME = 'regional_guidance_layer.rect';
 export const INITIAL_IMAGE_LAYER_NAME = 'initial_image_layer';
 export const INITIAL_IMAGE_LAYER_IMAGE_NAME = 'initial_image_layer.image';
 export const LAYER_BBOX_NAME = 'layer.bbox';
+export const COMPOSITING_RECT_NAME = 'compositing-rect';
 
 // Getters for non-singleton layer and object IDs
 const getRGLayerId = (layerId: string) => `${RG_LAYER_NAME}_${layerId}`;

invokeai/frontend/web/src/features/controlLayers/store/types.ts

@@ -72,6 +72,7 @@ export type RegionalGuidanceLayer = RenderableLayerBase & {
   previewColor: RgbColor;
   autoNegative: ParameterAutoNegative;
   needsPixelBbox: boolean; // Needs the slower pixel-based bbox calculation - set to true when an there is an eraser object
+  uploadedMaskImage: ImageWithDims | null;
 };
 
 export type InitialImageLayer = RenderableLayerBase & {

invokeai/frontend/web/src/features/controlLayers/util/bbox.ts

@@ -123,7 +123,7 @@ export const getLayerBboxPixels = (layer: KonvaLayerType, preview: boolean = fal
   return correctedLayerBbox;
 };
 
-export const getLayerBboxFast = (layer: KonvaLayerType): IRect | null => {
+export const getLayerBboxFast = (layer: KonvaLayerType): IRect => {
   const bbox = layer.getClientRect(GET_CLIENT_RECT_CONFIG);
   return {
     x: Math.floor(bbox.x),

invokeai/frontend/web/src/features/controlLayers/util/renderers.ts

@@ -1,12 +1,13 @@
 import { getStore } from 'app/store/nanostores/store';
 import { rgbaColorToString, rgbColorToString } from 'features/canvas/util/colorToString';
-import { getScaledFlooredCursorPosition } from 'features/controlLayers/hooks/mouseEventHooks';
+import { getScaledFlooredCursorPosition, snapPosToStage } from 'features/controlLayers/hooks/mouseEventHooks';
 import {
   $tool,
   BACKGROUND_LAYER_ID,
   BACKGROUND_RECT_ID,
   CA_LAYER_IMAGE_NAME,
   CA_LAYER_NAME,
+  COMPOSITING_RECT_NAME,
   getCALayerImageId,
   getIILayerImageId,
   getLayerBboxId,
@@ -211,12 +212,13 @@ const renderToolPreview = (
   }
 
   if (cursorPos && lastMouseDownPos && tool === 'rect') {
+    const snappedPos = snapPosToStage(cursorPos, stage);
     const rectPreview = toolPreviewLayer.findOne<Konva.Rect>(`#${TOOL_PREVIEW_RECT_ID}`);
     rectPreview?.setAttrs({
-      x: Math.min(cursorPos.x, lastMouseDownPos.x),
+      x: Math.min(snappedPos.x, lastMouseDownPos.x),
-      y: Math.min(cursorPos.y, lastMouseDownPos.y),
+      y: Math.min(snappedPos.y, lastMouseDownPos.y),
-      width: Math.abs(cursorPos.x - lastMouseDownPos.x),
+      width: Math.abs(snappedPos.x - lastMouseDownPos.x),
-      height: Math.abs(cursorPos.y - lastMouseDownPos.y),
+      height: Math.abs(snappedPos.y - lastMouseDownPos.y),
     });
     rectPreview?.visible(true);
   } else {
@@ -323,6 +325,12 @@ const createVectorMaskRect = (reduxObject: VectorMaskRect, konvaGroup: Konva.Gro
   return vectorMaskRect;
 };
 
+const createCompositingRect = (konvaLayer: Konva.Layer): Konva.Rect => {
+  const compositingRect = new Konva.Rect({ name: COMPOSITING_RECT_NAME, listening: false });
+  konvaLayer.add(compositingRect);
+  return compositingRect;
+};
+
 /**
  * Renders a vector mask layer.
  * @param stage The konva stage to render on.
@@ -400,15 +408,53 @@ const renderRegionalGuidanceLayer = (
     groupNeedsCache = true;
   }
 
-  if (konvaObjectGroup.children.length === 0) {
+  if (konvaObjectGroup.getChildren().length === 0) {
     // No objects - clear the cache to reset the previous pixel data
     konvaObjectGroup.clearCache();
-  } else if (groupNeedsCache) {
+    return;
-    konvaObjectGroup.cache();
   }
 
+  const compositingRect =
+    konvaLayer.findOne<Konva.Rect>(`.${COMPOSITING_RECT_NAME}`) ?? createCompositingRect(konvaLayer);
+
+  /**
+   * When the group is selected, we use a rect of the selected preview color, composited over the shapes. This allows
+   * shapes to render as a "raster" layer with all pixels drawn at the same color and opacity.
+   *
+   * Without this special handling, each shape is drawn individually with the given opacity, atop the other shapes. The
+   * effect is like if you have a Photoshop Group consisting of many shapes, each of which has the given opacity.
+   * Overlapping shapes will have their colors blended together, and the final color is the result of all the shapes.
+   *
+   * Instead, with the special handling, the effect is as if you drew all the shapes at 100% opacity, flattened them to
+   * a single raster image, and _then_ applied the 50% opacity.
+   */
+  if (reduxLayer.isSelected && tool !== 'move') {
+    // We must clear the cache first so Konva will re-draw the group with the new compositing rect
+    if (konvaObjectGroup.isCached()) {
+      konvaObjectGroup.clearCache();
+    }
+    // The user is allowed to reduce mask opacity to 0, but we need the opacity for the compositing rect to work
+    konvaObjectGroup.opacity(1);
+
+    compositingRect.setAttrs({
+      // The rect should be the size of the layer - use the fast method bc it's OK if the rect is larger
+      ...getLayerBboxFast(konvaLayer),
+      fill: rgbColor,
+      opacity: globalMaskLayerOpacity,
+      // Draw this rect only where there are non-transparent pixels under it (e.g. the mask shapes)
+      globalCompositeOperation: 'source-in',
+      visible: true,
+      // This rect must always be on top of all other shapes
+      zIndex: konvaObjectGroup.getChildren().length,
+    });
+  } else {
+    // The compositing rect should only be shown when the layer is selected.
+    compositingRect.visible(false);
+    // Cache only if needed - or if we are on this code path and _don't_ have a cache
+    if (groupNeedsCache || !konvaObjectGroup.isCached()) {
+      konvaObjectGroup.cache();
+    }
     // Updating group opacity does not require re-caching
-  if (konvaObjectGroup.opacity() !== globalMaskLayerOpacity) {
     konvaObjectGroup.opacity(globalMaskLayerOpacity);
   }
 };

invokeai/frontend/web/src/features/gallery/components/ImageViewer/CurrentImagePreview.tsx

@@ -22,7 +22,21 @@ const selectLastSelectedImageName = createSelector(
   (lastSelectedImage) => lastSelectedImage?.image_name
 );
 
-const CurrentImagePreview = () => {
+type Props = {
+  isDragDisabled?: boolean;
+  isDropDisabled?: boolean;
+  withNextPrevButtons?: boolean;
+  withMetadata?: boolean;
+  alwaysShowProgress?: boolean;
+};
+
+const CurrentImagePreview = ({
+  isDragDisabled = false,
+  isDropDisabled = false,
+  withNextPrevButtons = true,
+  withMetadata = true,
+  alwaysShowProgress = false,
+}: Props) => {
   const { t } = useTranslation();
   const shouldShowImageDetails = useAppSelector((s) => s.ui.shouldShowImageDetails);
   const imageName = useAppSelector(selectLastSelectedImageName);
@@ -52,30 +66,35 @@ const CurrentImagePreview = () => {
   // Show and hide the next/prev buttons on mouse move
   const [shouldShowNextPrevButtons, setShouldShowNextPrevButtons] = useState<boolean>(false);
   const timeoutId = useRef(0);
-  const onMouseMove = useCallback(() => {
+  const onMouseOver = useCallback(() => {
     setShouldShowNextPrevButtons(true);
     window.clearTimeout(timeoutId.current);
+  }, []);
+  const onMouseOut = useCallback(() => {
     timeoutId.current = window.setTimeout(() => {
       setShouldShowNextPrevButtons(false);
-    }, 1000);
+    }, 500);
   }, []);
 
   return (
     <Flex
-      onMouseMove={onMouseMove}
+      onMouseOver={onMouseOver}
+      onMouseOut={onMouseOut}
       width="full"
       height="full"
       alignItems="center"
       justifyContent="center"
       position="relative"
     >
-      {hasDenoiseProgress && shouldShowProgressInViewer ? (
+      {hasDenoiseProgress && (shouldShowProgressInViewer || alwaysShowProgress) ? (
         <ProgressImage />
       ) : (
         <IAIDndImage
           imageDTO={imageDTO}
           droppableData={droppableData}
           draggableData={draggableData}
+          isDragDisabled={isDragDisabled}
+          isDropDisabled={isDropDisabled}
           isUploadDisabled={true}
           fitContainer
           useThumbailFallback
@@ -85,7 +104,7 @@ const CurrentImagePreview = () => {
         />
       )}
       <AnimatePresence>
-        {shouldShowImageDetails && imageDTO && (
+        {shouldShowImageDetails && imageDTO && withMetadata && (
           <Box
             as={motion.div}
             key="metadataViewer"
@@ -103,7 +122,7 @@ const CurrentImagePreview = () => {
         )}
       </AnimatePresence>
       <AnimatePresence>
-        {shouldShowNextPrevButtons && imageDTO && (
+        {withNextPrevButtons && shouldShowNextPrevButtons && imageDTO && (
           <Box
             as={motion.div}
             key="nextPrevButtons"

invokeai/frontend/web/src/features/gallery/components/ImageViewer/EditorButton.tsx

@@ -4,19 +4,12 @@ import type { InvokeTabName } from 'features/ui/store/tabMap';
 import { activeTabNameSelector } from 'features/ui/store/uiSelectors';
 import { useMemo } from 'react';
 import { useTranslation } from 'react-i18next';
+import { PiArrowsDownUpBold } from 'react-icons/pi';
 
 import { useImageViewer } from './useImageViewer';
 
-const TAB_NAME_TO_TKEY: Record<InvokeTabName, string> = {
-  generation: 'ui.tabs.generationTab',
-  canvas: 'ui.tabs.canvasTab',
-  workflows: 'ui.tabs.workflowsTab',
-  models: 'ui.tabs.modelsTab',
-  queue: 'ui.tabs.queueTab',
-};
-
 const TAB_NAME_TO_TKEY_SHORT: Record<InvokeTabName, string> = {
-  generation: 'ui.tabs.generation',
+  generation: 'controlLayers.controlLayers',
   canvas: 'ui.tabs.canvas',
   workflows: 'ui.tabs.workflows',
   models: 'ui.tabs.models',
@@ -27,10 +20,19 @@ export const EditorButton = () => {
   const { t } = useTranslation();
   const { onClose } = useImageViewer();
   const activeTabName = useAppSelector(activeTabNameSelector);
-  const tooltip = useMemo(() => t('gallery.switchTo', { tab: t(TAB_NAME_TO_TKEY[activeTabName]) }), [t, activeTabName]);
+  const tooltip = useMemo(
+    () => t('gallery.switchTo', { tab: t(TAB_NAME_TO_TKEY_SHORT[activeTabName]) }),
+    [t, activeTabName]
+  );
 
   return (
-    <Button aria-label={tooltip} tooltip={tooltip} onClick={onClose} variant="ghost">
+    <Button
+      aria-label={tooltip}
+      tooltip={tooltip}
+      onClick={onClose}
+      variant="outline"
+      leftIcon={<PiArrowsDownUpBold />}
+    >
       {t(TAB_NAME_TO_TKEY_SHORT[activeTabName])}
     </Button>
   );

invokeai/frontend/web/src/features/gallery/components/ImageViewer/FloatingImageViewer.tsx

@@ -0,0 +1,184 @@
+import { Flex, IconButton, Spacer, Text, useShiftModifier } from '@invoke-ai/ui-library';
+import { useAppDispatch, useAppSelector } from 'app/store/storeHooks';
+import CurrentImagePreview from 'features/gallery/components/ImageViewer/CurrentImagePreview';
+import { isFloatingImageViewerOpenChanged } from 'features/gallery/store/gallerySlice';
+import { useCallback, useLayoutEffect, useRef } from 'react';
+import { flushSync } from 'react-dom';
+import { useTranslation } from 'react-i18next';
+import { PiHourglassBold, PiXBold } from 'react-icons/pi';
+import { Rnd } from 'react-rnd';
+
+const defaultDim = 256;
+const maxDim = 512;
+const defaultSize = { width: defaultDim, height: defaultDim + 24 };
+const maxSize = { width: maxDim, height: maxDim + 24 };
+const rndDefault = { x: 0, y: 0, ...defaultSize };
+
+const rndStyles = {
+  zIndex: 11,
+};
+
+const enableResizing = {
+  top: false,
+  right: false,
+  bottom: false,
+  left: false,
+  topRight: false,
+  bottomRight: true,
+  bottomLeft: false,
+  topLeft: false,
+};
+
+const FloatingImageViewerComponent = () => {
+  const { t } = useTranslation();
+  const dispatch = useAppDispatch();
+  const shift = useShiftModifier();
+  const rndRef = useRef<Rnd>(null);
+  const imagePreviewRef = useRef<HTMLDivElement>(null);
+  const onClose = useCallback(() => {
+    dispatch(isFloatingImageViewerOpenChanged(false));
+  }, [dispatch]);
+
+  const fitToScreen = useCallback(() => {
+    if (!imagePreviewRef.current || !rndRef.current) {
+      return;
+    }
+    const el = imagePreviewRef.current;
+    const rnd = rndRef.current;
+
+    const { top, right, bottom, left, width, height } = el.getBoundingClientRect();
+    const { innerWidth, innerHeight } = window;
+
+    const newPosition = rnd.getDraggablePosition();
+
+    if (top < 0) {
+      newPosition.y = 0;
+    }
+    if (left < 0) {
+      newPosition.x = 0;
+    }
+    if (bottom > innerHeight) {
+      newPosition.y = innerHeight - height;
+    }
+    if (right > innerWidth) {
+      newPosition.x = innerWidth - width;
+    }
+    rnd.updatePosition(newPosition);
+  }, []);
+
+  const onDoubleClick = useCallback(() => {
+    if (!rndRef.current || !imagePreviewRef.current) {
+      return;
+    }
+    const { width, height } = imagePreviewRef.current.getBoundingClientRect();
+    if (width === defaultSize.width && height === defaultSize.height) {
+      rndRef.current.updateSize(maxSize);
+    } else {
+      rndRef.current.updateSize(defaultSize);
+    }
+    flushSync(fitToScreen);
+  }, [fitToScreen]);
+
+  useLayoutEffect(() => {
+    window.addEventListener('resize', fitToScreen);
+    return () => {
+      window.removeEventListener('resize', fitToScreen);
+    };
+  }, [fitToScreen]);
+
+  useLayoutEffect(() => {
+    // Set the initial position
+    if (!imagePreviewRef.current || !rndRef.current) {
+      return;
+    }
+
+    const { width, height } = imagePreviewRef.current.getBoundingClientRect();
+
+    const initialPosition = {
+      // 54 = width of left-hand vertical bar of tab icons
+      // 430 = width of parameters panel
+      x: 54 + 430 / 2 - width / 2,
+      // 16 = just a reasonable bottom padding
+      y: window.innerHeight - height - 16,
+    };
+
+    rndRef.current.updatePosition(initialPosition);
+  }, [fitToScreen]);
+
+  return (
+    <Rnd
+      ref={rndRef}
+      default={rndDefault}
+      bounds="window"
+      lockAspectRatio={shift}
+      minWidth={defaultSize.width}
+      minHeight={defaultSize.height}
+      maxWidth={maxSize.width}
+      maxHeight={maxSize.height}
+      style={rndStyles}
+      enableResizing={enableResizing}
+    >
+      <Flex
+        ref={imagePreviewRef}
+        flexDir="column"
+        bg="base.850"
+        borderRadius="base"
+        w="full"
+        h="full"
+        borderWidth={1}
+        shadow="dark-lg"
+        cursor="move"
+      >
+        <Flex bg="base.800" w="full" p={1} onDoubleClick={onDoubleClick}>
+          <Text fontSize="sm" fontWeight="semibold" color="base.300" ps={2}>
+            {t('common.viewer')}
+          </Text>
+          <Spacer />
+          <IconButton aria-label={t('common.close')} icon={<PiXBold />} size="sm" variant="link" onClick={onClose} />
+        </Flex>
+        <Flex p={2} w="full" h="full">
+          <CurrentImagePreview
+            isDragDisabled={true}
+            isDropDisabled={true}
+            withNextPrevButtons={false}
+            withMetadata={false}
+            alwaysShowProgress
+          />
+        </Flex>
+      </Flex>
+    </Rnd>
+  );
+};
+
+export const FloatingImageViewer = () => {
+  const isOpen = useAppSelector((s) => s.gallery.isFloatingImageViewerOpen);
+
+  if (!isOpen) {
+    return null;
+  }
+
+  return <FloatingImageViewerComponent />;
+};
+
+export const ToggleFloatingImageViewerButton = () => {
+  const { t } = useTranslation();
+  const dispatch = useAppDispatch();
+  const isOpen = useAppSelector((s) => s.gallery.isFloatingImageViewerOpen);
+
+  const onToggle = useCallback(() => {
+    dispatch(isFloatingImageViewerOpenChanged(!isOpen));
+  }, [dispatch, isOpen]);
+
+  return (
+    <IconButton
+      tooltip={isOpen ? t('gallery.closeFloatingViewer') : t('gallery.openFloatingViewer')}
+      aria-label={isOpen ? t('gallery.closeFloatingViewer') : t('gallery.openFloatingViewer')}
+      icon={<PiHourglassBold fontSize={16} />}
+      size="sm"
+      onClick={onToggle}
+      variant="link"
+      colorScheme={isOpen ? 'invokeBlue' : 'base'}
+      boxSize={8}
+    />
+  );
+};

invokeai/frontend/web/src/features/gallery/components/ImageViewer/ImageViewer.tsx

@@ -42,8 +42,9 @@ export const ImageViewer = memo(() => {
   useHotkeys('z', onToggle, { enabled: isViewerEnabled }, [isViewerEnabled, onToggle]);
   useHotkeys('esc', onClose, { enabled: isViewerEnabled }, [isViewerEnabled, onClose]);
 
+  // The AnimatePresence mode must be wait - else framer can get confused if you spam the toggle button
   return (
-    <AnimatePresence>
+    <AnimatePresence mode="wait">
       {shouldShowViewer && (
         <Flex
           key="imageViewer"

invokeai/frontend/web/src/features/gallery/components/ImageViewer/ViewerButton.tsx

@@ -1,6 +1,7 @@
 import { Button } from '@invoke-ai/ui-library';
 import { useMemo } from 'react';
 import { useTranslation } from 'react-i18next';
+import { PiArrowsDownUpBold } from 'react-icons/pi';
 
 import { useImageViewer } from './useImageViewer';
 
@@ -10,7 +11,14 @@ export const ViewerButton = () => {
   const tooltip = useMemo(() => t('gallery.switchTo', { tab: t('common.viewer') }), [t]);
 
   return (
-    <Button aria-label={tooltip} tooltip={tooltip} onClick={onOpen} variant="ghost" pointerEvents="auto">
+    <Button
+      aria-label={tooltip}
+      tooltip={tooltip}
+      onClick={onOpen}
+      variant="outline"
+      pointerEvents="auto"
+      leftIcon={<PiArrowsDownUpBold />}
+    >
       {t('common.viewer')}
     </Button>
   );

invokeai/frontend/web/src/features/gallery/store/gallerySlice.ts

@@ -1,6 +1,7 @@
 import type { PayloadAction } from '@reduxjs/toolkit';
 import { createSlice, isAnyOf } from '@reduxjs/toolkit';
 import type { PersistConfig, RootState } from 'app/store/store';
+import { setActiveTab } from 'features/ui/store/uiSlice';
 import { uniqBy } from 'lodash-es';
 import { boardsApi } from 'services/api/endpoints/boards';
 import { imagesApi } from 'services/api/endpoints/images';
@@ -22,6 +23,7 @@ const initialGalleryState: GalleryState = {
   limit: INITIAL_IMAGE_LIMIT,
   offset: 0,
   isImageViewerOpen: false,
+  isFloatingImageViewerOpen: false,
 };
 
 export const gallerySlice = createSlice({
@@ -30,11 +32,9 @@ export const gallerySlice = createSlice({
   reducers: {
     imageSelected: (state, action: PayloadAction<ImageDTO | null>) => {
       state.selection = action.payload ? [action.payload] : [];
-      state.isImageViewerOpen = true;
     },
     selectionChanged: (state, action: PayloadAction<ImageDTO[]>) => {
       state.selection = uniqBy(action.payload, (i) => i.image_name);
-      state.isImageViewerOpen = true;
     },
     shouldAutoSwitchChanged: (state, action: PayloadAction<boolean>) => {
       state.shouldAutoSwitch = action.payload;
@@ -81,8 +81,14 @@ export const gallerySlice = createSlice({
     isImageViewerOpenChanged: (state, action: PayloadAction<boolean>) => {
       state.isImageViewerOpen = action.payload;
     },
+    isFloatingImageViewerOpenChanged: (state, action: PayloadAction<boolean>) => {
+      state.isFloatingImageViewerOpen = action.payload;
+    },
   },
   extraReducers: (builder) => {
+    builder.addCase(setActiveTab, (state) => {
+      state.isImageViewerOpen = false;
+    });
     builder.addMatcher(isAnyBoardDeleted, (state, action) => {
       const deletedBoardId = action.meta.arg.originalArgs;
       if (deletedBoardId === state.selectedBoardId) {
@@ -119,6 +125,7 @@ export const {
   moreImagesLoaded,
   alwaysShowImageSizeBadgeChanged,
   isImageViewerOpenChanged,
+  isFloatingImageViewerOpenChanged,
 } = gallerySlice.actions;
 
 const isAnyBoardDeleted = isAnyOf(

invokeai/frontend/web/src/features/gallery/store/types.ts

@@ -21,4 +21,5 @@ export type GalleryState = {
   limit: number;
   alwaysShowImageSizeBadge: boolean;
   isImageViewerOpen: boolean;
+  isFloatingImageViewerOpen: boolean;
 };

invokeai/frontend/web/src/features/nodes/util/graph/addControlLayersToGraph.ts

@@ -4,7 +4,9 @@ import {
   isControlAdapterLayer,
   isIPAdapterLayer,
   isRegionalGuidanceLayer,
+  rgLayerMaskImageUploaded,
 } from 'features/controlLayers/store/controlLayersSlice';
+import type { RegionalGuidanceLayer } from 'features/controlLayers/store/types';
 import {
   type ControlNetConfigV2,
   type ImageWithDims,
@@ -32,12 +34,13 @@ import {
 } from 'features/nodes/util/graph/constants';
 import { upsertMetadata } from 'features/nodes/util/graph/metadata';
 import { size } from 'lodash-es';
-import { imagesApi } from 'services/api/endpoints/images';
+import { getImageDTO, imagesApi } from 'services/api/endpoints/images';
 import type {
   CollectInvocation,
   ControlNetInvocation,
   CoreMetadataInvocation,
   Edge,
+  ImageDTO,
   IPAdapterInvocation,
   NonNullableGraph,
   S,
@@ -337,7 +340,6 @@ const addGlobalIPAdaptersToGraph = async (
 };
 
 export const addControlLayersToGraph = async (state: RootState, graph: NonNullableGraph, denoiseNodeId: string) => {
-  const { dispatch } = getStore();
   const mainModel = state.generation.model;
   assert(mainModel, 'Missing main model when building graph');
   const isSDXL = mainModel.base === 'sdxl';
@@ -404,10 +406,6 @@ export const addControlLayersToGraph = async (state: RootState, graph: NonNullab
       return hasTextPrompt || hasIPAdapter;
     });
 
-  const layerIds = rgLayers.map((l) => l.id);
-  const blobs = await getRegionalPromptLayerBlobs(layerIds);
-  assert(size(blobs) === size(layerIds), 'Mismatch between layer IDs and blobs');
-
   // TODO: We should probably just use conditioning collectors by default, and skip all this fanagling with re-routing
   // the existing conditioning nodes.
 
@@ -470,22 +468,15 @@ export const addControlLayersToGraph = async (state: RootState, graph: NonNullab
     },
   });
 
-  // Upload the blobs to the backend, add each to graph
+  const layerIds = rgLayers.map((l) => l.id);
-  // TODO: Store the uploaded image names in redux to reuse them, so long as the layer hasn't otherwise changed. This
+  const blobs = await getRegionalPromptLayerBlobs(layerIds);
-  // would be a great perf win - not only would we skip re-uploading the same image, but we'd be able to use the node
+  assert(size(blobs) === size(layerIds), 'Mismatch between layer IDs and blobs');
-  // cache (currently, when we re-use the same mask data, since it is a different image, the node cache is not used).
+
   for (const layer of rgLayers) {
     const blob = blobs[layer.id];
     assert(blob, `Blob for layer ${layer.id} not found`);
-
+    // Upload the mask image, or get the cached image if it exists
-    const file = new File([blob], `${layer.id}_mask.png`, { type: 'image/png' });
+    const { image_name } = await getMaskImage(layer, blob);
-    const req = dispatch(
-      imagesApi.endpoints.uploadImage.initiate({ file, image_category: 'mask', is_intermediate: true })
-    );
-    req.reset();
-
-    // TODO: This will raise on network error
-    const { image_name } = await req.unwrap();
 
     // The main mask-to-tensor node
     const maskToTensorNode: S['AlphaMaskToTensorInvocation'] = {
@@ -679,3 +670,23 @@ export const addControlLayersToGraph = async (state: RootState, graph: NonNullab
     }
   }
 };
+
+const getMaskImage = async (layer: RegionalGuidanceLayer, blob: Blob): Promise<ImageDTO> => {
+  if (layer.uploadedMaskImage) {
+    const imageDTO = await getImageDTO(layer.uploadedMaskImage.imageName);
+    if (imageDTO) {
+      return imageDTO;
+    }
+  }
+  const { dispatch } = getStore();
+  // No cached mask, or the cached image no longer exists - we need to upload the mask image
+  const file = new File([blob], `${layer.id}_mask.png`, { type: 'image/png' });
+  const req = dispatch(
+    imagesApi.endpoints.uploadImage.initiate({ file, image_category: 'mask', is_intermediate: true })
+  );
+  req.reset();
+
+  const imageDTO = await req.unwrap();
+  dispatch(rgLayerMaskImageUploaded({ layerId: layer.id, imageDTO }));
+  return imageDTO;
+};

invokeai/frontend/web/src/features/nodes/util/graph/addInitialImageToLinearGraph.ts

@@ -6,11 +6,14 @@ import { assert } from 'tsafe';
 
 import { IMAGE_TO_LATENTS, NOISE, RESIZE } from './constants';
 
+/**
+ * Returns true if an initial image was added, false if not.
+ */
 export const addInitialImageToLinearGraph = (
   state: RootState,
   graph: NonNullableGraph,
   denoiseNodeId: string
-): void => {
+): boolean => {
   // Remove Existing UNet Connections
   const { img2imgStrength, vaePrecision, model } = state.generation;
   const { refinerModel, refinerStart } = state.sdxl;
@@ -19,7 +22,7 @@ export const addInitialImageToLinearGraph = (
   const initialImage = initialImageLayer?.isEnabled ? initialImageLayer?.image : null;
 
   if (!initialImage) {
-    return;
+    return false;
   }
 
   const isSDXL = model?.base === 'sdxl';
@@ -122,4 +125,6 @@ export const addInitialImageToLinearGraph = (
     strength: img2imgStrength,
     init_image: initialImage.imageName,
   });
+
+  return true;
 };

invokeai/frontend/web/src/features/nodes/util/graph/buildAdHocUpscaleGraph.ts

@@ -1,5 +1,5 @@
 import type { RootState } from 'app/store/store';
-import { getBoardField, getIsIntermediate } from 'features/nodes/util/graph/graphBuilderUtils';
+import { getBoardField } from 'features/nodes/util/graph/graphBuilderUtils';
 import type { ESRGANInvocation, Graph, NonNullableGraph } from 'services/api/types';
 
 import { ESRGAN } from './constants';
@@ -18,7 +18,7 @@ export const buildAdHocUpscaleGraph = ({ image_name, state }: Arg): Graph => {
     type: 'esrgan',
     image: { image_name },
     model_name: esrganModelName,
-    is_intermediate: getIsIntermediate(state),
+    is_intermediate: false,
     board: getBoardField(state),
   };
 

invokeai/frontend/web/src/features/nodes/util/graph/buildGenerationTabGraph.ts

@@ -232,7 +232,7 @@ export const buildGenerationTabGraph = async (state: RootState): Promise<NonNull
     LATENTS_TO_IMAGE
   );
 
-  addInitialImageToLinearGraph(state, graph, DENOISE_LATENTS);
+  const didAddInitialImage = addInitialImageToLinearGraph(state, graph, DENOISE_LATENTS);
 
   // Add Seamless To Graph
   if (seamlessXAxis || seamlessYAxis) {
@@ -249,7 +249,7 @@ export const buildGenerationTabGraph = async (state: RootState): Promise<NonNull
   await addControlLayersToGraph(state, graph, DENOISE_LATENTS);
 
   // High resolution fix.
-  if (state.hrf.hrfEnabled) {
+  if (state.hrf.hrfEnabled && !didAddInitialImage) {
     addHrfToGraph(state, graph);
   }
 

invokeai/frontend/web/src/features/system/components/HotkeysModal/useHotkeyData.ts

@@ -141,14 +141,9 @@ export const useHotkeyData = (): HotkeyGroup[] => {
           hotkeys: [['Arrow Right']],
         },
         {
-          title: t('hotkeys.openImageViewer.title'),
+          title: t('hotkeys.toggleViewer.title'),
-          desc: t('hotkeys.openImageViewer.desc'),
+          desc: t('hotkeys.toggleViewer.desc'),
-          hotkeys: [['I']],
+          hotkeys: [['Z']],
-        },
-        {
-          title: t('hotkeys.backToEditor.title'),
-          desc: t('hotkeys.backToEditor.desc'),
-          hotkeys: [['Esc']],
         },
       ],
     }),

invokeai/frontend/web/src/features/ui/components/InvokeTabs.tsx

@@ -4,6 +4,7 @@ import { createMemoizedSelector } from 'app/store/createMemoizedSelector';
 import { $customNavComponent } from 'app/store/nanostores/customNavComponent';
 import { useAppDispatch, useAppSelector } from 'app/store/storeHooks';
 import ImageGalleryContent from 'features/gallery/components/ImageGalleryContent';
+import { ToggleFloatingImageViewerButton } from 'features/gallery/components/ImageViewer/FloatingImageViewer';
 import { ImageViewer } from 'features/gallery/components/ImageViewer/ImageViewer';
 import NodeEditorPanelGroup from 'features/nodes/components/sidePanel/NodeEditorPanelGroup';
 import InvokeAILogoComponent from 'features/system/components/InvokeAILogoComponent';
@@ -223,6 +224,7 @@ const InvokeTabs = () => {
         </TabList>
         <Spacer />
         <StatusIndicator />
+        <ToggleFloatingImageViewerButton />
         {customNavComponent ? customNavComponent : <SettingsMenu />}
       </Flex>
       <PanelGroup
@@ -254,11 +256,11 @@ const InvokeTabs = () => {
             />
           </>
         )}
-        <Panel style={{ position: 'relative' }} id="main-panel" order={1} minSize={20}>
+        <Panel id="main-panel" order={1} minSize={20}>
-          <TabPanels w="full" h="full">
+          <TabPanels w="full" h="full" position="relative">
             {tabPanels}
-          </TabPanels>
             <ImageViewer />
+          </TabPanels>
         </Panel>
         {shouldShowGalleryPanel && (
           <>

invokeai/frontend/web/src/features/ui/components/ParametersPanelTextToImage.tsx

@@ -24,27 +24,16 @@ const overlayScrollbarsStyles: CSSProperties = {
   width: '100%',
 };
 
-const unselectedStyles: ChakraProps['sx'] = {
+const baseStyles: ChakraProps['sx'] = {
-  bg: 'none',
-  color: 'base.300',
   fontWeight: 'semibold',
   fontSize: 'sm',
-  w: '50%',
+  color: 'base.300',
-  borderWidth: 1,
-  borderRadius: 'base',
 };
 
 const selectedStyles: ChakraProps['sx'] = {
+  borderColor: 'base.800',
+  borderBottomColor: 'base.900',
   color: 'invokeBlue.300',
-  borderColor: 'invokeBlueAlpha.400',
-  _hover: {
-    color: 'invokeBlue.200',
-  },
-};
-
-const hoverStyles: ChakraProps['sx'] = {
-  bg: 'base.850',
-  color: 'base.100',
 };
 
 const ParametersPanelTextToImage = () => {
@@ -61,12 +50,12 @@ const ParametersPanelTextToImage = () => {
           <OverlayScrollbarsComponent defer style={overlayScrollbarsStyles} options={overlayScrollbarsParams.options}>
             <Flex gap={2} flexDirection="column" h="full" w="full">
               {isSDXL ? <SDXLPrompts /> : <Prompts />}
-              <Tabs variant="unstyled" display="flex" flexDir="column" w="full" h="full" gap={2}>
+              <Tabs variant="enclosed" display="flex" flexDir="column" w="full" h="full" gap={2}>
-                <TabList gap={2}>
+                <TabList gap={2} fontSize="sm" borderColor="base.800">
-                  <Tab _hover={hoverStyles} _selected={selectedStyles} sx={unselectedStyles}>
+                  <Tab sx={baseStyles} _selected={selectedStyles}>
                     {t('common.settingsLabel')}
                   </Tab>
-                  <Tab _hover={hoverStyles} _selected={selectedStyles} sx={unselectedStyles}>
+                  <Tab sx={baseStyles} _selected={selectedStyles}>
                     {controlLayersTitle}
                   </Tab>
                 </TabList>