mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
3.1 Documentation Updates (#4318)
* Updating Nodes documentation * Restructured nodes docs * Comfy to Invoke Overview * Corrections to Comfy -> Invoke Mappings * Adding GA4 to docs * Hiding CLI status * Node doc updates * File path updates * Updates based on lstein's feedback * Fix broken links * Fix broken links * Update comfy to invoke nodes list * Updated prompts documenation * Fix formatting
This commit is contained in:
Millun Atluri
GitHub
parent
44b6adfb9f
commit
4ee65d179c
@ -43,7 +43,7 @@ Web Interface, interactive Command Line Interface, and also serves as
|
||||
the foundation for multiple commercial products.
|
||||
|
||||
**Quick links**: [[How to
|
||||
Install](https://invoke-ai.github.io/InvokeAI/#installation)] [<a
|
||||
Install](https://invoke-ai.github.io/InvokeAI/installation/INSTALLATION/)] [<a
|
||||
href="https://discord.gg/ZmtBAhwWhy">Discord Server</a>] [<a
|
||||
href="https://invoke-ai.github.io/InvokeAI/">Documentation and
|
||||
Tutorials</a>] [<a
|
||||
@ -81,7 +81,7 @@ Table of Contents 📝
|
||||
## Quick Start
|
||||
|
||||
For full installation and upgrade instructions, please see:
|
||||
[InvokeAI Installation Overview](https://invoke-ai.github.io/InvokeAI/installation/)
|
||||
[InvokeAI Installation Overview](https://invoke-ai.github.io/InvokeAI/installation/INSTALLATION/)
|
||||
|
||||
If upgrading from version 2.3, please read [Migrating a 2.3 root
|
||||
directory to 3.0](#migrating-to-3) first.
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 335 KiB After Width: | Height: | Size: 319 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 179 KiB After Width: | Height: | Size: 197 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 501 KiB After Width: | Height: | Size: 421 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 473 KiB After Width: | Height: | Size: 585 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 557 KiB After Width: | Height: | Size: 598 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 340 KiB After Width: | Height: | Size: 438 KiB |
@ -14,11 +14,14 @@ To join, just raise your hand on the InvokeAI Discord server (#dev-chat) or the
|
||||
#### Development
|
||||
If you’d like to help with development, please see our [development guide](contribution_guides/development.md). If you’re unfamiliar with contributing to open source projects, there is a tutorial contained within the development guide.
|
||||
|
||||
#### Nodes
|
||||
If you’d like to help with development, please see our [nodes contribution guide](/nodes/contributingNodes). If you’re unfamiliar with contributing to open source projects, there is a tutorial contained within the development guide.
|
||||
|
||||
#### Documentation
|
||||
If you’d like to help with documentation, please see our [documentation guide](contribution_guides/documenation.md).
|
||||
If you’d like to help with documentation, please see our [documentation guide](contribution_guides/documentation.md).
|
||||
|
||||
#### Translation
|
||||
If you'd like to help with translation, please see our [translation guide](docs/contributing/.contribution_guides/translation.md).
|
||||
If you'd like to help with translation, please see our [translation guide](contribution_guides/translation.md).
|
||||
|
||||
#### Tutorials
|
||||
Please reach out to @imic or @hipsterusername on [Discord](https://discord.gg/ZmtBAhwWhy) to help create tutorials for InvokeAI.
|
||||
|
||||
@ -270,9 +270,12 @@ new Invocation ready to be used.
|
||||
|
||||
|
||||
|
||||
# Advanced
|
||||
## Contributing Nodes
|
||||
Once you've created a Node, the next step is to share it with the community! The best way to do this is to submit a Pull Request to add the Node to the [Community Nodes](nodes/communityNodes) list. If you're not sure how to do that, take a look a at our [contributing nodes overview](contributingNodes).
|
||||
|
||||
## Custom Input Fields
|
||||
## Advanced
|
||||
|
||||
### Custom Input Fields
|
||||
|
||||
Now that you know how to create your own Invocations, let us dive into slightly
|
||||
more advanced topics.
|
||||
@ -352,7 +355,7 @@ input field.
|
||||
We will discuss the `Config` class in extra detail later in this guide and how
|
||||
you can use it to make your Invocations more robust.
|
||||
|
||||
## Custom Output Types
|
||||
### Custom Output Types
|
||||
|
||||
Like with custom inputs, sometimes you might find yourself needing custom
|
||||
outputs that InvokeAI does not provide. We can easily set one up.
|
||||
@ -396,7 +399,7 @@ All set. We now have an output type that requires what we need to create a
|
||||
blank_image. And if you noticed it, we even used the `Config` class to ensure
|
||||
the fields are required.
|
||||
|
||||
## Custom Configuration
|
||||
### Custom Configuration
|
||||
|
||||
As you might have noticed when making inputs and outputs, we used a class called
|
||||
`Config` from _pydantic_ to further customize them. Because our inputs and
|
||||
@ -492,7 +495,7 @@ later time.
|
||||
|
||||
# **[TODO]**
|
||||
|
||||
## Custom Components For Frontend
|
||||
### Custom Components For Frontend
|
||||
|
||||
Every backend input type should have a corresponding frontend component so the
|
||||
UI knows what to render when you use a particular field type.
|
||||
@ -513,7 +516,7 @@ now.
|
||||
|
||||
---
|
||||
|
||||
# OLD -- TO BE DELETED OR MOVED LATER
|
||||
<!-- # OLD -- TO BE DELETED OR MOVED LATER
|
||||
|
||||
---
|
||||
|
||||
@ -787,4 +790,5 @@ With the customization in place, the schema will now show these properties as
|
||||
required, obviating the need for extensive null checks in client code.
|
||||
|
||||
See this `pydantic` issue for discussion on this solution:
|
||||
<https://github.com/pydantic/pydantic/discussions/4577>
|
||||
<https://github.com/pydantic/pydantic/discussions/4577> -->
|
||||
|
||||
|
||||
@ -1,208 +0,0 @@
|
||||
# Nodes Editor (Experimental)
|
||||
|
||||
🚨
|
||||
*The node editor is experimental. We've made it accessible because we use it to develop the application, but we have not addressed the many known rough edges. It's very easy to shoot yourself in the foot, and we cannot offer support for it until it sees full release (ETA v3.1). Everything is subject to change without warning.*
|
||||
🚨
|
||||
|
||||
The nodes editor is a blank canvas allowing for the use of individual functions and image transformations to control the image generation workflow. The node processing flow is usually done from left (inputs) to right (outputs), though linearity can become abstracted the more complex the node graph becomes. Nodes inputs and outputs are connected by dragging connectors from node to node.
|
||||
|
||||
To better understand how nodes are used, think of how an electric power bar works. It takes in one input (electricity from a wall outlet) and passes it to multiple devices through multiple outputs. Similarly, a node could have multiple inputs and outputs functioning at the same (or different) time, but all node outputs pass information onward like a power bar passes electricity. Not all outputs are compatible with all inputs, however - Each node has different constraints on how it is expecting to input/output information. In general, node outputs are colour-coded to match compatible inputs of other nodes.
|
||||
|
||||
## Anatomy of a Node
|
||||
|
||||
Individual nodes are made up of the following:
|
||||
|
||||
- Inputs: Edge points on the left side of the node window where you connect outputs from other nodes.
|
||||
- Outputs: Edge points on the right side of the node window where you connect to inputs on other nodes.
|
||||
- Options: Various options which are either manually configured, or overridden by connecting an output from another node to the input.
|
||||
|
||||
## Diffusion Overview
|
||||
|
||||
Taking the time to understand the diffusion process will help you to understand how to set up your nodes in the nodes editor.
|
||||
|
||||
There are two main spaces Stable Diffusion works in: image space and latent space.
|
||||
|
||||
Image space represents images in pixel form that you look at. Latent space represents compressed inputs. It’s in latent space that Stable Diffusion processes images. A VAE (Variational Auto Encoder) is responsible for compressing and encoding inputs into latent space, as well as decoding outputs back into image space.
|
||||
|
||||
When you generate an image using text-to-image, multiple steps occur in latent space:
|
||||
1. Random noise is generated at the chosen height and width. The noise’s characteristics are dictated by the chosen (or not chosen) seed. This noise tensor is passed into latent space. We’ll call this noise A.
|
||||
1. Using a model’s U-Net, a noise predictor examines noise A, and the words tokenized by CLIP from your prompt (conditioning). It generates its own noise tensor to predict what the final image might look like in latent space. We’ll call this noise B.
|
||||
1. Noise B is subtracted from noise A in an attempt to create a final latent image indicative of the inputs. This step is repeated for the number of sampler steps chosen.
|
||||
1. The VAE decodes the final latent image from latent space into image space.
|
||||
|
||||
image-to-image is a similar process, with only step 1 being different:
|
||||
1. The input image is decoded from image space into latent space by the VAE. Noise is then added to the input latent image. Denoising Strength dictates how much noise is added, 0 being none, and 1 being all-encompassing. We’ll call this noise A. The process is then the same as steps 2-4 in the text-to-image explanation above.
|
||||
|
||||
Furthermore, a model provides the CLIP prompt tokenizer, the VAE, and a U-Net (where noise prediction occurs given a prompt and initial noise tensor).
|
||||
|
||||
A noise scheduler (eg. DPM++ 2M Karras) schedules the subtraction of noise from the latent image across the sampler steps chosen (step 3 above). Less noise is usually subtracted at higher sampler steps.
|
||||
|
||||
## Node Types (Base Nodes)
|
||||
|
||||
| Node <img width=160 align="right"> | Function |
|
||||
| ---------------------------------- | --------------------------------------------------------------------------------------|
|
||||
| Add | Adds two numbers |
|
||||
| CannyImageProcessor | Canny edge detection for ControlNet |
|
||||
| ClipSkip | Skip layers in clip text_encoder model |
|
||||
| Collect | Collects values into a collection |
|
||||
| Prompt (Compel) | Parse prompt using compel package to conditioning |
|
||||
| ContentShuffleImageProcessor | Applies content shuffle processing to image |
|
||||
| ControlNet | Collects ControlNet info to pass to other nodes |
|
||||
| CvInpaint | Simple inpaint using opencv |
|
||||
| Divide | Divides two numbers |
|
||||
| DynamicPrompt | Parses a prompt using adieyal/dynamic prompt's random or combinatorial generator |
|
||||
| FloatLinearRange | Creates a range |
|
||||
| HedImageProcessor | Applies HED edge detection to image |
|
||||
| ImageBlur | Blurs an image |
|
||||
| ImageChannel | Gets a channel from an image |
|
||||
| ImageCollection | Load a collection of images and provide it as output |
|
||||
| ImageConvert | Converts an image to a different mode |
|
||||
| ImageCrop | Crops an image to a specified box. The box can be outside of the image. |
|
||||
| ImageInverseLerp | Inverse linear interpolation of all pixels of an image |
|
||||
| ImageLerp | Linear interpolation of all pixels of an image |
|
||||
| ImageMultiply | Multiplies two images together using `PIL.ImageChops.Multiply()` |
|
||||
| ImageNSFWBlurInvocation | Detects and blurs images that may contain sexually explicit content |
|
||||
| ImagePaste | Pastes an image into another image |
|
||||
| ImageProcessor | Base class for invocations that reprocess images for ControlNet |
|
||||
| ImageResize | Resizes an image to specific dimensions |
|
||||
| ImageScale | Scales an image by a factor |
|
||||
| ImageToLatents | Scales latents by a given factor |
|
||||
| ImageWatermarkInvocation | Adds an invisible watermark to images |
|
||||
| InfillColor | Infills transparent areas of an image with a solid color |
|
||||
| InfillPatchMatch | Infills transparent areas of an image using the PatchMatch algorithm |
|
||||
| InfillTile | Infills transparent areas of an image with tiles of the image |
|
||||
| Inpaint | Generates an image using inpaint |
|
||||
| Iterate | Iterates over a list of items |
|
||||
| LatentsToImage | Generates an image from latents |
|
||||
| LatentsToLatents | Generates latents using latents as base image |
|
||||
| LeresImageProcessor | Applies leres processing to image |
|
||||
| LineartAnimeImageProcessor | Applies line art anime processing to image |
|
||||
| LineartImageProcessor | Applies line art processing to image |
|
||||
| LoadImage | Load an image and provide it as output |
|
||||
| Lora Loader | Apply selected lora to unet and text_encoder |
|
||||
| Model Loader | Loads a main model, outputting its submodels |
|
||||
| MaskFromAlpha | Extracts the alpha channel of an image as a mask |
|
||||
| MediapipeFaceProcessor | Applies mediapipe face processing to image |
|
||||
| MidasDepthImageProcessor | Applies Midas depth processing to image |
|
||||
| MlsdImageProcessor | Applied MLSD processing to image |
|
||||
| Multiply | Multiplies two numbers |
|
||||
| Noise | Generates latent noise |
|
||||
| NormalbaeImageProcessor | Applies NormalBAE processing to image |
|
||||
| OpenposeImageProcessor | Applies Openpose processing to image |
|
||||
| ParamFloat | A float parameter |
|
||||
| ParamInt | An integer parameter |
|
||||
| PidiImageProcessor | Applies PIDI processing to an image |
|
||||
| Progress Image | Displays the progress image in the Node Editor |
|
||||
| RandomInit | Outputs a single random integer |
|
||||
| RandomRange | Creates a collection of random numbers |
|
||||
| Range | Creates a range of numbers from start to stop with step |
|
||||
| RangeOfSize | Creates a range from start to start + size with step |
|
||||
| ResizeLatents | Resizes latents to explicit width/height (in pixels). Provided dimensions are floor-divided by 8. |
|
||||
| RestoreFace | Restores faces in the image |
|
||||
| ScaleLatents | Scales latents by a given factor |
|
||||
| SegmentAnythingProcessor | Applies segment anything processing to image |
|
||||
| ShowImage | Displays a provided image, and passes it forward in the pipeline |
|
||||
| StepParamEasing | Experimental per-step parameter for easing for denoising steps |
|
||||
| Subtract | Subtracts two numbers |
|
||||
| TextToLatents | Generates latents from conditionings |
|
||||
| TileResampleProcessor | Bass class for invocations that preprocess images for ControlNet |
|
||||
| Upscale | Upscales an image |
|
||||
| VAE Loader | Loads a VAE model, outputting a VaeLoaderOutput |
|
||||
| ZoeDepthImageProcessor | Applies Zoe depth processing to image |
|
||||
|
||||
## Node Grouping Concepts
|
||||
|
||||
There are several node grouping concepts that can be examined with a narrow focus. These (and other) groupings can be pieced together to make up functional graph setups, and are important to understanding how groups of nodes work together as part of a whole. Note that the screenshots below aren't examples of complete functioning node graphs (see Examples).
|
||||
|
||||
### Noise
|
||||
|
||||
As described, an initial noise tensor is necessary for the latent diffusion process. As a result, all non-image *ToLatents nodes require a noise node input.
|
||||
|
||||
|
||||
|
||||
### Conditioning
|
||||
|
||||
As described, conditioning is necessary for the latent diffusion process, whether empty or not. As a result, all non-image *ToLatents nodes require positive and negative conditioning inputs. Conditioning is reliant on a CLIP tokenizer provided by the Model Loader node.
|
||||
|
||||
|
||||
|
||||
### Image Space & VAE
|
||||
|
||||
The ImageToLatents node doesn't require a noise node input, but requires a VAE input to convert the image from image space into latent space. In reverse, the LatentsToImage node requires a VAE input to convert from latent space back into image space.
|
||||
|
||||
|
||||
|
||||
### Defined & Random Seeds
|
||||
|
||||
It is common to want to use both the same seed (for continuity) and random seeds (for variance). To define a seed, simply enter it into the 'Seed' field on a noise node. Conversely, the RandomInt node generates a random integer between 'Low' and 'High', and can be used as input to the 'Seed' edge point on a noise node to randomize your seed.
|
||||
|
||||
|
||||
|
||||
### Control
|
||||
|
||||
Control means to guide the diffusion process to adhere to a defined input or structure. Control can be provided as input to non-image *ToLatents nodes from ControlNet nodes. ControlNet nodes usually require an image processor which converts an input image for use with ControlNet.
|
||||
|
||||
|
||||
|
||||
### LoRA
|
||||
|
||||
The Lora Loader node lets you load a LoRA (say that ten times fast) and pass it as output to both the Prompt (Compel) and non-image *ToLatents nodes. A model's CLIP tokenizer is passed through the LoRA into Prompt (Compel), where it affects conditioning. A model's U-Net is also passed through the LoRA into a non-image *ToLatents node, where it affects noise prediction.
|
||||
|
||||
|
||||
|
||||
### Scaling
|
||||
|
||||
Use the ImageScale, ScaleLatents, and Upscale nodes to upscale images and/or latent images. The chosen method differs across contexts. However, be aware that latents are already noisy and compressed at their original resolution; scaling an image could produce more detailed results.
|
||||
|
||||
|
||||
|
||||
### Iteration + Multiple Images as Input
|
||||
|
||||
Iteration is a common concept in any processing, and means to repeat a process with given input. In nodes, you're able to use the Iterate node to iterate through collections usually gathered by the Collect node. The Iterate node has many potential uses, from processing a collection of images one after another, to varying seeds across multiple image generations and more. This screenshot demonstrates how to collect several images and pass them out one at a time.
|
||||
|
||||
|
||||
|
||||
### Multiple Image Generation + Random Seeds
|
||||
|
||||
Multiple image generation in the node editor is done using the RandomRange node. In this case, the 'Size' field represents the number of images to generate. As RandomRange produces a collection of integers, we need to add the Iterate node to iterate through the collection.
|
||||
|
||||
To control seeds across generations takes some care. The first row in the screenshot will generate multiple images with different seeds, but using the same RandomRange parameters across invocations will result in the same group of random seeds being used across the images, producing repeatable results. In the second row, adding the RandomInt node as input to RandomRange's 'Seed' edge point will ensure that seeds are varied across all images across invocations, producing varied results.
|
||||
|
||||
|
||||
|
||||
## Examples
|
||||
|
||||
With our knowledge of node grouping and the diffusion process, let’s break down some basic graphs in the nodes editor. Note that a node's options can be overridden by inputs from other nodes. These examples aren't strict rules to follow and only demonstrate some basic configurations.
|
||||
|
||||
### Basic text-to-image Node Graph
|
||||
|
||||
|
||||
|
||||
- Model Loader: A necessity to generating images (as we’ve read above). We choose our model from the dropdown. It outputs a U-Net, CLIP tokenizer, and VAE.
|
||||
- Prompt (Compel): Another necessity. Two prompt nodes are created. One will output positive conditioning (what you want, ‘dog’), one will output negative (what you don’t want, ‘cat’). They both input the CLIP tokenizer that the Model Loader node outputs.
|
||||
- Noise: Consider this noise A from step one of the text-to-image explanation above. Choose a seed number, width, and height.
|
||||
- TextToLatents: This node takes many inputs for converting and processing text & noise from image space into latent space, hence the name TextTo**Latents**. In this setup, it inputs positive and negative conditioning from the prompt nodes for processing (step 2 above). It inputs noise from the noise node for processing (steps 2 & 3 above). Lastly, it inputs a U-Net from the Model Loader node for processing (step 2 above). It outputs latents for use in the next LatentsToImage node. Choose number of sampler steps, CFG scale, and scheduler.
|
||||
- LatentsToImage: This node takes in processed latents from the TextToLatents node, and the model’s VAE from the Model Loader node which is responsible for decoding latents back into the image space, hence the name LatentsTo**Image**. This node is the last stop, and once the image is decoded, it is saved to the gallery.
|
||||
|
||||
### Basic image-to-image Node Graph
|
||||
|
||||
|
||||
|
||||
- Model Loader: Choose a model from the dropdown.
|
||||
- Prompt (Compel): Two prompt nodes. One positive (dog), one negative (dog). Same CLIP inputs from the Model Loader node as before.
|
||||
- ImageToLatents: Upload a source image directly in the node window, via drag'n'drop from the gallery, or passed in as input. The ImageToLatents node inputs the VAE from the Model Loader node to decode the chosen image from image space into latent space, hence the name ImageTo**Latents**. It outputs latents for use in the next LatentsToLatents node. It also outputs the source image's width and height for use in the next Noise node if the final image is to be the same dimensions as the source image.
|
||||
- Noise: A noise tensor is created with the width and height of the source image, and connected to the next LatentsToLatents node. Notice the width and height fields are overridden by the input from the ImageToLatents width and height outputs.
|
||||
- LatentsToLatents: The inputs and options are nearly identical to TextToLatents, except that LatentsToLatents also takes latents as an input. Considering our source image is already converted to latents in the last ImageToLatents node, and text + noise are no longer the only inputs to process, we use the LatentsToLatents node.
|
||||
- LatentsToImage: Like previously, the LatentsToImage node will use the VAE from the Model Loader as input to decode the latents from LatentsToLatents into image space, and save it to the gallery.
|
||||
|
||||
### Basic ControlNet Node Graph
|
||||
|
||||
|
||||
|
||||
- Model Loader
|
||||
- Prompt (Compel)
|
||||
- Noise: Width and height of the CannyImageProcessor ControlNet image is passed in to set the dimensions of the noise passed to TextToLatents.
|
||||
- CannyImageProcessor: The CannyImageProcessor node is used to process the source image being used as a ControlNet. Each ControlNet processor node applies control in different ways, and has some different options to configure. Width and height are passed to noise, as mentioned. The processed ControlNet image is output to the ControlNet node.
|
||||
- ControlNet: Select the type of control model. In this case, canny is chosen as the CannyImageProcessor was used to generate the ControlNet image. Configure the control node options, and pass the control output to TextToLatents.
|
||||
- TextToLatents: Similar to the basic text-to-image example, except ControlNet is passed to the control input edge point.
|
||||
- LatentsToImage
|
||||
@ -4,80 +4,6 @@ title: Prompting-Features
|
||||
|
||||
# :octicons-command-palette-24: Prompting-Features
|
||||
|
||||
## **Negative and Unconditioned Prompts**
|
||||
|
||||
Any words between a pair of square brackets will instruct Stable
|
||||
Diffusion to attempt to ban the concept from the generated image. The
|
||||
same effect is achieved by placing words in the "Negative Prompts"
|
||||
textbox in the Web UI.
|
||||
|
||||
```text
|
||||
this is a test prompt [not really] to make you understand [cool] how this works.
|
||||
```
|
||||
|
||||
In the above statement, the words 'not really cool` will be ignored by Stable
|
||||
Diffusion.
|
||||
|
||||
Here's a prompt that depicts what it does.
|
||||
|
||||
original prompt:
|
||||
|
||||
`#!bash "A fantastical translucent pony made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve"`
|
||||
|
||||
`#!bash parameters: steps=20, dimensions=512x768, CFG=7.5, Scheduler=k_euler_a, seed=1654590180`
|
||||
|
||||
<figure markdown>
|
||||
|
||||
|
||||
|
||||
</figure>
|
||||
|
||||
That image has a woman, so if we want the horse without a rider, we can
|
||||
influence the image not to have a woman by putting [woman] in the prompt, like
|
||||
this:
|
||||
|
||||
`#!bash "A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman]"`
|
||||
(same parameters as above)
|
||||
|
||||
<figure markdown>
|
||||
|
||||
|
||||
|
||||
</figure>
|
||||
|
||||
That's nice - but say we also don't want the image to be quite so blue. We can
|
||||
add "blue" to the list of negative prompts, so it's now [woman blue]:
|
||||
|
||||
`#!bash "A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman blue]"`
|
||||
(same parameters as above)
|
||||
|
||||
<figure markdown>
|
||||
|
||||
|
||||
|
||||
</figure>
|
||||
|
||||
Getting close - but there's no sense in having a saddle when our horse doesn't
|
||||
have a rider, so we'll add one more negative prompt: [woman blue saddle].
|
||||
|
||||
`#!bash "A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman blue saddle]"`
|
||||
(same parameters as above)
|
||||
|
||||
<figure markdown>
|
||||
|
||||
|
||||
|
||||
</figure>
|
||||
|
||||
!!! notes "Notes about this feature:"
|
||||
|
||||
* The only requirement for words to be ignored is that they are in between a pair of square brackets.
|
||||
* You can provide multiple words within the same bracket.
|
||||
* You can provide multiple brackets with multiple words in different places of your prompt. That works just fine.
|
||||
* To improve typical anatomy problems, you can add negative prompts like `[bad anatomy, extra legs, extra arms, extra fingers, poorly drawn hands, poorly drawn feet, disfigured, out of frame, tiling, bad art, deformed, mutated]`.
|
||||
|
||||
---
|
||||
|
||||
## **Prompt Syntax Features**
|
||||
|
||||
The InvokeAI prompting language has the following features:
|
||||
@ -102,9 +28,6 @@ The following syntax is recognised:
|
||||
`a tall thin man (picking (apricots)1.3)1.1`. (`+` is equivalent to 1.1, `++`
|
||||
is pow(1.1,2), `+++` is pow(1.1,3), etc; `-` means 0.9, `--` means pow(0.9,2),
|
||||
etc.)
|
||||
- attention also applies to `[unconditioning]` so
|
||||
`a tall thin man picking apricots [(ladder)0.01]` will _very gently_ nudge SD
|
||||
away from trying to draw the man on a ladder
|
||||
|
||||
You can use this to increase or decrease the amount of something. Starting from
|
||||
this prompt of `a man picking apricots from a tree`, let's see what happens if
|
||||
@ -150,7 +73,7 @@ Or, alternatively, with more man:
|
||||
| ---------------------------------------------- | ---------------------------------------------- | ---------------------------------------------- | ---------------------------------------------- |
|
||||
|  |  |  |  |
|
||||
|
||||
### Blending between prompts
|
||||
### Prompt Blending
|
||||
|
||||
- `("a tall thin man picking apricots", "a tall thin man picking pears").blend(1,1)`
|
||||
- The existing prompt blending using `:<weight>` will continue to be supported -
|
||||
@ -168,6 +91,24 @@ Or, alternatively, with more man:
|
||||
See the section below on "Prompt Blending" for more information about how this
|
||||
works.
|
||||
|
||||
### Prompt Conjunction
|
||||
Join multiple clauses together to create a conjoined prompt. Each clause will be passed to CLIP separately.
|
||||
|
||||
For example, the prompt:
|
||||
|
||||
```bash
|
||||
"A mystical valley surround by towering granite cliffs, watercolor, warm"
|
||||
```
|
||||
|
||||
Can be used with .and():
|
||||
```bash
|
||||
("A mystical valley", "surround by towering granite cliffs", "watercolor", "warm").and()
|
||||
```
|
||||
|
||||
Each will give you different results - try them out and see what you prefer!
|
||||
|
||||
|
||||
|
||||
### Cross-Attention Control ('prompt2prompt')
|
||||
|
||||
Sometimes an image you generate is almost right, and you just want to change one
|
||||
@ -190,7 +131,7 @@ For example, consider the prompt `a cat.swap(dog) playing with a ball in the for
|
||||
|
||||
- For multiple word swaps, use parentheses: `a (fluffy cat).swap(barking dog) playing with a ball in the forest`.
|
||||
- To swap a comma, use quotes: `a ("fluffy, grey cat").swap("big, barking dog") playing with a ball in the forest`.
|
||||
- Supports options `t_start` and `t_end` (each 0-1) loosely corresponding to bloc97's `prompt_edit_tokens_start/_end` but with the math swapped to make it easier to
|
||||
- Supports options `t_start` and `t_end` (each 0-1) loosely corresponding to (bloc97's)[(https://github.com/bloc97/CrossAttentionControl)] `prompt_edit_tokens_start/_end` but with the math swapped to make it easier to
|
||||
intuitively understand. `t_start` and `t_end` are used to control on which steps cross-attention control should run. With the default values `t_start=0` and `t_end=1`, cross-attention control is active on every step of image generation. Other values can be used to turn cross-attention control off for part of the image generation process.
|
||||
- For example, if doing a diffusion with 10 steps for the prompt is `a cat.swap(dog, t_start=0.3, t_end=1.0) playing with a ball in the forest`, the first 3 steps will be run as `a cat playing with a ball in the forest`, while the last 7 steps will run as `a dog playing with a ball in the forest`, but the pixels that represent `dog` will be locked to the pixels that would have represented `cat` if the `cat` prompt had been used instead.
|
||||
- Conversely, for `a cat.swap(dog, t_start=0, t_end=0.7) playing with a ball in the forest`, the first 7 steps will run as `a dog playing with a ball in the forest` with the pixels that represent `dog` locked to the same pixels that would have represented `cat` if the `cat` prompt was being used instead. The final 3 steps will just run `a cat playing with a ball in the forest`.
|
||||
@ -201,7 +142,7 @@ Prompt2prompt `.swap()` is not compatible with xformers, which will be temporari
|
||||
The `prompt2prompt` code is based off
|
||||
[bloc97's colab](https://github.com/bloc97/CrossAttentionControl).
|
||||
|
||||
### Escaping parantheses () and speech marks ""
|
||||
### Escaping parentheses () and speech marks ""
|
||||
|
||||
If the model you are using has parentheses () or speech marks "" as part of its
|
||||
syntax, you will need to "escape" these using a backslash, so that`(my_keyword)`
|
||||
@ -212,23 +153,16 @@ the parentheses as part of the prompt syntax and it will get confused.
|
||||
|
||||
## **Prompt Blending**
|
||||
|
||||
You may blend together different sections of the prompt to explore the AI's
|
||||
You may blend together prompts to explore the AI's
|
||||
latent semantic space and generate interesting (and often surprising!)
|
||||
variations. The syntax is:
|
||||
|
||||
```bash
|
||||
blue sphere:0.25 red cube:0.75 hybrid
|
||||
("prompt #1", "prompt #2").blend(0.25, 0.75)
|
||||
```
|
||||
|
||||
This will tell the sampler to blend 25% of the concept of a blue sphere with 75%
|
||||
of the concept of a red cube. The blend weights can use any combination of
|
||||
integers and floating point numbers, and they do not need to add up to 1.
|
||||
Everything to the left of the `:XX` up to the previous `:XX` is used for
|
||||
merging, so the overall effect is:
|
||||
|
||||
```bash
|
||||
0.25 * "blue sphere" + 0.75 * "white duck" + hybrid
|
||||
```
|
||||
This will tell the sampler to blend 25% of the concept of prompt #1 with 75%
|
||||
of the concept of prompt #2. It is recommended to keep the sum of the weights to around 1.0, but interesting things might happen if you go outside of this range.
|
||||
|
||||
Because you are exploring the "mind" of the AI, the AI's way of mixing two
|
||||
concepts may not match yours, leading to surprising effects. To illustrate, here
|
||||
@ -236,13 +170,14 @@ are three images generated using various combinations of blend weights. As
|
||||
usual, unless you fix the seed, the prompts will give you different results each
|
||||
time you run them.
|
||||
|
||||
<figure markdown>
|
||||
Let's examine how this affects image generation results:
|
||||
|
||||
### "blue sphere, red cube, hybrid"
|
||||
|
||||
</figure>
|
||||
```bash
|
||||
"blue sphere, red cube, hybrid"
|
||||
```
|
||||
|
||||
This example doesn't use melding at all and represents the default way of mixing
|
||||
This example doesn't use blending at all and represents the default way of mixing
|
||||
concepts.
|
||||
|
||||
<figure markdown>
|
||||
@ -251,55 +186,47 @@ concepts.
|
||||
|
||||
</figure>
|
||||
|
||||
It's interesting to see how the AI expressed the concept of "cube" as the four
|
||||
quadrants of the enclosing frame. If you look closely, there is depth there, so
|
||||
the enclosing frame is actually a cube.
|
||||
It's interesting to see how the AI expressed the concept of "cube" within the sphere. If you look closely, there is depth there, so the enclosing frame is actually a cube.
|
||||
|
||||
<figure markdown>
|
||||
|
||||
### "blue sphere:0.25 red cube:0.75 hybrid"
|
||||
```bash
|
||||
("blue sphere", "red cube").blend(0.25, 0.75)
|
||||
```
|
||||
|
||||
|
||||
|
||||
</figure>
|
||||
|
||||
Now that's interesting. We get neither a blue sphere nor a red cube, but a red
|
||||
sphere embedded in a brick wall, which represents a melding of concepts within
|
||||
the AI's "latent space" of semantic representations. Where is Ludwig
|
||||
Wittgenstein when you need him?
|
||||
Now that's interesting. We get an image with a resemblance of a red cube, with a hint of blue shadows which represents a melding of concepts within the AI's "latent space" of semantic representations.
|
||||
|
||||
<figure markdown>
|
||||
|
||||
### "blue sphere:0.75 red cube:0.25 hybrid"
|
||||
```bash
|
||||
("blue sphere", "red cube").blend(0.75, 0.25)
|
||||
```
|
||||
|
||||
|
||||
|
||||
</figure>
|
||||
|
||||
Definitely more blue-spherey. The cube is gone entirely, but it's really cool
|
||||
abstract art.
|
||||
Definitely more blue-spherey.
|
||||
|
||||
<figure markdown>
|
||||
|
||||
### "blue sphere:0.5 red cube:0.5 hybrid"
|
||||
```bash
|
||||
("blue sphere", "red cube").blend(0.5, 0.5)
|
||||
```
|
||||
</figure>
|
||||
|
||||
<figure markdown>
|
||||
|
||||
|
||||
</figure>
|
||||
|
||||
Whoa...! I see blue and red, but no spheres or cubes. Is the word "hybrid"
|
||||
summoning up the concept of some sort of scifi creature? Let's find out.
|
||||
|
||||
<figure markdown>
|
||||
Whoa...! I see blue and red, and if I squint, spheres and cubes.
|
||||
|
||||
### "blue sphere:0.5 red cube:0.5"
|
||||
|
||||
|
||||
|
||||
</figure>
|
||||
|
||||
Indeed, removing the word "hybrid" produces an image that is more like what we'd
|
||||
expect.
|
||||
|
||||
## Dynamic Prompts
|
||||
|
||||
|
||||
@ -30,10 +30,6 @@ image output.
|
||||
### * [Image-to-Image Guide](IMG2IMG.md)
|
||||
Use a seed image to build new creations in the CLI.
|
||||
|
||||
### * [Generating Variations](VARIATIONS.md)
|
||||
Have an image you like and want to generate many more like it? Variations
|
||||
are the ticket.
|
||||
|
||||
## Model Management
|
||||
|
||||
### * [Model Installation](../installation/050_INSTALLING_MODELS.md)
|
||||
|
||||
27
docs/help/diffusion.md
Normal file
27
docs/help/diffusion.md
Normal file
@ -0,0 +1,27 @@
|
||||
Taking the time to understand the diffusion process will help you to understand how to more effectively use InvokeAI.
|
||||
|
||||
There are two main ways Stable Diffusion works - with images, and latents.
|
||||
|
||||
Image space represents images in pixel form that you look at. Latent space represents compressed inputs. It’s in latent space that Stable Diffusion processes images. A VAE (Variational Auto Encoder) is responsible for compressing and encoding inputs into latent space, as well as decoding outputs back into image space.
|
||||
|
||||
To fully understand the diffusion process, we need to understand a few more terms: UNet, CLIP, and conditioning.
|
||||
|
||||
A U-Net is a model trained on a large number of latent images with with known amounts of random noise added. This means that the U-Net can be given a slightly noisy image and it will predict the pattern of noise needed to subtract from the image in order to recover the original.
|
||||
|
||||
CLIP is a model that tokenizes and encodes text into conditioning. This conditioning guides the model during the denoising steps to produce a new image.
|
||||
|
||||
The U-Net and CLIP work together during the image generation process at each denoising step, with the U-Net removing noise in such a way that the result is similar to images in the U-Net’s training set, while CLIP guides the U-Net towards creating images that are most similar to the prompt.
|
||||
|
||||
|
||||
When you generate an image using text-to-image, multiple steps occur in latent space:
|
||||
1. Random noise is generated at the chosen height and width. The noise’s characteristics are dictated by seed. This noise tensor is passed into latent space. We’ll call this noise A.
|
||||
2. Using a model’s U-Net, a noise predictor examines noise A, and the words tokenized by CLIP from your prompt (conditioning). It generates its own noise tensor to predict what the final image might look like in latent space. We’ll call this noise B.
|
||||
3. Noise B is subtracted from noise A in an attempt to create a latent image consistent with the prompt. This step is repeated for the number of sampler steps chosen.
|
||||
4. The VAE decodes the final latent image from latent space into image space.
|
||||
|
||||
Image-to-image is a similar process, with only step 1 being different:
|
||||
1. The input image is encoded from image space into latent space by the VAE. Noise is then added to the input latent image. Denoising Strength dictates how may noise steps are added, and the amount of noise added at each step. A Denoising Strength of 0 means there are 0 steps and no noise added, resulting in an unchanged image, while a Denoising Strength of 1 results in the image being completely replaced with noise and a full set of denoising steps are performance. The process is then the same as steps 2-4 in the text-to-image process.
|
||||
|
||||
Furthermore, a model provides the CLIP prompt tokenizer, the VAE, and a U-Net (where noise prediction occurs given a prompt and initial noise tensor).
|
||||
|
||||
A noise scheduler (eg. DPM++ 2M Karras) schedules the subtraction of noise from the latent image across the sampler steps chosen (step 3 above). Less noise is usually subtracted at higher sampler steps.
|
||||
@ -49,9 +49,9 @@ title: Home
|
||||
[![github stars badge]][github stars link]
|
||||
[![github forks badge]][github forks link]
|
||||
|
||||
[![CI checks on main badge]][ci checks on main link]
|
||||
<!-- [![CI checks on main badge]][ci checks on main link]
|
||||
[![CI checks on dev badge]][ci checks on dev link]
|
||||
<!-- [![latest commit to dev badge]][latest commit to dev link] -->
|
||||
[![latest commit to dev badge]][latest commit to dev link] -->
|
||||
|
||||
[![github open issues badge]][github open issues link]
|
||||
[![github open prs badge]][github open prs link]
|
||||
|
||||
@ -8,9 +8,9 @@ title: Installing Manually
|
||||
|
||||
</figure>
|
||||
|
||||
!!! warning "This is for advanced Users"
|
||||
!!! warning "This is for Advanced Users"
|
||||
|
||||
**python experience is mandatory**
|
||||
**Python experience is mandatory**
|
||||
|
||||
## Introduction
|
||||
|
||||
|
||||
@ -4,9 +4,9 @@ title: Installing with Docker
|
||||
|
||||
# :fontawesome-brands-docker: Docker
|
||||
|
||||
!!! warning "For end users"
|
||||
!!! warning "For most users"
|
||||
|
||||
We highly recommend to Install InvokeAI locally using [these instructions](index.md)
|
||||
We highly recommend to Install InvokeAI locally using [these instructions](INSTALLATION.md)
|
||||
|
||||
!!! tip "For developers"
|
||||
|
||||
|
||||
7
docs/javascripts/tablesort.js
Normal file
7
docs/javascripts/tablesort.js
Normal file
@ -0,0 +1,7 @@
|
||||
document$.subscribe(function() {
|
||||
var tables = document.querySelectorAll("article table:not([class])")
|
||||
tables.forEach(function(table) {
|
||||
new Tablesort(table)
|
||||
})
|
||||
})
|
||||
|
||||
68
docs/nodes/NODES.md
Normal file
68
docs/nodes/NODES.md
Normal file
@ -0,0 +1,68 @@
|
||||
# Using the Node Editor
|
||||
|
||||
The nodes editor is a blank canvas allowing for the use of individual functions and image transformations to control the image generation workflow. Nodes take in inputs on the left side of the node, and return an output on the right side of the node. A node graph is composed of multiple nodes that are connected together to create a workflow. Nodes' inputs and outputs are connected by dragging connectors from node to node. Inputs and outputs are color coded for ease of use.
|
||||
|
||||
To better understand how nodes are used, think of how an electric power bar works. It takes in one input (electricity from a wall outlet) and passes it to multiple devices through multiple outputs. Similarly, a node could have multiple inputs and outputs functioning at the same (or different) time, but all node outputs pass information onward like a power bar passes electricity. Not all outputs are compatible with all inputs, however - Each node has different constraints on how it is expecting to input/output information. In general, node outputs are colour-coded to match compatible inputs of other nodes.
|
||||
|
||||
|
||||
If you're not familiar with Diffusion, take a look at our [Diffusion Overview.](../help/diffusion.md) Understanding how diffusion works will enable you to more easily use the Nodes Editor and build workflows to suit your needs.
|
||||
|
||||
## Important Concepts
|
||||
|
||||
There are several node grouping concepts that can be examined with a narrow focus. These (and other) groupings can be pieced together to make up functional graph setups, and are important to understanding how groups of nodes work together as part of a whole. Note that the screenshots below aren't examples of complete functioning node graphs (see Examples).
|
||||
|
||||
### Noise
|
||||
|
||||
An initial noise tensor is necessary for the latent diffusion process. As a result, the Denoising node requires a noise node input.
|
||||
|
||||
|
||||
|
||||
### Text Prompt Conditioning
|
||||
|
||||
Conditioning is necessary for the latent diffusion process, whether empty or not. As a result, the Denoising node requires positive and negative conditioning inputs. Conditioning is reliant on a CLIP text encoder provided by the Model Loader node.
|
||||
|
||||
|
||||
|
||||
### Image to Latents & VAE
|
||||
|
||||
The ImageToLatents node takes in a pixel image and a VAE and outputs a latents. The LatentsToImage node does the opposite, taking in a latents and a VAE and outpus a pixel image.
|
||||
|
||||
|
||||
|
||||
### Defined & Random Seeds
|
||||
|
||||
It is common to want to use both the same seed (for continuity) and random seeds (for variety). To define a seed, simply enter it into the 'Seed' field on a noise node. Conversely, the RandomInt node generates a random integer between 'Low' and 'High', and can be used as input to the 'Seed' edge point on a noise node to randomize your seed.
|
||||
|
||||
|
||||
|
||||
### ControlNet
|
||||
|
||||
The ControlNet node outputs a Control, which can be provided as input to non-image *ToLatents nodes. Depending on the type of ControlNet desired, ControlNet nodes usually require an image processor node, such as a Canny Processor or Depth Processor, which prepares an input image for use with ControlNet.
|
||||
|
||||
|
||||
|
||||
### LoRA
|
||||
|
||||
The Lora Loader node lets you load a LoRA and pass it as output.A LoRA provides fine-tunes to the UNet and text encoder weights that augment the base model’s image and text vocabularies.
|
||||
|
||||
|
||||
|
||||
### Scaling
|
||||
|
||||
Use the ImageScale, ScaleLatents, and Upscale nodes to upscale images and/or latent images. Upscaling is the process of enlarging an image and adding more detail. The chosen method differs across contexts. However, be aware that latents are already noisy and compressed at their original resolution; scaling an image could produce more detailed results.
|
||||
|
||||
|
||||
|
||||
### Iteration + Multiple Images as Input
|
||||
|
||||
Iteration is a common concept in any processing, and means to repeat a process with given input. In nodes, you're able to use the Iterate node to iterate through collections usually gathered by the Collect node. The Iterate node has many potential uses, from processing a collection of images one after another, to varying seeds across multiple image generations and more. This screenshot demonstrates how to collect several images and use them in an image generation workflow.
|
||||
|
||||
|
||||
|
||||
### Multiple Image Generation + Random Seeds
|
||||
|
||||
Multiple image generation in the node editor is done using the RandomRange node. In this case, the 'Size' field represents the number of images to generate. As RandomRange produces a collection of integers, we need to add the Iterate node to iterate through the collection.
|
||||
|
||||
To control seeds across generations takes some care. The first row in the screenshot will generate multiple images with different seeds, but using the same RandomRange parameters across invocations will result in the same group of random seeds being used across the images, producing repeatable results. In the second row, adding the RandomInt node as input to RandomRange's 'Seed' edge point will ensure that seeds are varied across all images across invocations, producing varied results.
|
||||
|
||||
|
||||
80
docs/nodes/comfyToInvoke.md
Normal file
80
docs/nodes/comfyToInvoke.md
Normal file
@ -0,0 +1,80 @@
|
||||
# ComfyUI to InvokeAI
|
||||
|
||||
If you're coming to InvokeAI from ComfyUI, welcome! You'll find things are similar but different - the good news is that you already know how things should work, and it's just a matter of wiring them up!
|
||||
|
||||
Some things to note:
|
||||
|
||||
- InvokeAI's nodes tend to be more granular than default nodes in Comfy. This means each node in Invoke will do a specific task and you might need to use multiple nodes to achieve the same result. The added granularity improves the control you have have over your workflows.
|
||||
- InvokeAI's backend and ComfyUI's backend are very different which means Comfy workflows are not able to be imported into InvokeAI. However, we have created a [list of popular workflows](exampleWorkflows.md) for you to get started with Nodes in InvokeAI!
|
||||
|
||||
## Node Equivalents:
|
||||
|
||||
| Comfy UI Category | ComfyUI Node | Invoke Equivalent |
|
||||
|:---------------------------------- |:---------------------------------- | :----------------------------------|
|
||||
| Sampling |KSampler |Denoise Latents|
|
||||
| Sampling |Ksampler Advanced|Denoise Latents |
|
||||
| Loaders |Load Checkpoint | Main Model Loader _or_ SDXL Main Model Loader|
|
||||
| Loaders |Load VAE | VAE Loader |
|
||||
| Loaders |Load Lora | LoRA Loader _or_ SDXL Lora Loader|
|
||||
| Loaders |Load ControlNet Model | ControlNet|
|
||||
| Loaders |Load ControlNet Model (diff) | ControlNet|
|
||||
| Loaders |Load Style Model | Reference Only ControlNet will be coming in a future version of InvokeAI|
|
||||
| Loaders |unCLIPCheckpointLoader | N/A |
|
||||
| Loaders |GLIGENLoader | N/A |
|
||||
| Loaders |Hypernetwork Loader | N/A |
|
||||
| Loaders |Load Upscale Model | Occurs within "Upscale (RealESRGAN)"|
|
||||
|Conditioning |CLIP Text Encode (Prompt) | Compel (Prompt) or SDXL Compel (Prompt) |
|
||||
|Conditioning |CLIP Set Last Layer | CLIP Skip|
|
||||
|Conditioning |Conditioning (Average) | Use the .blend() feature of prompts |
|
||||
|Conditioning |Conditioning (Combine) | N/A |
|
||||
|Conditioning |Conditioning (Concat) | See the Prompt Tools Community Node|
|
||||
|Conditioning |Conditioning (Set Area) | N/A |
|
||||
|Conditioning |Conditioning (Set Mask) | Mask Edge |
|
||||
|Conditioning |CLIP Vision Encode | N/A |
|
||||
|Conditioning |unCLIPConditioning | N/A |
|
||||
|Conditioning |Apply ControlNet | ControlNet |
|
||||
|Conditioning |Apply ControlNet (Advanced) | ControlNet |
|
||||
|Latent |VAE Decode | Latents to Image|
|
||||
|Latent |VAE Encode | Image to Latents |
|
||||
|Latent |Empty Latent Image | Noise |
|
||||
|Latent |Upscale Latent |Resize Latents |
|
||||
|Latent |Upscale Latent By |Scale Latents |
|
||||
|Latent |Latent Composite | Blend Latents |
|
||||
|Latent |LatentCompositeMasked | N/A |
|
||||
|Image |Save Image | Image |
|
||||
|Image |Preview Image |Current |
|
||||
|Image |Load Image | Image|
|
||||
|Image |Empty Image| Blank Image |
|
||||
|Image |Invert Image | Invert Lerp Image |
|
||||
|Image |Batch Images | Link "Image" nodes into an "Image Collection" node |
|
||||
|Image |Pad Image for Outpainting | Outpainting is easily accomplished in the Unified Canvas |
|
||||
|Image |ImageCompositeMasked | Paste Image |
|
||||
|Image | Upscale Image | Resize Image |
|
||||
|Image | Upscale Image By | Upscale Image |
|
||||
|Image | Upscale Image (using Model) | Upscale Image |
|
||||
|Image | ImageBlur | Blur Image |
|
||||
|Image | ImageQuantize | N/A |
|
||||
|Image | ImageSharpen | N/A |
|
||||
|Image | Canny | Canny Processor |
|
||||
|Mask |Load Image (as Mask) | Image |
|
||||
|Mask |Convert Mask to Image | Image|
|
||||
|Mask |Convert Image to Mask | Image |
|
||||
|Mask |SolidMask | N/A |
|
||||
|Mask |InvertMask |Invert Lerp Image |
|
||||
|Mask |CropMask | Crop Image |
|
||||
|Mask |MaskComposite | Combine Mask |
|
||||
|Mask |FeatherMask | Blur Image |
|
||||
|Advanced | Load CLIP | Main Model Loader _or_ SDXL Main Model Loader|
|
||||
|Advanced | UNETLoader | Main Model Loader _or_ SDXL Main Model Loader|
|
||||
|Advanced | DualCLIPLoader | Main Model Loader _or_ SDXL Main Model Loader|
|
||||
|Advanced | Load Checkpoint | Main Model Loader _or_ SDXL Main Model Loader |
|
||||
|Advanced | ConditioningZeroOut | N/A |
|
||||
|Advanced | ConditioningSetTimestepRange | N/A |
|
||||
|Advanced | CLIPTextEncodeSDXLRefiner | Compel (Prompt) or SDXL Compel (Prompt) |
|
||||
|Advanced | CLIPTextEncodeSDXL |Compel (Prompt) or SDXL Compel (Prompt) |
|
||||
|Advanced | ModelMergeSimple | Model Merging is available in the Model Manager |
|
||||
|Advanced | ModelMergeBlocks | Model Merging is available in the Model Manager|
|
||||
|Advanced | CheckpointSave | Model saving is available in the Model Manager|
|
||||
|Advanced | CLIPMergeSimple | N/A |
|
||||
|
||||
|
||||
@ -2,17 +2,13 @@
|
||||
|
||||
These are nodes that have been developed by the community, for the community. If you're not sure what a node is, you can learn more about nodes [here](overview.md).
|
||||
|
||||
If you'd like to submit a node for the community, please refer to the [node creation overview](./overview.md#contributing-nodes).
|
||||
If you'd like to submit a node for the community, please refer to the [node creation overview](contributingNodes.md).
|
||||
|
||||
To download a node, simply download the `.py` node file from the link and add it to the `invokeai/app/invocations/` folder in your Invoke AI install location. Along with the node, an example node graph should be provided to help you get started with the node.
|
||||
To download a node, simply download the `.py` node file from the link and add it to the `invokeai/app/invocations` folder in your Invoke AI install location. Along with the node, an example node graph should be provided to help you get started with the node.
|
||||
|
||||
To use a community node graph, download the the `.json` node graph file and load it into Invoke AI via the **Load Nodes** button on the Node Editor.
|
||||
|
||||
## Disclaimer
|
||||
|
||||
The nodes linked below have been developed and contributed by members of the Invoke AI community. While we strive to ensure the quality and safety of these contributions, we do not guarantee the reliability or security of the nodes. If you have issues or concerns with any of the nodes below, please raise it on GitHub or in the Discord.
|
||||
|
||||
## List of Nodes
|
||||
## Community Nodes
|
||||
|
||||
### FaceTools
|
||||
|
||||
@ -47,7 +43,12 @@ The nodes linked below have been developed and contributed by members of the Inv
|
||||
|
||||
{: style="height:115px;width:240px"}
|
||||
|
||||
|
||||
## Disclaimer
|
||||
|
||||
The nodes linked have been developed and contributed by members of the Invoke AI community. While we strive to ensure the quality and safety of these contributions, we do not guarantee the reliability or security of the nodes. If you have issues or concerns with any of the nodes below, please raise it on GitHub or in the Discord.
|
||||
|
||||
|
||||
## Help
|
||||
If you run into any issues with a node, please post in the [InvokeAI Discord](https://discord.gg/ZmtBAhwWhy).
|
||||
|
||||
|
||||
|
||||
27
docs/nodes/contributingNodes.md
Normal file
27
docs/nodes/contributingNodes.md
Normal file
@ -0,0 +1,27 @@
|
||||
# Contributing Nodes
|
||||
|
||||
To learn about the specifics of creating a new node, please visit our [Node creation documentation](../contributing/INVOCATIONS.md).
|
||||
|
||||
Once you’ve created a node and confirmed that it behaves as expected locally, follow these steps:
|
||||
|
||||
- Make sure the node is contained in a new Python (.py) file
|
||||
- Submit a pull request with a link to your node in GitHub against the `nodes` branch to add the node to the [Community Nodes](Community Nodes) list
|
||||
- Make sure you are following the template below and have provided all relevant details about the node and what it does.
|
||||
- A maintainer will review the pull request and node. If the node is aligned with the direction of the project, you might be asked for permission to include it in the core project.
|
||||
|
||||
### Community Node Template
|
||||
|
||||
```markdown
|
||||
--------------------------------
|
||||
### Super Cool Node Template
|
||||
|
||||
**Description:** This node allows you to do super cool things with InvokeAI.
|
||||
|
||||
**Node Link:** https://github.com/invoke-ai/InvokeAI/fake_node.py
|
||||
|
||||
**Example Node Graph:** https://github.com/invoke-ai/InvokeAI/fake_node_graph.json
|
||||
|
||||
**Output Examples**
|
||||
|
||||
|
||||
```
|
||||
97
docs/nodes/defaultNodes.md
Normal file
97
docs/nodes/defaultNodes.md
Normal file
@ -0,0 +1,97 @@
|
||||
# List of Default Nodes
|
||||
|
||||
The table below contains a list of the default nodes shipped with InvokeAI and their descriptions.
|
||||
|
||||
| Node <img width=160 align="right"> | Function |
|
||||
|: ---------------------------------- | :--------------------------------------------------------------------------------------|
|
||||
|Add Integers | Adds two numbers|
|
||||
|Boolean Primitive Collection | A collection of boolean primitive values|
|
||||
|Boolean Primitive | A boolean primitive value|
|
||||
|Canny Processor | Canny edge detection for ControlNet|
|
||||
|CLIP Skip | Skip layers in clip text_encoder model.|
|
||||
|Collect | Collects values into a collection|
|
||||
|Color Correct | Shifts the colors of a target image to match the reference image, optionally using a mask to only color-correct certain regions of the target image.|
|
||||
|Color Primitive | A color primitive value|
|
||||
|Compel Prompt | Parse prompt using compel package to conditioning.|
|
||||
|Conditioning Primitive Collection | A collection of conditioning tensor primitive values|
|
||||
|Conditioning Primitive | A conditioning tensor primitive value|
|
||||
|Content Shuffle Processor | Applies content shuffle processing to image|
|
||||
|ControlNet | Collects ControlNet info to pass to other nodes|
|
||||
|OpenCV Inpaint | Simple inpaint using opencv.|
|
||||
|Denoise Latents | Denoises noisy latents to decodable images|
|
||||
|Divide Integers | Divides two numbers|
|
||||
|Dynamic Prompt | Parses a prompt using adieyal/dynamicprompts' random or combinatorial generator|
|
||||
|Upscale (RealESRGAN) | Upscales an image using RealESRGAN.|
|
||||
|Float Primitive Collection | A collection of float primitive values|
|
||||
|Float Primitive | A float primitive value|
|
||||
|Float Range | Creates a range|
|
||||
|HED (softedge) Processor | Applies HED edge detection to image|
|
||||
|Blur Image | Blurs an image|
|
||||
|Extract Image Channel | Gets a channel from an image.|
|
||||
|Image Primitive Collection | A collection of image primitive values|
|
||||
|Convert Image Mode | Converts an image to a different mode.|
|
||||
|Crop Image | Crops an image to a specified box. The box can be outside of the image.|
|
||||
|Image Hue Adjustment | Adjusts the Hue of an image.|
|
||||
|Inverse Lerp Image | Inverse linear interpolation of all pixels of an image|
|
||||
|Image Primitive | An image primitive value|
|
||||
|Lerp Image | Linear interpolation of all pixels of an image|
|
||||
|Image Luminosity Adjustment | Adjusts the Luminosity (Value) of an image.|
|
||||
|Multiply Images | Multiplies two images together using `PIL.ImageChops.multiply()`.|
|
||||
|Blur NSFW Image | Add blur to NSFW-flagged images|
|
||||
|Paste Image | Pastes an image into another image.|
|
||||
|ImageProcessor | Base class for invocations that preprocess images for ControlNet|
|
||||
|Resize Image | Resizes an image to specific dimensions|
|
||||
|Image Saturation Adjustment | Adjusts the Saturation of an image.|
|
||||
|Scale Image | Scales an image by a factor|
|
||||
|Image to Latents | Encodes an image into latents.|
|
||||
|Add Invisible Watermark | Add an invisible watermark to an image|
|
||||
|Solid Color Infill | Infills transparent areas of an image with a solid color|
|
||||
|PatchMatch Infill | Infills transparent areas of an image using the PatchMatch algorithm|
|
||||
|Tile Infill | Infills transparent areas of an image with tiles of the image|
|
||||
|Integer Primitive Collection | A collection of integer primitive values|
|
||||
|Integer Primitive | An integer primitive value|
|
||||
|Iterate | Iterates over a list of items|
|
||||
|Latents Primitive Collection | A collection of latents tensor primitive values|
|
||||
|Latents Primitive | A latents tensor primitive value|
|
||||
|Latents to Image | Generates an image from latents.|
|
||||
|Leres (Depth) Processor | Applies leres processing to image|
|
||||
|Lineart Anime Processor | Applies line art anime processing to image|
|
||||
|Lineart Processor | Applies line art processing to image|
|
||||
|LoRA Loader | Apply selected lora to unet and text_encoder.|
|
||||
|Main Model Loader | Loads a main model, outputting its submodels.|
|
||||
|Combine Mask | Combine two masks together by multiplying them using `PIL.ImageChops.multiply()`.|
|
||||
|Mask Edge | Applies an edge mask to an image|
|
||||
|Mask from Alpha | Extracts the alpha channel of an image as a mask.|
|
||||
|Mediapipe Face Processor | Applies mediapipe face processing to image|
|
||||
|Midas (Depth) Processor | Applies Midas depth processing to image|
|
||||
|MLSD Processor | Applies MLSD processing to image|
|
||||
|Multiply Integers | Multiplies two numbers|
|
||||
|Noise | Generates latent noise.|
|
||||
|Normal BAE Processor | Applies NormalBae processing to image|
|
||||
|ONNX Latents to Image | Generates an image from latents.|
|
||||
|ONNX Prompt (Raw) | A node to process inputs and produce outputs. May use dependency injection in __init__ to receive providers.|
|
||||
|ONNX Text to Latents | Generates latents from conditionings.|
|
||||
|ONNX Model Loader | Loads a main model, outputting its submodels.|
|
||||
|Openpose Processor | Applies Openpose processing to image|
|
||||
|PIDI Processor | Applies PIDI processing to image|
|
||||
|Prompts from File | Loads prompts from a text file|
|
||||
|Random Integer | Outputs a single random integer.|
|
||||
|Random Range | Creates a collection of random numbers|
|
||||
|Integer Range | Creates a range of numbers from start to stop with step|
|
||||
|Integer Range of Size | Creates a range from start to start + size with step|
|
||||
|Resize Latents | Resizes latents to explicit width/height (in pixels). Provided dimensions are floor-divided by 8.|
|
||||
|SDXL Compel Prompt | Parse prompt using compel package to conditioning.|
|
||||
|SDXL LoRA Loader | Apply selected lora to unet and text_encoder.|
|
||||
|SDXL Main Model Loader | Loads an sdxl base model, outputting its submodels.|
|
||||
|SDXL Refiner Compel Prompt | Parse prompt using compel package to conditioning.|
|
||||
|SDXL Refiner Model Loader | Loads an sdxl refiner model, outputting its submodels.|
|
||||
|Scale Latents | Scales latents by a given factor.|
|
||||
|Segment Anything Processor | Applies segment anything processing to image|
|
||||
|Show Image | Displays a provided image, and passes it forward in the pipeline.|
|
||||
|Step Param Easing | Experimental per-step parameter easing for denoising steps|
|
||||
|String Primitive Collection | A collection of string primitive values|
|
||||
|String Primitive | A string primitive value|
|
||||
|Subtract Integers | Subtracts two numbers|
|
||||
|Tile Resample Processor | Tile resampler processor|
|
||||
|VAE Loader | Loads a VAE model, outputting a VaeLoaderOutput|
|
||||
|Zoe (Depth) Processor | Applies Zoe depth processing to image|
|
||||
15
docs/nodes/exampleWorkflows.md
Normal file
15
docs/nodes/exampleWorkflows.md
Normal file
@ -0,0 +1,15 @@
|
||||
# Example Workflows
|
||||
|
||||
TODO: Will update once uploading workflows is available.
|
||||
|
||||
## Text2Image
|
||||
|
||||
## Image2Image
|
||||
|
||||
## ControlNet
|
||||
|
||||
## Upscaling
|
||||
|
||||
## Inpainting / Outpainting
|
||||
|
||||
## LoRAs
|
||||
@ -1,42 +1,26 @@
|
||||
# Nodes
|
||||
|
||||
## What are Nodes?
|
||||
An Node is simply a single operation that takes in some inputs and gives
|
||||
out some outputs. We can then chain multiple nodes together to create more
|
||||
An Node is simply a single operation that takes in inputs and returns
|
||||
out outputs. Multiple nodes can be linked together to create more
|
||||
complex functionality. All InvokeAI features are added through nodes.
|
||||
|
||||
This means nodes can be used to easily extend the image generation capabilities of InvokeAI, and allow you build workflows to suit your needs.
|
||||
### Anatomy of a Node
|
||||
|
||||
You can read more about nodes and the node editor [here](../features/NODES.md).
|
||||
Individual nodes are made up of the following:
|
||||
|
||||
- Inputs: Edge points on the left side of the node window where you connect outputs from other nodes.
|
||||
- Outputs: Edge points on the right side of the node window where you connect to inputs on other nodes.
|
||||
- Options: Various options which are either manually configured, or overridden by connecting an output from another node to the input.
|
||||
|
||||
|
||||
## Downloading Nodes
|
||||
To download a new node, visit our list of [Community Nodes](communityNodes.md). These are nodes that have been created by the community, for the community.
|
||||
With nodes, you can can easily extend the image generation capabilities of InvokeAI, and allow you build workflows that suit your needs.
|
||||
|
||||
You can read more about nodes and the node editor [here](../nodes/NODES.md).
|
||||
|
||||
To get started with nodes, take a look at some of our examples for [common workflows](../nodes/exampleWorkflows.md)
|
||||
|
||||
## Downloading New Nodes
|
||||
To download a new node, visit our list of [Community Nodes](../nodes/communityNodes.md). These are nodes that have been created by the community, for the community.
|
||||
|
||||
|
||||
## Contributing Nodes
|
||||
|
||||
To learn about creating a new node, please visit our [Node creation documenation](../contributing/INVOCATIONS.md).
|
||||
|
||||
Once you’ve created a node and confirmed that it behaves as expected locally, follow these steps:
|
||||
* Make sure the node is contained in a new Python (.py) file
|
||||
* Submit a pull request with a link to your node in GitHub against the `nodes` branch to add the node to the [Community Nodes](Community Nodes) list
|
||||
* Make sure you are following the template below and have provided all relevant details about the node and what it does.
|
||||
* A maintainer will review the pull request and node. If the node is aligned with the direction of the project, you might be asked for permission to include it in the core project.
|
||||
|
||||
### Community Node Template
|
||||
|
||||
```markdown
|
||||
--------------------------------
|
||||
### Super Cool Node Template
|
||||
|
||||
**Description:** This node allows you to do super cool things with InvokeAI.
|
||||
|
||||
**Node Link:** https://github.com/invoke-ai/InvokeAI/fake_node.py
|
||||
|
||||
**Example Node Graph:** https://github.com/invoke-ai/InvokeAI/fake_node_graph.json
|
||||
|
||||
**Output Examples**
|
||||
|
||||
|
||||
```
|
||||
|
||||
25
mkdocs.yml
25
mkdocs.yml
@ -98,6 +98,15 @@ plugins:
|
||||
'installation/INSTALLING_MODELS.md': 'installation/050_INSTALLING_MODELS.md'
|
||||
'installation/INSTALL_PATCHMATCH.md': 'installation/060_INSTALL_PATCHMATCH.md'
|
||||
|
||||
extra_javascript:
|
||||
- https://unpkg.com/tablesort@5.3.0/dist/tablesort.min.js
|
||||
- javascripts/tablesort.js
|
||||
|
||||
extra:
|
||||
analytics:
|
||||
provider: google
|
||||
property: G-2X4JR4S4FB
|
||||
|
||||
nav:
|
||||
- Home: 'index.md'
|
||||
- Installation:
|
||||
@ -118,9 +127,14 @@ nav:
|
||||
- Manual Installation on Windows: 'installation/deprecated_documentation/INSTALL_WINDOWS.md'
|
||||
- Installing Invoke with pip: 'installation/deprecated_documentation/INSTALL_PCP.md'
|
||||
- Source Installer: 'installation/deprecated_documentation/INSTALL_SOURCE.md'
|
||||
- Community Nodes:
|
||||
- Nodes:
|
||||
- Community Nodes: 'nodes/communityNodes.md'
|
||||
- Overview: 'nodes/overview.md'
|
||||
- Example Workflows: 'nodes/exampleWorkflows.md'
|
||||
- Nodes Overview: 'nodes/overview.md'
|
||||
- List of Default Nodes: 'nodes/defaultNodes.md'
|
||||
- Node Editor Usage: 'nodes/NODES.md'
|
||||
- ComfyUI to InvokeAI: 'nodes/comfyToInvoke.md'
|
||||
- Contributing Nodes: 'nodes/contributingNodes.md'
|
||||
- Features:
|
||||
- Overview: 'features/index.md'
|
||||
- New to InvokeAI?: 'help/gettingStartedWithAI.md'
|
||||
@ -130,13 +144,12 @@ nav:
|
||||
- Image-to-Image: 'features/IMG2IMG.md'
|
||||
- Controlling Logging: 'features/LOGGING.md'
|
||||
- Model Merging: 'features/MODEL_MERGING.md'
|
||||
- Nodes Editor (Experimental): 'features/NODES.md'
|
||||
- NSFW Checker: 'features/NSFW.md'
|
||||
- Using Nodes : './nodes/overview'
|
||||
- NSFW Checker: 'features/WATERMARK+NSFW.md'
|
||||
- Postprocessing: 'features/POSTPROCESS.md'
|
||||
- Prompting Features: 'features/PROMPTS.md'
|
||||
- Training: 'features/TRAINING.md'
|
||||
- Unified Canvas: 'features/UNIFIED_CANVAS.md'
|
||||
- Variations: 'features/VARIATIONS.md'
|
||||
- InvokeAI Web Server: 'features/WEB.md'
|
||||
- WebUI Hotkeys: "features/WEBUIHOTKEYS.md"
|
||||
- Other: 'features/OTHER.md'
|
||||
@ -148,6 +161,7 @@ nav:
|
||||
- Frontend Documentation: 'contributing/contribution_guides/development_guides/contributingToFrontend.md'
|
||||
- Local Development: 'contributing/LOCAL_DEVELOPMENT.md'
|
||||
- Documentation: 'contributing/contribution_guides/documentation.md'
|
||||
- Nodes: 'contributing/INVOCATIONS.md'
|
||||
- Translation: 'contributing/contribution_guides/translation.md'
|
||||
- Tutorials: 'contributing/contribution_guides/tutorials.md'
|
||||
- Changelog: 'CHANGELOG.md'
|
||||
@ -158,6 +172,7 @@ nav:
|
||||
- Outpainting: 'deprecated/OUTPAINTING.md'
|
||||
- Help:
|
||||
- Getting Started: 'help/gettingStartedWithAI.md'
|
||||
- Diffusion Overview: 'help/diffusion.md'
|
||||
- Sampler Convergence: 'help/SAMPLER_CONVERGENCE.md'
|
||||
- Other:
|
||||
- Contributors: 'other/CONTRIBUTORS.md'
|
||||
|
||||
Loading…
Reference in New Issue
Block a user