Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

3.1 Documentation Updates (#4318)

Browse Source 
* Updating Nodes documentation

* Restructured nodes docs

* Comfy to Invoke Overview

* Corrections to Comfy -> Invoke Mappings

* Adding GA4 to docs

* Hiding CLI status

* Node doc updates

* File path updates

* Updates based on lstein's feedback

* Fix broken links

* Fix broken links

* Update comfy to invoke nodes list

* Updated prompts documenation

* Fix formatting
This commit is contained in:
Millun Atluri
2023-08-25 11:59:46 +10:00
committed by GitHub
parent 44b6adfb9f
commit 4ee65d179c
25 changed files with 435 additions and 392 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
docs/nodes/NODES.md Normal file
View File

@ -0,0 +1,68 @@
# Using the Node Editor
The nodes editor is a blank canvas allowing for the use of individual functions and image transformations to control the image generation workflow. Nodes take in inputs on the left side of the node, and return an output on the right side of the node. A node graph is composed of multiple nodes that are connected together to create a workflow. Nodes' inputs and outputs are connected by dragging connectors from node to node. Inputs and outputs are color coded for ease of use.
To better understand how nodes are used, think of how an electric power bar works. It takes in one input (electricity from a wall outlet) and passes it to multiple devices through multiple outputs. Similarly, a node could have multiple inputs and outputs functioning at the same (or different) time, but all node outputs pass information onward like a power bar passes electricity. Not all outputs are compatible with all inputs, however - Each node has different constraints on how it is expecting to input/output information. In general, node outputs are colour-coded to match compatible inputs of other nodes.
If you're not familiar with Diffusion, take a look at our [Diffusion Overview.](../help/diffusion.md) Understanding how diffusion works will enable you to more easily use the Nodes Editor and build workflows to suit your needs.
## Important Concepts
There are several node grouping concepts that can be examined with a narrow focus. These (and other) groupings can be pieced together to make up functional graph setups, and are important to understanding how groups of nodes work together as part of a whole. Note that the screenshots below aren't examples of complete functioning node graphs (see Examples).
### Noise
An initial noise tensor is necessary for the latent diffusion process. As a result, the Denoising node requires a noise node input.
![groupsnoise](../assets/nodes/groupsnoise.png)
### Text Prompt Conditioning
Conditioning is necessary for the latent diffusion process, whether empty or not. As a result, the Denoising node requires positive and negative conditioning inputs. Conditioning is reliant on a CLIP text encoder provided by the Model Loader node.
![groupsconditioning](../assets/nodes/groupsconditioning.png)
### Image to Latents & VAE
The ImageToLatents node takes in a pixel image and a VAE and outputs a latents. The LatentsToImage node does the opposite, taking in a latents and a VAE and outpus a pixel image.
![groupsimgvae](../assets/nodes/groupsimgvae.png)
### Defined & Random Seeds
It is common to want to use both the same seed (for continuity) and random seeds (for variety). To define a seed, simply enter it into the 'Seed' field on a noise node. Conversely, the RandomInt node generates a random integer between 'Low' and 'High', and can be used as input to the 'Seed' edge point on a noise node to randomize your seed.
![groupsrandseed](../assets/nodes/groupsrandseed.png)
### ControlNet
The ControlNet node outputs a Control, which can be provided as input to non-image *ToLatents nodes. Depending on the type of ControlNet desired, ControlNet nodes usually require an image processor node, such as a Canny Processor or Depth Processor, which prepares an input image for use with ControlNet.
![groupscontrol](../assets/nodes/groupscontrol.png)
### LoRA
The Lora Loader node lets you load a LoRA and pass it as output.A LoRA provides fine-tunes to the UNet and text encoder weights that augment the base model’s image and text vocabularies.
![groupslora](../assets/nodes/groupslora.png)
### Scaling
Use the ImageScale, ScaleLatents, and Upscale nodes to upscale images and/or latent images. Upscaling is the process of enlarging an image and adding more detail. The chosen method differs across contexts. However, be aware that latents are already noisy and compressed at their original resolution; scaling an image could produce more detailed results.
![groupsallscale](../assets/nodes/groupsallscale.png)
### Iteration + Multiple Images as Input
Iteration is a common concept in any processing, and means to repeat a process with given input. In nodes, you're able to use the Iterate node to iterate through collections usually gathered by the Collect node. The Iterate node has many potential uses, from processing a collection of images one after another, to varying seeds across multiple image generations and more. This screenshot demonstrates how to collect several images and use them in an image generation workflow.
![groupsiterate](../assets/nodes/groupsiterate.png)
### Multiple Image Generation + Random Seeds
Multiple image generation in the node editor is done using the RandomRange node. In this case, the 'Size' field represents the number of images to generate. As RandomRange produces a collection of integers, we need to add the Iterate node to iterate through the collection.
To control seeds across generations takes some care. The first row in the screenshot will generate multiple images with different seeds, but using the same RandomRange parameters across invocations will result in the same group of random seeds being used across the images, producing repeatable results. In the second row, adding the RandomInt node as input to RandomRange's 'Seed' edge point will ensure that seeds are varied across all images across invocations, producing varied results.
![groupsmultigenseeding](../assets/nodes/groupsmultigenseeding.png)

80
docs/nodes/comfyToInvoke.md Normal file
View File

@ -0,0 +1,80 @@
# ComfyUI to InvokeAI
If you're coming to InvokeAI from ComfyUI, welcome! You'll find things are similar but different - the good news is that you already know how things should work, and it's just a matter of wiring them up!
Some things to note:
- InvokeAI's nodes tend to be more granular than default nodes in Comfy. This means each node in Invoke will do a specific task and you might need to use multiple nodes to achieve the same result. The added granularity improves the control you have have over your workflows.
- InvokeAI's backend and ComfyUI's backend are very different which means Comfy workflows are not able to be imported into InvokeAI. However, we have created a [list of popular workflows](exampleWorkflows.md) for you to get started with Nodes in InvokeAI!
## Node Equivalents:
| Comfy UI Category | ComfyUI Node | Invoke Equivalent |
|:---------------------------------- |:---------------------------------- | :----------------------------------|
| Sampling |KSampler |Denoise Latents|
| Sampling |Ksampler Advanced|Denoise Latents |
| Loaders |Load Checkpoint | Main Model Loader _or_ SDXL Main Model Loader|
| Loaders |Load VAE | VAE Loader |
| Loaders |Load Lora | LoRA Loader _or_ SDXL Lora Loader|
| Loaders |Load ControlNet Model | ControlNet|
| Loaders |Load ControlNet Model (diff) | ControlNet|
| Loaders |Load Style Model | Reference Only ControlNet will be coming in a future version of InvokeAI|
| Loaders |unCLIPCheckpointLoader | N/A |
| Loaders |GLIGENLoader | N/A |
| Loaders |Hypernetwork Loader | N/A |
| Loaders |Load Upscale Model | Occurs within "Upscale (RealESRGAN)"|
|Conditioning |CLIP Text Encode (Prompt) | Compel (Prompt) or SDXL Compel (Prompt) |
|Conditioning |CLIP Set Last Layer | CLIP Skip|
|Conditioning |Conditioning (Average) | Use the .blend() feature of prompts |
|Conditioning |Conditioning (Combine) | N/A |
|Conditioning |Conditioning (Concat) | See the Prompt Tools Community Node|
|Conditioning |Conditioning (Set Area) | N/A |
|Conditioning |Conditioning (Set Mask) | Mask Edge |
|Conditioning |CLIP Vision Encode | N/A |
|Conditioning |unCLIPConditioning | N/A |
|Conditioning |Apply ControlNet | ControlNet |
|Conditioning |Apply ControlNet (Advanced) | ControlNet |
|Latent |VAE Decode | Latents to Image|
|Latent |VAE Encode | Image to Latents |
|Latent |Empty Latent Image | Noise |
|Latent |Upscale Latent |Resize Latents |
|Latent |Upscale Latent By |Scale Latents |
|Latent |Latent Composite | Blend Latents |
|Latent |LatentCompositeMasked | N/A |
|Image |Save Image | Image |
|Image |Preview Image |Current |
|Image |Load Image | Image|
|Image |Empty Image| Blank Image |
|Image |Invert Image | Invert Lerp Image |
|Image |Batch Images | Link "Image" nodes into an "Image Collection" node |
|Image |Pad Image for Outpainting | Outpainting is easily accomplished in the Unified Canvas |
|Image |ImageCompositeMasked | Paste Image |
|Image | Upscale Image | Resize Image |
|Image | Upscale Image By | Upscale Image |
|Image | Upscale Image (using Model) | Upscale Image |
|Image | ImageBlur | Blur Image |
|Image | ImageQuantize | N/A |
|Image | ImageSharpen | N/A |
|Image | Canny | Canny Processor |
|Mask |Load Image (as Mask) | Image |
|Mask |Convert Mask to Image | Image|
|Mask |Convert Image to Mask | Image |
|Mask |SolidMask | N/A |
|Mask |InvertMask |Invert Lerp Image |
|Mask |CropMask | Crop Image |
|Mask |MaskComposite | Combine Mask |
|Mask |FeatherMask | Blur Image |
|Advanced | Load CLIP | Main Model Loader _or_ SDXL Main Model Loader|
|Advanced | UNETLoader | Main Model Loader _or_ SDXL Main Model Loader|
|Advanced | DualCLIPLoader | Main Model Loader _or_ SDXL Main Model Loader|
|Advanced | Load Checkpoint | Main Model Loader _or_ SDXL Main Model Loader |
|Advanced | ConditioningZeroOut | N/A |
|Advanced | ConditioningSetTimestepRange | N/A |
|Advanced | CLIPTextEncodeSDXLRefiner | Compel (Prompt) or SDXL Compel (Prompt) |
|Advanced | CLIPTextEncodeSDXL |Compel (Prompt) or SDXL Compel (Prompt) |
|Advanced | ModelMergeSimple | Model Merging is available in the Model Manager |
|Advanced | ModelMergeBlocks | Model Merging is available in the Model Manager|
|Advanced | CheckpointSave | Model saving is available in the Model Manager|
|Advanced | CLIPMergeSimple | N/A |

17
docs/nodes/communityNodes.md
View File

@ -2,17 +2,13 @@
These are nodes that have been developed by the community, for the community. If you're not sure what a node is, you can learn more about nodes [here](overview.md).
If you'd like to submit a node for the community, please refer to the [node creation overview](./overview.md#contributing-nodes).
If you'd like to submit a node for the community, please refer to the [node creation overview](contributingNodes.md).
To download a node, simply download the `.py` node file from the link and add it to the `invokeai/app/invocations/` folder in your Invoke AI install location. Along with the node, an example node graph should be provided to help you get started with the node.
To download a node, simply download the `.py` node file from the link and add it to the `invokeai/app/invocations` folder in your Invoke AI install location. Along with the node, an example node graph should be provided to help you get started with the node.
To use a community node graph, download the the `.json` node graph file and load it into Invoke AI via the **Load Nodes** button on the Node Editor.
## Disclaimer
The nodes linked below have been developed and contributed by members of the Invoke AI community. While we strive to ensure the quality and safety of these contributions, we do not guarantee the reliability or security of the nodes. If you have issues or concerns with any of the nodes below, please raise it on GitHub or in the Discord.
## List of Nodes
## Community Nodes
### FaceTools
@ -47,7 +43,12 @@ The nodes linked below have been developed and contributed by members of the Inv
![Example Image](https://invoke-ai.github.io/InvokeAI/assets/invoke_ai_banner.png){: style="height:115px;width:240px"}
## Disclaimer
The nodes linked have been developed and contributed by members of the Invoke AI community. While we strive to ensure the quality and safety of these contributions, we do not guarantee the reliability or security of the nodes. If you have issues or concerns with any of the nodes below, please raise it on GitHub or in the Discord.
## Help
If you run into any issues with a node, please post in the [InvokeAI Discord](https://discord.gg/ZmtBAhwWhy).

27
docs/nodes/contributingNodes.md Normal file
View File

@ -0,0 +1,27 @@
# Contributing Nodes
To learn about the specifics of creating a new node, please visit our [Node creation documentation](../contributing/INVOCATIONS.md).
Once you’ve created a node and confirmed that it behaves as expected locally, follow these steps:
- Make sure the node is contained in a new Python (.py) file
- Submit a pull request with a link to your node in GitHub against the `nodes` branch to add the node to the [Community Nodes](Community Nodes) list
- Make sure you are following the template below and have provided all relevant details about the node and what it does.
- A maintainer will review the pull request and node. If the node is aligned with the direction of the project, you might be asked for permission to include it in the core project.
### Community Node Template
```markdown
--------------------------------
### Super Cool Node Template
**Description:** This node allows you to do super cool things with InvokeAI.
**Node Link:** https://github.com/invoke-ai/InvokeAI/fake_node.py
**Example Node Graph:** https://github.com/invoke-ai/InvokeAI/fake_node_graph.json
**Output Examples**
![InvokeAI](https://invoke-ai.github.io/InvokeAI/assets/invoke_ai_banner.png)
```

97
docs/nodes/defaultNodes.md Normal file
View File

@ -0,0 +1,97 @@
# List of Default Nodes
The table below contains a list of the default nodes shipped with InvokeAI and their descriptions.
| Node <img width=160 align="right"> | Function |
|: ---------------------------------- | :--------------------------------------------------------------------------------------|
|Add Integers | Adds two numbers|
|Boolean Primitive Collection | A collection of boolean primitive values|
|Boolean Primitive | A boolean primitive value|
|Canny Processor | Canny edge detection for ControlNet|
|CLIP Skip | Skip layers in clip text_encoder model.|
|Collect | Collects values into a collection|
|Color Correct | Shifts the colors of a target image to match the reference image, optionally using a mask to only color-correct certain regions of the target image.|
|Color Primitive | A color primitive value|
|Compel Prompt | Parse prompt using compel package to conditioning.|
|Conditioning Primitive Collection | A collection of conditioning tensor primitive values|
|Conditioning Primitive | A conditioning tensor primitive value|
|Content Shuffle Processor | Applies content shuffle processing to image|
|ControlNet | Collects ControlNet info to pass to other nodes|
|OpenCV Inpaint | Simple inpaint using opencv.|
|Denoise Latents | Denoises noisy latents to decodable images|
|Divide Integers | Divides two numbers|
|Dynamic Prompt | Parses a prompt using adieyal/dynamicprompts' random or combinatorial generator|
|Upscale (RealESRGAN) | Upscales an image using RealESRGAN.|
|Float Primitive Collection | A collection of float primitive values|
|Float Primitive | A float primitive value|
|Float Range | Creates a range|
|HED (softedge) Processor | Applies HED edge detection to image|
|Blur Image | Blurs an image|
|Extract Image Channel | Gets a channel from an image.|
|Image Primitive Collection | A collection of image primitive values|
|Convert Image Mode | Converts an image to a different mode.|
|Crop Image | Crops an image to a specified box. The box can be outside of the image.|
|Image Hue Adjustment | Adjusts the Hue of an image.|
|Inverse Lerp Image | Inverse linear interpolation of all pixels of an image|
|Image Primitive | An image primitive value|
|Lerp Image | Linear interpolation of all pixels of an image|
|Image Luminosity Adjustment | Adjusts the Luminosity (Value) of an image.|
|Multiply Images | Multiplies two images together using `PIL.ImageChops.multiply()`.|
|Blur NSFW Image | Add blur to NSFW-flagged images|
|Paste Image | Pastes an image into another image.|
|ImageProcessor | Base class for invocations that preprocess images for ControlNet|
|Resize Image | Resizes an image to specific dimensions|
|Image Saturation Adjustment | Adjusts the Saturation of an image.|
|Scale Image | Scales an image by a factor|
|Image to Latents | Encodes an image into latents.|
|Add Invisible Watermark | Add an invisible watermark to an image|
|Solid Color Infill | Infills transparent areas of an image with a solid color|
|PatchMatch Infill | Infills transparent areas of an image using the PatchMatch algorithm|
|Tile Infill | Infills transparent areas of an image with tiles of the image|
|Integer Primitive Collection | A collection of integer primitive values|
|Integer Primitive | An integer primitive value|
|Iterate | Iterates over a list of items|
|Latents Primitive Collection | A collection of latents tensor primitive values|
|Latents Primitive | A latents tensor primitive value|
|Latents to Image | Generates an image from latents.|
|Leres (Depth) Processor | Applies leres processing to image|
|Lineart Anime Processor | Applies line art anime processing to image|
|Lineart Processor | Applies line art processing to image|
|LoRA Loader | Apply selected lora to unet and text_encoder.|
|Main Model Loader | Loads a main model, outputting its submodels.|
|Combine Mask | Combine two masks together by multiplying them using `PIL.ImageChops.multiply()`.|
|Mask Edge | Applies an edge mask to an image|
|Mask from Alpha | Extracts the alpha channel of an image as a mask.|
|Mediapipe Face Processor | Applies mediapipe face processing to image|
|Midas (Depth) Processor | Applies Midas depth processing to image|
|MLSD Processor | Applies MLSD processing to image|
|Multiply Integers | Multiplies two numbers|
|Noise | Generates latent noise.|
|Normal BAE Processor | Applies NormalBae processing to image|
|ONNX Latents to Image | Generates an image from latents.|
|ONNX Prompt (Raw) | A node to process inputs and produce outputs. May use dependency injection in __init__ to receive providers.|
|ONNX Text to Latents | Generates latents from conditionings.|
|ONNX Model Loader | Loads a main model, outputting its submodels.|
|Openpose Processor | Applies Openpose processing to image|
|PIDI Processor | Applies PIDI processing to image|
|Prompts from File | Loads prompts from a text file|
|Random Integer | Outputs a single random integer.|
|Random Range | Creates a collection of random numbers|
|Integer Range | Creates a range of numbers from start to stop with step|
|Integer Range of Size | Creates a range from start to start + size with step|
|Resize Latents | Resizes latents to explicit width/height (in pixels). Provided dimensions are floor-divided by 8.|
|SDXL Compel Prompt | Parse prompt using compel package to conditioning.|
|SDXL LoRA Loader | Apply selected lora to unet and text_encoder.|
|SDXL Main Model Loader | Loads an sdxl base model, outputting its submodels.|
|SDXL Refiner Compel Prompt | Parse prompt using compel package to conditioning.|
|SDXL Refiner Model Loader | Loads an sdxl refiner model, outputting its submodels.|
|Scale Latents | Scales latents by a given factor.|
|Segment Anything Processor | Applies segment anything processing to image|
|Show Image | Displays a provided image, and passes it forward in the pipeline.|
|Step Param Easing | Experimental per-step parameter easing for denoising steps|
|String Primitive Collection | A collection of string primitive values|
|String Primitive | A string primitive value|
|Subtract Integers | Subtracts two numbers|
|Tile Resample Processor | Tile resampler processor|
|VAE Loader | Loads a VAE model, outputting a VaeLoaderOutput|
|Zoe (Depth) Processor | Applies Zoe depth processing to image|

15
docs/nodes/exampleWorkflows.md Normal file
View File

@ -0,0 +1,15 @@
# Example Workflows
TODO: Will update once uploading workflows is available.
## Text2Image
## Image2Image
## ControlNet
## Upscaling
## Inpainting / Outpainting
## LoRAs

48
docs/nodes/overview.md
View File

@ -1,42 +1,26 @@
# Nodes
## What are Nodes?
An Node is simply a single operation that takes in some inputs and gives
out some outputs. We can then chain multiple nodes together to create more
An Node is simply a single operation that takes in inputs and returns
out outputs. Multiple nodes can be linked together to create more
complex functionality. All InvokeAI features are added through nodes.
This means nodes can be used to easily extend the image generation capabilities of InvokeAI, and allow you build workflows to suit your needs.
### Anatomy of a Node
You can read more about nodes and the node editor [here](../features/NODES.md).
Individual nodes are made up of the following:
- Inputs: Edge points on the left side of the node window where you connect outputs from other nodes.
- Outputs: Edge points on the right side of the node window where you connect to inputs on other nodes.
- Options: Various options which are either manually configured, or overridden by connecting an output from another node to the input.
## Downloading Nodes
To download a new node, visit our list of [Community Nodes](communityNodes.md). These are nodes that have been created by the community, for the community.
With nodes, you can can easily extend the image generation capabilities of InvokeAI, and allow you build workflows that suit your needs.
You can read more about nodes and the node editor [here](../nodes/NODES.md).
To get started with nodes, take a look at some of our examples for [common workflows](../nodes/exampleWorkflows.md)
## Downloading New Nodes
To download a new node, visit our list of [Community Nodes](../nodes/communityNodes.md). These are nodes that have been created by the community, for the community.
## Contributing Nodes
To learn about creating a new node, please visit our [Node creation documenation](../contributing/INVOCATIONS.md).
Once you’ve created a node and confirmed that it behaves as expected locally, follow these steps:
* Make sure the node is contained in a new Python (.py) file
* Submit a pull request with a link to your node in GitHub against the `nodes` branch to add the node to the [Community Nodes](Community Nodes) list
* Make sure you are following the template below and have provided all relevant details about the node and what it does.
* A maintainer will review the pull request and node. If the node is aligned with the direction of the project, you might be asked for permission to include it in the core project.
### Community Node Template
```markdown
--------------------------------
### Super Cool Node Template
**Description:** This node allows you to do super cool things with InvokeAI.
**Node Link:** https://github.com/invoke-ai/InvokeAI/fake_node.py
**Example Node Graph:** https://github.com/invoke-ai/InvokeAI/fake_node_graph.json
**Output Examples**
![InvokeAI](https://invoke-ai.github.io/InvokeAI/assets/invoke_ai_banner.png)
```