1aaad9336f
# Remove node dependencies on generate.py
This is a draft PR in which I am replacing `generate.py` with a cleaner,
more structured interface to the underlying image generation routines.
The basic code pattern to generate an image using the new API is this:
```
from invokeai.backend import ModelManager, Txt2Img, Img2Img
manager = ModelManager('/data/lstein/invokeai-main/configs/models.yaml')
model = manager.get_model('stable-diffusion-1.5')
txt2img = Txt2Img(model)
outputs = txt2img.generate(prompt='banana sushi', steps=12, scheduler='k_euler_a', iterations=5)
# generate() returns an iterator
for next_output in outputs:
print(next_output.image, next_output.seed)
outputs = Img2Img(model).generate(prompt='strawberry` sushi', init_img='./banana_sushi.png')
output = next(outputs)
output.image.save('strawberries.png')
```
### model management
The `ModelManager` handles model selection and initialization. Its
`get_model()` method will return a `dict` with the following keys:
`model`, `model_name`,`hash`, `width`, and `height`, where `model` is
the actual StableDiffusionGeneratorPIpeline. If `get_model()` is called
without a model name, it will return whatever is defined as the default
in `models.yaml`, or the first entry if no default is designated.
### InvokeAIGenerator
The abstract base class `InvokeAIGenerator` is subclassed into into
`Txt2Img`, `Img2Img`, `Inpaint` and `Embiggen`. The constructor for
these classes takes the model dict returned by
`model_manager.get_model()` and optionally an
`InvokeAIGeneratorBasicParams` object, which encapsulates all the
parameters in common among `Txt2Img`, `Img2Img` etc. If you don't
provide the basic params, a reasonable set of defaults will be chosen.
Any of these parameters can be overridden at `generate()` time.
These classes are defined in `invokeai.backend.generator`, but they are
also exported by `invokeai.backend` as shown in the example below.
```
from invokeai.backend import InvokeAIGeneratorBasicParams, Img2Img
params = InvokeAIGeneratorBasicParams(
perlin = 0.15
steps = 30
scheduler = 'k_lms'
)
img2img = Img2Img(model, params)
outputs = img2img.generate(scheduler='k_heun')
```
Note that we were able to override the basic params in the call to
`generate()`
The `generate()` method will returns an iterator over a series of
`InvokeAIGeneratorOutput` objects. These objects contain the PIL image,
the seed, the model name and hash, and attributes for all the parameters
used to generate the object (you can also get these as a dict). The
`iterations` argument controls how many objects will be returned,
defaulting to 1. Pass `None` to get an infinite iterator.
Given the proposed use of `compel` to generate a templated series of
prompts, I thought the API would benefit from a style that lets you loop
over the output results indefinitely. I did consider returning a single
`InvokeAIGeneratorOutput` object in the event that `iterations=1`, but I
think it's dangerous for a method to return different types of result
under different circumstances.
Changing the model is as easy as this:
```
model = manager.get_model('inkspot-2.0`)
txt2img = Txt2Img(model)
```
### Node and legacy support
With respect to `Nodes`, I have written `model_manager_initializer` and
`restoration_services` modules that return `model_manager` and
`restoration` services respectively. The latter is used by the face
reconstruction and upscaling nodes. There is no longer any reference to
`Generate` in the `app` tree.
I have confirmed that `txt2img` and `img2img` work in the nodes client.
I have not tested `embiggen` or `inpaint` yet. pytests are passing, with
some warnings that I don't think are related to what I did.
The legacy WebUI and CLI are still working off `Generate` (which has not
yet been removed from the source tree) and fully functional.
I've finished all the tasks on my TODO list:
- [x] Update the pytests, which are failing due to dangling references
to `generate`
- [x] Rewrite the `reconstruct.py` and `upscale.py` nodes to call
directly into the postprocessing modules rather than going through
`Generate`
- [x] Update the pytests, which are failing due to dangling references
to `generate`
|
||
|---|---|---|
| .dev_scripts | ||
| .github | ||
| binary_installer | ||
| docker | ||
| docs | ||
| installer | ||
| invokeai | ||
| notebooks | ||
| scripts | ||
| static | ||
| tests | ||
| .coveragerc | ||
| .dockerignore | ||
| .editorconfig | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| .prettierrc.yaml | ||
| .pytest.ini | ||
| CODE_OF_CONDUCT.md | ||
| InvokeAI_Statement_of_Values.md | ||
| LICENSE | ||
| LICENSE-ModelWeights.txt | ||
| mkdocs.yml | ||
| pyproject.toml | ||
| README.md | ||
| shell.nix | ||
| Stable_Diffusion_v1_Model_Card.md | ||
InvokeAI: A Stable Diffusion Toolkit
InvokeAI is a leading creative engine built to empower professionals and enthusiasts alike. Generate and create stunning visual media using the latest AI-driven technologies. InvokeAI offers an industry leading Web Interface, interactive Command Line Interface, and also serves as the foundation for multiple commercial products.
Quick links: [How to Install] [Discord Server] [Documentation and Tutorials] [Code and Downloads] [Bug Reports] [Discussion, Ideas & Q&A]
Note: InvokeAI is rapidly evolving. Please use the Issues tab to report bugs and make feature requests. Be sure to use the provided templates. They will help us diagnose issues faster.
Table of Contents
- Quick Start
- Installation
- Hardware Requirements
- Features
- Latest Changes
- Troubleshooting
- Contributing
- Contributors
- Support
- Further Reading
Getting Started with InvokeAI
For full installation and upgrade instructions, please see: InvokeAI Installation Overview
Automatic Installer (suggested for 1st time users)
-
Go to the bottom of the Latest Release Page
-
Download the .zip file for your OS (Windows/macOS/Linux).
-
Unzip the file.
-
If you are on Windows, double-click on the
install.batscript. On macOS, open a Terminal window, drag the fileinstall.shfrom Finder into the Terminal, and press return. On Linux, runinstall.sh. -
You'll be asked to confirm the location of the folder in which to install InvokeAI and its image generation model files. Pick a location with at least 15 GB of free memory. More if you plan on installing lots of models.
-
Wait while the installer does its thing. After installing the software, the installer will launch a script that lets you configure InvokeAI and select a set of starting image generaiton models.
-
Find the folder that InvokeAI was installed into (it is not the same as the unpacked zip file directory!) The default location of this folder (if you didn't change it in step 5) is
~/invokeaion Linux/Mac systems, andC:\Users\YourName\invokeaion Windows. This directory will contain launcher scripts namedinvoke.shandinvoke.bat. -
On Windows systems, double-click on the
invoke.batfile. On macOS, open a Terminal window, draginvoke.shfrom the folder into the Terminal, and press return. On Linux, runinvoke.sh -
Press 2 to open the "browser-based UI", press enter/return, wait a minute or two for Stable Diffusion to start up, then open your browser and go to http://localhost:9090.
-
Type
banana sushiin the box on the top left and clickInvoke
Command-Line Installation (for users familiar with Terminals)
You must have Python 3.9 or 3.10 installed on your machine. Earlier or later versions are not supported.
-
Open a command-line window on your machine. The PowerShell is recommended for Windows.
-
Create a directory to install InvokeAI into. You'll need at least 15 GB of free space:
mkdir invokeai -
Create a virtual environment named
.venvinside this directory and activate it:cd invokeai python -m venv .venv --prompt InvokeAI -
Activate the virtual environment (do it every time you run InvokeAI)
For Linux/Mac users:
source .venv/bin/activateFor Windows users:
.venv\Scripts\activate -
Install the InvokeAI module and its dependencies. Choose the command suited for your platform & GPU.
For Windows/Linux with an NVIDIA GPU:
pip install InvokeAI[xformers] --use-pep517 --extra-index-url https://download.pytorch.org/whl/cu117For Linux with an AMD GPU:
pip install InvokeAI --use-pep517 --extra-index-url https://download.pytorch.org/whl/rocm5.2For Macintoshes, either Intel or M1/M2:
pip install InvokeAI --use-pep517 -
Configure InvokeAI and install a starting set of image generation models (you only need to do this once):
invokeai-configure -
Launch the web server (do it every time you run InvokeAI):
invokeai --web -
Point your browser to http://localhost:9090 to bring up the web interface.
-
Type
banana sushiin the box on the top left and clickInvoke.
Be sure to activate the virtual environment each time before re-launching InvokeAI,
using source .venv/bin/activate or .venv\Scripts\activate.
Detailed Installation Instructions
This fork is supported across Linux, Windows and Macintosh. Linux users can use either an Nvidia-based card (with CUDA support) or an AMD card (using the ROCm driver). For full installation and upgrade instructions, please see: InvokeAI Installation Overview
Hardware Requirements
InvokeAI is supported across Linux, Windows and macOS. Linux users can use either an Nvidia-based card (with CUDA support) or an AMD card (using the ROCm driver).
System
You will need one of the following:
- An NVIDIA-based graphics card with 4 GB or more VRAM memory.
- An Apple computer with an M1 chip.
- An AMD-based graphics card with 4GB or more VRAM memory. (Linux only)
We do not recommend the GTX 1650 or 1660 series video cards. They are unable to run in half-precision mode and do not have sufficient VRAM to render 512x512 images.
Memory
- At least 12 GB Main Memory RAM.
Disk
- At least 12 GB of free disk space for the machine learning model, Python, and all its dependencies.
Features
Feature documentation can be reviewed by navigating to the InvokeAI Documentation page
Web Server & UI
InvokeAI offers a locally hosted Web Server & React Frontend, with an industry leading user experience. The Web-based UI allows for simple and intuitive workflows, and is responsive for use on mobile devices and tablets accessing the web server.
Unified Canvas
The Unified Canvas is a fully integrated canvas implementation with support for all core generation capabilities, in/outpainting, brush tools, and more. This creative tool unlocks the capability for artists to create with AI as a creative collaborator, and can be used to augment AI-generated imagery, sketches, photography, renders, and more.
Advanced Prompt Syntax
InvokeAI's advanced prompt syntax allows for token weighting, cross-attention control, and prompt blending, allowing for fine-tuned tweaking of your invocations and exploration of the latent space.
Command Line Interface
For users utilizing a terminal-based environment, or who want to take advantage of CLI features, InvokeAI offers an extensive and actively supported command-line interface that provides the full suite of generation functionality available in the tool.
Other features
- Support for both ckpt and diffusers models
- SD 2.0, 2.1 support
- Noise Control & Tresholding
- Popular Sampler Support
- Upscaling & Face Restoration Tools
- Embedding Manager & Support
- Model Manager & Support
Coming Soon
- Node-Based Architecture & UI
- And more...
Latest Changes
For our latest changes, view our Release Notes and the CHANGELOG.
Troubleshooting
Please check out our Q&A to get solutions for common installation problems and other issues.
Contributing
Anyone who wishes to contribute to this project, whether documentation, features, bug fixes, code cleanup, testing, or code reviews, is very much encouraged to do so.
To join, just raise your hand on the InvokeAI Discord server (#dev-chat) or the GitHub discussion board.
If you'd like to help with translation, please see our translation guide.
If you are unfamiliar with how to contribute to GitHub projects, here is a Getting Started Guide. A full set of contribution guidelines, along with templates, are in progress. You can make your pull request against the "main" branch.
We hope you enjoy using our software as much as we enjoy creating it, and we hope that some of those of you who are reading this will elect to become part of our community.
Welcome to InvokeAI!
Contributors
This fork is a combined effort of various people from across the world. Check out the list of all these amazing people. We thank them for their time, hard work and effort.
Thanks to Weblate for generously providing translation services to this project.
Support
For support, please use this repository's GitHub Issues tracking service, or join the Discord.
Original portions of the software are Copyright (c) 2023 by respective contributors.