Beta branch containing documentation enhancements, minor bug fix (#3831)
The HF access token was not being saved by the configure script. This fixes that.
BIN
docs/assets/control-panel-2.png
Normal file
After Width: | Height: | Size: 415 KiB |
BIN
docs/assets/installing-models/model-installer-controlnet.png
Normal file
After Width: | Height: | Size: 57 KiB |
BIN
docs/assets/invoke-control-panel-1.png
Normal file
After Width: | Height: | Size: 37 KiB |
Before Width: | Height: | Size: 101 KiB After Width: | Height: | Size: 22 KiB |
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 16 KiB |
Before Width: | Height: | Size: 148 KiB After Width: | Height: | Size: 76 KiB |
Before Width: | Height: | Size: 637 KiB After Width: | Height: | Size: 729 KiB |
BIN
docs/assets/lora-example-0.png
Normal file
After Width: | Height: | Size: 530 KiB |
BIN
docs/assets/lora-example-1.png
Normal file
After Width: | Height: | Size: 24 KiB |
BIN
docs/assets/lora-example-2.png
Normal file
After Width: | Height: | Size: 8.5 KiB |
BIN
docs/assets/lora-example-3.png
Normal file
After Width: | Height: | Size: 409 KiB |
BIN
docs/assets/send-to-icon.png
Normal file
After Width: | Height: | Size: 41 KiB |
BIN
docs/assets/upscaling.png
Normal file
After Width: | Height: | Size: 637 KiB |
@ -1,8 +1,8 @@
|
||||
---
|
||||
title: Concepts
|
||||
title: Textual Inversion Embeddings and LoRAs
|
||||
---
|
||||
|
||||
# :material-library-shelves: The Hugging Face Concepts Library and Importing Textual Inversion files
|
||||
# :material-library-shelves: Textual Inversions and LoRAs
|
||||
|
||||
With the advances in research, many new capabilities are available to customize the knowledge and understanding of novel concepts not originally contained in the base model.
|
||||
|
||||
@ -64,21 +64,25 @@ select the embedding you'd like to use. This UI has type-ahead support, so you c
|
||||
|
||||
## Using LoRAs
|
||||
|
||||
LoRA files are models that customize the output of Stable Diffusion image generation.
|
||||
Larger than embeddings, but much smaller than full models, they augment SD with improved
|
||||
understanding of subjects and artistic styles.
|
||||
LoRA files are models that customize the output of Stable Diffusion
|
||||
image generation. Larger than embeddings, but much smaller than full
|
||||
models, they augment SD with improved understanding of subjects and
|
||||
artistic styles.
|
||||
|
||||
Unlike TI files, LoRAs do not introduce novel vocabulary into the model's known tokens. Instead,
|
||||
LoRAs augment the model's weights that are applied to generate imagery. LoRAs may be supplied
|
||||
with a "trigger" word that they have been explicitly trained on, or may simply apply their
|
||||
effect without being triggered.
|
||||
Unlike TI files, LoRAs do not introduce novel vocabulary into the
|
||||
model's known tokens. Instead, LoRAs augment the model's weights that
|
||||
are applied to generate imagery. LoRAs may be supplied with a
|
||||
"trigger" word that they have been explicitly trained on, or may
|
||||
simply apply their effect without being triggered.
|
||||
|
||||
LoRAs are typically stored in .safetensors files, which are the most secure way to store and transmit
|
||||
these types of weights. You may install any number of `.safetensors` LoRA files simply by copying them into
|
||||
the `lora` directory of the corresponding InvokeAI models directory (usually `invokeai`
|
||||
in your home directory). For example, you can simply move a Stable Diffusion 1.5 LoRA file to
|
||||
the `sd-1/lora` folder.
|
||||
LoRAs are typically stored in .safetensors files, which are the most
|
||||
secure way to store and transmit these types of weights. You may
|
||||
install any number of `.safetensors` LoRA files simply by copying them
|
||||
into the `autoimport/lora` directory of the corresponding InvokeAI models
|
||||
directory (usually `invokeai` in your home directory).
|
||||
|
||||
To use these when generating, open the LoRA menu item in the options panel, select the LoRAs you want to apply
|
||||
and ensure that they have the appropriate weight recommended by the model provider. Typically, most LoRAs perform best at a weight of .75-1.
|
||||
To use these when generating, open the LoRA menu item in the options
|
||||
panel, select the LoRAs you want to apply and ensure that they have
|
||||
the appropriate weight recommended by the model provider. Typically,
|
||||
most LoRAs perform best at a weight of .75-1.
|
||||
|
||||
|
@ -8,20 +8,64 @@ title: ControlNet
|
||||
|
||||
ControlNet
|
||||
|
||||
ControlNet is a powerful set of features developed by the open-source community (notably, Stanford researcher [**@ilyasviel**](https://github.com/lllyasviel)) that allows you to apply a secondary neural network model to your image generation process in Invoke.
|
||||
ControlNet is a powerful set of features developed by the open-source
|
||||
community (notably, Stanford researcher
|
||||
[**@ilyasviel**](https://github.com/lllyasviel)) that allows you to
|
||||
apply a secondary neural network model to your image generation
|
||||
process in Invoke.
|
||||
|
||||
With ControlNet, you can get more control over the output of your image generation, providing you with a way to direct the network towards generating images that better fit your desired style or outcome.
|
||||
With ControlNet, you can get more control over the output of your
|
||||
image generation, providing you with a way to direct the network
|
||||
towards generating images that better fit your desired style or
|
||||
outcome.
|
||||
|
||||
|
||||
### How it works
|
||||
|
||||
ControlNet works by analyzing an input image, pre-processing that image to identify relevant information that can be interpreted by each specific ControlNet model, and then inserting that control information into the generation process. This can be used to adjust the style, composition, or other aspects of the image to better achieve a specific result.
|
||||
ControlNet works by analyzing an input image, pre-processing that
|
||||
image to identify relevant information that can be interpreted by each
|
||||
specific ControlNet model, and then inserting that control information
|
||||
into the generation process. This can be used to adjust the style,
|
||||
composition, or other aspects of the image to better achieve a
|
||||
specific result.
|
||||
|
||||
|
||||
### Models
|
||||
|
||||
As part of the model installation, ControlNet models can be selected including a variety of pre-trained models that have been added to achieve different effects or styles in your generated images. Further ControlNet models may require additional code functionality to also be incorporated into Invoke's Invocations folder. You should expect to follow any installation instructions for ControlNet models loaded outside the default models provided by Invoke. The default models include:
|
||||
InvokeAI provides access to a series of ControlNet models that provide
|
||||
different effects or styles in your generated images. Currently
|
||||
InvokeAI only supports "diffuser" style ControlNet models. These are
|
||||
folders that contain the files `config.json` and/or
|
||||
`diffusion_pytorch_model.safetensors` and
|
||||
`diffusion_pytorch_model.fp16.safetensors`. The name of the folder is
|
||||
the name of the model.
|
||||
|
||||
***InvokeAI does not currently support checkpoint-format
|
||||
ControlNets. These come in the form of a single file with the
|
||||
extension `.safetensors`.***
|
||||
|
||||
Diffuser-style ControlNet models are available at HuggingFace
|
||||
(http://huggingface.co) and accessed via their repo IDs (identifiers
|
||||
in the format "author/modelname"). The easiest way to install them is
|
||||
to use the InvokeAI model installer application. Use the
|
||||
`invoke.sh`/`invoke.bat` launcher to select item [5] and then navigate
|
||||
to the CONTROLNETS section. Select the models you wish to install and
|
||||
press "APPLY CHANGES". You may also enter additional HuggingFace
|
||||
repo_ids in the "Additional models" textbox:
|
||||
|
||||
![Model Installer -
|
||||
Controlnetl](../assets/installing-models/model-installer-controlnet.png){:width="640px"}
|
||||
|
||||
Command-line users can launch the model installer using the command
|
||||
`invokeai-model-install`.
|
||||
|
||||
_Be aware that some ControlNet models require additional code
|
||||
functionality in order to work properly, so just installing a
|
||||
third-party ControlNet model may not have the desired effect._ Please
|
||||
read and follow the documentation for installing a third party model
|
||||
not currently included among InvokeAI's default list.
|
||||
|
||||
The models currently supported include:
|
||||
|
||||
**Canny**:
|
||||
|
||||
|
@ -4,15 +4,19 @@ title: InvokeAI Web Server
|
||||
|
||||
# :material-web: InvokeAI Web Server
|
||||
|
||||
As of version 2.0.0, this distribution comes with a full-featured web server
|
||||
(see screenshot).
|
||||
## Quick guided walkthrough of the WebUI's features
|
||||
|
||||
To use it, launch the `invoke.sh`/`invoke.bat` script and select
|
||||
option (2). Alternatively, with the InvokeAI environment active, run
|
||||
the `invokeai` script by adding the `--web` option:
|
||||
While most of the WebUI's features are intuitive, here is a guided walkthrough
|
||||
through its various components.
|
||||
|
||||
### Launching the WebUI
|
||||
|
||||
To run the InvokeAI web server, start the `invoke.sh`/`invoke.bat`
|
||||
script and select option (1). Alternatively, with the InvokeAI
|
||||
environment active, run `invokeai-web`:
|
||||
|
||||
```bash
|
||||
invokeai --web
|
||||
invokeai-web
|
||||
```
|
||||
|
||||
You can then connect to the server by pointing your web browser at
|
||||
@ -28,33 +32,32 @@ invoke.sh --host 0.0.0.0
|
||||
or
|
||||
|
||||
```bash
|
||||
invokeai --web --host 0.0.0.0
|
||||
invokeai-web --host 0.0.0.0
|
||||
```
|
||||
|
||||
## Quick guided walkthrough of the WebUI's features
|
||||
|
||||
While most of the WebUI's features are intuitive, here is a guided walkthrough
|
||||
through its various components.
|
||||
### The InvokeAI Web Interface
|
||||
|
||||
![Invoke Web Server - Major Components](../assets/invoke-web-server-1.png){:width="640px"}
|
||||
|
||||
The screenshot above shows the Text to Image tab of the WebUI. There are three
|
||||
main sections:
|
||||
|
||||
1. A **control panel** on the left, which contains various settings for text to
|
||||
image generation. The most important part is the text field (currently
|
||||
showing `strawberry sushi`) for entering the text prompt, and the camera icon
|
||||
directly underneath that will render the image. We'll call this the _Invoke_
|
||||
button from now on.
|
||||
1. A **control panel** on the left, which contains various settings
|
||||
for text to image generation. The most important part is the text
|
||||
field (currently showing `fantasy painting, horned demon`) for
|
||||
entering the positive text prompt, another text field right below it for an
|
||||
optional negative text prompt (concepts to exclude), and a _Invoke_ button
|
||||
to begin the image rendering process.
|
||||
|
||||
2. The **current image** section in the middle, which shows a large format
|
||||
version of the image you are currently working on. A series of buttons at the
|
||||
top ("image to image", "Use All", "Use Seed", etc) lets you modify the image
|
||||
in various ways.
|
||||
2. The **current image** section in the middle, which shows a large
|
||||
format version of the image you are currently working on. A series
|
||||
of buttons at the top lets you modify and manipulate the image in
|
||||
various ways.
|
||||
|
||||
3. A \*_gallery_ section on the left that contains a history of the images you
|
||||
3. A **gallery** section on the left that contains a history of the images you
|
||||
have generated. These images are read and written to the directory specified
|
||||
at launch time in `--outdir`.
|
||||
in the `INVOKEAIROOT/invokeai.yaml` initialization file, usually a directory
|
||||
named `outputs` in `INVOKEAIROOT`.
|
||||
|
||||
In addition to these three elements, there are a series of icons for changing
|
||||
global settings, reporting bugs, and changing the theme on the upper right.
|
||||
@ -76,15 +79,11 @@ From top to bottom, these are:
|
||||
with outpainting,and modify interior portions of the image with
|
||||
inpainting, erase portions of a starting image and have the AI fill in
|
||||
the erased region from a text prompt.
|
||||
4. Node Editor - this panel allows you to create
|
||||
4. Node Editor - (experimental) this panel allows you to create
|
||||
pipelines of common operations and combine them into workflows.
|
||||
5. Model Manager - this panel allows you to import and configure new
|
||||
models using URLs, local paths, or HuggingFace diffusers repo_ids.
|
||||
|
||||
The inpainting, outpainting and postprocessing tabs are currently in
|
||||
development. However, limited versions of their features can already be accessed
|
||||
through the Text to Image and Image to Image tabs.
|
||||
|
||||
## Walkthrough
|
||||
|
||||
The following walkthrough will exercise most (but not all) of the WebUI's
|
||||
@ -92,43 +91,54 @@ feature set.
|
||||
|
||||
### Text to Image
|
||||
|
||||
1. Launch the WebUI using `python scripts/invoke.py --web` and connect to it
|
||||
with your browser by accessing `http://localhost:9090`. If the browser and
|
||||
server are running on different machines on your LAN, add the option
|
||||
`--host 0.0.0.0` to the launch command line and connect to the machine
|
||||
hosting the web server using its IP address or domain name.
|
||||
1. Launch the WebUI using launcher option [1] and connect to it with
|
||||
your browser by accessing `http://localhost:9090`. If the browser
|
||||
and server are running on different machines on your LAN, add the
|
||||
option `--host 0.0.0.0` to the `invoke.sh` launch command line and connect to
|
||||
the machine hosting the web server using its IP address or domain
|
||||
name.
|
||||
|
||||
2. If all goes well, the WebUI should come up and you'll see a green
|
||||
`connected` message on the upper right.
|
||||
2. If all goes well, the WebUI should come up and you'll see a green dot
|
||||
meaning `connected` on the upper right.
|
||||
|
||||
![Invoke Web Server - Control Panel](../assets/invoke-control-panel-1.png){ align=right width=300px }
|
||||
|
||||
#### Basics
|
||||
|
||||
1. Generate an image by typing _strawberry sushi_ into the large prompt field
|
||||
on the upper left and then clicking on the Invoke button (the one with the
|
||||
Camera icon). After a short wait, you'll see a large image of sushi in the
|
||||
1. Generate an image by typing _bluebird_ into the large prompt field
|
||||
on the upper left and then clicking on the Invoke button or pressing
|
||||
the return button.
|
||||
After a short wait, you'll see a large image of a bluebird in the
|
||||
image panel, and a new thumbnail in the gallery on the right.
|
||||
|
||||
If you need more room on the screen, you can turn the gallery off by
|
||||
clicking on the **x** to the right of "Your Invocations". You can turn it
|
||||
back on later by clicking the image icon that appears in the gallery's
|
||||
place.
|
||||
If you need more room on the screen, you can turn the gallery off
|
||||
by typing the **g** hotkey. You can turn it back on later by clicking the
|
||||
image icon that appears in the gallery's place. The list of hotkeys can
|
||||
be found by clicking on the keyboard icon above the image gallery.
|
||||
|
||||
The images are written into the directory indicated by the `--outdir` option
|
||||
provided at script launch time. By default, this is `outputs/img-samples`
|
||||
under the InvokeAI directory.
|
||||
|
||||
2. Generate a bunch of strawberry sushi images by increasing the number of
|
||||
requested images by adjusting the Images counter just below the Camera
|
||||
2. Generate a bunch of bluebird images by increasing the number of
|
||||
requested images by adjusting the Images counter just below the Invoke
|
||||
button. As each is generated, it will be added to the gallery. You can
|
||||
switch the active image by clicking on the gallery thumbnails.
|
||||
|
||||
If you'd like to watch the image generation progress, click the hourglass
|
||||
icon above the main image area. As generation progresses, you'll see
|
||||
increasingly detailed versions of the ultimate image.
|
||||
|
||||
3. Try playing with different settings, including image width and height, the
|
||||
Sampler, the Steps and the CFG scale.
|
||||
3. Try playing with different settings, including changing the main
|
||||
model, the image width and height, the Scheduler, the Steps and
|
||||
the CFG scale.
|
||||
|
||||
The _Model_ changes the main model. Thousands of custom models are
|
||||
now available, which generate a variety of image styles and
|
||||
subjects. While InvokeAI comes with a few starter models, it is
|
||||
easy to import new models into the application. See [Installing
|
||||
Models](../installation/050_INSTALLING_MODELS.md) for more details.
|
||||
|
||||
Image _Width_ and _Height_ do what you'd expect. However, be aware that
|
||||
larger images consume more VRAM memory and take longer to generate.
|
||||
|
||||
The _Sampler_ controls how the AI selects the image to display. Some
|
||||
The _Scheduler_ controls how the AI selects the image to display. Some
|
||||
samplers are more "creative" than others and will produce a wider range of
|
||||
variations (see next section). Some samplers run faster than others.
|
||||
|
||||
@ -142,17 +152,27 @@ feature set.
|
||||
to the input prompt. You can go as high or low as you like, but generally
|
||||
values greater than 20 won't improve things much, and values lower than 5
|
||||
will produce unexpected images. There are complex interactions between
|
||||
_Steps_, _CFG Scale_ and the _Sampler_, so experiment to find out what works
|
||||
_Steps_, _CFG Scale_ and the _Scheduler_, so experiment to find out what works
|
||||
for you.
|
||||
|
||||
The _Seed_ controls the series of values returned by InvokeAI's
|
||||
random number generator. Each unique seed value will generate a different
|
||||
image. To regenerate a previous image, simply use the original image's
|
||||
seed value. A slider to the right of the _Seed_ field will change the
|
||||
seed each time an image is generated.
|
||||
|
||||
4. To regenerate a previously-generated image, select the image you want and
|
||||
click _Use All_. This loads the text prompt and other original settings into
|
||||
the control panel. If you then press _Invoke_ it will regenerate the image
|
||||
exactly. You can also selectively modify the prompt or other settings to
|
||||
tweak the image.
|
||||
![Invoke Web Server - Control Panel 2](../assets/control-panel-2.png){ align=right width=400px }
|
||||
|
||||
Alternatively, you may click on _Use Seed_ to load just the image's seed,
|
||||
and leave other settings unchanged.
|
||||
4. To regenerate a previously-generated image, select the image you
|
||||
want and click the asterisk ("*") button at the top of the
|
||||
image. This loads the text prompt and other original settings into
|
||||
the control panel. If you then press _Invoke_ it will regenerate
|
||||
the image exactly. You can also selectively modify the prompt or
|
||||
other settings to tweak the image.
|
||||
|
||||
Alternatively, you may click on the "sprouting plant icon" to load
|
||||
just the image's seed, and leave other settings unchanged or the
|
||||
quote icon to load just the positive and negative prompts.
|
||||
|
||||
5. To regenerate a Stable Diffusion image that was generated by another SD
|
||||
package, you need to know its text prompt and its _Seed_. Copy-paste the
|
||||
@ -161,62 +181,22 @@ feature set.
|
||||
you Invoke, you will get something similar to the original image. It will
|
||||
not be exact unless you also set the correct values for the original
|
||||
sampler, CFG, steps and dimensions, but it will (usually) be close.
|
||||
|
||||
6. To save an image, right click on it to bring up a menu that will
|
||||
let you download the image, save it to a named image gallery, and
|
||||
copy it to the clipboard, among other things.
|
||||
|
||||
#### Variations on a theme
|
||||
#### Upscaling
|
||||
|
||||
1. Let's try generating some variations. Select your favorite sushi image from
|
||||
the gallery to load it. Then select "Use All" from the list of buttons
|
||||
above. This will load up all the settings used to generate this image,
|
||||
including its unique seed.
|
||||
![Invoke Web Server - Upscaling](../assets/upscaling.png){ align=right width=400px }
|
||||
|
||||
Go down to the Variations section of the Control Panel and set the button to
|
||||
On. Set Variation Amount to 0.2 to generate a modest number of variations on
|
||||
the image, and also set the Image counter to `4`. Press the `invoke` button.
|
||||
This will generate a series of related images. To obtain smaller variations,
|
||||
just lower the Variation Amount. You may also experiment with changing the
|
||||
Sampler. Some samplers generate more variability than others. _k_euler_a_ is
|
||||
particularly creative, while _ddim_ is pretty conservative.
|
||||
|
||||
2. For even more variations, experiment with increasing the setting for
|
||||
_Perlin_. This adds a bit of noise to the image generation process. Note
|
||||
that values of Perlin noise greater than 0.15 produce poor images for
|
||||
several of the samplers.
|
||||
|
||||
#### Facial reconstruction and upscaling
|
||||
|
||||
Stable Diffusion frequently produces mangled faces, particularly when there are
|
||||
multiple figures in the same scene. Stable Diffusion has particular issues with
|
||||
generating reallistic eyes. InvokeAI provides the ability to reconstruct faces
|
||||
using either the GFPGAN or CodeFormer libraries. For more information see
|
||||
[POSTPROCESS](POSTPROCESS.md).
|
||||
|
||||
1. Invoke a prompt that generates a mangled face. A prompt that often gives
|
||||
this is "portrait of a lawyer, 3/4 shot" (this is not intended as a slur
|
||||
against lawyers!) Once you have an image that needs some touching up, load
|
||||
it into the Image panel, and press the button with the face icon
|
||||
(highlighted in the first screenshot below). A dialog box will appear. Leave
|
||||
_Strength_ at 0.8 and press \*Restore Faces". If all goes well, the eyes and
|
||||
other aspects of the face will be improved (see the second screenshot)
|
||||
|
||||
![Invoke Web Server - Original Image](../assets/invoke-web-server-3.png)
|
||||
|
||||
![Invoke Web Server - Retouched Image](../assets/invoke-web-server-4.png)
|
||||
|
||||
The facial reconstruction _Strength_ field adjusts how aggressively the face
|
||||
library will try to alter the face. It can be as high as 1.0, but be aware
|
||||
that this often softens the face airbrush style, losing some details. The
|
||||
default 0.8 is usually sufficient.
|
||||
|
||||
2. "Upscaling" is the process of increasing the size of an image while
|
||||
retaining the sharpness. InvokeAI uses an external library called "ESRGAN"
|
||||
to do this. To invoke upscaling, simply select an image and press the _HD_
|
||||
button above it. You can select between 2X and 4X upscaling, and adjust the
|
||||
upscaling strength, which has much the same meaning as in facial
|
||||
reconstruction. Try running this on one of your previously-generated images.
|
||||
|
||||
3. Finally, you can run facial reconstruction and/or upscaling automatically
|
||||
after each Invocation. Go to the Advanced Options section of the Control
|
||||
Panel and turn on _Restore Face_ and/or _Upscale_.
|
||||
"Upscaling" is the process of increasing the size of an image while
|
||||
retaining the sharpness. InvokeAI uses an external library called
|
||||
"ESRGAN" to do this. To invoke upscaling, simply select an image
|
||||
and press the "expanding arrows" button above it. You can select
|
||||
between 2X and 4X upscaling, and adjust the upscaling strength,
|
||||
which has much the same meaning as in facial reconstruction. Try
|
||||
running this on one of your previously-generated images.
|
||||
|
||||
### Image to Image
|
||||
|
||||
@ -224,24 +204,14 @@ InvokeAI lets you take an existing image and use it as the basis for a new
|
||||
creation. You can use any sort of image, including a photograph, a scanned
|
||||
sketch, or a digital drawing, as long as it is in PNG or JPEG format.
|
||||
|
||||
For this tutorial, we'll use files named
|
||||
[Lincoln-and-Parrot-512.png](../assets/Lincoln-and-Parrot-512.png), and
|
||||
[Lincoln-and-Parrot-512-transparent.png](../assets/Lincoln-and-Parrot-512-transparent.png).
|
||||
Download these images to your local machine now to continue with the
|
||||
walkthrough.
|
||||
For this tutorial, we'll use the file named
|
||||
[Lincoln-and-Parrot-512.png](../assets/Lincoln-and-Parrot-512.png).
|
||||
|
||||
1. Click on the _Image to Image_ tab icon, which is the second icon from the
|
||||
top on the left-hand side of the screen:
|
||||
1. Click on the _Image to Image_ tab icon, which is the second icon
|
||||
from the top on the left-hand side of the screen. This will bring
|
||||
you to a screen similar to the one shown here:
|
||||
|
||||
<figure markdown>
|
||||
![Invoke Web Server - Image to Image Icon](../assets/invoke-web-server-5.png)
|
||||
</figure>
|
||||
|
||||
This will bring you to a screen similar to the one shown here:
|
||||
|
||||
<figure markdown>
|
||||
![Invoke Web Server - Image to Image Tab](../assets/invoke-web-server-6.png){:width="640px"}
|
||||
</figure>
|
||||
![Invoke Web Server - Image to Image Tab](../assets/invoke-web-server-6.png){ width="640px" }
|
||||
|
||||
2. Drag-and-drop the Lincoln-and-Parrot image into the Image panel, or click
|
||||
the blank area to get an upload dialog. The image will load into an area
|
||||
@ -255,120 +225,99 @@ walkthrough.
|
||||
![Invoke Web Server - Image to Image example](../assets/invoke-web-server-7.png){:width="640px"}
|
||||
|
||||
4. Experiment with the different settings. The most influential one in Image to
|
||||
Image is _Image to Image Strength_ located about midway down the control
|
||||
Image is _Denoising Strength_ located about midway down the control
|
||||
panel. By default it is set to 0.75, but can range from 0.0 to 0.99. The
|
||||
higher the value, the more of the original image the AI will replace. A
|
||||
value of 0 will leave the initial image completely unchanged, while 0.99
|
||||
will replace it completely. However, the Sampler and CFG Scale also
|
||||
will replace it completely. However, the _Scheduler_ and _CFG Scale_ also
|
||||
influence the final result. You can also generate variations in the same way
|
||||
as described in Text to Image.
|
||||
|
||||
5. What if we only want to change certain part(s) of the image and leave the
|
||||
rest intact? This is called Inpainting, and a future version of the InvokeAI
|
||||
web server will provide an interactive painting canvas on which you can
|
||||
directly draw the areas you wish to Inpaint into. For now, you can achieve
|
||||
this effect by using an external photoeditor tool to make one or more
|
||||
regions of the image transparent as described in [INPAINTING.md] and
|
||||
uploading that.
|
||||
|
||||
The file
|
||||
[Lincoln-and-Parrot-512-transparent.png](../assets/Lincoln-and-Parrot-512-transparent.png)
|
||||
is a version of the earlier image in which the area around the parrot has
|
||||
been replaced with transparency. Click on the "x" in the upper right of the
|
||||
Initial Image and upload the transparent version. Using the same prompt "old
|
||||
sea captain with raven on shoulder" try Invoking an image. This time, only
|
||||
the parrot will be replaced, leaving the rest of the original image intact:
|
||||
|
||||
<figure markdown>
|
||||
![Invoke Web Server - Inpainting](../assets/invoke-web-server-8.png){:width="640px"}
|
||||
</figure>
|
||||
5. What if we only want to change certain part(s) of the image and
|
||||
leave the rest intact? This is called Inpainting, and you can do
|
||||
it in the [Unified Canvas](UNIFIED_CANVAS.md). The Unified Canvas
|
||||
also allows you to extend borders of the image and fill in the
|
||||
blank areas, a process called outpainting.
|
||||
|
||||
6. Would you like to modify a previously-generated image using the Image to
|
||||
Image facility? Easy! While in the Image to Image panel, hover over any of
|
||||
the gallery images to see a little menu of icons pop up. Click the picture
|
||||
icon to instantly send the selected image to Image to Image as the initial
|
||||
image.
|
||||
Image facility? Easy! While in the Image to Image panel, drag and drop any
|
||||
image in the gallery into the Initial Image area, and it will be ready for
|
||||
use. You can do the same thing with the main image display. Click on the
|
||||
_Send to_ icon to get a menu of
|
||||
commands and choose "Send to Image to Image".
|
||||
|
||||
![Send To Icon](../assets/send-to-icon.png)
|
||||
|
||||
You can do the same from the Text to Image tab by clicking on the picture icon
|
||||
above the central image panel. The screenshot below shows where the "use as
|
||||
initial image" icons are located.
|
||||
### Textual Inversion, LoRA and ControlNet
|
||||
|
||||
![Invoke Web Server - Use as Image Links](../assets/invoke-web-server-9.png){:width="640px"}
|
||||
InvokeAI supports several different types of model files that
|
||||
extending the capabilities of the main model by adding artistic
|
||||
styles, special effects, or subjects. By mixing and matching textual
|
||||
inversion, LoRA and ControlNet models, you can achieve many
|
||||
interesting and beautiful effects.
|
||||
|
||||
### Unified Canvas
|
||||
We will give an example using a LoRA model named "Ink Scenery". This
|
||||
LoRA, which can be downloaded from Civitai (civitai.com), is
|
||||
specialized to paint landscapes that look like they were made with
|
||||
dripping india ink. To install this LoRA, we first download it and
|
||||
put it into the `autoimport/lora` folder located inside the
|
||||
`invokeai` root directory. After restarting the web server, the
|
||||
LoRA will now become available for use.
|
||||
|
||||
See the [Unified Canvas Guide](UNIFIED_CANVAS.md)
|
||||
To see this LoRA at work, we'll first generate an image without it
|
||||
using the standard `stable-diffusion-v1-5` model. Choose this
|
||||
model and enter the prompt "mountains, ink". Here is a typical
|
||||
generated image, a mountain range rendered in ink and watercolor
|
||||
wash:
|
||||
|
||||
## Reference
|
||||
![Ink Scenery without LoRA](../assets/lora-example-0.png){ width=512px }
|
||||
|
||||
### Additional Options
|
||||
Now let's install and activate the Ink Scenery LoRA. Go to
|
||||
https://civitai.com/models/78605/ink-scenery-or and download the LoRA
|
||||
model file to `invokeai/autoimport/lora` and restart the web
|
||||
server. (Alternatively, you can use [InvokeAI's Web Model
|
||||
Manager](../installation/050_INSTALLING_MODELS.md) to download and
|
||||
install the LoRA directly by typing its URL into the _Import
|
||||
Models_->_Location_ field).
|
||||
|
||||
| parameter <img width=160 align="right"> | effect |
|
||||
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `--web_develop` | Starts the web server in development mode. |
|
||||
| `--web_verbose` | Enables verbose logging |
|
||||
| `--cors [CORS ...]` | Additional allowed origins, comma-separated |
|
||||
| `--host HOST` | Web server: Host or IP to listen on. Set to 0.0.0.0 to accept traffic from other devices on your network. |
|
||||
| `--port PORT` | Web server: Port to listen on |
|
||||
| `--certfile CERTFILE` | Web server: Path to certificate file to use for SSL. Use together with --keyfile |
|
||||
| `--keyfile KEYFILE` | Web server: Path to private key file to use for SSL. Use together with --certfile' |
|
||||
| `--gui` | Start InvokeAI GUI - This is the "desktop mode" version of the web app. It uses Flask to create a desktop app experience of the webserver. |
|
||||
Scroll down the control panel until you get to the LoRA accordion
|
||||
section, and open it:
|
||||
|
||||
### Web Specific Features
|
||||
![LoRA Section](../assets/lora-example-1.png){ width=512px }
|
||||
|
||||
The web experience offers an incredibly easy-to-use experience for interacting
|
||||
with the InvokeAI toolkit. For detailed guidance on individual features, see the
|
||||
Feature-specific help documents available in this directory. Note that the
|
||||
latest functionality available in the CLI may not always be available in the Web
|
||||
interface.
|
||||
Click the popup menu and select "Ink scenery". (If it isn't there, then
|
||||
the model wasn't installed to the right place, or perhaps you forgot
|
||||
to restart the web server.) The LoRA section will change to look like this:
|
||||
|
||||
#### Dark Mode & Light Mode
|
||||
![LoRA Section Loaded](../assets/lora-example-2.png){ width=512px }
|
||||
|
||||
The InvokeAI interface is available in a nano-carbon black & purple Dark Mode,
|
||||
and a "burn your eyes out Nosferatu" Light Mode. These can be toggled by
|
||||
clicking the Sun/Moon icons at the top right of the interface.
|
||||
Note that there is now a slider control for _Ink scenery_. The slider
|
||||
controls how much influence the LoRA model will have on the generated
|
||||
image.
|
||||
|
||||
![InvokeAI Web Server - Dark Mode](../assets/invoke_web_dark.png)
|
||||
Run the "mountains, ink" prompt again and observe the change in style:
|
||||
|
||||
![InvokeAI Web Server - Light Mode](../assets/invoke_web_light.png)
|
||||
![Ink Scenery](../assets/lora-example-3.png){ width=512px }
|
||||
|
||||
#### Invocation Toolbar
|
||||
Try adjusting the weight slider for larger and smaller weights and
|
||||
generate the image after each adjustment. The higher the weight, the
|
||||
more influence the LoRA will have.
|
||||
|
||||
The left side of the InvokeAI interface is available for customizing the prompt
|
||||
and the settings used for invoking your new image. Typing your prompt into the
|
||||
open text field and clicking the Invoke button will produce the image based on
|
||||
the settings configured in the toolbar.
|
||||
To remove the LoRA completely, just click on its trash can icon.
|
||||
|
||||
See below for additional documentation related to each feature:
|
||||
Multiple LoRAs can be added simultaneously and combined with textual
|
||||
inversions and ControlNet models. Please see [Textual Inversions and
|
||||
LoRAs](CONCEPTS.md) and [Using ControlNet](CONTROLNET.md) for details.
|
||||
|
||||
- [Variations](./VARIATIONS.md)
|
||||
- [Upscaling](./POSTPROCESS.md#upscaling)
|
||||
- [Image to Image](./IMG2IMG.md)
|
||||
- [Other](./OTHER.md)
|
||||
## Summary
|
||||
|
||||
#### Invocation Gallery
|
||||
|
||||
The currently selected --outdir (or the default outputs folder) will display all
|
||||
previously generated files on load. As new invocations are generated, these will
|
||||
be dynamically added to the gallery, and can be previewed by selecting them.
|
||||
Each image also has a simple set of actions (e.g., Delete, Use Seed, Use All
|
||||
Parameters, etc.) that can be accessed by hovering over the image.
|
||||
|
||||
#### Image Workspace
|
||||
|
||||
When an image from the Invocation Gallery is selected, or is generated, the
|
||||
image will be displayed within the center of the interface. A quickbar of common
|
||||
image interactions are displayed along the top of the image, including:
|
||||
|
||||
- Use image in the `Image to Image` workflow
|
||||
- Initialize Face Restoration on the selected file
|
||||
- Initialize Upscaling on the selected file
|
||||
- View File metadata and details
|
||||
- Delete the file
|
||||
This walkthrough just skims the surface of the many things InvokeAI
|
||||
can do. Please see [Features](index.md) for more detailed reference
|
||||
guides.
|
||||
|
||||
## Acknowledgements
|
||||
|
||||
A huge shout-out to the core team working to make this vision a reality,
|
||||
A huge shout-out to the core team working to make the Web GUI a reality,
|
||||
including [psychedelicious](https://github.com/psychedelicious),
|
||||
[Kyle0654](https://github.com/Kyle0654) and
|
||||
[blessedcoolant](https://github.com/blessedcoolant).
|
||||
|
@ -17,8 +17,12 @@ a single convenient digital artist-optimized user interface.
|
||||
### * [Prompt Engineering](PROMPTS.md)
|
||||
Get the images you want with the InvokeAI prompt engineering language.
|
||||
|
||||
## * The [Concepts Library](CONCEPTS.md)
|
||||
Add custom subjects and styles using HuggingFace's repository of embeddings.
|
||||
### * The [LoRA, LyCORIS and Textual Inversion Models](CONCEPTS.md)
|
||||
Add custom subjects and styles using a variety of fine-tuned models.
|
||||
|
||||
### * [ControlNet](CONTROLNET.md)
|
||||
Learn how to install and use ControlNet models for fine control over
|
||||
image output.
|
||||
|
||||
### * [Image-to-Image Guide](IMG2IMG.md)
|
||||
Use a seed image to build new creations in the CLI.
|
||||
@ -29,26 +33,28 @@ are the ticket.
|
||||
|
||||
## Model Management
|
||||
|
||||
## * [Model Installation](../installation/050_INSTALLING_MODELS.md)
|
||||
### * [Model Installation](../installation/050_INSTALLING_MODELS.md)
|
||||
Learn how to import third-party models and switch among them. This
|
||||
guide also covers optimizing models to load quickly.
|
||||
|
||||
## * [Merging Models](MODEL_MERGING.md)
|
||||
### * [Merging Models](MODEL_MERGING.md)
|
||||
Teach an old model new tricks. Merge 2-3 models together to create a
|
||||
new model that combines characteristics of the originals.
|
||||
|
||||
## * [Textual Inversion](TRAINING.md)
|
||||
### * [Textual Inversion](TRAINING.md)
|
||||
Personalize models by adding your own style or subjects.
|
||||
|
||||
# Other Features
|
||||
## Other Features
|
||||
|
||||
## * [The NSFW Checker](NSFW.md)
|
||||
### * [The NSFW Checker](NSFW.md)
|
||||
Prevent InvokeAI from displaying unwanted racy images.
|
||||
|
||||
## * [Controlling Logging](LOGGING.md)
|
||||
### * [Controlling Logging](LOGGING.md)
|
||||
Control how InvokeAI logs status messages.
|
||||
|
||||
## * [Miscellaneous](OTHER.md)
|
||||
<!-- OUT OF DATE
|
||||
### * [Miscellaneous](OTHER.md)
|
||||
Run InvokeAI on Google Colab, generate images with repeating patterns,
|
||||
batch process a file of prompts, increase the "creativity" of image
|
||||
generation by adding initial noise, and more!
|
||||
-->
|
||||
|
@ -145,6 +145,7 @@ This method is recommended for those familiar with running Docker containers
|
||||
### Model Management
|
||||
- [Installing](installation/050_INSTALLING_MODELS.md)
|
||||
- [Model Merging](features/MODEL_MERGING.md)
|
||||
- [ControlNet Models](features/CONTROLNET.md)
|
||||
- [Style/Subject Concepts and Embeddings](features/CONCEPTS.md)
|
||||
- [Not Safe for Work (NSFW) Checker](features/NSFW.md)
|
||||
<!-- seperator -->
|
||||
|
@ -560,7 +560,6 @@ def edit_opts(program_opts: Namespace, invokeai_opts: Namespace) -> argparse.Nam
|
||||
editApp.run()
|
||||
return editApp.new_opts()
|
||||
|
||||
|
||||
def default_startup_options(init_file: Path) -> Namespace:
|
||||
opts = InvokeAIAppConfig.get_config()
|
||||
if not init_file.exists():
|
||||
@ -664,6 +663,9 @@ def write_opts(opts: Namespace, init_file: Path):
|
||||
with open(init_file,'w', encoding='utf-8') as file:
|
||||
file.write(new_config.to_yaml())
|
||||
|
||||
if opts.hf_token:
|
||||
HfLogin(opts.hf_token)
|
||||
|
||||
# -------------------------------------
|
||||
def default_output_dir() -> Path:
|
||||
return config.root_path / "outputs"
|
||||
|
@ -938,20 +938,29 @@ class ModelManager(object):
|
||||
def models_found(self):
|
||||
return self.new_models_found
|
||||
|
||||
config = self.app_config
|
||||
|
||||
# LS: hacky
|
||||
# Patch in the SD VAE from core so that it is available for use by the UI
|
||||
try:
|
||||
self.heuristic_import({config.root_path / 'models/core/convert/sd-vae-ft-mse'})
|
||||
except:
|
||||
pass
|
||||
|
||||
installer = ModelInstall(config = self.app_config,
|
||||
model_manager = self,
|
||||
prediction_type_helper = ask_user_for_prediction_type,
|
||||
)
|
||||
config = self.app_config
|
||||
known_paths = {config.root_path / x['path'] for x in self.list_models()}
|
||||
directories = {config.root_path / x for x in [config.autoimport_dir,
|
||||
config.lora_dir,
|
||||
config.embedding_dir,
|
||||
config.controlnet_dir]
|
||||
config.controlnet_dir,
|
||||
]
|
||||
}
|
||||
scanner = ScanAndImport(directories, self.logger, ignore=known_paths, installer=installer)
|
||||
scanner.search()
|
||||
|
||||
return scanner.models_found()
|
||||
|
||||
def heuristic_import(self,
|
||||
|