mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
This commit is contained in:
Lincoln Stein
@ -4,75 +4,95 @@ title: Outpainting
|
||||
|
||||
# :octicons-paintbrush-16: Outpainting
|
||||
|
||||
## Continous outpainting
|
||||
## Outpainting and outcropping
|
||||
|
||||
This extension uses the inpainting code to extend an existing image to
|
||||
any direction of "top", "right", "bottom" or "left". To use it you
|
||||
need to provide an initial image with -I and an extension direction
|
||||
with -D (direction). When extending using outpainting a higher img2img
|
||||
strength value of 0.83 is the default.
|
||||
Outpainting is a process by which the AI generates parts of the image
|
||||
that are outside its original frame. It can be used to fix up images
|
||||
in which the subject is off center, or when some detail (often the top
|
||||
of someone's head!) is cut off.
|
||||
|
||||
The code is not foolproof. Sometimes it will do a good job extending
|
||||
the image, and other times it will generate ghost images and other
|
||||
artifacts. In addition, the code works best on images that were
|
||||
generated by dream.py, because it will be able to recover the original
|
||||
prompt that generated the file and "understand" what you are trying to
|
||||
achieve.
|
||||
InvokeAI supports two versions of outpainting, one called "outpaint"
|
||||
and the other "outcrop." They work slightly differently and each has
|
||||
its advantages and drawbacks.
|
||||
|
||||
### Basic Usage
|
||||
### Outcrop
|
||||
|
||||
To illustrate, consider this image generated with the prompt "fantasy
|
||||
portrait of eleven princess." It's nice, but rather annoying that the
|
||||
top of the head has been cropped off.
|
||||
The `outcrop` extension allows you to extend the image in 64 pixel
|
||||
increments in any dimension. You can apply the module to any image
|
||||
previously-generated by InvokeAI. Note that it will **not** work with
|
||||
arbitrary photographs or Stable Diffusion images created by other
|
||||
implementations.
|
||||
|
||||
|
||||
Consider this image:
|
||||
|
||||
We can fix that using the `!fix` command!
|
||||
|
||||
|
||||
Pretty nice, but it's annoying that the top of her head is cut
|
||||
off. She's also a bit off center. Let's fix that!
|
||||
|
||||
~~~~
|
||||
dream> !fix my_images/elven_princess.png -D top 50
|
||||
dream> !fix images/curly.png --outcrop top 64 right 64
|
||||
~~~~
|
||||
|
||||
This is telling dream.py to open up a rectangle 50 pixels high at the
|
||||
top of the image and outpaint into it. The result is:
|
||||
This is saying to apply the `outcrop` extension by extending the top
|
||||
of the image by 64 pixels, and the right of the image by the same
|
||||
amount. You can use any combination of top|left|right|bottom, and
|
||||
specify any number of pixels to extend. You can also abbreviate
|
||||
`--outcrop` to `-c`.
|
||||
|
||||
|
||||
The result looks like this:
|
||||
|
||||
Viola! You can similarly specify `bottom`, `left` or `right` to
|
||||
outpaint into these margins.
|
||||
|
||||
|
||||
There are some limitations to be aware of:
|
||||
The new image is actually slightly larger than the original (576x576,
|
||||
because 64 pixels were added to the top and right sides.)
|
||||
|
||||
1. You cannot change the size of the image rectangle. In the example,
|
||||
notice that the whole image is shifted downwards by 50 pixels, rather
|
||||
than the top being extended upwards.
|
||||
A number of caveats:
|
||||
|
||||
2. Attempting to outpaint larger areas will frequently give rise to ugly
|
||||
1. Although you can specify any pixel values, they will be rounded up
|
||||
to the nearest multiple of 64. Smaller values are better. Larger
|
||||
extensions are more likely to generate artefacts. However, if you wish
|
||||
you can run the !fix command repeatedly to cautiously expand the
|
||||
image.
|
||||
|
||||
2. The extension is stochastic, meaning that each time you run it
|
||||
you'll get a slightly different result. You can run it repeatedly
|
||||
until you get an image you like. Unfortunately `!fix` does not
|
||||
currently respect the `-n` (`--iterations`) argument.
|
||||
|
||||
## Outpaint
|
||||
|
||||
The `outpaint` extension does the same thing, but with subtle
|
||||
differences. Starting with the same image, here is how we would add an
|
||||
additional 64 pixels to the top of the image:
|
||||
|
||||
~~~
|
||||
dream> !fix images/curly.png --out_direction top 64
|
||||
~~~
|
||||
|
||||
(you can abbreviate ``--out_direction` as `-D`.
|
||||
|
||||
The result is shown here:
|
||||
|
||||
|
||||
|
||||
Although the effect is similar, there are significant differences from
|
||||
outcropping:
|
||||
|
||||
1. You can only specify one direction to extend at a time.
|
||||
2. The image is **not** resized. Instead, the image is shifted by the specified
|
||||
number of pixels. If you look carefully, you'll see that less of the lady's
|
||||
torso is visible in the image.
|
||||
3. Because the image dimensions remain the same, there's no rounding
|
||||
to multiples of 64.
|
||||
4. Attempting to outpaint larger areas will frequently give rise to ugly
|
||||
ghosting effects.
|
||||
|
||||
3. For best results, try increasing the step number.
|
||||
|
||||
4. If you don't specify a pixel value in -D, it will default to half
|
||||
5. For best results, try increasing the step number.
|
||||
6. If you don't specify a pixel value in -D, it will default to half
|
||||
of the whole image, which is likely not what you want.
|
||||
|
||||
You can do more with `!fix` including upscaling and facial
|
||||
reconstruction of previously-generated images. See
|
||||
[./UPSCALE.md#fixing-previously-generated-images] for the details.
|
||||
|
||||
### Advanced Usage
|
||||
|
||||
For more control over the outpaintihg process, you can provide the
|
||||
`-D` option at image generation time. This allows you to apply all the
|
||||
controls, including the ability to resize the image and apply face-fixing
|
||||
and upscaling. For example:
|
||||
|
||||
~~~~
|
||||
dream> man with cat on shoulder -I./images/man.png -D bottom 100 -W960 -H960 -fit
|
||||
~~~~
|
||||
|
||||
Or even shorter, since the prompt is read from the metadata of the old image:
|
||||
|
||||
~~~~
|
||||
dream> -I./images/man.png -D bottom 100 -W960 -H960 -fit -U2 -G1
|
||||
~~~~
|
||||
Neither `outpaint` nor `outcrop` are perfect, but we continue to tune
|
||||
and improve them. If one doesn't work, try the other. You may also
|
||||
wish to experiment with other `img2img` arguments, such as `-C`, `-f`
|
||||
and `-s`.
|
||||
|
||||
|
||||
@ -1,14 +1,18 @@
|
||||
|
||||
---
|
||||
title: Upscale
|
||||
title: Postprocessing
|
||||
---
|
||||
|
||||
## Intro
|
||||
|
||||
The script provides the ability to restore faces and upscale. You can apply
|
||||
these operations at the time you generate the images, or at any time to a
|
||||
previously-generated PNG file, using the
|
||||
[!fix](#fixing-previously-generated-images) command.
|
||||
This extension provides the ability to restore faces and upscale
|
||||
images.
|
||||
|
||||
Face restoration and upscaling can be applied at the time you generate
|
||||
the images, or at any later time against a previously-generated PNG
|
||||
file, using the [!fix](#fixing-previously-generated-images)
|
||||
command. [Outpainting and outcropping](OUTPAINTING.md) can only be
|
||||
applied after the fact.
|
||||
|
||||
## Face Fixing
|
||||
|
||||
@ -158,9 +162,9 @@ situations when there is very little facial data to work with.
|
||||
## Fixing Previously-Generated Images
|
||||
|
||||
It is easy to apply face restoration and/or upscaling to any
|
||||
previously-generated file. Just use the syntax
|
||||
`!fix path/to/file.png <options>`. For example, to apply GFPGAN at strength 0.8
|
||||
and upscale 2X for a file named `./outputs/img-samples/000044.2945021133.png`,
|
||||
previously-generated file. Just use the syntax `!fix path/to/file.png
|
||||
<options>`. For example, to apply GFPGAN at strength 0.8 and upscale
|
||||
2X for a file named `./outputs/img-samples/000044.2945021133.png`,
|
||||
just run:
|
||||
|
||||
```
|
||||
Reference in New Issue
Block a user