mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
6f9f848345
- When outcropping an image you can now add a `--new_prompt` option, to specify a new prompt to be used instead of the original one used to generate the image. - Similarly you can provide a new seed using `--seed` (or `-S`). A seed of zero will pick one randomly. - This PR also fixes the crash that happened when trying to outcrop an image that does not contain InvokeAI metadata.
172 lines
6.9 KiB
Markdown
172 lines
6.9 KiB
Markdown
---
|
|
title: Outpainting
|
|
---
|
|
|
|
# :octicons-paintbrush-16: Outpainting
|
|
|
|
## Outpainting and outcropping
|
|
|
|
Outpainting is a process by which the AI generates parts of the image that are
|
|
outside its original frame. It can be used to fix up images in which the subject
|
|
is off center, or when some detail (often the top of someone's head!) is cut
|
|
off.
|
|
|
|
InvokeAI supports two versions of outpainting, one called "outpaint" and the
|
|
other "outcrop." They work slightly differently and each has its advantages and
|
|
drawbacks.
|
|
|
|
### Outpainting
|
|
|
|
Outpainting is the same as inpainting, except that the painting occurs in the
|
|
regions outside of the original image. To outpaint using the `invoke.py` command
|
|
line script, prepare an image in which the borders to be extended are pure
|
|
black. Add an alpha channel (if there isn't one already), and make the borders
|
|
completely transparent and the interior completely opaque. If you wish to modify
|
|
the interior as well, you may create transparent holes in the transparency
|
|
layer, which `img2img` will paint into as usual.
|
|
|
|
Pass the image as the argument to the `-I` switch as you would for regular
|
|
inpainting:
|
|
|
|
```bash
|
|
invoke> a stream by a river -I /path/to/transparent_img.png
|
|
```
|
|
|
|
You'll likely be delighted by the results.
|
|
|
|
### Tips
|
|
|
|
1. Do not try to expand the image too much at once. Generally it is best to
|
|
expand the margins in 64-pixel increments. 128 pixels often works, but your
|
|
mileage may vary depending on the nature of the image you are trying to
|
|
outpaint into.
|
|
|
|
2. There are a series of switches that can be used to adjust how the inpainting
|
|
algorithm operates. In particular, you can use these to minimize the seam
|
|
that sometimes appears between the original image and the extended part.
|
|
These switches are:
|
|
|
|
| switch | default | description |
|
|
| -------------------------- | ------- | ---------------------------------------------------------------------- |
|
|
| `--seam_size SEAM_SIZE ` | `0` | Size of the mask around the seam between original and outpainted image |
|
|
| `--seam_blur SEAM_BLUR` | `0` | The amount to blur the seam inwards |
|
|
| `--seam_strength STRENGTH` | `0.7` | The img2img strength to use when filling the seam |
|
|
| `--seam_steps SEAM_STEPS` | `10` | The number of steps to use to fill the seam. |
|
|
| `--tile_size TILE_SIZE` | `32` | The tile size to use for filling outpaint areas |
|
|
|
|
### Outcrop
|
|
|
|
The `outcrop` extension gives you a convenient `!fix` postprocessing command
|
|
that allows you to extend a previously-generated image in 64 pixel increments in
|
|
any direction. You can apply the module to any image previously-generated by
|
|
InvokeAI. Note that it works with arbitrary PNG photographs, but not currently
|
|
with JPG or other formats. Outcropping is particularly effective when combined
|
|
with the
|
|
[runwayML custom inpainting model](INPAINTING.md#using-the-runwayml-inpainting-model).
|
|
|
|
Consider this image:
|
|
|
|
<figure markdown>
|
|
|
|
</figure>
|
|
|
|
Pretty nice, but it's annoying that the top of her head is cut off. She's also a
|
|
bit off center. Let's fix that!
|
|
|
|
```bash
|
|
invoke> !fix images/curly.png --outcrop top 128 right 64 bottom 64
|
|
```
|
|
|
|
This is saying to apply the `outcrop` extension by extending the top of the
|
|
image by 128 pixels, and the right and bottom of the image by 64 pixels. You can
|
|
use any combination of top|left|right|bottom, and specify any number of pixels
|
|
to extend. You can also abbreviate `--outcrop` to `-c`.
|
|
|
|
The result looks like this:
|
|
|
|
<figure markdown>
|
|
|
|
</figure>
|
|
|
|
The new image is larger than the original (576x704) because 64 pixels were added
|
|
to the top and right sides. You will need enough VRAM to process an image of
|
|
this size.
|
|
|
|
#### Outcropping non-InvokeAI images
|
|
|
|
You can outcrop an arbitrary image that was not generated by InvokeAI,
|
|
but your results will vary. The `inpainting-1.5` model is highly
|
|
recommended, but if not feasible, then you may be able to improve the
|
|
output by conditioning the outcropping with a text prompt that
|
|
describes the scene using the `--new_prompt` argument:
|
|
|
|
```bash
|
|
invoke> !fix images/vacation.png --outcrop top 128 --new_prompt "family vacation"
|
|
```
|
|
|
|
You may also provide a different seed for outcropping to use by passing
|
|
`-S<seed>`. A seed of "0" will generate a new random seed.
|
|
|
|
A number of caveats:
|
|
|
|
1. Although you can specify any pixel values, they will be rounded up to the
|
|
nearest multiple of 64. Smaller values are better. Larger extensions are more
|
|
likely to generate artefacts. However, if you wish you can run the !fix
|
|
command repeatedly to cautiously expand the image.
|
|
|
|
2. The extension is stochastic, meaning that each time you run it you'll get a
|
|
slightly different result. You can run it repeatedly until you get an image
|
|
you like. Unfortunately `!fix` does not currently respect the `-n`
|
|
(`--iterations`) argument.
|
|
|
|
3. Your results will be _much_ better if you use the `inpaint-1.5` model
|
|
released by runwayML and installed by default by `scripts/preload_models.py`.
|
|
This model was trained specifically to harmoniously fill in image gaps. The
|
|
standard model will work as well, but you may notice color discontinuities at
|
|
the border.
|
|
|
|
4. When using the `inpaint-1.5` model, you may notice subtle changes to the area
|
|
outside the masked region. This is because the model performs an
|
|
encoding/decoding on the image as a whole. This does not occur with the
|
|
standard model.
|
|
|
|
## Outpaint
|
|
|
|
The `outpaint` extension does the same thing, but with subtle differences.
|
|
Starting with the same image, here is how we would add an additional 64 pixels
|
|
to the top of the image:
|
|
|
|
```bash
|
|
invoke> !fix images/curly.png --out_direction top 64
|
|
```
|
|
|
|
(you can abbreviate `--out_direction` as `-D`.
|
|
|
|
The result is shown here:
|
|
|
|
<figure markdown>
|
|
|
|
</figure>
|
|
|
|
Although the effect is similar, there are significant differences from
|
|
outcropping:
|
|
|
|
- You can only specify one direction to extend at a time.
|
|
- The image is **not** resized. Instead, the image is shifted by the specified
|
|
number of pixels. If you look carefully, you'll see that less of the lady's
|
|
torso is visible in the image.
|
|
- Because the image dimensions remain the same, there's no rounding to multiples
|
|
of 64.
|
|
- Attempting to outpaint larger areas will frequently give rise to ugly ghosting
|
|
effects.
|
|
- For best results, try increasing the step number.
|
|
- If you don't specify a pixel value in `-D`, it will default to half of the
|
|
whole image, which is likely not what you want.
|
|
|
|
!!! tip
|
|
|
|
Neither `outpaint` nor `outcrop` are perfect, but we continue to tune
|
|
and improve them. If one doesn't work, try the other. You may also
|
|
wish to experiment with other `img2img` arguments, such as `-C`, `-f`
|
|
and `-s`.
|