2022-09-21 06:53:08 +00:00
|
|
|
---
|
|
|
|
title: Outpainting
|
|
|
|
---
|
|
|
|
|
|
|
|
# :octicons-paintbrush-16: Outpainting
|
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
## Outpainting and outcropping
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
Outpainting is a process by which the AI generates parts of the image
|
|
|
|
that are outside its original frame. It can be used to fix up images
|
|
|
|
in which the subject is off center, or when some detail (often the top
|
|
|
|
of someone's head!) is cut off.
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
InvokeAI supports two versions of outpainting, one called "outpaint"
|
|
|
|
and the other "outcrop." They work slightly differently and each has
|
|
|
|
its advantages and drawbacks.
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-27 22:42:51 +00:00
|
|
|
### Outpainting
|
|
|
|
|
|
|
|
Outpainting is the same as inpainting, except that the painting occurs
|
|
|
|
in the regions outside of the original image. To outpaint using the
|
|
|
|
`invoke.py` command line script, prepare an image in which the borders
|
|
|
|
to be extended are pure black. Add an alpha channel (if there isn't one
|
|
|
|
already), and make the borders completely transparent and the interior
|
|
|
|
completely opaque. If you wish to modify the interior as well, you may
|
|
|
|
create transparent holes in the transparency layer, which `img2img` will
|
|
|
|
paint into as usual.
|
|
|
|
|
|
|
|
Pass the image as the argument to the `-I` switch as you would for
|
|
|
|
regular inpainting:
|
|
|
|
|
|
|
|
invoke> a stream by a river -I /path/to/transparent_img.png
|
|
|
|
|
|
|
|
You'll likely be delighted by the results.
|
|
|
|
|
|
|
|
### Tips
|
|
|
|
|
|
|
|
1. Do not try to expand the image too much at once. Generally it is best
|
|
|
|
to expand the margins in 64-pixel increments. 128 pixels often works,
|
|
|
|
but your mileage may vary depending on the nature of the image you are
|
|
|
|
trying to outpaint into.
|
|
|
|
|
|
|
|
2. There are a series of switches that can be used to adjust how the
|
|
|
|
inpainting algorithm operates. In particular, you can use these to
|
|
|
|
minimize the seam that sometimes appears between the original image
|
|
|
|
and the extended part. These switches are:
|
|
|
|
|
|
|
|
--seam_size SEAM_SIZE Size of the mask around the seam between original and outpainted image (0)
|
|
|
|
--seam_blur SEAM_BLUR The amount to blur the seam inwards (0)
|
|
|
|
--seam_strength STRENGTH The img2img strength to use when filling the seam (0.7)
|
|
|
|
--seam_steps SEAM_STEPS The number of steps to use to fill the seam. (10)
|
|
|
|
--tile_size TILE_SIZE The tile size to use for filling outpaint areas (32)
|
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
### Outcrop
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-27 22:42:51 +00:00
|
|
|
The `outcrop` extension gives you a convenient `!fix` postprocessing
|
|
|
|
command that allows you to extend a previously-generated image in 64
|
|
|
|
pixel increments in any direction. You can apply the module to any
|
|
|
|
image previously-generated by InvokeAI. Note that it works with
|
|
|
|
arbitrary PNG photographs, but not currently with JPG or other
|
|
|
|
formats. Outcropping is particularly effective when combined with the
|
|
|
|
[runwayML custom inpainting
|
|
|
|
model](INPAINTING.md#using-the-runwayml-inpainting-model).
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
Consider this image:
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-17 22:40:54 +00:00
|
|
|
<figure markdown>
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
![curly_woman](../assets/outpainting/curly.png)
|
2022-10-17 22:40:54 +00:00
|
|
|
</figure>
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
|
|
|
|
Pretty nice, but it's annoying that the top of her head is cut
|
|
|
|
off. She's also a bit off center. Let's fix that!
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-11 20:10:35 +00:00
|
|
|
```bash
|
2022-11-03 19:44:06 +00:00
|
|
|
invoke> !fix images/curly.png --outcrop top 128 right 64 bottom 64
|
2022-10-11 20:10:35 +00:00
|
|
|
```
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
This is saying to apply the `outcrop` extension by extending the top
|
2022-11-03 19:44:06 +00:00
|
|
|
of the image by 128 pixels, and the right and bottom of the image by
|
|
|
|
64 pixels. You can use any combination of top|left|right|bottom, and
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
specify any number of pixels to extend. You can also abbreviate
|
|
|
|
`--outcrop` to `-c`.
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
The result looks like this:
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-17 22:40:54 +00:00
|
|
|
<figure markdown>
|
2022-11-03 19:44:06 +00:00
|
|
|
![curly_woman_outcrop](../assets/outpainting/curly-outcrop-2.png)
|
2022-10-17 22:40:54 +00:00
|
|
|
</figure>
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-11-03 19:44:06 +00:00
|
|
|
The new image is larger than the original (576x704)
|
|
|
|
because 64 pixels were added to the top and right sides. You will
|
|
|
|
need enough VRAM to process an image of this size.
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
A number of caveats:
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
1. Although you can specify any pixel values, they will be rounded up
|
|
|
|
to the nearest multiple of 64. Smaller values are better. Larger
|
|
|
|
extensions are more likely to generate artefacts. However, if you wish
|
|
|
|
you can run the !fix command repeatedly to cautiously expand the
|
|
|
|
image.
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
2. The extension is stochastic, meaning that each time you run it
|
|
|
|
you'll get a slightly different result. You can run it repeatedly
|
|
|
|
until you get an image you like. Unfortunately `!fix` does not
|
|
|
|
currently respect the `-n` (`--iterations`) argument.
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-11-03 19:44:06 +00:00
|
|
|
3. Your results will be _much_ better if you use the `inpaint-1.5`
|
|
|
|
model released by runwayML and installed by default by
|
|
|
|
`scripts/preload_models.py`. This model was trained specifically to
|
|
|
|
harmoniously fill in image gaps. The standard model will work as well,
|
|
|
|
but you may notice color discontinuities at the border.
|
|
|
|
|
|
|
|
4. When using the `inpaint-1.5` model, you may notice subtle changes
|
|
|
|
to the area within the original image. This is because the model
|
|
|
|
performs an encoding/decoding on the image as a whole. This does not
|
|
|
|
occur with the standard model.
|
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
## Outpaint
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
The `outpaint` extension does the same thing, but with subtle
|
|
|
|
differences. Starting with the same image, here is how we would add an
|
|
|
|
additional 64 pixels to the top of the image:
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-11 20:10:35 +00:00
|
|
|
```bash
|
2022-10-08 13:32:06 +00:00
|
|
|
invoke> !fix images/curly.png --out_direction top 64
|
2022-10-11 20:10:35 +00:00
|
|
|
```
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-09 22:46:23 +00:00
|
|
|
(you can abbreviate `--out_direction` as `-D`.
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
The result is shown here:
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-17 22:40:54 +00:00
|
|
|
<figure markdown>
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
![curly_woman_outpaint](../assets/outpainting/curly-outpaint.png)
|
2022-10-17 22:40:54 +00:00
|
|
|
</figure>
|
2022-09-21 06:53:08 +00:00
|
|
|
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
Although the effect is similar, there are significant differences from
|
|
|
|
outcropping:
|
|
|
|
|
2022-10-11 20:10:35 +00:00
|
|
|
- You can only specify one direction to extend at a time.
|
|
|
|
- The image is **not** resized. Instead, the image is shifted by the specified
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
number of pixels. If you look carefully, you'll see that less of the lady's
|
|
|
|
torso is visible in the image.
|
2022-10-11 20:10:35 +00:00
|
|
|
- Because the image dimensions remain the same, there's no rounding
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
to multiples of 64.
|
2022-10-11 20:10:35 +00:00
|
|
|
- Attempting to outpaint larger areas will frequently give rise to ugly
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
ghosting effects.
|
2022-10-11 20:10:35 +00:00
|
|
|
- For best results, try increasing the step number.
|
|
|
|
- If you don't specify a pixel value in `-D`, it will default to half
|
refactor how postprocessors work
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:
~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
{
"model": "stable diffusion",
"model_id": "stable-diffusion-1.4",
"model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
"app_id": "lstein/stable-diffusion",
"app_version": "v1.15",
"image": {
"height": 512,
"width": 512,
"steps": 50,
"cfg_scale": 7.5,
"seed": 942491079,
"prompt": [
{
"prompt": "pretty curly-haired redhead woman",
"weight": 1.0
}
],
"postprocessing": [
{
"tool": "outcrop",
"dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
},
{
"tool": "gfpgan",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
},
{
"tool": "upscale",
"dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
}
],
"sampler": "k_lms",
"variations": [],
"type": "txt2img"
}
}
~~~
2022-10-03 20:53:12 +00:00
|
|
|
of the whole image, which is likely not what you want.
|
|
|
|
|
2022-10-11 20:10:35 +00:00
|
|
|
!!! tip
|
2022-09-21 06:53:08 +00:00
|
|
|
|
2022-10-11 20:10:35 +00:00
|
|
|
Neither `outpaint` nor `outcrop` are perfect, but we continue to tune
|
|
|
|
and improve them. If one doesn't work, try the other. You may also
|
|
|
|
wish to experiment with other `img2img` arguments, such as `-C`, `-f`
|
|
|
|
and `-s`.
|