Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor how postprocessors work

Browse Source 
- similar call structures for outpainting, outcropping and face restoration modules
- added documentation for outcropping
- post-processing steps now leave a provenance chain (of sorts) in the sd-metadata field:

~~~
scripts/sd-metadata.py outputs/img-samples/curly.942491079.upscale.png
outputs/img-samples/curly.942491079.upscale.png:
 {
    "model": "stable diffusion",
    "model_id": "stable-diffusion-1.4",
    "model_hash": "fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556",
    "app_id": "lstein/stable-diffusion",
    "app_version": "v1.15",
    "image": {
        "height": 512,
        "width": 512,
        "steps": 50,
        "cfg_scale": 7.5,
        "seed": 942491079,
        "prompt": [
            {
                "prompt": "pretty curly-haired redhead woman",
                "weight": 1.0
            }
        ],
        "postprocessing": [
            {
                "tool": "outcrop",
                "dream_command": "!fix \"test-pictures/curly.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -c top 64 right 64"
            },
            {
                "tool": "gfpgan",
                "dream_command": "!fix \"outputs/img-samples/curly.942491079.outcrop-02.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -G 0.8"
            },
            {
                "tool": "upscale",
                "dream_command": "!fix \"outputs/img-samples/curly.942491079.gfpgan.png\" -s 50 -S 942491079 -W 512 -H 512 -C 7.5 -A k_lms -U 4.0 0.75"
            }
        ],
        "sampler": "k_lms",
        "variations": [],
        "type": "txt2img"
    }
}
~~~
This commit is contained in:
Lincoln Stein
2022-10-03 16:53:12 -04:00
parent 609983ffa8
commit 4c482fe24a
13 changed files with 218 additions and 170 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/assets/outpainting/curly-outcrop.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 500 KiB

BIN
docs/assets/outpainting/curly-outpaint.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 422 KiB

BIN
docs/assets/outpainting/curly.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 428 KiB

BIN
docs/assets/outpainting/elven_princess.outpainted.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 572 KiB

BIN
docs/assets/outpainting/elven_princess.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 538 KiB

126
docs/features/OUTPAINTING.md
View File

@ -4,75 +4,95 @@ title: Outpainting
# :octicons-paintbrush-16: Outpainting
## Continous outpainting
## Outpainting and outcropping
This extension uses the inpainting code to extend an existing image to
any direction of "top", "right", "bottom" or "left". To use it you
need to provide an initial image with -I and an extension direction
with -D (direction). When extending using outpainting a higher img2img
strength value of 0.83 is the default.
Outpainting is a process by which the AI generates parts of the image
that are outside its original frame. It can be used to fix up images
in which the subject is off center, or when some detail (often the top
of someone's head!) is cut off.
The code is not foolproof. Sometimes it will do a good job extending
the image, and other times it will generate ghost images and other
artifacts. In addition, the code works best on images that were
generated by dream.py, because it will be able to recover the original
prompt that generated the file and "understand" what you are trying to
achieve.
InvokeAI supports two versions of outpainting, one called "outpaint"
and the other "outcrop." They work slightly differently and each has
its advantages and drawbacks.
### Basic Usage
### Outcrop
To illustrate, consider this image generated with the prompt "fantasy
portrait of eleven princess." It's nice, but rather annoying that the
top of the head has been cropped off.
The `outcrop` extension allows you to extend the image in 64 pixel
increments in any dimension. You can apply the module to any image
previously-generated by InvokeAI. Note that it will **not** work with
arbitrary photographs or Stable Diffusion images created by other
implementations.
![elven_princess](../assets/outpainting/elven_princess.png)
Consider this image:
We can fix that using the `!fix` command!
![curly_woman](../assets/outpainting/curly.png)
Pretty nice, but it's annoying that the top of her head is cut
off. She's also a bit off center. Let's fix that!
~~~~
dream> !fix my_images/elven_princess.png -D top 50
dream> !fix images/curly.png --outcrop top 64 right 64
~~~~
This is telling dream.py to open up a rectangle 50 pixels high at the
top of the image and outpaint into it. The result is:
This is saying to apply the `outcrop` extension by extending the top
of the image by 64 pixels, and the right of the image by the same
amount. You can use any combination of top|left|right|bottom, and
specify any number of pixels to extend. You can also abbreviate
`--outcrop` to `-c`.
![elven_princess.fixed](../assets/outpainting/elven_princess.outpainted.png)
The result looks like this:
Viola! You can similarly specify `bottom`, `left` or `right` to
outpaint into these margins.
![curly_woman_outcrop](../assets/outpainting/curly-outcrop.png)
There are some limitations to be aware of:
The new image is actually slightly larger than the original (576x576,
because 64 pixels were added to the top and right sides.)
1. You cannot change the size of the image rectangle. In the example,
notice that the whole image is shifted downwards by 50 pixels, rather
than the top being extended upwards.
A number of caveats:
2. Attempting to outpaint larger areas will frequently give rise to ugly
1. Although you can specify any pixel values, they will be rounded up
to the nearest multiple of 64. Smaller values are better. Larger
extensions are more likely to generate artefacts. However, if you wish
you can run the !fix command repeatedly to cautiously expand the
image.
2. The extension is stochastic, meaning that each time you run it
you'll get a slightly different result. You can run it repeatedly
until you get an image you like. Unfortunately `!fix` does not
currently respect the `-n` (`--iterations`) argument.
## Outpaint
The `outpaint` extension does the same thing, but with subtle
differences. Starting with the same image, here is how we would add an
additional 64 pixels to the top of the image:
~~~
dream> !fix images/curly.png --out_direction top 64
~~~
(you can abbreviate ``--out_direction` as `-D`.
The result is shown here:
![curly_woman_outpaint](../assets/outpainting/curly-outpaint.png)
Although the effect is similar, there are significant differences from
outcropping:
1. You can only specify one direction to extend at a time.
2. The image is **not** resized. Instead, the image is shifted by the specified
number of pixels. If you look carefully, you'll see that less of the lady's
torso is visible in the image.
3. Because the image dimensions remain the same, there's no rounding
to multiples of 64.
4. Attempting to outpaint larger areas will frequently give rise to ugly
ghosting effects.
3. For best results, try increasing the step number.
4. If you don't specify a pixel value in -D, it will default to half
5. For best results, try increasing the step number.
6. If you don't specify a pixel value in -D, it will default to half
of the whole image, which is likely not what you want.
You can do more with `!fix` including upscaling and facial
reconstruction of previously-generated images. See
[./UPSCALE.md#fixing-previously-generated-images] for the details.
### Advanced Usage
For more control over the outpaintihg process, you can provide the
`-D` option at image generation time. This allows you to apply all the
controls, including the ability to resize the image and apply face-fixing
and upscaling. For example:
~~~~
dream> man with cat on shoulder -I./images/man.png -D bottom 100 -W960 -H960 -fit
~~~~
Or even shorter, since the prompt is read from the metadata of the old image:
~~~~
dream> -I./images/man.png -D bottom 100 -W960 -H960 -fit -U2 -G1
~~~~
Neither `outpaint` nor `outcrop` are perfect, but we continue to tune
and improve them. If one doesn't work, try the other. You may also
wish to experiment with other `img2img` arguments, such as `-C`, `-f`
and `-s`.

20
docs/features/UPSCALE.md → docs/features/POSTPROCESS.md
View File

@ -1,14 +1,18 @@
---
title: Upscale
title: Postprocessing
---
## Intro
The script provides the ability to restore faces and upscale. You can apply
these operations at the time you generate the images, or at any time to a
previously-generated PNG file, using the
[!fix](#fixing-previously-generated-images) command.
This extension provides the ability to restore faces and upscale
images.
Face restoration and upscaling can be applied at the time you generate
the images, or at any later time against a previously-generated PNG
file, using the [!fix](#fixing-previously-generated-images)
command. [Outpainting and outcropping](OUTPAINTING.md) can only be
applied after the fact.
## Face Fixing
@ -158,9 +162,9 @@ situations when there is very little facial data to work with.
## Fixing Previously-Generated Images
It is easy to apply face restoration and/or upscaling to any
previously-generated file. Just use the syntax
`!fix path/to/file.png <options>`. For example, to apply GFPGAN at strength 0.8
and upscale 2X for a file named `./outputs/img-samples/000044.2945021133.png`,
previously-generated file. Just use the syntax `!fix path/to/file.png
<options>`. For example, to apply GFPGAN at strength 0.8 and upscale
2X for a file named `./outputs/img-samples/000044.2945021133.png`,
just run:
```