mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
Upgrade GFPGAN to Version 1.4
This commit is contained in:
blessedcoolant
committed by
Lincoln Stein
Lincoln Stein
parent
d117d23469
commit
53b4c3cc60
@ -8,20 +8,23 @@ hide:
|
||||
|
||||
## **Interactive Command Line Interface**
|
||||
|
||||
The `dream.py` script, located in `scripts/dream.py`, provides an interactive interface to image
|
||||
generation similar to the "dream mothership" bot that Stable AI provided on its Discord server.
|
||||
The `dream.py` script, located in `scripts/dream.py`, provides an interactive
|
||||
interface to image generation similar to the "dream mothership" bot that Stable
|
||||
AI provided on its Discord server.
|
||||
|
||||
Unlike the `txt2img.py` and `img2img.py` scripts provided in the original
|
||||
[CompVis/stable-diffusion](https://github.com/CompVis/stable-diffusion) source code repository, the
|
||||
time-consuming initialization of the AI model initialization only happens once. After that image
|
||||
generation from the command-line interface is very fast.
|
||||
[CompVis/stable-diffusion](https://github.com/CompVis/stable-diffusion) source
|
||||
code repository, the time-consuming initialization of the AI model
|
||||
initialization only happens once. After that image generation from the
|
||||
command-line interface is very fast.
|
||||
|
||||
The script uses the readline library to allow for in-line editing, command history (++up++ and
|
||||
++down++), autocompletion, and more. To help keep track of which prompts generated which images, the
|
||||
script writes a log file of image names and prompts to the selected output directory.
|
||||
The script uses the readline library to allow for in-line editing, command
|
||||
history (++up++ and ++down++), autocompletion, and more. To help keep track of
|
||||
which prompts generated which images, the script writes a log file of image
|
||||
names and prompts to the selected output directory.
|
||||
|
||||
In addition, as of version 1.02, it also writes the prompt into the PNG file's metadata where it can
|
||||
be retrieved using `scripts/images2prompt.py`
|
||||
In addition, as of version 1.02, it also writes the prompt into the PNG file's
|
||||
metadata where it can be retrieved using `scripts/images2prompt.py`
|
||||
|
||||
The script is confirmed to work on Linux, Windows and Mac systems.
|
||||
|
||||
@ -56,21 +59,24 @@ dream> q
|
||||
|
||||
|
||||
|
||||
The `dream>` prompt's arguments are pretty much identical to those used in the Discord bot, except
|
||||
you don't need to type "!dream" (it doesn't hurt if you do). A significant change is that creation
|
||||
of individual images is now the default unless `--grid` (`-g`) is given. A full list is given in
|
||||
The `dream>` prompt's arguments are pretty much identical to those used in the
|
||||
Discord bot, except you don't need to type "!dream" (it doesn't hurt if you do).
|
||||
A significant change is that creation of individual images is now the default
|
||||
unless `--grid` (`-g`) is given. A full list is given in
|
||||
[List of prompt arguments](#list-of-prompt-arguments).
|
||||
|
||||
## Arguments
|
||||
|
||||
The script itself also recognizes a series of command-line switches that will change important
|
||||
global defaults, such as the directory for image outputs and the location of the model weight files.
|
||||
The script itself also recognizes a series of command-line switches that will
|
||||
change important global defaults, such as the directory for image outputs and
|
||||
the location of the model weight files.
|
||||
|
||||
### List of arguments recognized at the command line
|
||||
|
||||
These command-line arguments can be passed to `dream.py` when you first run it from the Windows, Mac
|
||||
or Linux command line. Some set defaults that can be overridden on a per-prompt basis (see [List of
|
||||
prompt arguments] (#list-of-prompt-arguments). Others
|
||||
These command-line arguments can be passed to `dream.py` when you first run it
|
||||
from the Windows, Mac or Linux command line. Some set defaults that can be
|
||||
overridden on a per-prompt basis (see [List of prompt arguments]
|
||||
(#list-of-prompt-arguments). Others
|
||||
|
||||
| Argument <img width="240" align="right"/> | Shortcut <img width="100" align="right"/> | Default <img width="320" align="right"/> | Description |
|
||||
| ----------------------------------------- | ----------------------------------------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
|
||||
@ -90,7 +96,7 @@ prompt arguments] (#list-of-prompt-arguments). Others
|
||||
| `--seamless` | | `False` | Create interesting effects by tiling elements of the image. |
|
||||
| `--embedding_path <path>` | | `None` | Path to pre-trained embedding manager checkpoints, for custom models |
|
||||
| `--gfpgan_dir` | | `src/gfpgan` | Path to where GFPGAN is installed. |
|
||||
| `--gfpgan_model_path` | | `experiments/pretrained_models/GFPGANv1.3.pth` | Path to GFPGAN model file, relative to `--gfpgan_dir`. |
|
||||
| `--gfpgan_model_path` | | `experiments/pretrained_models/GFPGANv1.4.pth` | Path to GFPGAN model file, relative to `--gfpgan_dir`. |
|
||||
| `--device <device>` | `-d<device>` | `torch.cuda.current_device()` | Device to run SD on, e.g. "cuda:0" |
|
||||
|
||||
#### deprecated
|
||||
@ -115,9 +121,10 @@ These arguments are deprecated but still work:
|
||||
|
||||
### List of prompt arguments
|
||||
|
||||
After the `dream.py` script initializes, it will present you with a **`dream>`** prompt. Here you
|
||||
can enter information to generate images from text (txt2img), to embellish an existing image or
|
||||
sketch (img2img), or to selectively alter chosen regions of the image (inpainting).
|
||||
After the `dream.py` script initializes, it will present you with a **`dream>`**
|
||||
prompt. Here you can enter information to generate images from text (txt2img),
|
||||
to embellish an existing image or sketch (img2img), or to selectively alter
|
||||
chosen regions of the image (inpainting).
|
||||
|
||||
#### txt2img
|
||||
|
||||
@ -171,12 +178,13 @@ Those are the `dream` commands that apply to txt2img:
|
||||
than 640x480. Otherwise the image size will be identical to the provided photo and you may run out
|
||||
of memory if it is large.
|
||||
|
||||
Repeated chaining of img2img on an image can result in significant color shifts in the output,
|
||||
especially if run with lower strength. Color correction can be run against a reference image to fix
|
||||
this issue. Use the original input image to the chain as the the reference image for each step in
|
||||
the chain.
|
||||
Repeated chaining of img2img on an image can result in significant color shifts
|
||||
in the output, especially if run with lower strength. Color correction can be
|
||||
run against a reference image to fix this issue. Use the original input image to
|
||||
the chain as the the reference image for each step in the chain.
|
||||
|
||||
In addition to the command-line options recognized by txt2img, img2img accepts additional options:
|
||||
In addition to the command-line options recognized by txt2img, img2img accepts
|
||||
additional options:
|
||||
|
||||
| Argument <img width="160" align="right"/> | Shortcut | Default | Description |
|
||||
| ----------------------------------------- | ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
@ -198,8 +206,8 @@ In addition to the command-line options recognized by txt2img, img2img accepts a
|
||||
the areas to overpaint made transparent, but you must be careful not to destroy the pixels
|
||||
underneath when you create the transparent areas. See [Inpainting](./INPAINTING.md) for details.
|
||||
|
||||
Inpainting accepts all the arguments used for txt2img and img2img, as well as the `--mask` (`-M`)
|
||||
argument:
|
||||
Inpainting accepts all the arguments used for txt2img and img2img, as well as
|
||||
the `--mask` (`-M`) argument:
|
||||
|
||||
| Argument <img width="100" align="right"/> | Shortcut | Default | Description |
|
||||
| ----------------------------------------- | ---------- | ------- | ------------------------------------------------------------------------------------------------ |
|
||||
@ -207,37 +215,42 @@ argument:
|
||||
|
||||
## Command-line editing and completion
|
||||
|
||||
If you are on a Macintosh or Linux machine, the command-line offers convenient history tracking,
|
||||
editing, and command completion.
|
||||
If you are on a Macintosh or Linux machine, the command-line offers convenient
|
||||
history tracking, editing, and command completion.
|
||||
|
||||
- To scroll through previous commands and potentially edit/reuse them, use the ++up++ and ++down++
|
||||
cursor keys.
|
||||
- To edit the current command, use the ++left++ and ++right++ cursor keys to position the cursor,
|
||||
and then ++backspace++, ++delete++ or ++insert++ characters.
|
||||
- To move to the very beginning of the command, type ++ctrl+a++ (or ++command+a++ on the Mac)
|
||||
- To scroll through previous commands and potentially edit/reuse them, use the
|
||||
++up++ and ++down++ cursor keys.
|
||||
- To edit the current command, use the ++left++ and ++right++ cursor keys to
|
||||
position the cursor, and then ++backspace++, ++delete++ or ++insert++
|
||||
characters.
|
||||
- To move to the very beginning of the command, type ++ctrl+a++ (or
|
||||
++command+a++ on the Mac)
|
||||
- To move to the end of the command, type ++ctrl+e++.
|
||||
- To cut a section of the command, position the cursor where you want to start cutting and type
|
||||
++ctrl+k++.
|
||||
- To paste a cut section back in, position the cursor where you want to paste, and type ++ctrl+y++
|
||||
- To cut a section of the command, position the cursor where you want to start
|
||||
cutting and type ++ctrl+k++.
|
||||
- To paste a cut section back in, position the cursor where you want to paste,
|
||||
and type ++ctrl+y++
|
||||
|
||||
Windows users can get similar, but more limited, functionality if they launch `dream.py` with the
|
||||
"winpty" program:
|
||||
Windows users can get similar, but more limited, functionality if they launch
|
||||
`dream.py` with the "winpty" program:
|
||||
|
||||
```batch
|
||||
winpty python scripts\dream.py
|
||||
```
|
||||
|
||||
On the Mac and Linux platforms, when you exit `dream.py`, the last 1000 lines of your command-line
|
||||
history will be saved. When you restart `dream.py`, you can access the saved history using the
|
||||
++up++ key.
|
||||
On the Mac and Linux platforms, when you exit `dream.py`, the last 1000 lines of
|
||||
your command-line history will be saved. When you restart `dream.py`, you can
|
||||
access the saved history using the ++up++ key.
|
||||
|
||||
In addition, limited command-line completion is installed. In various contexts, you can start typing
|
||||
your command and press tab. A list of potential completions will be presented to you. You can then
|
||||
type a little more, hit tab again, and eventually autocomplete what you want.
|
||||
In addition, limited command-line completion is installed. In various contexts,
|
||||
you can start typing your command and press tab. A list of potential completions
|
||||
will be presented to you. You can then type a little more, hit tab again, and
|
||||
eventually autocomplete what you want.
|
||||
|
||||
When specifying file paths using the one-letter shortcuts, the CLI will attempt to complete
|
||||
pathnames for you. This is most handy for the `-I` (init image) and `-M` (init mask) paths. To
|
||||
initiate completion, start the path with a slash `/` or `./`, for example:
|
||||
When specifying file paths using the one-letter shortcuts, the CLI will attempt
|
||||
to complete pathnames for you. This is most handy for the `-I` (init image) and
|
||||
`-M` (init mask) paths. To initiate completion, start the path with a slash `/`
|
||||
or `./`, for example:
|
||||
|
||||
```bash
|
||||
dream> "zebra with a mustache" -I./test-pictures<TAB>
|
||||
|
||||
@ -4,37 +4,42 @@ title: Upscale
|
||||
|
||||
## Intro
|
||||
|
||||
The script provides the ability to restore faces and upscale. You can apply these operations
|
||||
at the time you generate the images, or at any time to a previously-generated PNG file, using
|
||||
the [!fix](#fixing-previously-generated-images) command.
|
||||
The script provides the ability to restore faces and upscale. You can apply
|
||||
these operations at the time you generate the images, or at any time to a
|
||||
previously-generated PNG file, using the
|
||||
[!fix](#fixing-previously-generated-images) command.
|
||||
|
||||
## Face Fixing
|
||||
|
||||
The default face restoration module is GFPGAN. The default upscale is Real-ESRGAN. For an alternative
|
||||
face restoration module, see [CodeFormer Support] below.
|
||||
The default face restoration module is GFPGAN. The default upscale is
|
||||
Real-ESRGAN. For an alternative face restoration module, see [CodeFormer
|
||||
Support] below.
|
||||
|
||||
As of version 1.14, environment.yaml will install the Real-ESRGAN package into the standard install
|
||||
location for python packages, and will put GFPGAN into a subdirectory of "src" in the
|
||||
stable-diffusion directory. (The reason for this is that the standard GFPGAN distribution has a
|
||||
minor bug that adversely affects image color.) Upscaling with Real-ESRGAN should "just work" without
|
||||
further intervention. Simply pass the --upscale (-U) option on the dream> command line, or indicate
|
||||
the desired scale on the popup in the Web GUI.
|
||||
As of version 1.14, environment.yaml will install the Real-ESRGAN package into
|
||||
the standard install location for python packages, and will put GFPGAN into a
|
||||
subdirectory of "src" in the stable-diffusion directory. (The reason for this is
|
||||
that the standard GFPGAN distribution has a minor bug that adversely affects
|
||||
image color.) Upscaling with Real-ESRGAN should "just work" without further
|
||||
intervention. Simply pass the --upscale (-U) option on the dream> command line,
|
||||
or indicate the desired scale on the popup in the Web GUI.
|
||||
|
||||
For **GFPGAN** to work, there is one additional step needed. You will need to download and copy the
|
||||
GFPGAN [models file](https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth)
|
||||
into **src/gfpgan/experiments/pretrained_models**. On Mac and Linux systems, here's how you'd do it
|
||||
using **wget**:
|
||||
For **GFPGAN** to work, there is one additional step needed. You will need to
|
||||
download and copy the GFPGAN
|
||||
[models file](https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth)
|
||||
into **src/gfpgan/experiments/pretrained_models**. On Mac and Linux systems,
|
||||
here's how you'd do it using **wget**:
|
||||
|
||||
```bash
|
||||
> wget https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth src/gfpgan/experiments/pretrained_models/
|
||||
> wget https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth src/gfpgan/experiments/pretrained_models/
|
||||
```
|
||||
|
||||
Make sure that you're in the stable-diffusion directory when you do this.
|
||||
|
||||
Alternatively, if you have GFPGAN installed elsewhere, or if you are using an earlier version of
|
||||
this package which asked you to install GFPGAN in a sibling directory, you may use the
|
||||
`--gfpgan_dir` argument with `dream.py` to set a custom path to your GFPGAN directory. _There are
|
||||
other GFPGAN related boot arguments if you wish to customize further._
|
||||
Alternatively, if you have GFPGAN installed elsewhere, or if you are using an
|
||||
earlier version of this package which asked you to install GFPGAN in a sibling
|
||||
directory, you may use the `--gfpgan_dir` argument with `dream.py` to set a
|
||||
custom path to your GFPGAN directory. _There are other GFPGAN related boot
|
||||
arguments if you wish to customize further._
|
||||
|
||||
!!! warning "Internet connection needed"
|
||||
|
||||
@ -52,13 +57,14 @@ You will now have access to two new prompt arguments.
|
||||
|
||||
`-U : <upscaling_factor> <upscaling_strength>`
|
||||
|
||||
The upscaling prompt argument takes two values. The first value is a scaling factor and should be
|
||||
set to either `2` or `4` only. This will either scale the image 2x or 4x respectively using
|
||||
different models.
|
||||
The upscaling prompt argument takes two values. The first value is a scaling
|
||||
factor and should be set to either `2` or `4` only. This will either scale the
|
||||
image 2x or 4x respectively using different models.
|
||||
|
||||
You can set the scaling stength between `0` and `1.0` to control intensity of the of the scaling.
|
||||
This is handy because AI upscalers generally tend to smooth out texture details. If you wish to
|
||||
retain some of those for natural looking results, we recommend using values between `0.5 to 0.8`.
|
||||
You can set the scaling stength between `0` and `1.0` to control intensity of
|
||||
the of the scaling. This is handy because AI upscalers generally tend to smooth
|
||||
out texture details. If you wish to retain some of those for natural looking
|
||||
results, we recommend using values between `0.5 to 0.8`.
|
||||
|
||||
If you do not explicitly specify an upscaling_strength, it will default to 0.75.
|
||||
|
||||
@ -66,18 +72,19 @@ If you do not explicitly specify an upscaling_strength, it will default to 0.75.
|
||||
|
||||
`-G : <gfpgan_strength>`
|
||||
|
||||
This prompt argument controls the strength of the face restoration that is being applied. Similar to
|
||||
upscaling, values between `0.5 to 0.8` are recommended.
|
||||
This prompt argument controls the strength of the face restoration that is being
|
||||
applied. Similar to upscaling, values between `0.5 to 0.8` are recommended.
|
||||
|
||||
You can use either one or both without any conflicts. In cases where you use both, the image will be
|
||||
first upscaled and then the face restoration process will be executed to ensure you get the highest
|
||||
quality facial features.
|
||||
You can use either one or both without any conflicts. In cases where you use
|
||||
both, the image will be first upscaled and then the face restoration process
|
||||
will be executed to ensure you get the highest quality facial features.
|
||||
|
||||
`--save_orig`
|
||||
|
||||
When you use either `-U` or `-G`, the final result you get is upscaled or face modified. If you want
|
||||
to save the original Stable Diffusion generation, you can use the `-save_orig` prompt argument to
|
||||
save the original unaffected version too.
|
||||
When you use either `-U` or `-G`, the final result you get is upscaled or face
|
||||
modified. If you want to save the original Stable Diffusion generation, you can
|
||||
use the `-save_orig` prompt argument to save the original unaffected version
|
||||
too.
|
||||
|
||||
### Example Usage
|
||||
|
||||
@ -102,60 +109,69 @@ dream> a man wearing a pineapple hat -I path/to/your/file.png -U 2 0.5 -G 0.6
|
||||
process is complete. While the image generation is taking place, you will still be able to preview
|
||||
the base images.
|
||||
|
||||
If you wish to stop during the image generation but want to upscale or face restore a particular
|
||||
generated image, pass it again with the same prompt and generated seed along with the `-U` and `-G`
|
||||
prompt arguments to perform those actions.
|
||||
If you wish to stop during the image generation but want to upscale or face
|
||||
restore a particular generated image, pass it again with the same prompt and
|
||||
generated seed along with the `-U` and `-G` prompt arguments to perform those
|
||||
actions.
|
||||
|
||||
## CodeFormer Support
|
||||
|
||||
This repo also allows you to perform face restoration using
|
||||
[CodeFormer](https://github.com/sczhou/CodeFormer).
|
||||
|
||||
In order to setup CodeFormer to work, you need to download the models like with GFPGAN. You can do
|
||||
this either by running `preload_models.py` or by manually downloading the
|
||||
[model file](https://github.com/sczhou/CodeFormer/releases/download/v0.1.0/codeformer.pth) and
|
||||
saving it to `ldm/restoration/codeformer/weights` folder.
|
||||
In order to setup CodeFormer to work, you need to download the models like with
|
||||
GFPGAN. You can do this either by running `preload_models.py` or by manually
|
||||
downloading the
|
||||
[model file](https://github.com/sczhou/CodeFormer/releases/download/v0.1.0/codeformer.pth)
|
||||
and saving it to `ldm/restoration/codeformer/weights` folder.
|
||||
|
||||
You can use `-ft` prompt argument to swap between CodeFormer and the default GFPGAN. The above
|
||||
mentioned `-G` prompt argument will allow you to control the strength of the restoration effect.
|
||||
You can use `-ft` prompt argument to swap between CodeFormer and the default
|
||||
GFPGAN. The above mentioned `-G` prompt argument will allow you to control the
|
||||
strength of the restoration effect.
|
||||
|
||||
### Usage:
|
||||
|
||||
The following command will perform face restoration with CodeFormer instead of the default gfpgan.
|
||||
The following command will perform face restoration with CodeFormer instead of
|
||||
the default gfpgan.
|
||||
|
||||
`<prompt> -G 0.8 -ft codeformer`
|
||||
|
||||
### Other Options:
|
||||
|
||||
- `-cf` - cf or CodeFormer Fidelity takes values between `0` and `1`. 0 produces high quality
|
||||
results but low accuracy and 1 produces lower quality results but higher accuacy to your original
|
||||
face.
|
||||
- `-cf` - cf or CodeFormer Fidelity takes values between `0` and `1`. 0 produces
|
||||
high quality results but low accuracy and 1 produces lower quality results but
|
||||
higher accuacy to your original face.
|
||||
|
||||
The following command will perform face restoration with CodeFormer. CodeFormer will output a result
|
||||
that is closely matching to the input face.
|
||||
The following command will perform face restoration with CodeFormer. CodeFormer
|
||||
will output a result that is closely matching to the input face.
|
||||
|
||||
`<prompt> -G 1.0 -ft codeformer -cf 0.9`
|
||||
|
||||
The following command will perform face restoration with CodeFormer. CodeFormer will output a result
|
||||
that is the best restoration possible. This may deviate slightly from the original face. This is an
|
||||
excellent option to use in situations when there is very little facial data to work with.
|
||||
The following command will perform face restoration with CodeFormer. CodeFormer
|
||||
will output a result that is the best restoration possible. This may deviate
|
||||
slightly from the original face. This is an excellent option to use in
|
||||
situations when there is very little facial data to work with.
|
||||
|
||||
`<prompt> -G 1.0 -ft codeformer -cf 0.1`
|
||||
|
||||
## Fixing Previously-Generated Images
|
||||
|
||||
It is easy to apply face restoration and/or upscaling to any previously-generated file. Just use the
|
||||
syntax `!fix path/to/file.png <options>`. For example, to apply GFPGAN at strength 0.8 and upscale 2X
|
||||
for a file named `./outputs/img-samples/000044.2945021133.png`, just run:
|
||||
It is easy to apply face restoration and/or upscaling to any
|
||||
previously-generated file. Just use the syntax
|
||||
`!fix path/to/file.png <options>`. For example, to apply GFPGAN at strength 0.8
|
||||
and upscale 2X for a file named `./outputs/img-samples/000044.2945021133.png`,
|
||||
just run:
|
||||
|
||||
~~~~
|
||||
```
|
||||
dream> !fix ./outputs/img-samples/000044.2945021133.png -G 0.8 -U 2
|
||||
~~~~
|
||||
```
|
||||
|
||||
A new file named `000044.2945021133.fixed.png` will be created in the output directory. Note that
|
||||
the `!fix` command does not replace the original file, unlike the behavior at generate time.
|
||||
A new file named `000044.2945021133.fixed.png` will be created in the output
|
||||
directory. Note that the `!fix` command does not replace the original file,
|
||||
unlike the behavior at generate time.
|
||||
|
||||
### Disabling:
|
||||
|
||||
If, for some reason, you do not wish to load the GFPGAN and/or ESRGAN libraries, you can disable them
|
||||
on the dream.py command line with the `--no_restore` and `--no_upscale` options, respectively.
|
||||
If, for some reason, you do not wish to load the GFPGAN and/or ESRGAN libraries,
|
||||
you can disable them on the dream.py command line with the `--no_restore` and
|
||||
`--no_upscale` options, respectively.
|
||||
|
||||
Reference in New Issue
Block a user