Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'development' into development

Browse Source
This commit is contained in:
Peter Baylies 2022-09-20 17:40:21 -04:00 committed by GitHub
commit d5209965bc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
48 changed files with 1068 additions and 540 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.dev_scripts/test_regression_txt2img_dream_v1_4.sh
View File

@ -5,8 +5,7 @@ SAMPLES_DIR=${OUT_DIR}
python scripts/dream.py \
--from_file ${PROMPT_FILE} \
--outdir ${OUT_DIR} \
--sampler plms \
--full_precision
--sampler plms
# original output by CompVis/stable-diffusion
IMAGE1=".dev_scripts/images/v1_4_astronaut_rides_horse_plms_step50_seed42.png"

4
.github/workflows/mkdocs-flow.yml vendored
View File

@ -12,7 +12,9 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Build
uses: Tiryoh/actions-mkdocs@v0
with:

4
.github/workflows/test-dream-conda.yml vendored
View File

@ -85,9 +85,9 @@ jobs:
fi
# Utterly hacky, but I don't know how else to do this
if [[ ${{ github.ref }} == 'refs/heads/master' ]]; then
time ${{ steps.vars.outputs.PYTHON_BIN }} scripts/dream.py --from_file tests/preflight_prompts.txt --full_precision
time ${{ steps.vars.outputs.PYTHON_BIN }} scripts/dream.py --from_file tests/preflight_prompts.txt
elif [[ ${{ github.ref }} == 'refs/heads/development' ]]; then
time ${{ steps.vars.outputs.PYTHON_BIN }} scripts/dream.py --from_file tests/dev_prompts.txt --full_precision
time ${{ steps.vars.outputs.PYTHON_BIN }} scripts/dream.py --from_file tests/dev_prompts.txt
fi
mkdir -p outputs/img-samples
- name: Archive results

3
.gitignore vendored
View File

@ -3,6 +3,9 @@ outputs/
models/ldm/stable-diffusion-v1/model.ckpt
ldm/restoration/codeformer/weights
# ignore the Anaconda/Miniconda installer used while building Docker image
anaconda.sh
# ignore a directory which serves as a place for initial images
inputs/

4
.prettierrc.yaml
View File

@ -5,9 +5,9 @@ singleQuote: true
quoteProps: as-needed
embeddedLanguageFormatting: auto
overrides:
- files: "*.md"
- files: '*.md'
options:
proseWrap: always
printWidth: 100
printWidth: 80
parser: markdown
cursorOffset: -1

22
README.md
View File

@ -1,4 +1,4 @@
<div align="center">
<h1 align='center'><b>InvokeAI: A Stable Diffusion Toolkit</b></h1>
# Stable Diffusion Dream Script
@ -8,7 +8,7 @@
<a href="https://discord.gg/ZmtBAhwWhy"><img src="docs/assets/join-us-on-discord-image.png"/></a>
</p>
# **Stable Diffusion Dream Script**
# **InvokeAI - A Stable Diffusion Toolkit**
[![discord badge]][discord link]
[![latest release badge]][latest release link] [![github stars badge]][github stars link] [![github forks badge]][github forks link]
@ -86,17 +86,14 @@ You wil need one of the following:
- At least 6 GB of free disk space for the machine learning model, Python, and all its dependencies.
> Note
>
> If you have an Nvidia 10xx series card (e.g. the 1080ti), please run the dream script in
> full-precision mode as shown below.
#### Note
Similarly, specify full-precision mode on Apple M1 hardware.
To run in full-precision mode, start `dream.py` with the `--full_precision` flag:
Precision is auto configured based on the device. If however you encounter
errors like 'expected type Float but found Half' or 'not implemented for Half'
you can try starting `dream.py` with the `--precision=float32` flag:
```bash
(ldm) ~/stable-diffusion$ python scripts/dream.py --full_precision
(ldm) ~/stable-diffusion$ python scripts/dream.py --precision=float32
```
### Features
@ -126,6 +123,11 @@ To run in full-precision mode, start `dream.py` with the `--full_precision` flag
### Latest Changes
- vNEXT (TODO 2022)
- Deprecated `--full_precision` / `-F`. Simply omit it and `dream.py` will auto
configure. To switch away from auto use the new flag like `--precision=float32`.
- v1.14 (11 September 2022)
- Memory optimizations for small-RAM cards. 512x512 now possible on 4 GB GPUs.

57
docker-build/Dockerfile Normal file
View File

@ -0,0 +1,57 @@
FROM debian
ARG gsd
ENV GITHUB_STABLE_DIFFUSION $gsd
ARG rsd
ENV REQS $rsd
ARG cs
ENV CONDA_SUBDIR $cs
ENV PIP_EXISTS_ACTION="w"
# TODO: Optimize image size
SHELL ["/bin/bash", "-c"]
WORKDIR /
RUN apt update && apt upgrade -y \
&& apt install -y \
git \
libgl1-mesa-glx \
libglib2.0-0 \
pip \
python3 \
&& git clone $GITHUB_STABLE_DIFFUSION
# Install Anaconda or Miniconda
COPY anaconda.sh .
RUN bash anaconda.sh -b -u -p /anaconda && /anaconda/bin/conda init bash
# SD
WORKDIR /stable-diffusion
RUN source ~/.bashrc \
&& conda create -y --name ldm && conda activate ldm \
&& conda config --env --set subdir $CONDA_SUBDIR \
&& pip3 install -r $REQS \
&& pip3 install basicsr facexlib realesrgan \
&& mkdir models/ldm/stable-diffusion-v1 \
&& ln -s "/data/sd-v1-4.ckpt" models/ldm/stable-diffusion-v1/model.ckpt
# Face restoreation
# by default expected in a sibling directory to stable-diffusion
WORKDIR /
RUN git clone https://github.com/TencentARC/GFPGAN.git
WORKDIR /GFPGAN
RUN pip3 install -r requirements.txt \
&& python3 setup.py develop \
&& ln -s "/data/GFPGANv1.3.pth" experiments/pretrained_models/GFPGANv1.3.pth
WORKDIR /stable-diffusion
RUN python3 scripts/preload_models.py
WORKDIR /
COPY entrypoint.sh .
ENTRYPOINT ["/entrypoint.sh"]

10
docker-build/entrypoint.sh Executable file
View File

@ -0,0 +1,10 @@
#!/bin/bash
cd /stable-diffusion
if [ $# -eq 0 ]; then
python3 scripts/dream.py --full_precision -o /data
# bash
else
python3 scripts/dream.py --full_precision -o /data "$@"
fi

0
assets/stable-samples/img2img/mountains-2.png → docs/assets/stable-samples/img2img/mountains-2.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 643 KiB

After

Width:  |  Height:  |  Size: 643 KiB

0
assets/stable-samples/img2img/mountains-3.png → docs/assets/stable-samples/img2img/mountains-3.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 641 KiB

After

Width:  |  Height:  |  Size: 641 KiB

0
assets/stable-samples/img2img/sketch-mountains-input.jpg → docs/assets/stable-samples/img2img/sketch-mountains-input.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 174 KiB

After

Width:  |  Height:  |  Size: 174 KiB

0
assets/stable-samples/txt2img/merged-0005.png → docs/assets/stable-samples/txt2img/merged-0005.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.5 MiB

After

Width:  |  Height:  |  Size: 2.5 MiB

0
assets/stable-samples/txt2img/merged-0006.png → docs/assets/stable-samples/txt2img/merged-0006.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.5 MiB

After

Width:  |  Height:  |  Size: 2.5 MiB

0
assets/stable-samples/txt2img/merged-0007.png → docs/assets/stable-samples/txt2img/merged-0007.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.3 MiB

After

Width:  |  Height:  |  Size: 2.3 MiB

0
assets/v1-variants-scores.jpg → docs/assets/v1-variants-scores.jpg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 70 KiB

After

Width:  |  Height:  |  Size: 70 KiB

2
docs/features/CHANGELOG.md
View File

@ -2,6 +2,8 @@
title: Changelog
---
# :octicons-log-16: Changelog
## v1.13 <small>(in process)</small>
- Supports a Google Colab notebook for a standalone server running on Google

267
docs/features/CLI.md
View File

@ -1,27 +1,34 @@
---
title: CLI
hide:
- toc
---
# :material-bash: CLI
## **Interactive Command Line Interface**
The `dream.py` script, located in `scripts/dream.py`, provides an interactive interface to image
generation similar to the "dream mothership" bot that Stable AI provided on its Discord server.
Unlike the txt2img.py and img2img.py scripts provided in the original CompViz/stable-diffusion
source code repository, the time-consuming initialization of the AI model initialization only
happens once. After that image generation from the command-line interface is very fast.
Unlike the `txt2img.py` and `img2img.py` scripts provided in the original
[CompVis/stable-diffusion](https://github.com/CompVis/stable-diffusion) source code repository, the
time-consuming initialization of the AI model initialization only happens once. After that image
generation from the command-line interface is very fast.
The script uses the readline library to allow for in-line editing, command history (up and down
arrows), autocompletion, and more. To help keep track of which prompts generated which images, the
The script uses the readline library to allow for in-line editing, command history (++up++ and
++down++), autocompletion, and more. To help keep track of which prompts generated which images, the
script writes a log file of image names and prompts to the selected output directory.
In addition, as of version 1.02, it also writes the prompt into the PNG file's metadata where it can
be retrieved using scripts/images2prompt.py
be retrieved using `scripts/images2prompt.py`
The script is confirmed to work on Linux, Windows and Mac systems.
_Note:_ This script runs from the command-line or can be used as a Web application. The Web GUI is
currently rudimentary, but a much better replacement is on its way.
!!! note
This script runs from the command-line or can be used as a Web application. The Web GUI is
currently rudimentary, but a much better replacement is on its way.
```bash
(ldm) ~/stable-diffusion$ python3 ./scripts/dream.py
@ -47,185 +54,197 @@ dream> q
00011.png: "there's a fly in my soup" -n6 -g -S 2685670268
```
<p align='center'>
<img src="../assets/dream-py-demo.png"/>
</p>
![dream-py-demo](../assets/dream-py-demo.png)
The `dream>` prompt's arguments are pretty much identical to those used in the Discord bot, except
you don't need to type "!dream" (it doesn't hurt if you do). A significant change is that creation
of individual images is now the default unless --grid (-g) is given. A full list is given in [List
of prompt arguments] (#list-of-prompt-arguments).
of individual images is now the default unless `--grid` (`-g`) is given. A full list is given in
[List of prompt arguments](#list-of-prompt-arguments).
## Arguments
The script itself also recognizes a series of command-line switches that will change important
global defaults, such as the directory for image outputs and the location of the model weight files.
## List of arguments recognized at the command line
### List of arguments recognized at the command line
These command-line arguments can be passed to dream.py when you first run it from the Windows, Mac
These command-line arguments can be passed to `dream.py` when you first run it from the Windows, Mac
or Linux command line. Some set defaults that can be overridden on a per-prompt basis (see [List of
prompt arguments] (#list-of-prompt-arguments). Others
| Argument | Shortcut | Default | Description |
| :---------------------- | :---------: | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------- |
| --help | -h | | Print a concise help message. |
| --outdir <path> | -o<path> | outputs/img_samples | Location for generated images. |
| --prompt_as_dir | -p | False | Name output directories using the prompt text. |
| --from_file <path> | | None | Read list of prompts from a file. Use "-" to read from standard input |
| --model <modelname> | | stable-diffusion-1.4 | Loads model specified in configs/models.yaml. Currently one of "stable-diffusion-1.4" or "laion400m" |
| --full_precision | -F | False | Run in slower full-precision mode. Needed for Macintosh M1/M2 hardware and some older video cards. |
| --web | | False | Start in web server mode |
| --host <ip addr> | | localhost | Which network interface web server should listen on. Set to 0.0.0.0 to listen on any. |
| --port <port> | | 9090 | Which port web server should listen for requests on. |
| --config <path> | | configs/models.yaml | Configuration file for models and their weights. |
| --iterations <int> | -n<int> | 1 | How many images to generate per prompt. |
| --grid | -g | False | Save all image series as a grid rather than individually. |
| --sampler <sampler> | -A<sampler> | k_lms | Sampler to use. Use -h to get list of available samplers. |
| --seamless | | False | Create interesting effects by tiling elements of the image. |
| --embedding_path <path> | | None | Path to pre-trained embedding manager checkpoints, for custom models |
| --gfpgan_dir | | src/gfpgan | Path to where GFPGAN is installed. |
| --gfpgan_model_path | | experiments/pretrained_models<br>/GFPGANv1.3.pth | Path to GFPGAN model file, relative to --gfpgan_dir. |
| --device <device> | -d<device> | torch.cuda.current_device() | Device to run SD on, e.g. "cuda:0" |
| Argument <img width="240" align="right"/> | Shortcut <img width="100" align="right"/> | Default <img width="320" align="right"/> | Description |
| ----------------------------------------- | ----------------------------------------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `--help` | `-h` | | Print a concise help message. |
| `--outdir <path>` | `-o<path>` | `outputs/img_samples` | Location for generated images. |
| `--prompt_as_dir` | `-p` | `False` | Name output directories using the prompt text. |
| `--from_file <path>` | | `None` | Read list of prompts from a file. Use `-` to read from standard input |
| `--model <modelname>` | | `stable-diffusion-1.4` | Loads model specified in configs/models.yaml. Currently one of "stable-diffusion-1.4" or "laion400m" |
| `--full_precision` | `-F` | `False` | Run in slower full-precision mode. Needed for Macintosh M1/M2 hardware and some older video cards. |
| `--web` | | `False` | Start in web server mode |
| `--host <ip addr>` | | `localhost` | Which network interface web server should listen on. Set to 0.0.0.0 to listen on any. |
| `--port <port>` | | `9090` | Which port web server should listen for requests on. |
| `--config <path>` | | `configs/models.yaml` | Configuration file for models and their weights. |
| `--iterations <int>` | `-n<int>` | `1` | How many images to generate per prompt. |
| `--grid` | `-g` | `False` | Save all image series as a grid rather than individually. |
| `--sampler <sampler>` | `-A<sampler>` | `k_lms` | Sampler to use. Use `-h` to get list of available samplers. |
| `--seamless` | | `False` | Create interesting effects by tiling elements of the image. |
| `--embedding_path <path>` | | `None` | Path to pre-trained embedding manager checkpoints, for custom models |
| `--gfpgan_dir` | | `src/gfpgan` | Path to where GFPGAN is installed. |
| `--gfpgan_model_path` | | `experiments/pretrained_models/GFPGANv1.3.pth` | Path to GFPGAN model file, relative to `--gfpgan_dir`. |
| `--device <device>` | `-d<device>` | `torch.cuda.current_device()` | Device to run SD on, e.g. "cuda:0" |
#### deprecated
These arguments are deprecated but still work:
| Argument | Shortcut | Default | Description |
| ---------------- | -------- | ------- | --------------------------------------------------------------- |
| --weights <path> | | None | Pth to weights file; use `--model stable-diffusion-1.4` instead |
| --laion400m | -l | False | Use older LAION400m weights; use `--model=laion400m` instead |
<figure markdown>
### **A note on path names:**
| Argument | Shortcut | Default | Description |
| ------------------ | -------- | ------- | --------------------------------------------------------------- |
| `--weights <path>` | | `None` | Pth to weights file; use `--model stable-diffusion-1.4` instead |
| `--laion400m` | `-l` | `False` | Use older LAION400m weights; use `--model=laion400m` instead |
On Windows systems, you may run into problems when passing the dream script standard backslashed
path names because the Python interpreter treats "\" as an escape. You can either double your
slashes (ick): `C:\\\\path\\\\to\\\\my\\\\file`, or use Linux/Mac style forward slashes (better):
`C:/path/to/my/file`.
</figure>
!!! note
On Windows systems, you may run into problems when passing the dream script standard backslashed
path names because the Python interpreter treats `\` as an escape. You can either double your
slashes (ick): `C:\\path\\to\\my\\file`, or use Linux/Mac style forward slashes (better):
`C:/path/to/my/file`.
### List of prompt arguments
After the dream.py script initializes, it will present you with a **dream>** prompt. Here you can
enter information to generate images from text (txt2img), to embellish an existing image or sketch
(img2img), or to selectively alter chosen regions of the image (inpainting).
After the `dream.py` script initializes, it will present you with a **`dream>`** prompt. Here you
can enter information to generate images from text (txt2img), to embellish an existing image or
sketch (img2img), or to selectively alter chosen regions of the image (inpainting).
### This is an example of txt2img
#### txt2img
```bash
dream> "waterfall and rainbow" -W640 -H480
```
!!! example
This will create the requested image with the dimensions 640 (width) and 480 (height).
```bash
dream> "waterfall and rainbow" -W640 -H480
```
This will create the requested image with the dimensions 640 (width) and 480 (height).
Those are the `dream` commands that apply to txt2img:
| Argument | Shortcut | Default | Description |
| --------------------------- | ---------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| "my prompt" | | | Text prompt to use. The quotation marks are optional. |
| --width <int> | -W<int> | 512 | Width of generated image |
| --height <int> | -H<int> | 512 | Height of generated image |
| --iterations <int> | -n<int> | 1 | How many images to generate from this prompt |
| --steps <int> | -s<int> | 50 | How many steps of refinement to apply |
| --cfg_scale <float> | -C<float> | 7.5 | How hard to try to match the prompt to the generated image; any number greater than 0.0 works, but the useful range is roughly 5.0 to 20.0 |
| --seed <int> | -S<int> | None | Set the random seed for the next series of images. This can be used to recreate an image generated previously. |
| --sampler <sampler> | -A<sampler> | k_lms | Sampler to use. Use -h to get list of available samplers. |
| --grid | -g | False | Turn on grid mode to return a single image combining all the images generated by this prompt |
| --individual | -i | True | Turn off grid mode (deprecated; leave off --grid instead) |
| --outdir <path> | -o<path> | outputs/img_samples | Temporarily change the location of these images |
| --seamless | | False | Activate seamless tiling for interesting effects |
| --log_tokenization | -t | False | Display a color-coded list of the parsed tokens derived from the prompt |
| --skip_normalization | -x | False | Weighted subprompts will not be normalized. See [Weighted Prompts](./OTHER.md#weighted-prompts) |
| --upscale <int> <float> | -U <int> <float> | -U 1 0.75 | Upscale image by magnification factor (2, 4), and set strength of upscaling (0.0-1.0). If strength not set, will default to 0.75. |
| --gfpgan_strength <float> | -G <float> | -G0 | Fix faces using the GFPGAN algorithm; argument indicates how hard the algorithm should try (0.0-1.0) |
| --save_original | -save_orig | False | When upscaling or fixing faces, this will cause the original image to be saved rather than replaced. |
| --variation <float> | -v<float> | 0.0 | Add a bit of noise (0.0=none, 1.0=high) to the image in order to generate a series of variations. Usually used in combination with -S<seed> and -n<int> to generate a series a riffs on a starting image. See [Variations](./VARIATIONS.md). |
| --with_variations <pattern> | -V<pattern> | None | Combine two or more variations. See [Variations](./VARIATIONS.md) for now to use this. |
| Argument <img width="680" align="right"/> | Shortcut <img width="420" align="right"/> | Default <img width="480" align="right"/> | Description |
| ----------------------------------------- | ----------------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `"my prompt"` | | | Text prompt to use. The quotation marks are optional. |
| `--width <int>` | `-W<int>` | `512` | Width of generated image |
| `--height <int>` | `-H<int>` | `512` | Height of generated image |
| `--iterations <int>` | `-n<int>` | `1` | How many images to generate from this prompt |
| `--steps <int>` | `-s<int>` | `50` | How many steps of refinement to apply |
| `--cfg_scale <float>` | `-C<float>` | `7.5` | How hard to try to match the prompt to the generated image; any number greater than 0.0 works, but the useful range is roughly 5.0 to 20.0 |
| `--seed <int>` | `-S<int>` | `None` | Set the random seed for the next series of images. This can be used to recreate an image generated previously. |
| `--sampler <sampler>` | `-A<sampler>` | `k_lms` | Sampler to use. Use `-h` to get list of available samplers. |
| `--grid` | `-g` | `False` | Turn on grid mode to return a single image combining all the images generated by this prompt |
| `--individual` | `-i` | `True` | Turn off grid mode (deprecated; leave off `--grid` instead) |
| `--outdir <path>` | `-o<path>` | `outputs/img_samples` | Temporarily change the location of these images |
| `--seamless` | | `False` | Activate seamless tiling for interesting effects |
| `--log_tokenization` | `-t` | `False` | Display a color-coded list of the parsed tokens derived from the prompt |
| `--skip_normalization` | `-x` | `False` | Weighted subprompts will not be normalized. See [Weighted Prompts](./OTHER.md#weighted-prompts) |
| `--upscale <int> <float>` | `-U <int> <float>` | `-U 1 0.75` | Upscale image by magnification factor (2, 4), and set strength of upscaling (0.0-1.0). If strength not set, will default to 0.75. |
| `--gfpgan_strength <float>` | `-G <float>` | `-G0` | Fix faces using the GFPGAN algorithm; argument indicates how hard the algorithm should try (0.0-1.0) |
| `--save_original` | `-save_orig` | `False` | When upscaling or fixing faces, this will cause the original image to be saved rather than replaced. |
| `--variation <float>` | `-v<float>` | `0.0` | Add a bit of noise (0.0=none, 1.0=high) to the image in order to generate a series of variations. Usually used in combination with `-S<seed>` and `-n<int>` to generate a series a riffs on a starting image. See [Variations](./VARIATIONS.md). |
| `--with_variations <pattern>` | `-V<pattern>` | `None` | Combine two or more variations. See [Variations](./VARIATIONS.md) for now to use this. |
Note that the width and height of the image must be multiples of 64. You can provide different
values, but they will be rounded down to the nearest multiple of 64.
!!! note
### This is an example of img2img
The width and height of the image must be multiples of 64. You can provide different
values, but they will be rounded down to the nearest multiple of 64.
```bash
dream> waterfall and rainbow -I./vacation-photo.png -W640 -H480 --fit
```
#### img2img
This will modify the indicated vacation photograph by making it more like the prompt. Results will
vary greatly depending on what is in the image. We also ask to --fit the image into a box no bigger
than 640x480. Otherwise the image size will be identical to the provided photo and you may run out
of memory if it is large.
!!! example
Repeated chaining of img2img on an image can result in significant color shifts
in the output, especially if run with lower strength. Color correction can be
run against a reference image to fix this issue. Use the original input image to the
chain as the the reference image for each step in the chain.
```bash
dream> "waterfall and rainbow" -I./vacation-photo.png -W640 -H480 --fit
```
This will modify the indicated vacation photograph by making it more like the prompt. Results will
vary greatly depending on what is in the image. We also ask to --fit the image into a box no bigger
than 640x480. Otherwise the image size will be identical to the provided photo and you may run out
of memory if it is large.
Repeated chaining of img2img on an image can result in significant color shifts in the output,
especially if run with lower strength. Color correction can be run against a reference image to fix
this issue. Use the original input image to the chain as the the reference image for each step in
the chain.
In addition to the command-line options recognized by txt2img, img2img accepts additional options:
| Argument | Shortcut | Default | Description |
| ------------------ | --------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| --init_img <path> | -I<path> | None | Path to the initialization image |
| --init_color <path> | | None | Path to reference image for color correction |
| --fit | -F | False | Scale the image to fit into the specified -H and -W dimensions |
| --strength <float> | -s<float> | 0.75 | How hard to try to match the prompt to the initial image. Ranges from 0.0-0.99, with higher values replacing the initial image completely. |
| Argument <img width="160" align="right"/> | Shortcut | Default | Description |
| ----------------------------------------- | ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `--init_img <path>` | `-I<path>` | `None` | Path to the initialization image |
| `--init_color <path>` | | `None` | Path to reference image for color correction |
| `--fit` | `-F` | `False` | Scale the image to fit into the specified -H and -W dimensions |
| `--strength <float>` | `-f<float>` | `0.75` | How hard to try to match the prompt to the initial image. Ranges from 0.0-0.99, with higher values replacing the initial image completely. |
### This is an example of inpainting
#### Inpainting
```bash
dream> "waterfall and rainbow" -I./vacation-photo.png -M./vacation-mask.png -W640 -H480 --fit
```
!!! example
This will do the same thing as img2img, but image alterations will only occur within transparent
areas defined by the mask file specified by -M. You may also supply just a single initial image with
the areas to overpaint made transparent, but you must be careful not to destroy the pixels
underneath when you create the transparent areas. See [Inpainting](./INPAINTING.md) for details.
```bash
dream> "waterfall and rainbow" -I./vacation-photo.png -M./vacation-mask.png -W640 -H480 --fit
```
inpainting accepts all the arguments used for txt2img and img2img, as well as the --mask (-M)
This will do the same thing as img2img, but image alterations will only occur within transparent
areas defined by the mask file specified by `-M`. You may also supply just a single initial image with
the areas to overpaint made transparent, but you must be careful not to destroy the pixels
underneath when you create the transparent areas. See [Inpainting](./INPAINTING.md) for details.
Inpainting accepts all the arguments used for txt2img and img2img, as well as the `--mask` (`-M`)
argument:
| Argument | Shortcut | Default | Description |
| ------------------ | -------- | ------- | ------------------------------------------------------------------------------------------------ |
| --init_mask <path> | -M<path> | None | Path to an image the same size as the initial_image, with areas for inpainting made transparent. |
| Argument <img width="100" align="right"/> | Shortcut | Default | Description |
| ----------------------------------------- | ---------- | ------- | ------------------------------------------------------------------------------------------------ |
| `--init_mask <path>` | `-M<path>` | `None` | Path to an image the same size as the initial_image, with areas for inpainting made transparent. |
## Command-line editing and completion
If you are on a Macintosh or Linux machine, the command-line offers convenient history tracking,
editing, and command completion.
- To scroll through previous commands and potentially edit/reuse them, use the up and down cursor
keys.
- To edit the current command, use the left and right cursor keys to position the cursor, and then
backspace, delete or insert characters.
- To move to the very beginning of the command, type CTRL-A (or command-A on the Mac)
- To move to the end of the command, type CTRL-E.
- To scroll through previous commands and potentially edit/reuse them, use the ++up++ and ++down++
cursor keys.
- To edit the current command, use the ++left++ and ++right++ cursor keys to position the cursor,
and then ++backspace++, ++delete++ or ++insert++ characters.
- To move to the very beginning of the command, type ++ctrl+a++ (or ++command+a++ on the Mac)
- To move to the end of the command, type ++ctrl+e++.
- To cut a section of the command, position the cursor where you want to start cutting and type
CTRL-K.
- To paste a cut section back in, position the cursor where you want to paste, and type CTRL-Y
++ctrl+k++.
- To paste a cut section back in, position the cursor where you want to paste, and type ++ctrl+y++
Windows users can get similar, but more limited, functionality if they launch dream.py with the
Windows users can get similar, but more limited, functionality if they launch `dream.py` with the
"winpty" program:
```
> winpty python scripts\dream.py
```batch
winpty python scripts\dream.py
```
On the Mac and Linux platforms, when you exit dream.py, the last 1000 lines of your command-line
history will be saved. When you restart dream.py, you can access the saved history using the
up-arrow key.
On the Mac and Linux platforms, when you exit `dream.py`, the last 1000 lines of your command-line
history will be saved. When you restart `dream.py`, you can access the saved history using the
++up++ key.
In addition, limited command-line completion is installed. In various contexts, you can start typing
your command and press tab. A list of potential completions will be presented to you. You can then
type a little more, hit tab again, and eventually autocomplete what you want.
When specifying file paths using the one-letter shortcuts, the CLI will attempt to complete
pathnames for you. This is most handy for the -I (init image) and -M (init mask) paths. To initiate
completion, start the path with a slash ("/") or "./". For example:
pathnames for you. This is most handy for the `-I` (init image) and `-M` (init mask) paths. To
initiate completion, start the path with a slash `/` or `./`, for example:
```
dream> zebra with a mustache -I./test-pictures<TAB>
```bash
dream> "zebra with a mustache" -I./test-pictures<TAB>
-I./test-pictures/Lincoln-and-Parrot.png -I./test-pictures/zebra.jpg -I./test-pictures/madonna.png
-I./test-pictures/bad-sketch.png -I./test-pictures/man_with_eagle/
```
You can then type "z", hit tab again, and it will autofill to "zebra.jpg".
You can then type ++z++, hit ++tab++ again, and it will autofill to `zebra.jpg`.
More text completion features (such as autocompleting seeds) are on their way.

47
docs/features/EMBIGGEN.md
View File

@ -1,4 +1,10 @@
# **Embiggen -- upscale your images on limited memory machines**
---
title: Embiggen
---
# :material-loupe: Embiggen
**upscale your images on limited memory machines**
GFPGAN and Real-ESRGAN are both memory intensive. In order to avoid
crashes and memory overloads during the Stable Diffusion process,
@ -16,7 +22,7 @@ face restore a particular generated image, pass it again with the same
prompt and generated seed along with the `-U` and `-G` prompt
arguments to perform those actions.
## Embiggen
## Embiggen
If you wanted to be able to do more (pixels) without running out of VRAM,
or you want to upscale with details that couldn't possibly appear
@ -37,7 +43,7 @@ it's similar to that, except it can work up to an arbitrarily large size
has extra logic to re-run any number of the tile sub-sections of the image
if for example a small part of a huge run got messed up.
**Usage**
## Usage
`-embiggen <scaling_factor> <esrgan_strength> <overlap_ratio OR overlap_pixels>`
@ -55,7 +61,6 @@ and it can also be less than one if the init_img is too big.
Esrgan_strength defaults to 0.75, and the overlap_ratio defaults to
0.25, both are optional.
Unlike Img2Img, the `--width` (`-W`) and `--height` (`-H`) arguments
do not control the size of the image as a whole, but the size of the
tiles used to Embiggen the image.
@ -95,12 +100,12 @@ Tiles are numbered starting with one, and left-to-right,
top-to-bottom. So, if you are generating a 3x3 tiled image, the
middle row would be `4 5 6`.
**Example Usage**
## Example Usage
Running Embiggen with 512x512 tiles on an existing image, scaling up by a factor of 2.5x;
and doing the same again (default ESRGAN strength is 0.75, default overlap between tiles is 0.25):
```
```bash
dream > a photo of a forest at sunset -s 100 -W 512 -H 512 -I outputs/forest.png -f 0.4 -embiggen 2.5
dream > a photo of a forest at sunset -s 100 -W 512 -H 512 -I outputs/forest.png -f 0.4 -embiggen 2.5 0.75 0.25
```
@ -112,23 +117,23 @@ If there weren't enough clouds in the sky of that forest you just made
512x512 tiles with 0.25 overlaps wide) we can replace that top row of
tiles:
```
```bash
dream> a photo of puffy clouds over a forest at sunset -s 100 -W 512 -H 512 -I outputs/000002.seed.png -f 0.5 -embiggen_tiles 1 2 3
```
**Note**
!!! note
Because the same prompt is used on all the tiled images, and the model
doesn't have the context of anything outside the tile being run - it
can end up creating repeated pattern (also called 'motifs') across all
the tiles based on that prompt. The best way to combat this is
lowering the `--strength` (`-f`) to stay more true to the init image,
and increasing the number of steps so there is more compute-time to
create the detail. Anecdotally `--strength` 0.35-0.45 works pretty
well on most things. It may also work great in some examples even with
the `--strength` set high for patterns, landscapes, or subjects that
are more abstract. Because this is (relatively) fast, you can also
always create a few Embiggen'ed images and manually composite them to
preserve the best parts from each.
Because the same prompt is used on all the tiled images, and the model
doesn't have the context of anything outside the tile being run - it
can end up creating repeated pattern (also called 'motifs') across all
the tiles based on that prompt. The best way to combat this is
lowering the `--strength` (`-f`) to stay more true to the init image,
and increasing the number of steps so there is more compute-time to
create the detail. Anecdotally `--strength` 0.35-0.45 works pretty
well on most things. It may also work great in some examples even with
the `--strength` set high for patterns, landscapes, or subjects that
are more abstract. Because this is (relatively) fast, you can also
always create a few Embiggen'ed images and manually composite them to
preserve the best parts from each.
Author: [Travco](https://github.com/travco)
Author: [Travco](https://github.com/travco)

3
docs/features/IMG2IMG.md
View File

@ -2,7 +2,8 @@
title: Image-to-Image
---
## **IMG2IMG**
# :material-image-multiple: **IMG2IMG**
This script also provides an `img2img` feature that lets you seed your creations with an initial
drawing or photo. This is a really cool feature that tells stable diffusion to build the prompt on
top of the image you provide, preserving the original's basic shape and layout. To use it, provide

56
docs/features/INPAINTING.md
View File

@ -2,6 +2,8 @@
title: Inpainting
---
# :octicons-paintbrush-16: Inpainting
## **Creating Transparent Regions for Inpainting**
Inpainting is really cool. To do it, you start with an initial image and use a photoeditor to make
@ -26,6 +28,8 @@ dream> "man with cat on shoulder" -I./images/man.png -M./images/man-transparent.
We are hoping to get rid of the need for this workaround in an upcoming release.
---
## Recipe for GIMP
[GIMP](https://www.gimp.org/) is a popular Linux photoediting tool.
@ -34,7 +38,7 @@ We are hoping to get rid of the need for this workaround in an upcoming release.
2. Layer->Transparency->Add Alpha Channel
3. Use lasoo tool to select region to mask
4. Choose Select -> Float to create a floating selection
5. Open the Layers toolbar (^L) and select "Floating Selection"
5. Open the Layers toolbar (++ctrl+l++) and select "Floating Selection"
6. Set opacity to 0%
7. Export as PNG
8. In the export dialogue, Make sure the "Save colour values from
@ -44,37 +48,41 @@ We are hoping to get rid of the need for this workaround in an upcoming release.
## Recipe for Adobe Photoshop
1. Open image in Photoshop
<p align='left'>
<img src="../assets/step1.png"/>
</p>
<figure markdown>
![step1](../assets/step1.png)
</figure>
2. Use any of the selection tools (Marquee, Lasso, or Wand) to select the area you desire to inpaint.
<p align='left'>
<img src="../assets/step2.png"/>
</p>
3. Because we'll be applying a mask over the area we want to preserve, you should now select the inverse by using the Shift + Ctrl + I shortcut, or right clicking and using the "Select Inverse" option.
<figure markdown>
![step2](../assets/step2.png)
</figure>
4. You'll now create a mask by selecting the image layer, and Masking the selection. Make sure that you don't delete any of the underlying image, or your inpainting results will be dramatically impacted.
<p align='left'>
<img src="../assets/step4.png"/>
</p>
3. Because we'll be applying a mask over the area we want to preserve, you should now select the inverse by using the ++shift+ctrl+i++ shortcut, or right clicking and using the "Select Inverse" option.
4. You'll now create a mask by selecting the image layer, and Masking the selection. Make sure that you don't delete any of the undrlying image, or your inpainting results will be dramatically impacted.
<figure markdown>
![step4](../assets/step4.png)
</figure>
5. Make sure to hide any background layers that are present. You should see the mask applied to your image layer, and the image on your canvas should display the checkered background.
<p align='left'>
<img src="../assets/step5.png"/>
</p>
<p align='left'>
<img src="../assets/step6.png"/>
</p>
<figure markdown>
![step5](../assets/step5.png)
</figure>
6. Save the image as a transparent PNG by using the "Save a Copy" option in the File menu, or using the Alt + Ctrl + S keyboard shortcut.
6. Save the image as a transparent PNG by using the "Save a Copy" option in the File menu, or using the Alt + Ctrl + S keyboard shortcut
<figure markdown>
![step6](../assets/step6.png)
</figure>
7. After following the inpainting instructions above (either through the CLI or the Web UI), marvel at your newfound ability to selectively dream. Lookin' good!
<p align='left'>
<img src="../assets/step7.png"/>
</p>
8. In the export dialogue, Make sure the "Save colour values from transparent pixels" checkbox is
selected.
<figure markdown>
![step7](../assets/step7.png)
</figure>
8. In the export dialogue, Make sure the "Save colour values from transparent pixels" checkbox is selected.

2
docs/features/OTHER.md
View File

@ -2,6 +2,8 @@
title: Others
---
# :fontawesome-regular-share-from-square: Others
## **Google Colab**
Stable Diffusion AI Notebook: <a

52
docs/features/PROMPTS.md
View File

@ -1,4 +1,8 @@
# Prompting Features
---
title: Prompting Features
---
# :octicons-command-palette-24: Prompting Features
## **Reading Prompts from a File**
@ -54,43 +58,41 @@ In the above statement, the words 'not really cool` will be ignored by Stable Di
Here's a prompt that depicts what it does.
original prompt:
original prompt:
```bash
"A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve" -s 20 -W 512 -H 768 -C 7.5 -A k_euler_a -S 1654590180
```
`#!bash "A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve" -s 20 -W 512 -H 768 -C 7.5 -A k_euler_a -S 1654590180`
![step1](../assets/variation_walkthru/step1.png)
<figure markdown>
![step1](../assets/negative_prompt_walkthru/step1.png)
</figure>
That image has a woman, so if we want the horse without a rider, we can influence the image not to have a woman by putting [woman] in the prompt, like this:
```bash
"A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman]" -s 20 -W 512 -H 768 -C 7.5 -A k_euler_a -S 1654590180
```
`#!bash "A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman]" -s 20 -W 512 -H 768 -C 7.5 -A k_euler_a -S 1654590180`
![step2](../assets/variation_walkthru/step2.png)
<figure markdown>
![step2](../assets/negative_prompt_walkthru/step2.png)
</figure>
That's nice - but say we also don't want the image to be quite so blue. We can add "blue" to the list of negative prompts, so it's now [woman blue]:
```bash
"A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman blue]" -s 20 -W 512 -H 768 -C 7.5 -A k_euler_a -S 1654590180
```
![step3](../assets/variation_walkthru/step3.png)
`#!bash "A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman blue]" -s 20 -W 512 -H 768 -C 7.5 -A k_euler_a -S 1654590180`
<figure markdown>
![step3](../assets/negative_prompt_walkthru/step3.png)
</figure>
Getting close - but there's no sense in having a saddle when our horse doesn't have a rider, so we'll add one more negative prompt: [woman blue saddle].
```bash
"A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman blue saddle]" -s 20 -W 512 -H 768 -C 7.5 -A k_euler_a -S 1654590180
```
`#!bash "A fantastical translucent poney made of water and foam, ethereal, radiant, hyperalism, scottish folklore, digital painting, artstation, concept art, smooth, 8 k frostbite 3 engine, ultra detailed, art by artgerm and greg rutkowski and magali villeneuve [woman blue saddle]" -s 20 -W 512 -H 768 -C 7.5 -A k_euler_a -S 1654590180`
![step4](../assets/variation_walkthru/step4.png)
<figure markdown>
![step4](../assets/negative_prompt_walkthru/step4.png)
</figure>
!!! notes "Notes about this feature:"
Notes about this feature:
* The only requirement for words to be ignored is that they are in between a pair of square brackets.
* You can provide multiple words within the same bracket.
* You can provide multiple brackets with multiple words in different places of your prompt. That works just fine.
* To improve typical anatomy problems, you can add negative prompts like [bad anatomy, extra legs, extra arms, extra fingers, poorly drawn hands, poorly drawn feet, disfigured, out of frame, tiling, bad art, deformed, mutated].
* The only requirement for words to be ignored is that they are in between a pair of square brackets.
* You can provide multiple words within the same bracket.
* You can provide multiple brackets with multiple words in different places of your prompt. That works just fine.
* To improve typical anatomy problems, you can add negative prompts like `[bad anatomy, extra legs, extra arms, extra fingers, poorly drawn hands, poorly drawn feet, disfigured, out of frame, tiling, bad art, deformed, mutated]`.

8
docs/features/TEXTUAL_INVERSION.md
View File

@ -2,6 +2,8 @@
title: TEXTUAL_INVERSION
---
# :material-file-document-plus-outline: TEXTUAL_INVERSION
## **Personalizing Text-to-Image Generation**
You may personalize the generated images to provide your own styles or objects
@ -39,7 +41,7 @@ and one with the init word provided.
On a RTX3090, the process for SD will take ~1h @1.6 iterations/sec.
!!! Info _Note_
!!! note
According to the associated paper, the optimal number of
images is 3-5. Your model may not converge if you use more images than
@ -57,9 +59,7 @@ Once the model is trained, specify the trained .pt or .bin file when starting
dream using
```bash
python3 ./scripts/dream.py \
--embedding_path /path/to/embedding.pt \
--full_precision
python3 ./scripts/dream.py --embedding_path /path/to/embedding.pt
```
Then, to utilize your subject at the dream prompt

30
docs/features/UPSCALE.md
View File

@ -2,6 +2,8 @@
title: Upscale
---
# :material-image-size-select-large: Upscale
## **GFPGAN and Real-ESRGAN Support**
The script also provides the ability to do face restoration and upscaling with the help of GFPGAN
@ -30,11 +32,13 @@ this package which asked you to install GFPGAN in a sibling directory, you may u
`--gfpgan_dir` argument with `dream.py` to set a custom path to your GFPGAN directory. _There are
other GFPGAN related boot arguments if you wish to customize further._
**Note: Internet connection needed:** Users whose GPU machines are isolated from the Internet (e.g.
on a University cluster) should be aware that the first time you run dream.py with GFPGAN and
Real-ESRGAN turned on, it will try to download model files from the Internet. To rectify this, you
may run `python3 scripts/preload_models.py` after you have installed GFPGAN and all its
dependencies.
!!! warning "Internet connection needed"
Users whose GPU machines are isolated from the Internet (e.g.
on a University cluster) should be aware that the first time you run dream.py with GFPGAN and
Real-ESRGAN turned on, it will try to download model files from the Internet. To rectify this, you
may run `python3 scripts/preload_models.py` after you have installed GFPGAN and all its
dependencies.
## **Usage**
@ -83,16 +87,16 @@ This also works with img2img:
dream> a man wearing a pineapple hat -I path/to/your/file.png -U 2 0.5 -G 0.6
```
### **Note**
!!! note
GFPGAN and Real-ESRGAN are both memory intensive. In order to avoid crashes and memory overloads
during the Stable Diffusion process, these effects are applied after Stable Diffusion has completed
its work.
GFPGAN and Real-ESRGAN are both memory intensive. In order to avoid crashes and memory overloads
during the Stable Diffusion process, these effects are applied after Stable Diffusion has completed
its work.
In single image generations, you will see the output right away but when you are using multiple
iterations, the images will first be generated and then upscaled and face restored after that
process is complete. While the image generation is taking place, you will still be able to preview
the base images.
In single image generations, you will see the output right away but when you are using multiple
iterations, the images will first be generated and then upscaled and face restored after that
process is complete. While the image generation is taking place, you will still be able to preview
the base images.
If you wish to stop during the image generation but want to upscale or face restore a particular
generated image, pass it again with the same prompt and generated seed along with the `-U` and `-G`

34
docs/features/VARIATIONS.md
View File

@ -2,6 +2,10 @@
title: Variations
---
# :material-tune-variant: Variations
## Intro
Release 1.13 of SD-Dream adds support for image variations.
You are able to do the following:
@ -29,7 +33,7 @@ This will be indicated as `prompt` in the examples below.
First we let SD create a series of images in the usual way, in this case
requesting six iterations:
```
```bash
dream> lucy lawless as xena, warrior princess, character portrait, high resolution -n6
...
Outputs:
@ -41,9 +45,10 @@ Outputs:
./outputs/Xena/000001.3357757885.png: "prompt" -s50 -W512 -H512 -C7.5 -Ak_lms -S3357757885
```
The one with seed 3357757885 looks nice:
![var1](../assets/variation_walkthru/000001.3357757885.png)
<figure markdown>
![var1](../assets/variation_walkthru/000001.3357757885.png)
<figcaption>Seed 3357757885 looks nice</figcaption>
</figure>
---
@ -75,15 +80,21 @@ used to generate it.
This gives us a series of closely-related variations, including the two shown
here.
![var2](../assets/variation_walkthru/000002.3647897225.png)
<figure markdown>
![var2](../assets/variation_walkthru/000002.3647897225.png)
<figcaption>subseed 3647897225</figcaption>
</figure>
![var3](../assets/variation_walkthru/000002.1614299449.png)
<figure markdown>
![var3](../assets/variation_walkthru/000002.1614299449.png)
<figcaption>subseed 1614299449</figcaption>
</figure>
I like the expression on Xena's face in the first one (subseed 3647897225), and
the armor on her shoulder in the second one (subseed 1614299449). Can we combine
them to get the best of both worlds?
We combine the two variations using `-V` (--with_variations). Again, we must
We combine the two variations using `-V` (`--with_variations`). Again, we must
provide the seed for the originally-chosen image in order for this to work.
```bash
@ -95,7 +106,9 @@ Outputs:
Here we are providing equal weights (0.1 and 0.1) for both the subseeds. The
resulting image is close, but not exactly what I wanted:
![var4](../assets/variation_walkthru/000003.1614299449.png)
<figure markdown>
![var4](../assets/variation_walkthru/000003.1614299449.png)
</figure>
We could either try combining the images with different weights, or we can
generate more variations around the almost-but-not-quite image. We do the
@ -116,7 +129,10 @@ Outputs:
This produces six images, all slight variations on the combination of the chosen
two images. Here's the one I like best:
![var5](../assets/variation_walkthru/000004.3747154981.png)
<figure markdown>
![var5](../assets/variation_walkthru/000004.3747154981.png)
<figcaption>000004.3747154981.png</figcaption>
</figure>
As you can see, this is a very powerful tool, which when combined with subprompt
weighting, gives you great control over the content and quality of your

4
docs/features/WEB.md
View File

@ -2,8 +2,10 @@
title: Barebones Web Server
---
# :material-web: Barebones Web Server
As of version 1.10, this distribution comes with a bare bones web server (see
screenshot). To use it, run the `dream.py` script by adding the `**--web**`
screenshot). To use it, run the `dream.py` script by adding the `--web`
option.
```bash

2
docs/help/TROUBLESHOOT.md
View File

@ -2,6 +2,8 @@
title: F.A.Q.
---
# :material-frequently-asked-questions: F.A.Q.
## **Frequently-Asked-Questions**
Here are a few common installation problems and their solutions. Often these are caused by

130
docs/index.md
View File

@ -1,102 +1,106 @@
---
title: Home
template: main.html
---
<!--
The Docs you find here (/docs/*) are built and deployed via mkdocs. If you want to do so from local it is pretty strait forward:
The Docs you find here (/docs/*) are built and deployed via mkdocs. If you want to run a local version to verify your changes, it's as simple as::
```bash
pip install -r requirements-mkdocs.txt
mkdocs serve -a localhost:8080
mkdocs serve
```
-->
<div align="center" markdown>
<h1 align='center'><b>Stable Diffusion Dream Script</b></h1>
# :material-script-text-outline: Stable Diffusion Dream Script
<p align='center'>
<img src="./assets/logo.png"/>
</p>
![project logo](assets/logo.png)
<p align="center">
<img src="https://img.shields.io/github/last-commit/lstein/stable-diffusion?logo=Python&logoColor=green&style=for-the-badge" alt="last-commit"/>
<img src="https://img.shields.io/github/stars/lstein/stable-diffusion?logo=GitHub&style=for-the-badge" alt="stars"/>
<br>
<img src="https://img.shields.io/github/issues/lstein/stable-diffusion?logo=GitHub&style=for-the-badge" alt="issues"/>
<img src="https://img.shields.io/github/issues-pr/lstein/stable-diffusion?logo=GitHub&style=for-the-badge" alt="pull-requests"/>
</p>
[![discord badge]][discord link]
[![latest release badge]][latest release link] [![github stars badge]][github stars link] [![github forks badge]][github forks link]
[![CI checks on main badge]][CI checks on main link] [![CI checks on dev badge]][CI checks on dev link] [![latest commit to dev badge]][latest commit to dev link]
[![github open issues badge]][github open issues link] [![github open prs badge]][github open prs link]
[CI checks on dev badge]: https://flat.badgen.net/github/checks/lstein/stable-diffusion/development?label=CI%20status%20on%20dev&cache=900&icon=github
[CI checks on dev link]: https://github.com/lstein/stable-diffusion/actions?query=branch%3Adevelopment
[CI checks on main badge]: https://flat.badgen.net/github/checks/lstein/stable-diffusion/main?label=CI%20status%20on%20main&cache=900&icon=github
[CI checks on main link]: https://github.com/lstein/stable-diffusion/actions/workflows/test-dream-conda.yml
[discord badge]: https://flat.badgen.net/discord/members/htRgbc7e?icon=discord
[discord link]: https://discord.com/invite/htRgbc7e
[github forks badge]: https://flat.badgen.net/github/forks/lstein/stable-diffusion?icon=github
[github forks link]: https://useful-forks.github.io/?repo=lstein%2Fstable-diffusion
[github open issues badge]: https://flat.badgen.net/github/open-issues/lstein/stable-diffusion?icon=github
[github open issues link]: https://github.com/lstein/stable-diffusion/issues?q=is%3Aissue+is%3Aopen
[github open prs badge]: https://flat.badgen.net/github/open-prs/lstein/stable-diffusion?icon=github
[github open prs link]: https://github.com/lstein/stable-diffusion/pulls?q=is%3Apr+is%3Aopen
[github stars badge]: https://flat.badgen.net/github/stars/lstein/stable-diffusion?icon=github
[github stars link]: https://github.com/lstein/stable-diffusion/stargazers
[latest commit to dev badge]: https://flat.badgen.net/github/last-commit/lstein/stable-diffusion/development?icon=github&color=yellow&label=last%20dev%20commit&cache=900
[latest commit to dev link]: https://github.com/lstein/stable-diffusion/commits/development
[latest release badge]: https://flat.badgen.net/github/release/lstein/stable-diffusion/development?icon=github
[latest release link]: https://github.com/lstein/stable-diffusion/releases
</div>
This is a fork of [CompVis/stable-diffusion](https://github.com/CompVis/stable-diffusion), the open
source text-to-image generator. It provides a streamlined process with various new features and
options to aid the image generation process. It runs on Windows, Mac and Linux machines, and runs on
GPU cards with as little as 4 GB or RAM.
_Note: This fork is rapidly evolving. Please use the
[Issues](https://github.com/lstein/stable-diffusion/issues) tab to report bugs and make feature
requests. Be sure to use the provided templates. They will help aid diagnose issues faster._
!!! note
## Installation
This fork is rapidly evolving. Please use the
[Issues](https://github.com/lstein/stable-diffusion/issues) tab to report bugs and make feature
requests. Be sure to use the provided templates. They will help aid diagnose issues faster.
## :octicons-package-dependencies-24: Installation
This fork is supported across multiple platforms. You can find individual installation instructions
below.
- [Linux](installation/INSTALL_LINUX.md)
- [Windows](installation/INSTALL_WINDOWS.md)
- [Macintosh](installation/INSTALL_MAC.md)
- :fontawesome-brands-linux: [Linux](installation/INSTALL_LINUX.md)
- :fontawesome-brands-windows: [Windows](installation/INSTALL_WINDOWS.md)
- :fontawesome-brands-apple: [Macintosh](installation/INSTALL_MAC.md)
## Hardware Requirements
## :fontawesome-solid-computer: Hardware Requirements
### System
### :octicons-cpu-24: System
You wil need one of the following:
- An NVIDIA-based graphics card with 4 GB or more VRAM memory.
- An Apple computer with an M1 chip.
- :simple-nvidia: An NVIDIA-based graphics card with 4 GB or more VRAM memory.
- :fontawesome-brands-apple: An Apple computer with an M1 chip.
### Memory
### :fontawesome-solid-memory: Memory
- At least 12 GB Main Memory RAM.
### Disk
### :fontawesome-regular-hard-drive: Disk
- At least 6 GB of free disk space for the machine learning model, Python, and all its dependencies.
### Note
!!! note
If you are have a Nvidia 10xx series card (e.g. the 1080ti), please run the dream script in
full-precision mode as shown below.
If you are have a Nvidia 10xx series card (e.g. the 1080ti), please run the dream script in
full-precision mode as shown below.
Similarly, specify full-precision mode on Apple M1 hardware.
Similarly, specify full-precision mode on Apple M1 hardware.
To run in full-precision mode, start `dream.py` with the `--full_precision` flag:
To run in full-precision mode, start `dream.py` with the `--full_precision` flag:
```bash
(ldm) ~/stable-diffusion$ python scripts/dream.py --full_precision
```
```bash
(ldm) ~/stable-diffusion$ python scripts/dream.py --full_precision
```
## :octicons-log-16: Latest Changes
## Features
### vNEXT <small>(TODO 2022)</small>
### Major Features
- [Interactive Command Line Interface](features/CLI.md)
- [Image To Image](features/IMG2IMG.md)
- [Inpainting Support](features/INPAINTING.md)
- [GFPGAN and Real-ESRGAN Support](features/UPSCALE.md)
- [Seamless Tiling](features/OTHER.md#seamless-tiling)
- [Google Colab](features/OTHER.md#google-colab)
- [Web Server](features/WEB.md)
- [Reading Prompts From File](features/OTHER.md#reading-prompts-from-a-file)
- [Shortcut: Reusing Seeds](features/OTHER.md#shortcuts-reusing-seeds)
- [Weighted Prompts](features/OTHER.md#weighted-prompts)
- [Variations](features/VARIATIONS.md)
- [Personalizing Text-to-Image Generation](features/TEXTUAL_INVERSION.md)
- [Simplified API for text to image generation](features/OTHER.md#simplified-api)
### Other Features
- [Creating Transparent Regions for Inpainting](features/INPAINTING.md#creating-transparent-regions-for-inpainting)
- [Preload Models](features/OTHER.md#preload-models)
## Latest Changes
- Deprecated `--full_precision` / `-F`. Simply omit it and `dream.py` will auto
configure. To switch away from auto use the new flag like `--precision=float32`.
### v1.14 <small>(11 September 2022)</small>
@ -127,12 +131,12 @@ To run in full-precision mode, start `dream.py` with the `--full_precision` flag
For older changelogs, please visit the **[CHANGELOG](features/CHANGELOG.md)**.
## Troubleshooting
## :material-target: Troubleshooting
Please check out our **[Q&A](help/TROUBLESHOOT.md)** to get solutions for common installation
Please check out our **[:material-frequently-asked-questions: Q&A](help/TROUBLESHOOT.md)** to get solutions for common installation
problems and other issues.
## Contributing
## :octicons-repo-push-24: Contributing
Anyone who wishes to contribute to this project, whether documentation, features, bug fixes, code
cleanup, testing, or code reviews, is very much encouraged to do so. If you are unfamiliar with how
@ -144,13 +148,13 @@ important thing is to **make your pull request against the "development" branch*
"main". This will help keep public breakage to a minimum and will allow you to propose more radical
changes.
## Contributors
## :octicons-person-24: Contributors
This fork is a combined effort of various people from across the world.
[Check out the list of all these amazing people](other/CONTRIBUTORS.md). We thank them for their
time, hard work and effort.
## Support
## :octicons-question-24: Support
For support, please use this repository's GitHub Issues tracking service. Feel free to send me an
email if you use and like the script.
@ -158,7 +162,7 @@ email if you use and like the script.
Original portions of the software are Copyright (c) 2020
[Lincoln D. Stein](https://github.com/lstein)
## Further Reading
## :octicons-book-24: Further Reading
Please see the original README for more information on this software and underlying algorithm,
located in the file [README-CompViz.md](other/README-CompViz.md).

183
docs/installation/INSTALL_DOCKER.md Normal file
View File

@ -0,0 +1,183 @@
# Before you begin
- For end users: Install Stable Diffusion locally using the instructions for your OS.
- For developers: For container-related development tasks or for enabling easy deployment to other environments (on-premises or cloud), follow these instructions. For general use, install locally to leverage your machine's GPU.
# Why containers?
They provide a flexible, reliable way to build and deploy Stable Diffusion. You'll also use a Docker volume to store the largest model files and image outputs as a first step in decoupling storage and compute. Future enhancements can do this for other assets. See [Processes](https://12factor.net/processes) under the Twelve-Factor App methodology for details on why running applications in such a stateless fashion is important.
You can specify the target platform when building the image and running the container. You'll also need to specify the Stable Diffusion requirements file that matches the container's OS and the architecture it will run on.
Developers on Apple silicon (M1/M2): You [can't access your GPU cores from Docker containers](https://github.com/pytorch/pytorch/issues/81224) and performance is reduced compared with running it directly on macOS but for development purposes it's fine. Once you're done with development tasks on your laptop you can build for the target platform and architecture and deploy to another environment with NVIDIA GPUs on-premises or in the cloud.
# Installation on a Linux container
## Prerequisites
### Get the data files
Go to [Hugging Face](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original), and click "Access repository" to Download the model file ```sd-v1-4.ckpt``` (~4 GB) to ```~/Downloads```. You'll need to create an account but it's quick and free.
Also download the face restoration model.
```Shell
cd ~/Downloads
wget https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth
```
### Install [Docker](https://github.com/santisbon/guides#docker)
On the Docker Desktop app, go to Preferences, Resources, Advanced. Increase the CPUs and Memory to avoid this [Issue](https://github.com/lstein/stable-diffusion/issues/342). You may need to increase Swap and Disk image size too.
## Setup
Set the fork you want to use and other variables.
```Shell
TAG_STABLE_DIFFUSION="santisbon/stable-diffusion"
PLATFORM="linux/arm64"
GITHUB_STABLE_DIFFUSION="-b orig-gfpgan https://github.com/santisbon/stable-diffusion.git"
REQS_STABLE_DIFFUSION="requirements-linux-arm64.txt"
CONDA_SUBDIR="osx-arm64"
echo $TAG_STABLE_DIFFUSION
echo $PLATFORM
echo $GITHUB_STABLE_DIFFUSION
echo $REQS_STABLE_DIFFUSION
echo $CONDA_SUBDIR
```
Create a Docker volume for the downloaded model files.
```Shell
docker volume create my-vol
```
Copy the data files to the Docker volume using a lightweight Linux container. We'll need the models at run time. You just need to create the container with the mountpoint; no need to run this dummy container.
```Shell
cd ~/Downloads # or wherever you saved the files
docker create --platform $PLATFORM --name dummy --mount source=my-vol,target=/data alpine
docker cp sd-v1-4.ckpt dummy:/data
docker cp GFPGANv1.3.pth dummy:/data
```
Get the repo and download the Miniconda installer (we'll need it at build time). Replace the URL with the version matching your container OS and the architecture it will run on.
```Shell
cd ~
git clone $GITHUB_STABLE_DIFFUSION
cd stable-diffusion/docker-build
chmod +x entrypoint.sh
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -O anaconda.sh && chmod +x anaconda.sh
```
Build the Docker image. Give it any tag ```-t``` that you want.
Choose the Linux container's host platform: x86-64/Intel is ```amd64```. Apple silicon is ```arm64```. If deploying the container to the cloud to leverage powerful GPU instances you'll be on amd64 hardware but if you're just trying this out locally on Apple silicon choose arm64.
The application uses libraries that need to match the host environment so use the appropriate requirements file.
Tip: Check that your shell session has the env variables set above.
```Shell
docker build -t $TAG_STABLE_DIFFUSION \
--platform $PLATFORM \
--build-arg gsd=$GITHUB_STABLE_DIFFUSION \
--build-arg rsd=$REQS_STABLE_DIFFUSION \
--build-arg cs=$CONDA_SUBDIR \
.
```
Run a container using your built image.
Tip: Make sure you've created and populated the Docker volume (above).
```Shell
docker run -it \
--rm \
--platform $PLATFORM \
--name stable-diffusion \
--hostname stable-diffusion \
--mount source=my-vol,target=/data \
$TAG_STABLE_DIFFUSION
```
# Usage (time to have fun)
## Startup
If you're on a **Linux container** the ```dream``` script is **automatically started** and the output dir set to the Docker volume you created earlier.
If you're **directly on macOS follow these startup instructions**.
With the Conda environment activated (```conda activate ldm```), run the interactive interface that combines the functionality of the original scripts ```txt2img``` and ```img2img```:
Use the more accurate but VRAM-intensive full precision math because half-precision requires autocast and won't work.
By default the images are saved in ```outputs/img-samples/```.
```Shell
python3 scripts/dream.py --full_precision
```
You'll get the script's prompt. You can see available options or quit.
```Shell
dream> -h
dream> q
```
## Text to Image
For quick (but bad) image results test with 5 steps (default 50) and 1 sample image. This will let you know that everything is set up correctly.
Then increase steps to 100 or more for good (but slower) results.
The prompt can be in quotes or not.
```Shell
dream> The hulk fighting with sheldon cooper -s5 -n1
dream> "woman closeup highly detailed" -s 150
# Reuse previous seed and apply face restoration
dream> "woman closeup highly detailed" --steps 150 --seed -1 -G 0.75
```
You'll need to experiment to see if face restoration is making it better or worse for your specific prompt.
If you're on a container the output is set to the Docker volume. You can copy it wherever you want.
You can download it from the Docker Desktop app, Volumes, my-vol, data.
Or you can copy it from your Mac terminal. Keep in mind ```docker cp``` can't expand ```*.png``` so you'll need to specify the image file name.
On your host Mac (you can use the name of any container that mounted the volume):
```Shell
docker cp dummy:/data/000001.928403745.png /Users/<your-user>/Pictures
```
## Image to Image
You can also do text-guided image-to-image translation. For example, turning a sketch into a detailed drawing.
```strength``` is a value between 0.0 and 1.0 that controls the amount of noise that is added to the input image. Values that approach 1.0 allow for lots of variations but will also produce images that are not semantically consistent with the input. 0.0 preserves image exactly, 1.0 replaces it completely.
Make sure your input image size dimensions are multiples of 64 e.g. 512x512. Otherwise you'll get ```Error: product of dimension sizes > 2**31'```. If you still get the error [try a different size](https://support.apple.com/guide/preview/resize-rotate-or-flip-an-image-prvw2015/mac#:~:text=image's%20file%20size-,In%20the%20Preview%20app%20on%20your%20Mac%2C%20open%20the%20file,is%20shown%20at%20the%20bottom.) like 512x256.
If you're on a Docker container, copy your input image into the Docker volume
```Shell
docker cp /Users/<your-user>/Pictures/sketch-mountains-input.jpg dummy:/data/
```
Try it out generating an image (or more). The ```dream``` script needs absolute paths to find the image so don't use ```~```.
If you're on your Mac
```Shell
dream> "A fantasy landscape, trending on artstation" -I /Users/<your-user>/Pictures/sketch-mountains-input.jpg --strength 0.75 --steps 100 -n4
```
If you're on a Linux container on your Mac
```Shell
dream> "A fantasy landscape, trending on artstation" -I /data/sketch-mountains-input.jpg --strength 0.75 --steps 50 -n1
```
## Web Interface
You can use the ```dream``` script with a graphical web interface. Start the web server with:
```Shell
python3 scripts/dream.py --full_precision --web
```
If it's running on your Mac point your Mac web browser to http://127.0.0.1:9090
Press Control-C at the command line to stop the web server.
## Notes
Some text you can add at the end of the prompt to make it very pretty:
```Shell
cinematic photo, highly detailed, cinematic lighting, ultra-detailed, ultrarealistic, photorealism, Octane Rendering, cyberpunk lights, Hyper Detail, 8K, HD, Unreal Engine, V-Ray, full hd, cyberpunk, abstract, 3d octane render + 4k UHD + immense detail + dramatic lighting + well lit + black, purple, blue, pink, cerulean, teal, metallic colours, + fine details, ultra photoreal, photographic, concept art, cinematic composition, rule of thirds, mysterious, eerie, photorealism, breathtaking detailed, painting art deco pattern, by hsiao, ron cheng, john james audubon, bizarre compositions, exquisite detail, extremely moody lighting, painted by greg rutkowski makoto shinkai takashi takeuchi studio ghibli, akihiko yoshida
```
The original scripts should work as well.
```Shell
python3 scripts/orig_scripts/txt2img.py --help
python3 scripts/orig_scripts/txt2img.py --ddim_steps 100 --n_iter 1 --n_samples 1 --plms --prompt "new born baby kitten. Hyper Detail, Octane Rendering, Unreal Engine, V-Ray"
python3 scripts/orig_scripts/txt2img.py --ddim_steps 5 --n_iter 1 --n_samples 1 --plms --prompt "ocean" # or --klms
```

30
docs/installation/INSTALL_LINUX.md
View File

@ -2,6 +2,10 @@
title: Linux
---
# :fontawesome-brands-linux: Linux
## Installation
1. You will need to install the following prerequisites if they are not already
available. Use your operating system's preferred installer.
@ -55,9 +59,11 @@ title: Linux
(ldm) ~/stable-diffusion$ python3 scripts/preload_models.py
```
Note that this step is necessary because I modified the original just-in-time
model loading scheme to allow the script to work on GPU machines that are not
internet connected. See [Preload Models](../features/OTHER.md#preload-models)
!!! note
This step is necessary because I modified the original just-in-time
model loading scheme to allow the script to work on GPU machines that are not
internet connected. See [Preload Models](../features/OTHER.md#preload-models)
7. Now you need to install the weights for the stable diffusion model.
@ -96,15 +102,15 @@ title: Linux
launch the dream script (step 8). If you forget to activate the ldm
environment, the script will fail with multiple `ModuleNotFound` errors.
### Updating to newer versions of the script
## Updating to newer versions of the script
This distribution is changing rapidly. If you used the `git clone` method
(step 5) to download the stable-diffusion directory, then to update to the
latest and greatest version, launch the Anaconda window, enter
`stable-diffusion` and type:
This distribution is changing rapidly. If you used the `git clone` method
(step 5) to download the stable-diffusion directory, then to update to the
latest and greatest version, launch the Anaconda window, enter
`stable-diffusion` and type:
```bash
(ldm) ~/stable-diffusion$ git pull
```
```bash
(ldm) ~/stable-diffusion$ git pull
```
This will bring your local copy into sync with the remote one.
This will bring your local copy into sync with the remote one.

354
docs/installation/INSTALL_MAC.md
View File

@ -2,6 +2,8 @@
title: macOS
---
# :fontawesome-brands-apple: macOS
## Requirements
- macOS 12.3 Monterey or later
@ -9,18 +11,21 @@ title: macOS
- Patience
- Apple Silicon or Intel Mac
Things have moved really fast and so these instructions change often and are
often out-of-date. One of the problems is that there are so many different ways
to run this.
Things have moved really fast and so these instructions change often which makes
them outdated pretty fast. One of the problems is that there are so many
different ways to run this.
We are trying to build a testing setup so that when we make changes it doesn't
always break.
How to (this hasn't been 100% tested yet):
## How to
First get the weights checkpoint download started - it's big:
(this hasn't been 100% tested yet)
1. Sign up at https://huggingface.co
First get the weights checkpoint download started since it's big and will take
some time:
1. Sign up at [huggingface.co](https://huggingface.co)
2. Go to the
[Stable diffusion diffusion model page](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original)
3. Accept the terms and click Access Repository:
@ -28,116 +33,147 @@ First get the weights checkpoint download started - it's big:
[sd-v1-4.ckpt (4.27 GB)](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original/blob/main/sd-v1-4.ckpt)
and note where you have saved it (probably the Downloads folder)
While that is downloading, open Terminal and run the following commands one
at a time.
While that is downloading, open a Terminal and run the following commands:
```bash
# install brew (and Xcode command line tools):
!!! todo "Homebrew"
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
=== "no brew installation yet"
#
# Now there are two different routes to get the Python (miniconda) environment up and running:
# 1. Alongside pyenv
# 2. No pyenv
#
# If you don't know what we are talking about, choose 2.
#
# NOW EITHER DO
# 1. Installing alongside pyenv
```bash title="install brew (and Xcode command line tools)"
/bin/bash -c \
"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```
=== "brew is already installed"
Only if you installed protobuf in a previous version of this tutorial, otherwise skip
brew install rust pyenv-virtualenv # you might have this from before, no problem
pyenv install anaconda3-2022.05
pyenv virtualenv anaconda3-2022.05
eval "$(pyenv init -)"
pyenv activate anaconda3-2022.05
`#!bash brew uninstall protobuf`
# OR,
# 2. Installing standalone
# install python 3, git, cmake, protobuf:
brew install cmake rust
!!! todo "Conda Installation"
# install miniconda for M1 arm64:
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh -o Miniconda3-latest-MacOSX-arm64.sh
/bin/bash Miniconda3-latest-MacOSX-arm64.sh
Now there are two different ways to set up the Python (miniconda) environment:
1. Standalone
2. with pyenv
If you don't know what we are talking about, choose Standalone
# OR install miniconda for Intel:
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -o Miniconda3-latest-MacOSX-x86_64.sh
/bin/bash Miniconda3-latest-MacOSX-x86_64.sh
=== "Standalone"
```bash
# install cmake and rust:
brew install cmake rust
```
# EITHER WAY,
# continue from here
=== "M1 arm64"
```bash title="Install miniconda for M1 arm64"
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh \
-o Miniconda3-latest-MacOSX-arm64.sh
/bin/bash Miniconda3-latest-MacOSX-arm64.sh
```
=== "Intel x86_64"
```bash title="Install miniconda for Intel"
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh \
-o Miniconda3-latest-MacOSX-x86_64.sh
/bin/bash Miniconda3-latest-MacOSX-x86_64.sh
```
=== "with pyenv"
```{.bash .annotate}
brew install rust pyenv-virtualenv # (1)!
pyenv install anaconda3-2022.05
pyenv virtualenv anaconda3-2022.05
eval "$(pyenv init -)"
pyenv activate anaconda3-2022.05
```
1. You might already have this installed, if that is the case just continue.
```{.bash .annotate title="local repo setup"}
# clone the repo
git clone https://github.com/lstein/stable-diffusion.git
cd stable-diffusion
#
# wait until the checkpoint file has downloaded, then proceed
#
# create symlink to checkpoint
mkdir -p models/ldm/stable-diffusion-v1/
PATH_TO_CKPT="$HOME/Downloads" # or wherever you saved sd-v1-4.ckpt
PATH_TO_CKPT="$HOME/Downloads" # (1)!
ln -s "$PATH_TO_CKPT/sd-v1-4.ckpt" models/ldm/stable-diffusion-v1/model.ckpt
ln -s "$PATH_TO_CKPT/sd-v1-4.ckpt" \
models/ldm/stable-diffusion-v1/model.ckpt
```
# install packages for arm64
PIP_EXISTS_ACTION=w CONDA_SUBDIR=osx-arm64 conda env create -f environment-mac.yaml
conda activate ldm
1. or wherever you saved sd-v1-4.ckpt
# OR install packages for x86_64
PIP_EXISTS_ACTION=w CONDA_SUBDIR=osx-x86_64 conda env create -f environment-mac.yaml
conda activate ldm
!!! todo "create Conda Environment"
=== "M1 arm64"
```bash
PIP_EXISTS_ACTION=w CONDA_SUBDIR=osx-arm64 \
conda env create \
-f environment-mac.yaml \
&& conda activate ldm
```
=== "Intel x86_64"
```bash
PIP_EXISTS_ACTION=w CONDA_SUBDIR=osx-x86_64 \
conda env create \
-f environment-mac.yaml \
&& conda activate ldm
```
```{.bash .annotate title="preload models and run script"}
# only need to do this once
python scripts/preload_models.py
# run SD!
python scripts/dream.py --full_precision # half-precision requires autocast and won't work
# now you can run SD in CLI mode
python scripts/dream.py --full_precision # (1)!
# or run the web interface!
python scripts/dream.py --web
# The original scripts should work as well.
python scripts/orig_scripts/txt2img.py \
--prompt "a photograph of an astronaut riding a horse" \
--plms
```
The original scripts should work as well.
1. half-precision requires autocast which is unfortunatelly incompatible
```bash
python scripts/orig_scripts/txt2img.py --prompt "a photograph of an astronaut riding a horse" --plms
```
!!! note
Note,
`#!bash export PIP_EXISTS_ACTION=w` is a precaution to fix a problem where
```bash
export PIP_EXISTS_ACTION=w
```
```bash
conda env create \
-f environment-mac.yaml
```
is a precaution to fix
```bash
conda env create -f environment-mac.yaml
```
never finishing in some situations. So it isn't required but wont hurt.
After you follow all the instructions and run dream.py you might get several
errors. Here's the errors I've seen and found solutions for.
did never finish in some situations. So it isn't required but wont hurt.
---
## Common problems
After you followed all the instructions and try to run dream.py, you might
get several errors. Here's the errors I've seen and found solutions for.
### Is it slow?
Be sure to specify 1 sample and 1 iteration.
```bash
```bash title="Be sure to specify 1 sample and 1 iteration."
python ./scripts/orig_scripts/txt2img.py \
--prompt "ocean" \
--ddim_steps 5 \
--n_samples 1 \
--n_iter 1
--prompt "ocean" \
--ddim_steps 5 \
--n_samples 1 \
--n_iter 1
```
---
@ -155,53 +191,75 @@ solution please
One debugging step is to update to the latest version of PyTorch nightly.
```bash
conda install pytorch torchvision torchaudio -c pytorch-nightly
conda install \
pytorch \
torchvision \
-c pytorch-nightly \
-n ldm
```
If it takes forever to run
```bash
conda env create -f environment-mac.yaml
conda env create \
-f environment-mac.yaml
```
you could try to run `git clean -f` followed by:
you could try to run:
`conda clean --yes --all`
```bash
git clean -f
conda clean \
--yes \
--all
```
Or you could try to completley reset Anaconda:
```bash
conda update --force-reinstall -y -n base -c defaults conda
conda update \
--force-reinstall \
-y \
-n base \
-c defaults conda
```
---
### "No module named cv2", torch, 'ldm', 'transformers', 'taming', etc
There are several causes of these errors.
There are several causes of these errors:
- First, did you remember to `conda activate ldm`? If your terminal prompt
begins with "(ldm)" then you activated it. If it begins with "(base)" or
something else you haven't.
1. Did you remember to `conda activate ldm`? If your terminal prompt begins with
"(ldm)" then you activated it. If it begins with "(base)" or something else
you haven't.
- Second, you might've run `./scripts/preload_models.py` or `./scripts/dream.py`
instead of `python ./scripts/preload_models.py` or
`python ./scripts/dream.py`. The cause of this error is long so it's below.
2. You might've run `./scripts/preload_models.py` or `./scripts/dream.py`
instead of `python ./scripts/preload_models.py` or
`python ./scripts/dream.py`. The cause of this error is long so it's below.
- Third, if it says you're missing taming you need to rebuild your virtual
environment.
<!-- I could not find out where the error is, otherwise would have marked it as a footnote -->
```bash
conda deactivate
conda env remove -n ldm
conda env create -f environment-mac.yaml
3. if it says you're missing taming you need to rebuild your virtual
environment.
Fourth, If you have activated the ldm virtual environment and tried rebuilding
it, maybe the problem could be that I have something installed that you don't
and you'll just need to manually install it. Make sure you activate the virtual
environment so it installs there instead of globally.
```bash
conda deactivate
conda env remove -n ldm
PIP_EXISTS_ACTION=w CONDA_SUBDIR=osx-arm64 \
conda env create \
-f environment-mac.yaml
```
`conda activate ldm pip install _name_`
4. If you have activated the ldm virtual environment and tried rebuilding it,
maybe the problem could be that I have something installed that you don't and
you'll just need to manually install it. Make sure you activate the virtual
environment so it installs there instead of globally.
```bash
conda activate ldm
pip install <package name>
```
You might also need to install Rust (I mention this again below).
@ -261,21 +319,20 @@ output of `python3 -V` and `python -V`.
/Users/name/miniforge3/envs/ldm/bin/python
```
The above is what you'll see if you have miniforge and you've correctly
activated the ldm environment, and you used option 2 in the setup instructions
above ("no pyenv").
The above is what you'll see if you have miniforge and correctly activated the
ldm environment, while usingd the standalone setup instructions above.
If you otherwise installed via pyenv, you will get this result:
```bash
(anaconda3-2022.05) % which python
/Users/name/.pyenv/shims/python
```
... and the above is what you'll see if you used option 1 ("Alongside pyenv").
It's all a mess and you should know
[how to modify the path environment variable](https://support.apple.com/guide/terminal/use-environment-variables-apd382cc5fa-4f58-4449-b20a-41c53c006f8f/mac)
if you want to fix it. Here's a brief hint of all the ways you can modify it
(don't really have the time to explain it all here).
if you want to fix it. Here's a brief hint of the most common ways you can
modify it (don't really have the time to explain it all here).
- ~/.zshrc
- ~/.bash_profile
@ -283,16 +340,21 @@ if you want to fix it. Here's a brief hint of all the ways you can modify it
- /etc/paths.d
- /etc/path
Which one you use will depend on what you have installed except putting a file
in /etc/paths.d is what I prefer to do.
Which one you use will depend on what you have installed, except putting a file
in /etc/paths.d - which also is the way I prefer to do.
Finally, to answer the question posed by this section's title, it may help to
list all of the `python` / `python3` things found in `$PATH` instead of just the
one that will be executed by default. To do that, add the `-a` switch to
`which`:
first hit. To do so, add the `-a` switch to `which`:
% which -a python3
...
```bash
% which -a python3
...
```
This will show a list of all binaries which are actually available in your PATH.
---
### Debugging?
@ -300,37 +362,56 @@ Tired of waiting for your renders to finish before you can see if it works?
Reduce the steps! The image quality will be horrible but at least you'll get
quick feedback.
python ./scripts/txt2img.py --prompt "ocean" --ddim_steps 5 --n_samples 1 --n_iter 1
```bash
python ./scripts/txt2img.py \
--prompt "ocean" \
--ddim_steps 5 \
--n_samples 1 \
--n_iter 1
```
### OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'...
---
python scripts/preload_models.py
### OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'
```bash
python scripts/preload_models.py
```
---
### "The operator [name] is not current implemented for the MPS device." (sic)
Example error.
!!! example "example error"
```
```bash
... NotImplementedError: The operator 'aten::_index_put_impl_' is not current
implemented for the MPS device. If you want this op to be added in priority
during the prototype phase of this feature, please comment on
https://github.com/pytorch/pytorch/issues/77764.
As a temporary fix, you can set the environment variable
`PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op.
WARNING: this will be slower than running natively on MPS.
```
... NotImplementedError: The operator 'aten::_index_put_impl_' is not current
implemented for the MPS device. If you want this op to be added in priority
during the prototype phase of this feature, please comment on
[https://github.com/pytorch/pytorch/issues/77764](https://github.com/pytorch/pytorch/issues/77764).
As a temporary fix, you can set the environment variable
`PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op.
WARNING: this will be slower than running natively on MPS.
```
The lstein branch includes this fix in
This fork already includes a fix for this in
[environment-mac.yaml](https://github.com/lstein/stable-diffusion/blob/main/environment-mac.yaml).
---
### "Could not build wheels for tokenizers"
I have not seen this error because I had Rust installed on my computer before I
started playing with Stable Diffusion. The fix is to install Rust.
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```bash
curl \
--proto '=https' \
--tlsv1.2 \
-sSf https://sh.rustup.rs | sh
```
---
### How come `--seed` doesn't work?
@ -347,7 +428,9 @@ still working on it.
### libiomp5.dylib error?
OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
```bash
OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
```
You are likely using an Intel package by mistake. Be sure to run conda with the
environment variable `CONDA_SUBDIR=osx-arm64`, like so:
@ -363,6 +446,8 @@ is a metapackage designed to prevent this, by making it impossible to install
Do _not_ use `os.environ['KMP_DUPLICATE_LIB_OK']='True'` or equivalents as this
masks the underlying issue of using Intel packages.
---
### Not enough memory
This seems to be a common problem and is probably the underlying problem for a
@ -374,6 +459,8 @@ how that would affect the quality of the images though.
See [this issue](https://github.com/CompVis/stable-diffusion/issues/71).
---
### "Error: product of dimension sizes > 2\*\*31'"
This error happens with img2img, which I haven't played with too much yet. But I
@ -388,6 +475,8 @@ BTW, 2\*\*31-1 =
is also 32-bit signed [LONG_MAX](https://en.wikipedia.org/wiki/C_data_types) in
C.
---
### I just got Rickrolled! Do I have a virus?
You don't have a virus. It's part of the project. Here's
@ -400,6 +489,8 @@ call this "computer vision", sheesh).
Actually, this could be happening because there's not enough RAM. You could try
the `model.half()` suggestion or specify smaller output images.
---
### My images come out black
We might have this fixed, we are still testing.
@ -428,7 +519,10 @@ This is a 32-bit vs 16-bit problem.
### The processor must support the Intel bla bla bla
What? Intel? On an Apple Silicon?
`bash Intel MKL FATAL ERROR: This system does not meet the minimum requirements for use of the Intel(R) Math Kernel Library. The processor must support the Intel(R) Supplemental Streaming SIMD Extensions 3 (Intel(R) SSSE3) instructions. The processor must support the Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) instructions. The processor must support the Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions. `
```bash
Intel MKL FATAL ERROR: This system does not meet the minimum requirements for use of the Intel(R) Math Kernel Library. The processor must support the Intel(R) Supplemental Streaming SIMD Extensions 3 (Intel(R) SSSE3) instructions. The processor must support the Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) instructions. The processor must support the Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
```
This is due to the Intel `mkl` package getting picked up when you try to install
something that depends on it-- Rosetta can translate some Intel instructions but
@ -453,5 +547,3 @@ Abort trap: 6
warnings.warn('resource_tracker: There appear to be %d '
```
Macs do not support `autocast/mixed-precision`, so you need to supply
`--full_precision` to use float32 everywhere.

31
docs/installation/INSTALL_WINDOWS.md
View File

@ -2,6 +2,8 @@
title: Windows
---
# :fontawesome-brands-windows: Windows
## **Notebook install (semi-automated)**
We have a
@ -28,17 +30,16 @@ in the wiki
### **Conda**
1. Install Anaconda3 (miniconda3 version) from here:
https://docs.anaconda.com/anaconda/install/windows/
1. Install Anaconda3 (miniconda3 version) from [here](https://docs.anaconda.com/anaconda/install/windows/)
2. Install Git from here: https://git-scm.com/download/win
2. Install Git from [here](https://git-scm.com/download/win)
3. Launch Anaconda from the Windows Start menu. This will bring up a command
window. Type all the remaining commands in this window.
4. Run the command:
```bash
```batch
git clone https://github.com/lstein/stable-diffusion.git
```
@ -48,15 +49,15 @@ in the wiki
5. Enter the newly-created stable-diffusion folder. From this step forward make
sure that you are working in the stable-diffusion directory!
```bash
```batch
cd stable-diffusion
```
6. Run the following two commands:
```bash
conda env create -f environment.yaml (step 6a)
conda activate ldm (step 6b)
```batch
conda env create -f environment.yaml
conda activate ldm
```
This will install all python requirements and activate the "ldm" environment
@ -64,7 +65,7 @@ in the wiki
7. Run the command:
```bash
```batch
python scripts\preload_models.py
```
@ -77,9 +78,9 @@ in the wiki
8. Now you need to install the weights for the big stable diffusion model.
- For running with the released weights, you will first need to set up an
acount with Hugging Face (https://huggingface.co).
acount with [Hugging Face](https://huggingface.co).
- Use your credentials to log in, and then point your browser at
https://huggingface.co/CompVis/stable-diffusion-v-1-4-original.
[https://huggingface.co/CompVis/stable-diffusion-v-1-4-original](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original).
- You may be asked to sign a license agreement at this point.
- Click on "Files and versions" near the top of the page, and then click on
the file named `sd-v1-4.ckpt`. You'll be taken to a page that prompts you
@ -90,7 +91,7 @@ in the wiki
Now run the following commands from **within the stable-diffusion directory**
to copy the weights file to the right place:
```bash
```batch
mkdir -p models\ldm\stable-diffusion-v1
copy C:\path\to\sd-v1-4.ckpt models\ldm\stable-diffusion-v1\model.ckpt
```
@ -102,7 +103,7 @@ in the wiki
9. Start generating images!
```bash
```batch
# for the pre-release weights
python scripts\dream.py -l
@ -122,14 +123,14 @@ in the wiki
---
### Updating to newer versions of the script
## Updating to newer versions of the script
This distribution is changing rapidly. If you used the `git clone` method
(step 5) to download the stable-diffusion directory, then to update to the
latest and greatest version, launch the Anaconda window, enter
`stable-diffusion`, and type:
```bash
```batch
git pull
conda env update -f environment.yaml
```

2
docs/other/CONTRIBUTORS.md
View File

@ -2,6 +2,8 @@
title: Contributors
---
# :octicons-person-24: Contributors
The list of all the amazing people who have contributed to the various features that you get to
experience in this fork.

18
ldm/dream/args.py
View File

@ -100,6 +100,13 @@ SAMPLER_CHOICES = [
'plms',
]
PRECISION_CHOICES = [
'auto',
'float32',
'autocast',
'float16',
]
# is there a way to pick this up during git commits?
APP_ID = 'lstein/stable-diffusion'
APP_VERSION = 'v1.15'
@ -322,7 +329,16 @@ class Args(object):
'--full_precision',
dest='full_precision',
action='store_true',
help='Use more memory-intensive full precision math for calculations',
help='Deprecated way to set --precision=float32',
)
model_group.add_argument(
'--precision',
dest='precision',
type=str,
choices=PRECISION_CHOICES,
metavar='PRECISION',
help=f'Set model precision. Defaults to auto selected based on device. Options: {", ".join(PRECISION_CHOICES)}',
default='auto',
)
file_group.add_argument(
'--from_file',

29
ldm/dream/devices.py
View File

@ -1,6 +1,6 @@
import torch
from torch import autocast
from contextlib import contextmanager, nullcontext
from contextlib import nullcontext
def choose_torch_device() -> str:
'''Convenience routine for guessing which GPU device to run model on'''
@ -10,15 +10,18 @@ def choose_torch_device() -> str:
return 'mps'
return 'cpu'
def choose_autocast_device(device):
'''Returns an autocast compatible device from a torch device'''
device_type = device.type # this returns 'mps' on M1
# autocast only for cuda, but GTX 16xx have issues with it
if device_type == 'cuda':
device_name = torch.cuda.get_device_name()
if 'GeForce GTX 1660' in device_name or 'GeForce GTX 1650' in device_name:
return device_type,nullcontext
else:
return device_type,autocast
else:
return 'cpu',nullcontext
def choose_precision(device) -> str:
'''Returns an appropriate precision for the given torch device'''
if device.type == 'cuda':
device_name = torch.cuda.get_device_name(device)
if not ('GeForce GTX 1660' in device_name or 'GeForce GTX 1650' in device_name):
return 'float16'
return 'float32'
def choose_autocast(precision):
'''Returns an autocast context or nullcontext for the given precision string'''
# float16 currently requires autocast to avoid errors like:
# 'expected scalar type Half but found Float'
if precision == 'autocast' or precision == 'float16':
return autocast
return nullcontext

9
ldm/dream/generator/base.py
View File

@ -9,14 +9,15 @@ from tqdm import tqdm, trange
from PIL import Image
from einops import rearrange, repeat
from pytorch_lightning import seed_everything
from ldm.dream.devices import choose_autocast_device
from ldm.dream.devices import choose_autocast
from ldm.util import rand_perlin_2d
downsampling = 8
class Generator():
def __init__(self,model):
def __init__(self, model, precision):
self.model = model
self.precision = precision
self.seed = None
self.latent_channels = model.channels
self.downsampling_factor = downsampling # BUG: should come from model or config
@ -39,7 +40,7 @@ class Generator():
def generate(self,prompt,init_image,width,height,iterations=1,seed=None,
image_callback=None, step_callback=None, threshold=0.0, perlin=0.0,
**kwargs):
device_type,scope = choose_autocast_device(self.model.device)
scope = choose_autocast(self.precision)
make_image = self.get_make_image(
prompt,
init_image = init_image,
@ -54,7 +55,7 @@ class Generator():
results = []
seed = seed if seed else self.new_seed()
seed, initial_noise = self.generate_initial_noise(seed, width, height)
with scope(device_type), self.model.ema_scope():
with scope(self.model.device.type), self.model.ema_scope():
for n in trange(iterations, desc='Generating'):
x_T = None
if self.variation_amount > 0:

4
ldm/dream/generator/embiggen.py
View File

@ -11,8 +11,8 @@ from ldm.models.diffusion.ddim import DDIMSampler
from ldm.dream.generator.img2img import Img2Img
class Embiggen(Generator):
def __init__(self,model):
super().__init__(model)
def __init__(self, model, precision):
super().__init__(model, precision)
self.init_latent = None
@torch.no_grad()

12
ldm/dream/generator/img2img.py
View File

@ -4,15 +4,15 @@ ldm.dream.generator.img2img descends from ldm.dream.generator
import torch
import numpy as np
from ldm.dream.devices import choose_autocast_device
from ldm.dream.devices import choose_autocast
from ldm.dream.generator.base import Generator
from ldm.models.diffusion.ddim import DDIMSampler
class Img2Img(Generator):
def __init__(self,model):
super().__init__(model)
def __init__(self, model, precision):
super().__init__(model, precision)
self.init_latent = None # by get_noise()
@torch.no_grad()
def get_make_image(self,prompt,sampler,steps,cfg_scale,ddim_eta,
conditioning,init_image,strength,step_callback=None,threshold=0.0,perlin=0.0,**kwargs):
@ -32,8 +32,8 @@ class Img2Img(Generator):
ddim_num_steps=steps, ddim_eta=ddim_eta, verbose=False
)
device_type,scope = choose_autocast_device(self.model.device)
with scope(device_type):
scope = choose_autocast(self.precision)
with scope(self.model.device.type):
self.init_latent = self.model.get_first_stage_encoding(
self.model.encode_first_stage(init_image)
) # move to latent space

12
ldm/dream/generator/inpaint.py
View File

@ -5,15 +5,15 @@ ldm.dream.generator.inpaint descends from ldm.dream.generator
import torch
import numpy as np
from einops import rearrange, repeat
from ldm.dream.devices import choose_autocast_device
from ldm.dream.devices import choose_autocast
from ldm.dream.generator.img2img import Img2Img
from ldm.models.diffusion.ddim import DDIMSampler
class Inpaint(Img2Img):
def __init__(self,model):
def __init__(self, model, precision):
self.init_latent = None
super().__init__(model)
super().__init__(model, precision)
@torch.no_grad()
def get_make_image(self,prompt,sampler,steps,cfg_scale,ddim_eta,
conditioning,init_image,mask_image,strength,
@ -38,8 +38,8 @@ class Inpaint(Img2Img):
ddim_num_steps=steps, ddim_eta=ddim_eta, verbose=False
)
device_type,scope = choose_autocast_device(self.model.device)
with scope(device_type):
scope = choose_autocast(self.precision)
with scope(self.model.device.type):
self.init_latent = self.model.get_first_stage_encoding(
self.model.encode_first_stage(init_image)
) # move to latent space

6
ldm/dream/generator/txt2img.py
View File

@ -7,9 +7,9 @@ import numpy as np
from ldm.dream.generator.base import Generator
class Txt2Img(Generator):
def __init__(self,model):
super().__init__(model)
def __init__(self, model, precision):
super().__init__(model, precision)
@torch.no_grad()
def get_make_image(self,prompt,sampler,steps,cfg_scale,ddim_eta,
conditioning,width,height,step_callback=None,threshold=0.0,perlin=0.0,**kwargs):

61
ldm/dream/log.py Normal file
View File

@ -0,0 +1,61 @@
"""
Functions for better format logging
write_log -- logs the name of the output image, prompt, and prompt args to the terminal and different types of file
1 write_log_message -- Writes a message to the console
2 write_log_files -- Writes a message to files
2.1 write_log_default -- File in plain text
2.2 write_log_txt -- File in txt format
2.3 write_log_markdown -- File in markdown format
"""
import os
def write_log(results, log_path, file_types, output_cntr):
"""
logs the name of the output image, prompt, and prompt args to the terminal and files
"""
output_cntr = write_log_message(results, output_cntr)
write_log_files(results, log_path, file_types)
return output_cntr
def write_log_message(results, output_cntr):
"""logs to the terminal"""
log_lines = [f"{path}: {prompt}\n" for path, prompt in results]
for l in log_lines:
output_cntr += 1
print(f"[{output_cntr}] {l}", end="")
return output_cntr
def write_log_files(results, log_path, file_types):
for file_type in file_types:
if file_type == "txt":
write_log_txt(log_path, results)
elif file_type == "md" or file_type == "markdown":
write_log_markdown(log_path, results)
else:
print(f"'{file_type}' format is not supported, so write in plain text")
write_log_default(log_path, results, file_type)
def write_log_default(log_path, results, file_type):
plain_txt_lines = [f"{path}: {prompt}\n" for path, prompt in results]
with open(log_path + "." + file_type, "a", encoding="utf-8") as file:
file.writelines(plain_txt_lines)
def write_log_txt(log_path, results):
txt_lines = [f"{path}: {prompt}\n" for path, prompt in results]
with open(log_path + ".txt", "a", encoding="utf-8") as file:
file.writelines(txt_lines)
def write_log_markdown(log_path, results):
md_lines = []
for path, prompt in results:
file_name = os.path.basename(path)
md_lines.append(f"## {file_name}\n![]({file_name})\n\n{prompt}\n")
with open(log_path + ".md", "a", encoding="utf-8") as file:
file.writelines(md_lines)

42
ldm/generate.py
View File

@ -29,7 +29,7 @@ from ldm.models.diffusion.plms import PLMSSampler
from ldm.models.diffusion.ksampler import KSampler
from ldm.dream.pngwriter import PngWriter
from ldm.dream.image_util import InitImageResizer
from ldm.dream.devices import choose_torch_device
from ldm.dream.devices import choose_torch_device, choose_precision
from ldm.dream.conditioning import get_uc_and_c
def fix_func(orig):
@ -104,7 +104,7 @@ gr = Generate(
# these values are set once and shouldn't be changed
conf = path to configuration file ('configs/models.yaml')
model = symbolic name of the model in the configuration file
full_precision = False
precision = float precision to be used
# this value is sticky and maintained between generation calls
sampler_name = ['ddim', 'k_dpm_2_a', 'k_dpm_2', 'k_euler_a', 'k_euler', 'k_heun', 'k_lms', 'plms'] // k_lms
@ -130,6 +130,7 @@ class Generate:
sampler_name = 'k_lms',
ddim_eta = 0.0, # deterministic
full_precision = False,
precision = 'auto',
# these are deprecated; if present they override values in the conf file
weights = None,
config = None,
@ -145,7 +146,7 @@ class Generate:
self.cfg_scale = 7.5
self.sampler_name = sampler_name
self.ddim_eta = 0.0 # same seed always produces same image
self.full_precision = True if choose_torch_device() == 'mps' else full_precision
self.precision = precision
self.strength = 0.75
self.seamless = False
self.embedding_path = embedding_path
@ -162,6 +163,14 @@ class Generate:
# it wasn't actually doing anything. This logic could be reinstated.
device_type = choose_torch_device()
self.device = torch.device(device_type)
if full_precision:
if self.precision != 'auto':
raise ValueError('Remove --full_precision / -F if using --precision')
print('Please remove deprecated --full_precision / -F')
print('If auto config does not work you can use --precision=float32')
self.precision = 'float32'
if self.precision == 'auto':
self.precision = choose_precision(self.device)
# for VRAM usage statistics
self.session_peakmem = torch.cuda.max_memory_allocated() if self._has_cuda else None
@ -450,25 +459,25 @@ class Generate:
def _make_img2img(self):
if not self.generators.get('img2img'):
from ldm.dream.generator.img2img import Img2Img
self.generators['img2img'] = Img2Img(self.model)
self.generators['img2img'] = Img2Img(self.model, self.precision)
return self.generators['img2img']
def _make_embiggen(self):
if not self.generators.get('embiggen'):
from ldm.dream.generator.embiggen import Embiggen
self.generators['embiggen'] = Embiggen(self.model)
self.generators['embiggen'] = Embiggen(self.model, self.precision)
return self.generators['embiggen']
def _make_txt2img(self):
if not self.generators.get('txt2img'):
from ldm.dream.generator.txt2img import Txt2Img
self.generators['txt2img'] = Txt2Img(self.model)
self.generators['txt2img'] = Txt2Img(self.model, self.precision)
return self.generators['txt2img']
def _make_inpaint(self):
if not self.generators.get('inpaint'):
from ldm.dream.generator.inpaint import Inpaint
self.generators['inpaint'] = Inpaint(self.model)
self.generators['inpaint'] = Inpaint(self.model, self.precision)
return self.generators['inpaint']
def load_model(self):
@ -479,7 +488,7 @@ class Generate:
model = self._load_model_from_config(self.config, self.weights)
if self.embedding_path is not None:
model.embedding_manager.load(
self.embedding_path, self.full_precision
self.embedding_path, self.precision == 'float32' or self.precision == 'autocast'
)
self.model = model.to(self.device)
# model.to doesn't change the cond_stage_model.device used to move the tokenizer output, so set it here
@ -629,16 +638,13 @@ class Generate:
sd = pl_sd['state_dict']
model = instantiate_from_config(c.model)
m, u = model.load_state_dict(sd, strict=False)
if self.full_precision:
print(
'>> Using slower but more accurate full-precision math (--full_precision)'
)
if self.precision == 'float16':
print('Using faster float16 precision')
model.to(torch.float16)
else:
print(
'>> Using half precision math. Call with --full_precision to use more accurate but VRAM-intensive full precision.'
)
model.half()
print('Using more accurate float32 precision')
model.to(self.device)
model.eval()

16
mkdocs.yml
View File

@ -1,6 +1,8 @@
# General
site_name: Dream Script Docs
site_url: https://lstein.github.io/stable-diffusion/
site_author: mauwii
dev_addr: "127.0.0.1:8080"
# Repository
repo_name: lstein/stable-diffusion
@ -13,18 +15,16 @@ copyright: Copyright &copy; 2022 Lincoln D. Stein <lincoln.stein@gmail.com>
# Configuration
theme:
name: material
features:
- toc.integrate
icon:
repo: fontawesome/brands/github
edit: material/file-document-edit-outline
palette:
- media: '(prefers-color-scheme: light)'
primary: blue
scheme: default
toggle:
icon: material/lightbulb
name: Switch to dark mode
- media: '(prefers-color-scheme: dark)'
primary: dark-blue
accent: white
scheme: slate
toggle:
icon: material/lightbulb-outline
@ -37,7 +37,6 @@ markdown_extensions:
- attr_list
- def_list
- footnotes
- meta
- md_in_html
- toc:
permalink: '#'
@ -56,7 +55,7 @@ markdown_extensions:
- pymdownx.keys
- pymdownx.magiclink:
repo_url_shorthand: true
user: 'Mauwii'
user: 'lstein'
repo: 'stable-diffusion'
- pymdownx.mark
- pymdownx.smartsymbols
@ -75,4 +74,5 @@ markdown_extensions:
plugins:
- search
- git-revision-date
- git-revision-date-localized:
enable_creation_date: true

25
requirements-linux-arm64.txt Normal file
View File

@ -0,0 +1,25 @@
albumentations==0.4.3
einops==0.3.0
huggingface-hub==0.8.1
imageio==2.9.0
imageio-ffmpeg==0.4.2
kornia==0.6.0
numpy==1.23.1
--pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu
omegaconf==2.1.1
opencv-python==4.6.0.66
pillow==9.2.0
pudb==2019.2
torch==1.12.1
torchvision==0.13.0
pytorch-lightning==1.4.2
streamlit==1.12.0
test-tube>=0.7.5
torch-fidelity==0.3.0
torchmetrics==0.6.0
transformers==4.19.2
-e git+https://github.com/openai/CLIP.git@main#egg=clip
-e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers
git+https://github.com/lstein/k-diffusion.git@master#egg=k-diffusion
-e git+https://github.com/lstein/GFPGAN@fix-dark-cast-images#egg=gfpgan
-e .

5
requirements-mkdocs.txt
View File

@ -1,2 +1,3 @@
mkdocs-material
mkdocs-git-revision-date-plugin>=0.3.2
mkdocs
mkdocs-material>=8, <9
mkdocs-git-revision-date-localized-plugin

19
scripts/dream.py
View File

@ -12,6 +12,7 @@ from ldm.dream.args import Args, metadata_dumps
from ldm.dream.pngwriter import PngWriter
from ldm.dream.server import DreamServer, ThreadingDreamServer
from ldm.dream.image_util import make_grid
from ldm.dream.log import write_log
from omegaconf import OmegaConf
# Placeholder to be replaced with proper class that tracks the
@ -53,6 +54,7 @@ def main():
sampler_name = opt.sampler_name,
embedding_path = opt.embedding_path,
full_precision = opt.full_precision,
precision = opt.precision,
)
except (FileNotFoundError, IOError, KeyError) as e:
print(f'{e}. Aborting.')
@ -292,8 +294,9 @@ def main_loop(gen, opt, infile):
continue
print('Outputs:')
log_path = os.path.join(current_outdir, 'dream_log.txt')
write_log_message(results, log_path)
log_path = os.path.join(current_outdir, 'dream_log')
global output_cntr
output_cntr = write_log(results, log_path ,('txt', 'md'), output_cntr)
print()
print('goodbye!')
@ -339,17 +342,5 @@ def dream_server_loop(gen, host, port, outdir):
dream_server.server_close()
def write_log_message(results, log_path):
"""logs the name of the output image, prompt, and prompt args to the terminal and log file"""
global output_cntr
log_lines = [f'{path}: {prompt}\n' for path, prompt in results]
for l in log_lines:
output_cntr += 1
print(f'[{output_cntr}] {l}',end='')
with open(log_path, 'a', encoding='utf-8') as file:
file.writelines(log_lines)
if __name__ == '__main__':
main()

2
server/application.py
View File

@ -119,7 +119,7 @@ def main():
# "height": height,
# "sampler_name": opt.sampler_name,
# "weights": weights,
# "full_precision": opt.full_precision,
# "precision": opt.precision,
# "config": config,
# "grid": opt.grid,
# "latent_diffusion_weights": opt.laion400m,

4
server/containers.py
View File

@ -23,14 +23,14 @@ class Container(containers.DeclarativeContainer):
model = config.model,
sampler_name = config.sampler_name,
embedding_path = config.embedding_path,
full_precision = config.full_precision
precision = config.precision
# config = config.model.config,
# width = config.model.width,
# height = config.model.height,
# sampler_name = config.model.sampler_name,
# weights = config.model.weights,
# full_precision = config.model.full_precision,
# precision = config.model.precision,
# grid = config.model.grid,
# seamless = config.model.seamless,
# embedding_path = config.model.embedding_path,