documentation enhancements (#1603)

- Add documentation for the Hugging Face concepts library and TI embedding.

- Fixup index.md to point to each of the feature documentation files,
  including ones that are pending.
This commit is contained in:
Lincoln Stein 2022-11-28 18:48:56 -05:00 committed by GitHub
parent 45e51bac9a
commit 6cc56043e2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 159 additions and 24 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 122 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 99 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 107 KiB

View File

@ -269,6 +269,12 @@ value, we are insisting on a more stringent classification.
invoke> a piece of cake -I /path/to/breakfast.png -tm bagel 0.6
```
### Custom Styles and Subjects
You can load and use hundreds of community-contributed Textual
Inversion models just by typing the appropriate trigger phrase. Please
see [Concepts Library](CONCEPTS.md) for more details.
# Other Commands
The CLI offers a number of commands that begin with "!".

132
docs/features/CONCEPTS.md Normal file
View File

@ -0,0 +1,132 @@
---
title: The Hugging Face Concepts Library and Importing Textual Inversion files
---
# :material-file-document: Concepts Library
## Using Textual Inversion Files
Textual inversion (TI) files are small models that customize the output of
Stable Diffusion image generation. They can augment SD with
specialized subjects and artistic styles. They are also known as
"embeds" in the machine learning world.
Each TI file introduces one or more vocabulary terms to the SD
model. These are known in InvokeAI as "triggers." Triggers are often,
but not always, denoted using angle brackets as in
"<trigger-phrase>". The two most common type of TI files that you'll
encounter are `.pt` and `.bin` files, which are produced by different
TI training packages. InvokeAI supports both formats, but its [built-in
TI training system](TEXTUAL_INVERSION.md) produces `.pt`.
The [Hugging Face company](https://huggingface.co/sd-concepts-library)
has amassed a large ligrary of >800 community-contributed TI files
covering a broad range of subjects and styles. InvokeAI has built-in
support for this library which downloads and merges TI files
automatically upon request. You can also install your own or others'
TI files by placing them in a designated directory.
### An Example
Here are a few examples to illustrate how it works. All these images
were generated using the command-line client and the Stable Diffusion
1.5 model:
Japanese gardener
<br>
<img src="../assets/concepts/image1.png">
Japanese gardener &lt;ghibli-face&gt;
<br>
<img src="../assets/concepts/image2.png">
Japanese gardener &lt;hoi4-leaders&gt;
<br>
<img src="../assets/concepts/image3.png">
Japanese gardener &lt;cartoona-animals&gt;
<br>
<img src="../assets/concepts/image4.png">
You can also combine styles and concepts:
A portrait of &lt;alf&gt; in &lt;cartoona-animal&gt; style
<br>
<img src="../assets/concepts/image5.png">
## Using a Hugging Face Concept
Hugging Face TI concepts are downloaded and installed automatically as
you require them. This requires your machine to be connected to the
Internet. To find out what each concept is for, you can browse the
[Hugging Face concepts
library](https://huggingface.co/sd-concepts-library) and look at
examples of what each concept produces.
When you have an idea of a concept you wish to try, go to the
command-line client (CLI) and type a "&lt;" character and the beginning
of the Hugging Face concept name you wish to load. Press the Tab key,
and the CLI will show you all matching concepts. You can also type "&lt;"
and Tab to get a listing of all ~800 concepts, but be prepared to
scroll up to see them all! If there is more than one match you can
continue to type and Tab until the concept is completed.
For example if you type "&lt;x" and Tab, you'll be prompted with the completions:
```
<xatu2> <xatu> <xbh> <xi> <xidiversity> <xioboma> <xuna> <xyz>
```
Now type "id" and press Tab. It will be autocompleted to
"&lt;xidiversity&gt;" because this is a unique match.
Finish your prompt and generate as usual. You may include multiple
concept terms in the prompt.
If you have never used this concept before, you will see a message
that the TI model is being downloaded and installed. After this, the
concept will be saved locally (in the `models/sd-concepts-library`
directory) for future use.
Several steps happen during downloading and
installation, including a scan of the file for malicious code. Should
any errors occur, you will be warned and the concept will fail to
load. Generation will then continue treating the trigger term as a
normal string of characters (e.g. as literal "&lt;ghibli-face&gt;").
Currently auto-installation of concepts is a feature only available on
the command-line client. Support for the WebUI is a work in progress.
## Installing your Own TI Files
You may install any number of `.pt` and `.bin` files simply by copying
them into the `embeddings` directory of the InvokeAI runtime directory
(usually `invokeai` in your home directory). You may create
subdirectories in order to organize the files in any way you wish. Be
careful not to overwrite one file with another. For example, TI files
generated by the Hugging Face toolkit share the named
`learned_embedding.bin`. You can use subdirectories to keep them
distinct.
At startup time, InvokeAI will scan the `embeddings` directory and
load any TI files it finds there. At startup you will see a message
similar to this one:
```
>> Current embedding manager terms: *, <HOI4-Leader>, <princess-knight>
```
Note the "*" trigger term. This is a placeholder term that many early
TI tutorials taught people to use rather than a more descriptive
term. Unfortunately, if you have multiple TI files that all use this
term, only the first one loaded will be triggered by use of the term.
To avoid this problem, you can use the `merge_embeddings.py` script to
merge two or more TI files together. If it encounters a collision of
terms, the script will prompt you to select new terms that do not
collide. See [Textual Inversion](TEXTUAL_INVERSION.md) for details.
## Further Reading
Please see [the repository](https://github.com/rinongal/textual_inversion) and
associated paper for details and limitations.

View File

@ -133,29 +133,6 @@ outputs = g.txt2img("a unicorn in manhattan")
Outputs is a list of lists in the format [filename1,seed1],[filename2,seed2]...].
Please see ldm/generate.py for more information. A set of example scripts is coming RSN.
Please see the documentation in ldm/generate.py for more information.
---
## **Preload Models**
In situations where you have limited internet connectivity or are blocked behind a firewall, you can
use the preload script to preload the required files for Stable Diffusion to run.
The preload script `scripts/preload_models.py` needs to be run once at least while connected to the
internet. In the following runs, it will load up the cached versions of the required files from the
`.cache` directory of the system.
```bash
(invokeai) ~/stable-diffusion$ python3 ./scripts/preload_models.py
preloading bert tokenizer...
Downloading: 100%|██████████████████████████████████| 28.0/28.0 [00:00<00:00, 49.3kB/s]
Downloading: 100%|██████████████████████████████████| 226k/226k [00:00<00:00, 2.79MB/s]
Downloading: 100%|██████████████████████████████████| 455k/455k [00:00<00:00, 4.36MB/s]
Downloading: 100%|██████████████████████████████████| 570/570 [00:00<00:00, 477kB/s]
...success
preloading kornia requirements...
Downloading: "https://github.com/DagnyT/hardnet/raw/master/pretrained/train_liberty_with_aug/checkpoint_liberty_with_aug.pth" to /u/lstein/.cache/torch/hub/checkpoints/checkpoint_liberty_with_aug.pth
100%|███████████████████████████████████████████████| 5.10M/5.10M [00:00<00:00, 101MB/s]
...success
```

View File

@ -119,6 +119,26 @@ You wil need one of the following:
```bash
(invokeai) ~/InvokeAI$ python scripts/invoke.py --full_precision
```
## :octicons-gift-24: InvokeAI Features
- [The InvokeAI Web Interface](features/WEB.md)
- [WebGUI hotkey reference guide](features/WEBUIHOTKEYS.md)
- [WebGUI Unified Canvas for Img2Img, inpainting and outpainting](features/UNIFIED_CANVAS.md)
- [The Command Line Interace](features/CLI.md)
- [Image2Image](features/IMG2IMG.md)
- [Inpainting][(features/INPAINTING.md)
- [Outpainting](features/OUTPAINTING.md)
- [Adding custom styles and subjects](features/CONCEPTS.md)
- [Upscaling and Face Reconstruction](features/POSTPROCESS.md)
- [Generating Variations](features/VARIATIONS.md)
- [Prompt Engineering](features/PROMPTS.md)
- Miscellaneous
- [Embiggen upscaling](features/EMBIGGEN.md)
- [Other](features/OTHER.md)
## :octicons-log-16: Latest Changes