mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
- Add documentation for the Hugging Face concepts library and TI embedding. - Fixup index.md to point to each of the feature documentation files, including ones that are pending.
133 lines
5.2 KiB
Markdown
133 lines
5.2 KiB
Markdown
---
|
|
title: The Hugging Face Concepts Library and Importing Textual Inversion files
|
|
---
|
|
|
|
# :material-file-document: Concepts Library
|
|
|
|
## Using Textual Inversion Files
|
|
|
|
Textual inversion (TI) files are small models that customize the output of
|
|
Stable Diffusion image generation. They can augment SD with
|
|
specialized subjects and artistic styles. They are also known as
|
|
"embeds" in the machine learning world.
|
|
|
|
Each TI file introduces one or more vocabulary terms to the SD
|
|
model. These are known in InvokeAI as "triggers." Triggers are often,
|
|
but not always, denoted using angle brackets as in
|
|
"<trigger-phrase>". The two most common type of TI files that you'll
|
|
encounter are `.pt` and `.bin` files, which are produced by different
|
|
TI training packages. InvokeAI supports both formats, but its [built-in
|
|
TI training system](TEXTUAL_INVERSION.md) produces `.pt`.
|
|
|
|
The [Hugging Face company](https://huggingface.co/sd-concepts-library)
|
|
has amassed a large ligrary of >800 community-contributed TI files
|
|
covering a broad range of subjects and styles. InvokeAI has built-in
|
|
support for this library which downloads and merges TI files
|
|
automatically upon request. You can also install your own or others'
|
|
TI files by placing them in a designated directory.
|
|
|
|
### An Example
|
|
|
|
Here are a few examples to illustrate how it works. All these images
|
|
were generated using the command-line client and the Stable Diffusion
|
|
1.5 model:
|
|
|
|
Japanese gardener
|
|
<br>
|
|
<img src="../assets/concepts/image1.png">
|
|
|
|
Japanese gardener <ghibli-face>
|
|
<br>
|
|
<img src="../assets/concepts/image2.png">
|
|
|
|
Japanese gardener <hoi4-leaders>
|
|
<br>
|
|
<img src="../assets/concepts/image3.png">
|
|
|
|
Japanese gardener <cartoona-animals>
|
|
<br>
|
|
<img src="../assets/concepts/image4.png">
|
|
|
|
You can also combine styles and concepts:
|
|
|
|
A portrait of <alf> in <cartoona-animal> style
|
|
<br>
|
|
<img src="../assets/concepts/image5.png">
|
|
|
|
## Using a Hugging Face Concept
|
|
|
|
Hugging Face TI concepts are downloaded and installed automatically as
|
|
you require them. This requires your machine to be connected to the
|
|
Internet. To find out what each concept is for, you can browse the
|
|
[Hugging Face concepts
|
|
library](https://huggingface.co/sd-concepts-library) and look at
|
|
examples of what each concept produces.
|
|
|
|
When you have an idea of a concept you wish to try, go to the
|
|
command-line client (CLI) and type a "<" character and the beginning
|
|
of the Hugging Face concept name you wish to load. Press the Tab key,
|
|
and the CLI will show you all matching concepts. You can also type "<"
|
|
and Tab to get a listing of all ~800 concepts, but be prepared to
|
|
scroll up to see them all! If there is more than one match you can
|
|
continue to type and Tab until the concept is completed.
|
|
|
|
For example if you type "<x" and Tab, you'll be prompted with the completions:
|
|
|
|
```
|
|
<xatu2> <xatu> <xbh> <xi> <xidiversity> <xioboma> <xuna> <xyz>
|
|
```
|
|
|
|
Now type "id" and press Tab. It will be autocompleted to
|
|
"<xidiversity>" because this is a unique match.
|
|
|
|
Finish your prompt and generate as usual. You may include multiple
|
|
concept terms in the prompt.
|
|
|
|
If you have never used this concept before, you will see a message
|
|
that the TI model is being downloaded and installed. After this, the
|
|
concept will be saved locally (in the `models/sd-concepts-library`
|
|
directory) for future use.
|
|
|
|
Several steps happen during downloading and
|
|
installation, including a scan of the file for malicious code. Should
|
|
any errors occur, you will be warned and the concept will fail to
|
|
load. Generation will then continue treating the trigger term as a
|
|
normal string of characters (e.g. as literal "<ghibli-face>").
|
|
|
|
Currently auto-installation of concepts is a feature only available on
|
|
the command-line client. Support for the WebUI is a work in progress.
|
|
|
|
## Installing your Own TI Files
|
|
|
|
You may install any number of `.pt` and `.bin` files simply by copying
|
|
them into the `embeddings` directory of the InvokeAI runtime directory
|
|
(usually `invokeai` in your home directory). You may create
|
|
subdirectories in order to organize the files in any way you wish. Be
|
|
careful not to overwrite one file with another. For example, TI files
|
|
generated by the Hugging Face toolkit share the named
|
|
`learned_embedding.bin`. You can use subdirectories to keep them
|
|
distinct.
|
|
|
|
At startup time, InvokeAI will scan the `embeddings` directory and
|
|
load any TI files it finds there. At startup you will see a message
|
|
similar to this one:
|
|
|
|
```
|
|
>> Current embedding manager terms: *, <HOI4-Leader>, <princess-knight>
|
|
```
|
|
|
|
Note the "*" trigger term. This is a placeholder term that many early
|
|
TI tutorials taught people to use rather than a more descriptive
|
|
term. Unfortunately, if you have multiple TI files that all use this
|
|
term, only the first one loaded will be triggered by use of the term.
|
|
|
|
To avoid this problem, you can use the `merge_embeddings.py` script to
|
|
merge two or more TI files together. If it encounters a collision of
|
|
terms, the script will prompt you to select new terms that do not
|
|
collide. See [Textual Inversion](TEXTUAL_INVERSION.md) for details.
|
|
|
|
## Further Reading
|
|
|
|
Please see [the repository](https://github.com/rinongal/textual_inversion) and
|
|
associated paper for details and limitations.
|