mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
132 lines
6.2 KiB
Markdown
132 lines
6.2 KiB
Markdown
---
|
|
title: Concepts Library
|
|
---
|
|
|
|
# :material-library-shelves: The Hugging Face Concepts Library and Importing Textual Inversion files
|
|
|
|
## Using Textual Inversion Files
|
|
|
|
Textual inversion (TI) files are small models that customize the output of
|
|
Stable Diffusion image generation. They can augment SD with specialized subjects
|
|
and artistic styles. They are also known as "embeds" in the machine learning
|
|
world.
|
|
|
|
Each TI file introduces one or more vocabulary terms to the SD model. These are
|
|
known in InvokeAI as "triggers." Triggers are often, but not always, denoted
|
|
using angle brackets as in "<trigger-phrase>". The two most common type of
|
|
TI files that you'll encounter are `.pt` and `.bin` files, which are produced by
|
|
different TI training packages. InvokeAI supports both formats, but its
|
|
[built-in TI training system](TEXTUAL_INVERSION.md) produces `.pt`.
|
|
|
|
The [Hugging Face company](https://huggingface.co/sd-concepts-library) has
|
|
amassed a large ligrary of >800 community-contributed TI files covering a
|
|
broad range of subjects and styles. InvokeAI has built-in support for this
|
|
library which downloads and merges TI files automatically upon request. You can
|
|
also install your own or others' TI files by placing them in a designated
|
|
directory.
|
|
|
|
### An Example
|
|
|
|
Here are a few examples to illustrate how it works. All these images were
|
|
generated using the command-line client and the Stable Diffusion 1.5 model:
|
|
|
|
| Japanese gardener | Japanese gardener <ghibli-face> | Japanese gardener <hoi4-leaders> | Japanese gardener <cartoona-animals> |
|
|
| :--------------------------------: | :-----------------------------------: | :------------------------------------: | :----------------------------------------: |
|
|
|  |  |  |  |
|
|
|
|
You can also combine styles and concepts:
|
|
|
|
<figure markdown>
|
|
| A portrait of <alf> in <cartoona-animal> style |
|
|
| :--------------------------------------------------------: |
|
|
|  |
|
|
</figure>
|
|
## Using a Hugging Face Concept
|
|
|
|
!!! warning "Authenticating to HuggingFace"
|
|
|
|
Some concepts require valid authentication to HuggingFace. Without it, they will not be downloaded
|
|
and will be silently ignored.
|
|
|
|
If you used an installer to install InvokeAI, you may have already set a HuggingFace token.
|
|
If you skipped this step, you can:
|
|
|
|
- run the InvokeAI configuration script again (if you used a manual installer): `invokeai-configure`
|
|
- set one of the `HUGGINGFACE_TOKEN` or `HUGGING_FACE_HUB_TOKEN` environment variables to contain your token
|
|
|
|
Finally, if you already used any HuggingFace library on your computer, you might already have a token
|
|
in your local cache. Check for a hidden `.huggingface` directory in your home folder. If it
|
|
contains a `token` file, then you are all set.
|
|
|
|
|
|
Hugging Face TI concepts are downloaded and installed automatically as you
|
|
require them. This requires your machine to be connected to the Internet. To
|
|
find out what each concept is for, you can browse the
|
|
[Hugging Face concepts library](https://huggingface.co/sd-concepts-library) and
|
|
look at examples of what each concept produces.
|
|
|
|
When you have an idea of a concept you wish to try, go to the command-line
|
|
client (CLI) and type a `<` character and the beginning of the Hugging Face
|
|
concept name you wish to load. Press ++tab++, and the CLI will show you all
|
|
matching concepts. You can also type `<` and hit ++tab++ to get a listing of all
|
|
~800 concepts, but be prepared to scroll up to see them all! If there is more
|
|
than one match you can continue to type and ++tab++ until the concept is
|
|
completed.
|
|
|
|
!!! example
|
|
|
|
if you type in `<x` and hit ++tab++, you'll be prompted with the completions:
|
|
|
|
```py
|
|
<xatu2> <xatu> <xbh> <xi> <xidiversity> <xioboma> <xuna> <xyz>
|
|
```
|
|
|
|
Now type `id` and press ++tab++. It will be autocompleted to `<xidiversity>`
|
|
because this is a unique match.
|
|
|
|
Finish your prompt and generate as usual. You may include multiple concept terms
|
|
in the prompt.
|
|
|
|
If you have never used this concept before, you will see a message that the TI
|
|
model is being downloaded and installed. After this, the concept will be saved
|
|
locally (in the `models/sd-concepts-library` directory) for future use.
|
|
|
|
Several steps happen during downloading and installation, including a scan of
|
|
the file for malicious code. Should any errors occur, you will be warned and the
|
|
concept will fail to load. Generation will then continue treating the trigger
|
|
term as a normal string of characters (e.g. as literal `<ghibli-face>`).
|
|
|
|
You can also use `<concept-names>` in the WebGUI's prompt textbox. There is no
|
|
autocompletion at this time.
|
|
|
|
## Installing your Own TI Files
|
|
|
|
You may install any number of `.pt` and `.bin` files simply by copying them into
|
|
the `embeddings` directory of the InvokeAI runtime directory (usually `invokeai`
|
|
in your home directory). You may create subdirectories in order to organize the
|
|
files in any way you wish. Be careful not to overwrite one file with another.
|
|
For example, TI files generated by the Hugging Face toolkit share the named
|
|
`learned_embedding.bin`. You can use subdirectories to keep them distinct.
|
|
|
|
At startup time, InvokeAI will scan the `embeddings` directory and load any TI
|
|
files it finds there. At startup you will see a message similar to this one:
|
|
|
|
```bash
|
|
>> Current embedding manager terms: *, <HOI4-Leader>, <princess-knight>
|
|
```
|
|
|
|
Note the `*` trigger term. This is a placeholder term that many early TI
|
|
tutorials taught people to use rather than a more descriptive term.
|
|
Unfortunately, if you have multiple TI files that all use this term, only the
|
|
first one loaded will be triggered by use of the term.
|
|
|
|
To avoid this problem, you can use the `merge_embeddings.py` script to merge two
|
|
or more TI files together. If it encounters a collision of terms, the script
|
|
will prompt you to select new terms that do not collide. See
|
|
[Textual Inversion](TEXTUAL_INVERSION.md) for details.
|
|
|
|
## Further Reading
|
|
|
|
Please see [the repository](https://github.com/rinongal/textual_inversion) and
|
|
associated paper for details and limitations.
|