mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
Update CONCEPTS.md
This commit is contained in:
parent
2eddd5db7d
commit
75b28eb79b
@ -1,9 +1,12 @@
|
|||||||
---
|
---
|
||||||
title: Concepts Library
|
title: Concepts
|
||||||
---
|
---
|
||||||
|
|
||||||
# :material-library-shelves: The Hugging Face Concepts Library and Importing Textual Inversion files
|
# :material-library-shelves: The Hugging Face Concepts Library and Importing Textual Inversion files
|
||||||
|
|
||||||
|
With the advances in research, many new capabilities are available to customize the knowledge and understanding of novel concepts not originally contained in the base model.
|
||||||
|
|
||||||
|
|
||||||
## Using Textual Inversion Files
|
## Using Textual Inversion Files
|
||||||
|
|
||||||
Textual inversion (TI) files are small models that customize the output of
|
Textual inversion (TI) files are small models that customize the output of
|
||||||
@ -12,18 +15,16 @@ and artistic styles. They are also known as "embeds" in the machine learning
|
|||||||
world.
|
world.
|
||||||
|
|
||||||
Each TI file introduces one or more vocabulary terms to the SD model. These are
|
Each TI file introduces one or more vocabulary terms to the SD model. These are
|
||||||
known in InvokeAI as "triggers." Triggers are often, but not always, denoted
|
known in InvokeAI as "triggers." Triggers are denoted using angle brackets
|
||||||
using angle brackets as in "<trigger-phrase>". The two most common type of
|
as in "<trigger-phrase>". The two most common type of
|
||||||
TI files that you'll encounter are `.pt` and `.bin` files, which are produced by
|
TI files that you'll encounter are `.pt` and `.bin` files, which are produced by
|
||||||
different TI training packages. InvokeAI supports both formats, but its
|
different TI training packages. InvokeAI supports both formats, but its
|
||||||
[built-in TI training system](TEXTUAL_INVERSION.md) produces `.pt`.
|
[built-in TI training system](TRAINING.md) produces `.pt`.
|
||||||
|
|
||||||
The [Hugging Face company](https://huggingface.co/sd-concepts-library) has
|
The [Hugging Face company](https://huggingface.co/sd-concepts-library) has
|
||||||
amassed a large ligrary of >800 community-contributed TI files covering a
|
amassed a large ligrary of >800 community-contributed TI files covering a
|
||||||
broad range of subjects and styles. InvokeAI has built-in support for this
|
broad range of subjects and styles. You can also install your own or others' TI files
|
||||||
library which downloads and merges TI files automatically upon request. You can
|
by placing them in the designated directory for the compatible model type
|
||||||
also install your own or others' TI files by placing them in a designated
|
|
||||||
directory.
|
|
||||||
|
|
||||||
### An Example
|
### An Example
|
||||||
|
|
||||||
@ -41,66 +42,43 @@ You can also combine styles and concepts:
|
|||||||
| :--------------------------------------------------------: |
|
| :--------------------------------------------------------: |
|
||||||
|  |
|
|  |
|
||||||
</figure>
|
</figure>
|
||||||
## Using a Hugging Face Concept
|
|
||||||
|
|
||||||
!!! warning "Authenticating to HuggingFace"
|
|
||||||
|
|
||||||
Some concepts require valid authentication to HuggingFace. Without it, they will not be downloaded
|
|
||||||
and will be silently ignored.
|
|
||||||
|
|
||||||
If you used an installer to install InvokeAI, you may have already set a HuggingFace token.
|
|
||||||
If you skipped this step, you can:
|
|
||||||
|
|
||||||
- run the InvokeAI configuration script again (if you used a manual installer): `invokeai-configure`
|
|
||||||
- set one of the `HUGGINGFACE_TOKEN` or `HUGGING_FACE_HUB_TOKEN` environment variables to contain your token
|
|
||||||
|
|
||||||
Finally, if you already used any HuggingFace library on your computer, you might already have a token
|
|
||||||
in your local cache. Check for a hidden `.huggingface` directory in your home folder. If it
|
|
||||||
contains a `token` file, then you are all set.
|
|
||||||
|
|
||||||
|
|
||||||
Hugging Face TI concepts are downloaded and installed automatically as you
|
|
||||||
require them. This requires your machine to be connected to the Internet. To
|
|
||||||
find out what each concept is for, you can browse the
|
|
||||||
[Hugging Face concepts library](https://huggingface.co/sd-concepts-library) and
|
|
||||||
look at examples of what each concept produces.
|
|
||||||
|
|
||||||
To load concepts, you will need to open the Web UI's configuration
|
|
||||||
dialogue and activate "Show Textual Inversions from HF Concepts
|
|
||||||
Library". This will then add a list of HF Concepts to the dropdown
|
|
||||||
"Add Textual Inversion" menu. Select the concept(s) of your choice and
|
|
||||||
they will be incorporated into the positive prompt. A few concepts are
|
|
||||||
designed for the negative prompt, in which case you can add them to
|
|
||||||
the negative prompt box by select the down arrow icon next to the
|
|
||||||
textual inversion menu.
|
|
||||||
|
|
||||||
There are nearly 1000 HF concepts, more than will fit into a menu. For
|
|
||||||
this reason we only show the most popular concepts (those which have
|
|
||||||
received 5 or more likes). If you wish to use a concept that is not on
|
|
||||||
the list, you may simply type its name surrounded by brackets. For
|
|
||||||
example, to load the concept named "xidiversity", add `<xidiversity>`
|
|
||||||
to the positive or negative prompt text.
|
|
||||||
|
|
||||||
## Installing your Own TI Files
|
## Installing your Own TI Files
|
||||||
|
|
||||||
You may install any number of `.pt` and `.bin` files simply by copying them into
|
You may install any number of `.pt` and `.bin` files simply by copying them into
|
||||||
the `embeddings` directory of the InvokeAI runtime directory (usually `invokeai`
|
the `embedding` directory of the corresponding InvokeAI models directory (usually `invokeai`
|
||||||
in your home directory). You may create subdirectories in order to organize the
|
in your home directory). For example, you can simply move a Stable Diffusion 1.5 embedding file to
|
||||||
files in any way you wish. Be careful not to overwrite one file with another.
|
the `sd-1/embedding` folder. Be careful not to overwrite one file with another.
|
||||||
For example, TI files generated by the Hugging Face toolkit share the named
|
For example, TI files generated by the Hugging Face toolkit share the named
|
||||||
`learned_embedding.bin`. You can use subdirectories to keep them distinct.
|
`learned_embedding.bin`. You can rename these, or use subdirectories to keep them distinct.
|
||||||
|
|
||||||
At startup time, InvokeAI will scan the `embeddings` directory and load any TI
|
At startup time, InvokeAI will scan the various `embedding` directories and load any TI
|
||||||
files it finds there. At startup you will see a message similar to this one:
|
files it finds there for compatible models. At startup you will see a message similar to this one:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
>> Current embedding manager terms: <HOI4-Leader>, <princess-knight>
|
>> Current embedding manager terms: <HOI4-Leader>, <princess-knight>
|
||||||
```
|
```
|
||||||
|
To use these when generating, simply type the `<` key in your prompt to open the Textual Inversion WebUI and
|
||||||
|
select the embedding you'd like to use. This UI has type-ahead support, so you can easily find supported embeddings.
|
||||||
|
|
||||||
The terms you can use will appear in the "Add Textual Inversion"
|
## Using LoRAs
|
||||||
dropdown menu above the HF Concepts.
|
|
||||||
|
|
||||||
## Further Reading
|
LoRA files are models that customize the output of Stable Diffusion image generation.
|
||||||
|
Larger than embeddings, but much smaller than full models, they augment SD with improved
|
||||||
|
understanding of subjects and artistic styles.
|
||||||
|
|
||||||
|
Unlike TI files, LoRAs do not introduce novel vocabulary into the model's known tokens. Instead,
|
||||||
|
LoRAs augment the model's weights that are applied to generate imagery. LoRAs may be supplied
|
||||||
|
with a "trigger" word that they have been explicitly trained on, or may simply apply their
|
||||||
|
effect without being triggered.
|
||||||
|
|
||||||
|
LoRAs are typically stored in .safetensors files, which are the most secure way to store and transmit
|
||||||
|
these types of weights. You may install any number of `.safetensors` LoRA files simply by copying them into
|
||||||
|
the `lora` directory of the corresponding InvokeAI models directory (usually `invokeai`
|
||||||
|
in your home directory). For example, you can simply move a Stable Diffusion 1.5 LoRA file to
|
||||||
|
the `sd-1/lora` folder.
|
||||||
|
|
||||||
|
To use these when generating, open the LoRA menu item in the options panel, select the LoRAs you want to apply
|
||||||
|
and ensure that they have the appropriate weight recommended by the model provider. Typically, most LoRAs perform best at a weight of .75-1.
|
||||||
|
|
||||||
Please see [the repository](https://github.com/rinongal/textual_inversion) and
|
|
||||||
associated paper for details and limitations.
|
|
||||||
|
Loading…
x
Reference in New Issue
Block a user