Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

update CONEPTS.md

Browse Source 
- use table with correct syntax for screenshots
- switch Title and first Headline to look better in ToC
This commit is contained in:
mauwii 2022-12-02 06:33:39 +01:00 committed by Lincoln Stein
parent bf399e303c
commit 470b267939
1 changed files with 70 additions and 90 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

160
docs/features/CONCEPTS.md
View File

@ -1,130 +1,110 @@
--- ---
title: The Hugging Face Concepts Library and Importing Textual Inversion files title: Concepts Library
--- ---
# :material-file-document: Concepts Library # :material-library-shelves: The Hugging Face Concepts Library and Importing Textual Inversion files
## Using Textual Inversion Files ## Using Textual Inversion Files
Textual inversion (TI) files are small models that customize the output of Textual inversion (TI) files are small models that customize the output of
Stable Diffusion image generation. They can augment SD with Stable Diffusion image generation. They can augment SD with specialized subjects
specialized subjects and artistic styles. They are also known as and artistic styles. They are also known as "embeds" in the machine learning
"embeds" in the machine learning world. world.
Each TI file introduces one or more vocabulary terms to the SD Each TI file introduces one or more vocabulary terms to the SD model. These are
model. These are known in InvokeAI as "triggers." Triggers are often, known in InvokeAI as "triggers." Triggers are often, but not always, denoted
but not always, denoted using angle brackets as in using angle brackets as in "<trigger-phrase>". The two most common type of
"<trigger-phrase>". The two most common type of TI files that you'll TI files that you'll encounter are `.pt` and `.bin` files, which are produced by
encounter are `.pt` and `.bin` files, which are produced by different different TI training packages. InvokeAI supports both formats, but its
TI training packages. InvokeAI supports both formats, but its [built-in [built-in TI training system](TEXTUAL_INVERSION.md) produces `.pt`.
TI training system](TEXTUAL_INVERSION.md) produces `.pt`.
The [Hugging Face company](https://huggingface.co/sd-concepts-library) The [Hugging Face company](https://huggingface.co/sd-concepts-library) has
has amassed a large ligrary of >800 community-contributed TI files amassed a large ligrary of >800 community-contributed TI files covering a
covering a broad range of subjects and styles. InvokeAI has built-in broad range of subjects and styles. InvokeAI has built-in support for this
support for this library which downloads and merges TI files library which downloads and merges TI files automatically upon request. You can
automatically upon request. You can also install your own or others' also install your own or others' TI files by placing them in a designated
TI files by placing them in a designated directory. directory.
### An Example ### An Example
Here are a few examples to illustrate how it works. All these images Here are a few examples to illustrate how it works. All these images were
were generated using the command-line client and the Stable Diffusion generated using the command-line client and the Stable Diffusion 1.5 model:
1.5 model:
Japanese gardener | Japanese gardener | Japanese gardener <ghibli-face> | Japanese gardener <hoi4-leaders> | Japanese gardener <cartoona-animals> |
<br> | :--------------------------------: | :-----------------------------------: | :------------------------------------: | :----------------------------------------: |
<img src="../assets/concepts/image1.png"> | ![](../assets/concepts/image1.png) | ![](../assets/concepts/image2.png) | ![](../assets/concepts/image3.png) | ![](../assets/concepts/image4.png) |
Japanese gardener &lt;ghibli-face&gt;
<br>
<img src="../assets/concepts/image2.png">
Japanese gardener &lt;hoi4-leaders&gt;
<br>
<img src="../assets/concepts/image3.png">
Japanese gardener &lt;cartoona-animals&gt;
<br>
<img src="../assets/concepts/image4.png">
You can also combine styles and concepts: You can also combine styles and concepts:
A portrait of &lt;alf&gt; in &lt;cartoona-animal&gt; style <figure markdown>
<br> ![](../assets/concepts/image5.png)
<img src="../assets/concepts/image5.png"> <figcaption>A portrait of &lt;alf&gt; in &lt;cartoona-animal&gt; style</figcaption>
</figure>
## Using a Hugging Face Concept ## Using a Hugging Face Concept
Hugging Face TI concepts are downloaded and installed automatically as Hugging Face TI concepts are downloaded and installed automatically as you
you require them. This requires your machine to be connected to the require them. This requires your machine to be connected to the Internet. To
Internet. To find out what each concept is for, you can browse the find out what each concept is for, you can browse the
[Hugging Face concepts [Hugging Face concepts library](https://huggingface.co/sd-concepts-library) and
library](https://huggingface.co/sd-concepts-library) and look at look at examples of what each concept produces.
examples of what each concept produces.
When you have an idea of a concept you wish to try, go to the When you have an idea of a concept you wish to try, go to the command-line
command-line client (CLI) and type a "&lt;" character and the beginning client (CLI) and type a "&lt;" character and the beginning of the Hugging Face
of the Hugging Face concept name you wish to load. Press the Tab key, concept name you wish to load. Press the Tab key, and the CLI will show you all
and the CLI will show you all matching concepts. You can also type "&lt;" matching concepts. You can also type "&lt;" and Tab to get a listing of all ~800
and Tab to get a listing of all ~800 concepts, but be prepared to concepts, but be prepared to scroll up to see them all! If there is more than
scroll up to see them all! If there is more than one match you can one match you can continue to type and Tab until the concept is completed.
continue to type and Tab until the concept is completed.
For example if you type "&lt;x" and Tab, you'll be prompted with the completions: For example if you type "&lt;x" and Tab, you'll be prompted with the
completions:
``` ```
<xatu2> <xatu> <xbh> <xi> <xidiversity> <xioboma> <xuna> <xyz> <xatu2> <xatu> <xbh> <xi> <xidiversity> <xioboma> <xuna> <xyz>
``` ```
Now type "id" and press Tab. It will be autocompleted to Now type "id" and press Tab. It will be autocompleted to "&lt;xidiversity&gt;"
"&lt;xidiversity&gt;" because this is a unique match. because this is a unique match.
Finish your prompt and generate as usual. You may include multiple Finish your prompt and generate as usual. You may include multiple concept terms
concept terms in the prompt. in the prompt.
If you have never used this concept before, you will see a message If you have never used this concept before, you will see a message that the TI
that the TI model is being downloaded and installed. After this, the model is being downloaded and installed. After this, the concept will be saved
concept will be saved locally (in the `models/sd-concepts-library` locally (in the `models/sd-concepts-library` directory) for future use.
directory) for future use.
Several steps happen during downloading and Several steps happen during downloading and installation, including a scan of
installation, including a scan of the file for malicious code. Should the file for malicious code. Should any errors occur, you will be warned and the
any errors occur, you will be warned and the concept will fail to concept will fail to load. Generation will then continue treating the trigger
load. Generation will then continue treating the trigger term as a term as a normal string of characters (e.g. as literal "&lt;ghibli-face&gt;").
normal string of characters (e.g. as literal "&lt;ghibli-face&gt;").
Currently auto-installation of concepts is a feature only available on Currently auto-installation of concepts is a feature only available on the
the command-line client. Support for the WebUI is a work in progress. command-line client. Support for the WebUI is a work in progress.
## Installing your Own TI Files ## Installing your Own TI Files
You may install any number of `.pt` and `.bin` files simply by copying You may install any number of `.pt` and `.bin` files simply by copying them into
them into the `embeddings` directory of the InvokeAI runtime directory the `embeddings` directory of the InvokeAI runtime directory (usually `invokeai`
(usually `invokeai` in your home directory). You may create in your home directory). You may create subdirectories in order to organize the
subdirectories in order to organize the files in any way you wish. Be files in any way you wish. Be careful not to overwrite one file with another.
careful not to overwrite one file with another. For example, TI files For example, TI files generated by the Hugging Face toolkit share the named
generated by the Hugging Face toolkit share the named `learned_embedding.bin`. You can use subdirectories to keep them distinct.
`learned_embedding.bin`. You can use subdirectories to keep them
distinct.
At startup time, InvokeAI will scan the `embeddings` directory and At startup time, InvokeAI will scan the `embeddings` directory and load any TI
load any TI files it finds there. At startup you will see a message files it finds there. At startup you will see a message similar to this one:
similar to this one:
``` ```bash
>> Current embedding manager terms: *, <HOI4-Leader>, <princess-knight> >> Current embedding manager terms: *, <HOI4-Leader>, <princess-knight>
``` ```
Note the "*" trigger term. This is a placeholder term that many early Note the `*` trigger term. This is a placeholder term that many early TI
TI tutorials taught people to use rather than a more descriptive tutorials taught people to use rather than a more descriptive term.
term. Unfortunately, if you have multiple TI files that all use this Unfortunately, if you have multiple TI files that all use this term, only the
term, only the first one loaded will be triggered by use of the term. first one loaded will be triggered by use of the term.
To avoid this problem, you can use the `merge_embeddings.py` script to To avoid this problem, you can use the `merge_embeddings.py` script to merge two
merge two or more TI files together. If it encounters a collision of or more TI files together. If it encounters a collision of terms, the script
terms, the script will prompt you to select new terms that do not will prompt you to select new terms that do not collide. See
collide. See [Textual Inversion](TEXTUAL_INVERSION.md) for details. [Textual Inversion](TEXTUAL_INVERSION.md) for details.
## Further Reading ## Further Reading