diff --git a/docs/assets/concepts/image1.png b/docs/assets/concepts/image1.png new file mode 100644 index 0000000000..f8c93efcf9 Binary files /dev/null and b/docs/assets/concepts/image1.png differ diff --git a/docs/assets/concepts/image2.png b/docs/assets/concepts/image2.png new file mode 100644 index 0000000000..a22411492e Binary files /dev/null and b/docs/assets/concepts/image2.png differ diff --git a/docs/assets/concepts/image3.png b/docs/assets/concepts/image3.png new file mode 100644 index 0000000000..e2213fc707 Binary files /dev/null and b/docs/assets/concepts/image3.png differ diff --git a/docs/assets/concepts/image4.png b/docs/assets/concepts/image4.png new file mode 100644 index 0000000000..052479019c Binary files /dev/null and b/docs/assets/concepts/image4.png differ diff --git a/docs/assets/concepts/image5.png b/docs/assets/concepts/image5.png new file mode 100644 index 0000000000..f3a4f76470 Binary files /dev/null and b/docs/assets/concepts/image5.png differ diff --git a/docs/features/CLI.md b/docs/features/CLI.md index 5139cdf145..b7d3eb3fe3 100644 --- a/docs/features/CLI.md +++ b/docs/features/CLI.md @@ -269,6 +269,12 @@ value, we are insisting on a more stringent classification. invoke> a piece of cake -I /path/to/breakfast.png -tm bagel 0.6 ``` +### Custom Styles and Subjects + +You can load and use hundreds of community-contributed Textual +Inversion models just by typing the appropriate trigger phrase. Please +see [Concepts Library](CONCEPTS.md) for more details. + # Other Commands The CLI offers a number of commands that begin with "!". diff --git a/docs/features/CONCEPTS.md b/docs/features/CONCEPTS.md new file mode 100644 index 0000000000..1b5a7472ea --- /dev/null +++ b/docs/features/CONCEPTS.md @@ -0,0 +1,132 @@ +--- +title: The Hugging Face Concepts Library and Importing Textual Inversion files +--- + +# :material-file-document: Concepts Library + +## Using Textual Inversion Files + +Textual inversion (TI) files are small models that customize the output of +Stable Diffusion image generation. They can augment SD with +specialized subjects and artistic styles. They are also known as +"embeds" in the machine learning world. + +Each TI file introduces one or more vocabulary terms to the SD +model. These are known in InvokeAI as "triggers." Triggers are often, +but not always, denoted using angle brackets as in +"<trigger-phrase>". The two most common type of TI files that you'll +encounter are `.pt` and `.bin` files, which are produced by different +TI training packages. InvokeAI supports both formats, but its [built-in +TI training system](TEXTUAL_INVERSION.md) produces `.pt`. + +The [Hugging Face company](https://huggingface.co/sd-concepts-library) +has amassed a large ligrary of >800 community-contributed TI files +covering a broad range of subjects and styles. InvokeAI has built-in +support for this library which downloads and merges TI files +automatically upon request. You can also install your own or others' +TI files by placing them in a designated directory. + +### An Example + +Here are a few examples to illustrate how it works. All these images +were generated using the command-line client and the Stable Diffusion +1.5 model: + +Japanese gardener +
+ + +Japanese gardener <ghibli-face> +
+ + +Japanese gardener <hoi4-leaders> +
+ + +Japanese gardener <cartoona-animals> +
+ + +You can also combine styles and concepts: + +A portrait of <alf> in <cartoona-animal> style +
+ + +## Using a Hugging Face Concept + +Hugging Face TI concepts are downloaded and installed automatically as +you require them. This requires your machine to be connected to the +Internet. To find out what each concept is for, you can browse the +[Hugging Face concepts +library](https://huggingface.co/sd-concepts-library) and look at +examples of what each concept produces. + +When you have an idea of a concept you wish to try, go to the +command-line client (CLI) and type a "<" character and the beginning +of the Hugging Face concept name you wish to load. Press the Tab key, +and the CLI will show you all matching concepts. You can also type "<" +and Tab to get a listing of all ~800 concepts, but be prepared to +scroll up to see them all! If there is more than one match you can +continue to type and Tab until the concept is completed. + +For example if you type "<x" and Tab, you'll be prompted with the completions: + +``` + +``` + +Now type "id" and press Tab. It will be autocompleted to +"<xidiversity>" because this is a unique match. + +Finish your prompt and generate as usual. You may include multiple +concept terms in the prompt. + +If you have never used this concept before, you will see a message +that the TI model is being downloaded and installed. After this, the +concept will be saved locally (in the `models/sd-concepts-library` +directory) for future use. + +Several steps happen during downloading and +installation, including a scan of the file for malicious code. Should +any errors occur, you will be warned and the concept will fail to +load. Generation will then continue treating the trigger term as a +normal string of characters (e.g. as literal "<ghibli-face>"). + +Currently auto-installation of concepts is a feature only available on +the command-line client. Support for the WebUI is a work in progress. + +## Installing your Own TI Files + +You may install any number of `.pt` and `.bin` files simply by copying +them into the `embeddings` directory of the InvokeAI runtime directory +(usually `invokeai` in your home directory). You may create +subdirectories in order to organize the files in any way you wish. Be +careful not to overwrite one file with another. For example, TI files +generated by the Hugging Face toolkit share the named +`learned_embedding.bin`. You can use subdirectories to keep them +distinct. + +At startup time, InvokeAI will scan the `embeddings` directory and +load any TI files it finds there. At startup you will see a message +similar to this one: + +``` +>> Current embedding manager terms: *, , +``` + +Note the "*" trigger term. This is a placeholder term that many early +TI tutorials taught people to use rather than a more descriptive +term. Unfortunately, if you have multiple TI files that all use this +term, only the first one loaded will be triggered by use of the term. + +To avoid this problem, you can use the `merge_embeddings.py` script to +merge two or more TI files together. If it encounters a collision of +terms, the script will prompt you to select new terms that do not +collide. See [Textual Inversion](TEXTUAL_INVERSION.md) for details. + +## Further Reading + +Please see [the repository](https://github.com/rinongal/textual_inversion) and +associated paper for details and limitations. diff --git a/docs/features/OTHER.md b/docs/features/OTHER.md index e2e1c746fb..2ba76e912a 100644 --- a/docs/features/OTHER.md +++ b/docs/features/OTHER.md @@ -133,29 +133,6 @@ outputs = g.txt2img("a unicorn in manhattan") Outputs is a list of lists in the format [filename1,seed1],[filename2,seed2]...]. -Please see ldm/generate.py for more information. A set of example scripts is coming RSN. +Please see the documentation in ldm/generate.py for more information. --- - -## **Preload Models** - -In situations where you have limited internet connectivity or are blocked behind a firewall, you can -use the preload script to preload the required files for Stable Diffusion to run. - -The preload script `scripts/preload_models.py` needs to be run once at least while connected to the -internet. In the following runs, it will load up the cached versions of the required files from the -`.cache` directory of the system. - -```bash -(invokeai) ~/stable-diffusion$ python3 ./scripts/preload_models.py -preloading bert tokenizer... -Downloading: 100%|██████████████████████████████████| 28.0/28.0 [00:00<00:00, 49.3kB/s] -Downloading: 100%|██████████████████████████████████| 226k/226k [00:00<00:00, 2.79MB/s] -Downloading: 100%|██████████████████████████████████| 455k/455k [00:00<00:00, 4.36MB/s] -Downloading: 100%|██████████████████████████████████| 570/570 [00:00<00:00, 477kB/s] -...success -preloading kornia requirements... -Downloading: "https://github.com/DagnyT/hardnet/raw/master/pretrained/train_liberty_with_aug/checkpoint_liberty_with_aug.pth" to /u/lstein/.cache/torch/hub/checkpoints/checkpoint_liberty_with_aug.pth -100%|███████████████████████████████████████████████| 5.10M/5.10M [00:00<00:00, 101MB/s] -...success -``` diff --git a/docs/index.md b/docs/index.md index 903c3a53bc..1e1bfc75dc 100644 --- a/docs/index.md +++ b/docs/index.md @@ -119,6 +119,26 @@ You wil need one of the following: ```bash (invokeai) ~/InvokeAI$ python scripts/invoke.py --full_precision ``` +## :octicons-gift-24: InvokeAI Features + +- [The InvokeAI Web Interface](features/WEB.md) + - [WebGUI hotkey reference guide](features/WEBUIHOTKEYS.md) + - [WebGUI Unified Canvas for Img2Img, inpainting and outpainting](features/UNIFIED_CANVAS.md) + +- [The Command Line Interace](features/CLI.md) + - [Image2Image](features/IMG2IMG.md) + - [Inpainting][(features/INPAINTING.md) + - [Outpainting](features/OUTPAINTING.md) + - [Adding custom styles and subjects](features/CONCEPTS.md) + - [Upscaling and Face Reconstruction](features/POSTPROCESS.md) + +- [Generating Variations](features/VARIATIONS.md) + +- [Prompt Engineering](features/PROMPTS.md) + +- Miscellaneous + - [Embiggen upscaling](features/EMBIGGEN.md) + - [Other](features/OTHER.md) ## :octicons-log-16: Latest Changes