1e1f871ee1
* add whole <style token> to vocab for concept library embeddings * add ability to load multiple concept .bin files * make --log_tokenization respect custom tokens * start working on concept downloading system * preliminary support for dynamic loading and merging of multiple embedded models - The embedding_manager is now enhanced with ldm.invoke.concepts_lib, which handles dynamic downloading and caching of embedded models from the Hugging Face concepts library (https://huggingface.co/sd-concepts-library) - Downloading of a embedded model is triggered by the presence of one or more <concept> tags in the prompt. - Once the embedded model is downloaded, its trigger phrase will be loaded into the embedding manager and the prompt's <concept> tag will be replaced with the <trigger_phrase> - The downloaded model stays on disk for fast loading later. - The CLI autocomplete will complete partial <concept> tags for you. Type a '<' and hit tab to get all ~700 concepts. BUGS AND LIMITATIONS: - MODEL NAME VS TRIGGER PHRASE You must use the name of the concept embed model from the SD library, and not the trigger phrase itself. Usually these are the same, but not always. For example, the model named "hoi4-leaders" corresponds to the trigger "<HOI4-Leader>" One reason for this design choice is that there is no apparent constraint on the uniqueness of the trigger phrases and one trigger phrase may map onto multiple models. So we use the model name instead. The second reason is that there is no way I know of to search Hugging Face for models with certain trigger phrases. So we'd have to download all 700 models to index the phrases. The problem this presents is that this may confuse users, who will want to reuse prompts from distributions that use the trigger phrase directly. Usually this will work, but not always. - WON'T WORK ON A FIREWALLED SYSTEM If the host running IAI has no internet connection, it can't download the concept libraries. I will add a script that allows users to preload a list of concept models. - BUG IN PROMPT REPLACEMENT WHEN MODEL NOT FOUND There's a small bug that occurs when the user provides an invalid model name. The <concept> gets replaced with <None> in the prompt. * fix loading .pt embeddings; allow multi-vector embeddings; warn on dupes * simplify replacement logic and remove cuda assumption * download list of concepts from hugging face * remove misleading customization of '*' placeholder the existing code as-is did not do anything; unclear what it was supposed to do. the obvious alternative -- setting using 'placeholder_strings' instead of 'placeholder_tokens' to match model.params.personalization_config.params.placeholder_strings -- caused a crash. i think this is because the passed string also needed to be handed over on init of the PersonalizedBase as the 'placeholder_token' argument. this is weird config dict magic and i don't want to touch it. put a breakpoint in personalzied.py line 116 (top of PersonalizedBase.__init__) if you want to have a crack at it yourself. * address all the issues raised by damian0815 in review of PR #1526 * actually resize the token_embeddings * multiple improvements to the concept loader based on code reviews 1. Activated the --embedding_directory option (alias --embedding_path) to load a single embedding or an entire directory of embeddings at startup time. 2. Can turn off automatic loading of embeddings using --no-embeddings. 3. Embedding checkpoints are scanned with the pickle scanner. 4. More informative error messages when a concept can't be loaded due either to a 404 not found error or a network error. * autocomplete terms end with ">" now * fix startup error and network unreachable 1. If the .invokeai file does not contain the --root and --outdir options, invoke.py will now fix it. 2. Catch and handle network problems when downloading hugging face textual inversion concepts. * fix misformatted error string Co-authored-by: Damian Stewart <d@damianstewart.com> |
||
|---|---|---|
| .dev_scripts | ||
| .github | ||
| assets | ||
| backend | ||
| configs | ||
| docker-build | ||
| docs | ||
| environments-and-requirements | ||
| frontend | ||
| installer | ||
| ldm | ||
| notebooks | ||
| scripts | ||
| server | ||
| source_installer | ||
| static | ||
| tests | ||
| .dockerignore | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| .prettierrc.yaml | ||
| CODE_OF_CONDUCT.md | ||
| LICENSE | ||
| LICENSE-ModelWeights.txt | ||
| main.py | ||
| mkdocs.yml | ||
| README.md | ||
| setup.py | ||
| shell.nix | ||
| Stable_Diffusion_v1_Model_Card.md | ||
InvokeAI: A Stable Diffusion Toolkit
Formerly known as lstein/stable-diffusion
This is a fork of CompVis/stable-diffusion, the open source text-to-image generator. It provides a streamlined process with various new features and options to aid the image generation process. It runs on Windows, Mac and Linux machines, with GPU cards with as little as 4 GB of RAM. It provides both a polished Web interface (see below), and an easy-to-use command-line interface.
Quick links: [Discord Server] [Documentation and Tutorials] [Code and Downloads] [Bug Reports] [Discussion, Ideas & Q&A]
Note: This fork is rapidly evolving. Please use the Issues tab to report bugs and make feature requests. Be sure to use the provided templates. They will help aid diagnose issues faster.
Table of Contents
- Installation
- Hardware Requirements
- Features
- Latest Changes
- Troubleshooting
- Contributing
- Contributors
- Support
- Further Reading
Installation
This fork is supported across Linux, Windows and Macintosh. Linux users can use either an Nvidia-based card (with CUDA support) or an AMD card (using the ROCm driver). For full installation and upgrade instructions, please see: InvokeAI Installation Overview
Hardware Requirements
System
You wil need one of the following:
- An NVIDIA-based graphics card with 4 GB or more VRAM memory.
- An Apple computer with an M1 chip.
Memory
- At least 12 GB Main Memory RAM.
Disk
- At least 12 GB of free disk space for the machine learning model, Python, and all its dependencies.
Note
If you have a Nvidia 10xx series card (e.g. the 1080ti), please run the dream script in full-precision mode as shown below.
Similarly, specify full-precision mode on Apple M1 hardware.
Precision is auto configured based on the device. If however you encounter
errors like 'expected type Float but found Half' or 'not implemented for Half'
you can try starting invoke.py with the --precision=float32 flag:
(invokeai) ~/InvokeAI$ python scripts/invoke.py --precision=float32
Features
Major Features
- Web Server
- Interactive Command Line Interface
- Image To Image
- Inpainting Support
- Outpainting Support
- Upscaling, face-restoration and outpainting
- Reading Prompts From File
- Prompt Blending
- Thresholding and Perlin Noise Initialization Options
- Negative/Unconditioned Prompts
- Variations
- Personalizing Text-to-Image Generation
- Simplified API for text to image generation
Other Features
Latest Changes
-
v2.0.1 (13 October 2022)
- fix noisy images at high step count when using k* samplers
- dream.py script now calls invoke.py module directly rather than via a new python process (which could break the environment)
-
v2.0.0 (9 October 2022)
dream.pyscript renamedinvoke.py. Adream.pyscript wrapper remains for backward compatibility.- Completely new WebGUI - launch with
python3 scripts/invoke.py --web - Support for inpainting and outpainting
- img2img runs on all k* samplers
- Support for negative prompts
- Support for CodeFormer face reconstruction
- Support for Textual Inversion on Macintoshes
- Support in both WebGUI and CLI for post-processing of previously-generated images
using facial reconstruction, ESRGAN upscaling, outcropping (similar to DALL-E infinite canvas),
and "embiggen" upscaling. See the
!fixcommand. - New
--hiresoption oninvoke>line allows larger images to be created without duplicating elements, at the cost of some performance. - New
--perlinand--thresholdoptions allow you to add and control variation during image generation (see Thresholding and Perlin Noise Initialization - Extensive metadata now written into PNG files, allowing reliable regeneration of images and tweaking of previous settings.
- Command-line completion in
invoke.pynow works on Windows, Linux and Mac platforms. - Improved command-line completion behavior.
New commands added:
- List command-line history with
!history - Search command-line history with
!search - Clear history with
!clear
- List command-line history with
- Deprecated
--full_precision/-F. Simply omit it andinvoke.pywill auto configure. To switch away from auto use the new flag like--precision=float32.
For older changelogs, please visit the CHANGELOG.
Troubleshooting
Please check out our Q&A to get solutions for common installation problems and other issues.
Contributing
Anyone who wishes to contribute to this project, whether documentation, features, bug fixes, code cleanup, testing, or code reviews, is very much encouraged to do so. If you are unfamiliar with how to contribute to GitHub projects, here is a Getting Started Guide.
A full set of contribution guidelines, along with templates, are in progress, but for now the most important thing is to make your pull request against the "development" branch, and not against "main". This will help keep public breakage to a minimum and will allow you to propose more radical changes.
Contributors
This fork is a combined effort of various people from across the world. Check out the list of all these amazing people. We thank them for their time, hard work and effort.
Support
For support, please use this repository's GitHub Issues tracking service. Feel free to send me an email if you use and like the script.
Original portions of the software are Copyright (c) 2020 Lincoln D. Stein
Further Reading
Please see the original README for more information on this software and underlying algorithm, located in the file README-CompViz.md.