mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
Merge branch 'main' into lstein-improve-ti-frontend
This commit is contained in:
Lincoln Stein
GitHub
77
docs/features/MODEL_MERGING.md
Normal file
77
docs/features/MODEL_MERGING.md
Normal file
@ -0,0 +1,77 @@
|
||||
---
|
||||
title: Model Merging
|
||||
---
|
||||
|
||||
# :material-image-off: Model Merging
|
||||
|
||||
## How to Merge Models
|
||||
|
||||
As of version 2.3, InvokeAI comes with a script that allows you to
|
||||
merge two or three diffusers-type models into a new merged model. The
|
||||
resulting model will combine characteristics of the original, and can
|
||||
be used to teach an old model new tricks.
|
||||
|
||||
You may run the merge script by starting the invoke launcher
|
||||
(`invoke.sh` or `invoke.bat`) and choosing the option for _merge
|
||||
models_. This will launch a text-based interactive user interface that
|
||||
prompts you to select the models to merge, how to merge them, and the
|
||||
merged model name.
|
||||
|
||||
Alternatively you may activate InvokeAI's virtual environment from the
|
||||
command line, and call the script via `merge_models_fe.py` (the "fe"
|
||||
stands for "front end"). There is also a version that accepts
|
||||
command-line arguments, which you can run with the command
|
||||
`merge_models.py`.
|
||||
|
||||
The user interface for the text-based interactive script is
|
||||
straightforward. It shows you a series of setting fields. Use control-N (^N)
|
||||
to move to the next field, and control-P (^P) to move to the previous
|
||||
one. You can also use TAB and shift-TAB to move forward and
|
||||
backward. Once you are in a multiple choice field, use the up and down
|
||||
cursor arrows to move to your desired selection, and press <SPACE> or
|
||||
<ENTER> to select it. Change text fields by typing in them, and adjust
|
||||
scrollbars using the left and right arrow keys.
|
||||
|
||||
Once you are happy with your settings, press the OK button. Note that
|
||||
there may be two pages of settings, depending on the height of your
|
||||
screen, and the OK button may be on the second page. Advance past the
|
||||
last field of the first page to get to the second page, and reverse
|
||||
this to get back.
|
||||
|
||||
If the merge runs successfully, it will create a new diffusers model
|
||||
under the selected name and register it with InvokeAI.
|
||||
|
||||
## The Settings
|
||||
|
||||
* Model Selection -- there are three multiple choice fields that
|
||||
display all the diffusers-style models that InvokeAI knows about.
|
||||
If you do not see the model you are looking for, then it is probably
|
||||
a legacy checkpoint model and needs to be converted using the
|
||||
`invoke.py` command-line client and its `!optimize` command. You
|
||||
must select at least two models to merge. The third can be left at
|
||||
"None" if you desire.
|
||||
|
||||
* Alpha -- This is the ratio to use when combining models. It ranges
|
||||
from 0 to 1. The higher the value, the more weight is given to the
|
||||
2d and (optionally) 3d models. So if you have two models named "A"
|
||||
and "B", an alpha value of 0.25 will give you a merged model that is
|
||||
25% A and 75% B.
|
||||
|
||||
* Interpolation Method -- This is the method used to combine
|
||||
weights. The options are "weighted_sum" (the default), "sigmoid",
|
||||
"inv_sigmoid" and "add_difference". Each produces slightly different
|
||||
results. When three models are in use, only "add_difference" is
|
||||
available. (TODO: cite a reference that describes what these
|
||||
interpolation methods actually do and how to decide among them).
|
||||
|
||||
* Force -- Not all models are compatible with each other. The merge
|
||||
script will check for compatibility and refuse to merge ones that
|
||||
are incompatible. Set this checkbox to try merging anyway.
|
||||
|
||||
* Name for merged model - This is the name for the new model. Please
|
||||
use InvokeAI conventions - only alphanumeric letters and the
|
||||
characters ".+-".
|
||||
|
||||
## Caveats
|
||||
|
||||
This is a new script and may contain bugs.
|
||||
@ -93,9 +93,15 @@ getting InvokeAI up and running on your system. For alternative installation and
|
||||
upgrade instructions, please see:
|
||||
[InvokeAI Installation Overview](installation/)
|
||||
|
||||
Linux users who wish to make use of the PyPatchMatch inpainting functions will
|
||||
need to perform a bit of extra work to enable this module. Instructions can be
|
||||
found at [Installing PyPatchMatch](installation/060_INSTALL_PATCHMATCH.md).
|
||||
Users who wish to make use of the **PyPatchMatch** inpainting functions
|
||||
will need to perform a bit of extra work to enable this
|
||||
module. Instructions can be found at [Installing
|
||||
PyPatchMatch](installation/060_INSTALL_PATCHMATCH.md).
|
||||
|
||||
If you have an NVIDIA card, you can benefit from the significant
|
||||
memory savings and performance benefits provided by Facebook Lab's
|
||||
**xFormers** module. Instructions for Linux and Windows users can be found
|
||||
at [Installing xFormers](installation/070_INSTALL_XFORMERS.md).
|
||||
|
||||
## :fontawesome-solid-computer: Hardware Requirements
|
||||
|
||||
@ -151,6 +157,8 @@ images in full-precision mode:
|
||||
<!-- seperator -->
|
||||
- [Prompt Engineering](features/PROMPTS.md)
|
||||
<!-- seperator -->
|
||||
- [Model Merging](features/MODEL_MERGING.md)
|
||||
<!-- seperator -->
|
||||
- Miscellaneous
|
||||
- [NSFW Checker](features/NSFW.md)
|
||||
- [Embiggen upscaling](features/EMBIGGEN.md)
|
||||
|
||||
206
docs/installation/070_INSTALL_XFORMERS.md
Normal file
206
docs/installation/070_INSTALL_XFORMERS.md
Normal file
@ -0,0 +1,206 @@
|
||||
---
|
||||
title: Installing xFormers
|
||||
---
|
||||
|
||||
# :material-image-size-select-large: Installing xformers
|
||||
|
||||
xFormers is toolbox that integrates with the pyTorch and CUDA
|
||||
libraries to provide accelerated performance and reduced memory
|
||||
consumption for applications using the transformers machine learning
|
||||
architecture. After installing xFormers, InvokeAI users who have
|
||||
CUDA GPUs will see a noticeable decrease in GPU memory consumption and
|
||||
an increase in speed.
|
||||
|
||||
xFormers can be installed into a working InvokeAI installation without
|
||||
any code changes or other updates. This document explains how to
|
||||
install xFormers.
|
||||
|
||||
## Pip Install
|
||||
|
||||
For both Windows and Linux, you can install `xformers` in just a
|
||||
couple of steps from the command line.
|
||||
|
||||
If you are used to launching `invoke.sh` or `invoke.bat` to start
|
||||
InvokeAI, then run the launcher and select the "developer's console"
|
||||
to get to the command line. If you run invoke.py directly from the
|
||||
command line, then just be sure to activate it's virtual environment.
|
||||
|
||||
Then run the following three commands:
|
||||
|
||||
```sh
|
||||
pip install xformers==0.0.16rc425
|
||||
pip install triton
|
||||
python -m xformers.info output
|
||||
```
|
||||
|
||||
The first command installs `xformers`, the second installs the
|
||||
`triton` training accelerator, and the third prints out the `xformers`
|
||||
installation status. If all goes well, you'll see a report like the
|
||||
following:
|
||||
|
||||
```sh
|
||||
xFormers 0.0.16rc425
|
||||
memory_efficient_attention.cutlassF: available
|
||||
memory_efficient_attention.cutlassB: available
|
||||
memory_efficient_attention.flshattF: available
|
||||
memory_efficient_attention.flshattB: available
|
||||
memory_efficient_attention.smallkF: available
|
||||
memory_efficient_attention.smallkB: available
|
||||
memory_efficient_attention.tritonflashattF: available
|
||||
memory_efficient_attention.tritonflashattB: available
|
||||
swiglu.fused.p.cpp: available
|
||||
is_triton_available: True
|
||||
is_functorch_available: False
|
||||
pytorch.version: 1.13.1+cu117
|
||||
pytorch.cuda: available
|
||||
gpu.compute_capability: 8.6
|
||||
gpu.name: NVIDIA RTX A2000 12GB
|
||||
build.info: available
|
||||
build.cuda_version: 1107
|
||||
build.python_version: 3.10.9
|
||||
build.torch_version: 1.13.1+cu117
|
||||
build.env.TORCH_CUDA_ARCH_LIST: 5.0+PTX 6.0 6.1 7.0 7.5 8.0 8.6
|
||||
build.env.XFORMERS_BUILD_TYPE: Release
|
||||
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS: None
|
||||
build.env.NVCC_FLAGS: None
|
||||
build.env.XFORMERS_PACKAGE_FROM: wheel-v0.0.16rc425
|
||||
source.privacy: open source
|
||||
```
|
||||
|
||||
## Source Builds
|
||||
|
||||
`xformers` is currently under active development and at some point you
|
||||
may wish to build it from sourcce to get the latest features and
|
||||
bugfixes.
|
||||
|
||||
### Source Build on Linux
|
||||
|
||||
Note that xFormers only works with true NVIDIA GPUs and will not work
|
||||
properly with the ROCm driver for AMD acceleration.
|
||||
|
||||
xFormers is not currently available as a pip binary wheel and must be
|
||||
installed from source. These instructions were written for a system
|
||||
running Ubuntu 22.04, but other Linux distributions should be able to
|
||||
adapt this recipe.
|
||||
|
||||
#### 1. Install CUDA Toolkit 11.7
|
||||
|
||||
You will need the CUDA developer's toolkit in order to compile and
|
||||
install xFormers. **Do not try to install Ubuntu's nvidia-cuda-toolkit
|
||||
package.** It is out of date and will cause conflicts among the NVIDIA
|
||||
driver and binaries. Instead install the CUDA Toolkit package provided
|
||||
by NVIDIA itself. Go to [CUDA Toolkit 11.7
|
||||
Downloads](https://developer.nvidia.com/cuda-11-7-0-download-archive)
|
||||
and use the target selection wizard to choose your platform and Linux
|
||||
distribution. Select an installer type of "runfile (local)" at the
|
||||
last step.
|
||||
|
||||
This will provide you with a recipe for downloading and running a
|
||||
install shell script that will install the toolkit and drivers. For
|
||||
example, the install script recipe for Ubuntu 22.04 running on a
|
||||
x86_64 system is:
|
||||
|
||||
```
|
||||
wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda_11.7.0_515.43.04_linux.run
|
||||
sudo sh cuda_11.7.0_515.43.04_linux.run
|
||||
```
|
||||
|
||||
Rather than cut-and-paste this example, We recommend that you walk
|
||||
through the toolkit wizard in order to get the most up to date
|
||||
installer for your system.
|
||||
|
||||
#### 2. Confirm/Install pyTorch 1.13 with CUDA 11.7 support
|
||||
|
||||
If you are using InvokeAI 2.3 or higher, these will already be
|
||||
installed. If not, you can check whether you have the needed libraries
|
||||
using a quick command. Activate the invokeai virtual environment,
|
||||
either by entering the "developer's console", or manually with a
|
||||
command similar to `source ~/invokeai/.venv/bin/activate` (depending
|
||||
on where your `invokeai` directory is.
|
||||
|
||||
Then run the command:
|
||||
|
||||
```sh
|
||||
python -c 'exec("import torch\nprint(torch.__version__)")'
|
||||
```
|
||||
|
||||
If it prints __1.13.1+cu117__ you're good. If not, you can install the
|
||||
most up to date libraries with this command:
|
||||
|
||||
```sh
|
||||
pip install --upgrade --force-reinstall torch torchvision
|
||||
```
|
||||
|
||||
#### 3. Install the triton module
|
||||
|
||||
This module isn't necessary for xFormers image inference optimization,
|
||||
but avoids a startup warning.
|
||||
|
||||
```sh
|
||||
pip install triton
|
||||
```
|
||||
|
||||
#### 4. Install source code build prerequisites
|
||||
|
||||
To build xFormers from source, you will need the `build-essentials`
|
||||
package. If you don't have it installed already, run:
|
||||
|
||||
```sh
|
||||
sudo apt install build-essential
|
||||
```
|
||||
|
||||
#### 5. Build xFormers
|
||||
|
||||
There is no pip wheel package for xFormers at this time (January
|
||||
2023). Although there is a conda package, InvokeAI no longer
|
||||
officially supports conda installations and you're on your own if you
|
||||
wish to try this route.
|
||||
|
||||
Following the recipe provided at the [xFormers GitHub
|
||||
page](https://github.com/facebookresearch/xformers), and with the
|
||||
InvokeAI virtual environment active (see step 1) run the following
|
||||
commands:
|
||||
|
||||
```sh
|
||||
pip install ninja
|
||||
export TORCH_CUDA_ARCH_LIST="6.0;6.1;6.2;7.0;7.2;7.5;8.0;8.6"
|
||||
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
|
||||
```
|
||||
|
||||
The TORCH_CUDA_ARCH_LIST is a list of GPU architectures to compile
|
||||
xFormer support for. You can speed up compilation by selecting
|
||||
the architecture specific for your system. You'll find the list of
|
||||
GPUs and their architectures at NVIDIA's [GPU Compute
|
||||
Capability](https://developer.nvidia.com/cuda-gpus) table.
|
||||
|
||||
If the compile and install completes successfully, you can check that
|
||||
xFormers is installed with this command:
|
||||
|
||||
```sh
|
||||
python -m xformers.info
|
||||
```
|
||||
|
||||
If suiccessful, the top of the listing should indicate "available" for
|
||||
each of the `memory_efficient_attention` modules, as shown here:
|
||||
|
||||
```sh
|
||||
memory_efficient_attention.cutlassF: available
|
||||
memory_efficient_attention.cutlassB: available
|
||||
memory_efficient_attention.flshattF: available
|
||||
memory_efficient_attention.flshattB: available
|
||||
memory_efficient_attention.smallkF: available
|
||||
memory_efficient_attention.smallkB: available
|
||||
memory_efficient_attention.tritonflashattF: available
|
||||
memory_efficient_attention.tritonflashattB: available
|
||||
[...]
|
||||
```
|
||||
|
||||
You can now launch InvokeAI and enjoy the benefits of xFormers.
|
||||
|
||||
### Windows
|
||||
|
||||
To come
|
||||
|
||||
|
||||
---
|
||||
(c) Copyright 2023 Lincoln Stein and the InvokeAI Development Team
|
||||
@ -18,7 +18,9 @@ experience and preferences.
|
||||
InvokeAI and its dependencies. We offer two recipes: one suited to
|
||||
those who prefer the `conda` tool, and one suited to those who prefer
|
||||
`pip` and Python virtual environments. In our hands the pip install
|
||||
is faster and more reliable, but your mileage may vary.
|
||||
is faster and more reliable, but your mileage may vary.
|
||||
Note that the conda installation method is currently deprecated and
|
||||
will not be supported at some point in the future.
|
||||
|
||||
This method is recommended for users who have previously used `conda`
|
||||
or `pip` in the past, developers, and anyone who wishes to remain on
|
||||
|
||||
Reference in New Issue
Block a user