mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
Add instructions for installing xFormers on linux (#2360)
I've written up the install procedure for xFormers on Linux systems. I need help with the Windows install; I don't know what the build dependencies (compiler, etc) are. This section of the docs is currently empty. Please see `docs/installation/070_INSTALL_XFORMERS.md`
This commit is contained in:
Lincoln Stein
GitHub
commit
f00f7095f9
@ -93,9 +93,15 @@ getting InvokeAI up and running on your system. For alternative installation and
|
||||
upgrade instructions, please see:
|
||||
[InvokeAI Installation Overview](installation/)
|
||||
|
||||
Linux users who wish to make use of the PyPatchMatch inpainting functions will
|
||||
need to perform a bit of extra work to enable this module. Instructions can be
|
||||
found at [Installing PyPatchMatch](installation/060_INSTALL_PATCHMATCH.md).
|
||||
Users who wish to make use of the **PyPatchMatch** inpainting functions
|
||||
will need to perform a bit of extra work to enable this
|
||||
module. Instructions can be found at [Installing
|
||||
PyPatchMatch](installation/060_INSTALL_PATCHMATCH.md).
|
||||
|
||||
If you have an NVIDIA card, you can benefit from the significant
|
||||
memory savings and performance benefits provided by Facebook Lab's
|
||||
**xFormers** module. Instructions for Linux and Windows users can be found
|
||||
at [Installing xFormers](installation/070_INSTALL_XFORMERS.md).
|
||||
|
||||
## :fontawesome-solid-computer: Hardware Requirements
|
||||
|
||||
|
||||
206
docs/installation/070_INSTALL_XFORMERS.md
Normal file
206
docs/installation/070_INSTALL_XFORMERS.md
Normal file
@ -0,0 +1,206 @@
|
||||
---
|
||||
title: Installing xFormers
|
||||
---
|
||||
|
||||
# :material-image-size-select-large: Installing xformers
|
||||
|
||||
xFormers is toolbox that integrates with the pyTorch and CUDA
|
||||
libraries to provide accelerated performance and reduced memory
|
||||
consumption for applications using the transformers machine learning
|
||||
architecture. After installing xFormers, InvokeAI users who have
|
||||
CUDA GPUs will see a noticeable decrease in GPU memory consumption and
|
||||
an increase in speed.
|
||||
|
||||
xFormers can be installed into a working InvokeAI installation without
|
||||
any code changes or other updates. This document explains how to
|
||||
install xFormers.
|
||||
|
||||
## Pip Install
|
||||
|
||||
For both Windows and Linux, you can install `xformers` in just a
|
||||
couple of steps from the command line.
|
||||
|
||||
If you are used to launching `invoke.sh` or `invoke.bat` to start
|
||||
InvokeAI, then run the launcher and select the "developer's console"
|
||||
to get to the command line. If you run invoke.py directly from the
|
||||
command line, then just be sure to activate it's virtual environment.
|
||||
|
||||
Then run the following three commands:
|
||||
|
||||
```sh
|
||||
pip install xformers==0.0.16rc425
|
||||
pip install triton
|
||||
python -m xformers.info output
|
||||
```
|
||||
|
||||
The first command installs `xformers`, the second installs the
|
||||
`triton` training accelerator, and the third prints out the `xformers`
|
||||
installation status. If all goes well, you'll see a report like the
|
||||
following:
|
||||
|
||||
```sh
|
||||
xFormers 0.0.16rc425
|
||||
memory_efficient_attention.cutlassF: available
|
||||
memory_efficient_attention.cutlassB: available
|
||||
memory_efficient_attention.flshattF: available
|
||||
memory_efficient_attention.flshattB: available
|
||||
memory_efficient_attention.smallkF: available
|
||||
memory_efficient_attention.smallkB: available
|
||||
memory_efficient_attention.tritonflashattF: available
|
||||
memory_efficient_attention.tritonflashattB: available
|
||||
swiglu.fused.p.cpp: available
|
||||
is_triton_available: True
|
||||
is_functorch_available: False
|
||||
pytorch.version: 1.13.1+cu117
|
||||
pytorch.cuda: available
|
||||
gpu.compute_capability: 8.6
|
||||
gpu.name: NVIDIA RTX A2000 12GB
|
||||
build.info: available
|
||||
build.cuda_version: 1107
|
||||
build.python_version: 3.10.9
|
||||
build.torch_version: 1.13.1+cu117
|
||||
build.env.TORCH_CUDA_ARCH_LIST: 5.0+PTX 6.0 6.1 7.0 7.5 8.0 8.6
|
||||
build.env.XFORMERS_BUILD_TYPE: Release
|
||||
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS: None
|
||||
build.env.NVCC_FLAGS: None
|
||||
build.env.XFORMERS_PACKAGE_FROM: wheel-v0.0.16rc425
|
||||
source.privacy: open source
|
||||
```
|
||||
|
||||
## Source Builds
|
||||
|
||||
`xformers` is currently under active development and at some point you
|
||||
may wish to build it from sourcce to get the latest features and
|
||||
bugfixes.
|
||||
|
||||
### Source Build on Linux
|
||||
|
||||
Note that xFormers only works with true NVIDIA GPUs and will not work
|
||||
properly with the ROCm driver for AMD acceleration.
|
||||
|
||||
xFormers is not currently available as a pip binary wheel and must be
|
||||
installed from source. These instructions were written for a system
|
||||
running Ubuntu 22.04, but other Linux distributions should be able to
|
||||
adapt this recipe.
|
||||
|
||||
#### 1. Install CUDA Toolkit 11.7
|
||||
|
||||
You will need the CUDA developer's toolkit in order to compile and
|
||||
install xFormers. **Do not try to install Ubuntu's nvidia-cuda-toolkit
|
||||
package.** It is out of date and will cause conflicts among the NVIDIA
|
||||
driver and binaries. Instead install the CUDA Toolkit package provided
|
||||
by NVIDIA itself. Go to [CUDA Toolkit 11.7
|
||||
Downloads](https://developer.nvidia.com/cuda-11-7-0-download-archive)
|
||||
and use the target selection wizard to choose your platform and Linux
|
||||
distribution. Select an installer type of "runfile (local)" at the
|
||||
last step.
|
||||
|
||||
This will provide you with a recipe for downloading and running a
|
||||
install shell script that will install the toolkit and drivers. For
|
||||
example, the install script recipe for Ubuntu 22.04 running on a
|
||||
x86_64 system is:
|
||||
|
||||
```
|
||||
wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda_11.7.0_515.43.04_linux.run
|
||||
sudo sh cuda_11.7.0_515.43.04_linux.run
|
||||
```
|
||||
|
||||
Rather than cut-and-paste this example, We recommend that you walk
|
||||
through the toolkit wizard in order to get the most up to date
|
||||
installer for your system.
|
||||
|
||||
#### 2. Confirm/Install pyTorch 1.13 with CUDA 11.7 support
|
||||
|
||||
If you are using InvokeAI 2.3 or higher, these will already be
|
||||
installed. If not, you can check whether you have the needed libraries
|
||||
using a quick command. Activate the invokeai virtual environment,
|
||||
either by entering the "developer's console", or manually with a
|
||||
command similar to `source ~/invokeai/.venv/bin/activate` (depending
|
||||
on where your `invokeai` directory is.
|
||||
|
||||
Then run the command:
|
||||
|
||||
```sh
|
||||
python -c 'exec("import torch\nprint(torch.__version__)")'
|
||||
```
|
||||
|
||||
If it prints __1.13.1+cu117__ you're good. If not, you can install the
|
||||
most up to date libraries with this command:
|
||||
|
||||
```sh
|
||||
pip install --upgrade --force-reinstall torch torchvision
|
||||
```
|
||||
|
||||
#### 3. Install the triton module
|
||||
|
||||
This module isn't necessary for xFormers image inference optimization,
|
||||
but avoids a startup warning.
|
||||
|
||||
```sh
|
||||
pip install triton
|
||||
```
|
||||
|
||||
#### 4. Install source code build prerequisites
|
||||
|
||||
To build xFormers from source, you will need the `build-essentials`
|
||||
package. If you don't have it installed already, run:
|
||||
|
||||
```sh
|
||||
sudo apt install build-essential
|
||||
```
|
||||
|
||||
#### 5. Build xFormers
|
||||
|
||||
There is no pip wheel package for xFormers at this time (January
|
||||
2023). Although there is a conda package, InvokeAI no longer
|
||||
officially supports conda installations and you're on your own if you
|
||||
wish to try this route.
|
||||
|
||||
Following the recipe provided at the [xFormers GitHub
|
||||
page](https://github.com/facebookresearch/xformers), and with the
|
||||
InvokeAI virtual environment active (see step 1) run the following
|
||||
commands:
|
||||
|
||||
```sh
|
||||
pip install ninja
|
||||
export TORCH_CUDA_ARCH_LIST="6.0;6.1;6.2;7.0;7.2;7.5;8.0;8.6"
|
||||
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
|
||||
```
|
||||
|
||||
The TORCH_CUDA_ARCH_LIST is a list of GPU architectures to compile
|
||||
xFormer support for. You can speed up compilation by selecting
|
||||
the architecture specific for your system. You'll find the list of
|
||||
GPUs and their architectures at NVIDIA's [GPU Compute
|
||||
Capability](https://developer.nvidia.com/cuda-gpus) table.
|
||||
|
||||
If the compile and install completes successfully, you can check that
|
||||
xFormers is installed with this command:
|
||||
|
||||
```sh
|
||||
python -m xformers.info
|
||||
```
|
||||
|
||||
If suiccessful, the top of the listing should indicate "available" for
|
||||
each of the `memory_efficient_attention` modules, as shown here:
|
||||
|
||||
```sh
|
||||
memory_efficient_attention.cutlassF: available
|
||||
memory_efficient_attention.cutlassB: available
|
||||
memory_efficient_attention.flshattF: available
|
||||
memory_efficient_attention.flshattB: available
|
||||
memory_efficient_attention.smallkF: available
|
||||
memory_efficient_attention.smallkB: available
|
||||
memory_efficient_attention.tritonflashattF: available
|
||||
memory_efficient_attention.tritonflashattB: available
|
||||
[...]
|
||||
```
|
||||
|
||||
You can now launch InvokeAI and enjoy the benefits of xFormers.
|
||||
|
||||
### Windows
|
||||
|
||||
To come
|
||||
|
||||
|
||||
---
|
||||
(c) Copyright 2023 Lincoln Stein and the InvokeAI Development Team
|
||||
Loading…
Reference in New Issue
Block a user