mirror of
https://github.com/invoke-ai/InvokeAI
synced 2024-08-30 20:32:17 +00:00
docs: merge INSTALL_TROUBLESHOOTING into FAQ
These two docs had overlap and were kinda the same thing.
This commit is contained in:
psychedelicious
blessedcoolant
parent
3655a47200
commit
237ac58dae
@ -395,9 +395,9 @@ For our latest changes, view our [Release
|
|||||||
Notes](https://github.com/invoke-ai/InvokeAI/releases) and the
|
Notes](https://github.com/invoke-ai/InvokeAI/releases) and the
|
||||||
[CHANGELOG](docs/CHANGELOG.md).
|
[CHANGELOG](docs/CHANGELOG.md).
|
||||||
|
|
||||||
### Troubleshooting
|
### Troubleshooting / FAQ
|
||||||
|
|
||||||
Please check out our **[Troubleshooting Guide](https://invoke-ai.github.io/InvokeAI/installation/INSTALL_TROUBLESHOOTING/)** to get solutions for common installation
|
Please check out our **[FAQ](https://invoke-ai.github.io/InvokeAI/help/FAQ/)** to get solutions for common installation
|
||||||
problems and other issues. For more help, please join our [Discord][discord link]
|
problems and other issues. For more help, please join our [Discord][discord link]
|
||||||
|
|
||||||
## Contributing
|
## Contributing
|
||||||
|
|||||||
179
docs/help/FAQ.md
179
docs/help/FAQ.md
@ -1,43 +1,172 @@
|
|||||||
# FAQs
|
# FAQ
|
||||||
|
|
||||||
**Where do I get started? How can I install Invoke?**
|
!!! info "How to Reinstall"
|
||||||
|
|
||||||
- You can download the latest installers [here](https://github.com/invoke-ai/InvokeAI/releases) - Note that any releases marked as *pre-release* are in a beta state. You may experience some issues, but we appreciate your help testing those! For stable/reliable installations, please install the **[Latest Release](https://github.com/invoke-ai/InvokeAI/releases/latest)**
|
Many issues can be resolved by re-installing the application. You won't lose any data by re-installing. We suggest downloading the [latest release](https://github.com/invoke-ai/InvokeAI/releases/latest) and using it to re-install the application. Consult the [installer guide](../installation/010_INSTALL_AUTOMATED.md) for more information.
|
||||||
|
|
||||||
**How can I download models? Can I use models I already have downloaded?**
|
When you run the installer, you'll have an option to select the version to install. If you aren't ready to upgrade, you choose the current version to fix a broken install.
|
||||||
|
|
||||||
- Models can be downloaded through the model manager, or through option [4] in the invoke.bat/invoke.sh launcher script. To download a model through the Model Manager, use the HuggingFace Repo ID by pressing the “Copy” button next to the repository name. Alternatively, to download a model from CivitAi, use the download link in the Model Manager.
|
If the troubleshooting steps on this page don't get you up and running, please either [create an issue] or hop on [discord] for help.
|
||||||
- Models that are already downloaded can be used by creating a symlink to the model location in the `autoimport` folder or by using the Model Manger’s “Scan for Models” function.
|
|
||||||
|
|
||||||
**My images are taking a long time to generate. How can I speed up generation?**
|
## How to Install
|
||||||
|
|
||||||
- A common solution is to reduce the size of your RAM & VRAM cache to 0.25. This ensures your system has enough memory to generate images.
|
You can download the latest installers [here](https://github.com/invoke-ai/InvokeAI/releases).
|
||||||
- Additionally, check the [hardware requirements](https://invoke-ai.github.io/InvokeAI/#hardware-requirements) to ensure that your system is capable of generating images.
|
|
||||||
- Lastly, double check your generations are happening on your GPU (if you have one). InvokeAI will log what is being used for generation upon startup.
|
|
||||||
|
|
||||||
**I’ve installed Python on Windows but the installer says it can’t find it?**
|
Note that any releases marked as _pre-release_ are in a beta state. You may experience some issues, but we appreciate your help testing those! For stable/reliable installations, please install the [latest release].
|
||||||
|
|
||||||
- Then ensure that you checked **'Add python.exe to PATH'** when installing Python. This can be found at the bottom of the Python Installer window. If you already have Python installed, this can be done with the modify / repair feature of the installer.
|
## Downloading models and using existing models
|
||||||
|
|
||||||
**I’ve installed everything successfully but I still get an error about Triton when starting Invoke?**
|
The Model Manager tab in the UI provides a few ways to install models, including using your already-downloaded models. You'll see a popup directing you there on first startup. For more information, see the [model install docs].
|
||||||
|
|
||||||
- This can be safely ignored. InvokeAI doesn't use Triton, but if you are on Linux and wish to dismiss the error, you can install Triton.
|
## Slow generation
|
||||||
|
|
||||||
**I updated to 3.4.0 and now xFormers can’t load C++/CUDA?**
|
- Check the [system requirements] to ensure that your system is capable of generating images.
|
||||||
|
- Check the `ram` setting in `invokeai.yaml`. This setting tells Invoke how much of your system RAM can be used to cache models. Having this too high or too low can slow things down. That said, it's generally safest to not set this at all and instead let Invoke manage it.
|
||||||
|
- Check the `vram` setting in `invokeai.yaml`. This setting tells Invoke how much of your GPU VRAM can be used to cache models. Counter-intuitively, if this setting is too high, Invoke will need to do a lot of shuffling of models as it juggles the VRAM cache and the currently-loaded model. The default value of 0.25 is generally works well for GPUs without 16GB or more VRAM. Even on a 24GB card, the default works well.
|
||||||
|
- Check that your generations are happening on your GPU (if you have one). InvokeAI will log what is being used for generation upon startup. If your GPU isn't used, re-install to ensure the correct versions of torch get installed.
|
||||||
|
|
||||||
- An issue occurred with your PyTorch update. Follow these steps to fix :
|
## Installer cannot find python (Windows)
|
||||||
1. Launch your invoke.bat / invoke.sh and select the option to open the developer console
|
|
||||||
2. Run:`pip install ".[xformers]" --upgrade --force-reinstall --extra-index-url https://download.pytorch.org/whl/cu121`
|
|
||||||
- If you run into an error with `typing_extensions`, re-open the developer console and run: `pip install -U typing-extensions`
|
|
||||||
|
|
||||||
**It says my pip is out of date - is that why my install isn't working?**
|
Ensure that you checked **Add python.exe to PATH** when installing Python. This can be found at the bottom of the Python Installer window. If you already have Python installed, you can re-run the python installer, choose the Modify option and check the box.
|
||||||
- An out of date won't cause an installation to fail. The cause of the error can likely be found above the message that says pip is out of date.
|
|
||||||
- If you saw that warning but the install went well, don't worry about it (but you can update pip afterwards if you'd like).
|
## Triton error on startup
|
||||||
|
|
||||||
|
This can be safely ignored. InvokeAI doesn't use Triton, but if you are on Linux and wish to dismiss the error, you can install Triton.
|
||||||
|
|
||||||
|
## Updated to 3.4.0 and xformers can’t load C++/CUDA
|
||||||
|
|
||||||
|
An issue occurred with your PyTorch update. Follow these steps to fix :
|
||||||
|
|
||||||
|
1. Launch your invoke.bat / invoke.sh and select the option to open the developer console
|
||||||
|
2. Run:`pip install ".[xformers]" --upgrade --force-reinstall --extra-index-url https://download.pytorch.org/whl/cu121`
|
||||||
|
- If you run into an error with `typing_extensions`, re-open the developer console and run: `pip install -U typing-extensions`
|
||||||
|
|
||||||
|
Note that v3.4.0 is an old, unsupported version. Please upgrade to the [latest release].
|
||||||
|
|
||||||
|
## Install failed and says `pip` is out of date
|
||||||
|
|
||||||
|
An out of date `pip` typically won't cause an installation to fail. The cause of the error can likely be found above the message that says `pip` is out of date.
|
||||||
|
|
||||||
|
If you saw that warning but the install went well, don't worry about it (but you can update `pip` afterwards if you'd like).
|
||||||
|
|
||||||
|
## Replicate image found online
|
||||||
|
|
||||||
**How can I generate the exact same that I found on the internet?**
|
|
||||||
Most example images with prompts that you'll find on the internet have been generated using different software, so you can't expect to get identical results. In order to reproduce an image, you need to replicate the exact settings and processing steps, including (but not limited to) the model, the positive and negative prompts, the seed, the sampler, the exact image size, any upscaling steps, etc.
|
Most example images with prompts that you'll find on the internet have been generated using different software, so you can't expect to get identical results. In order to reproduce an image, you need to replicate the exact settings and processing steps, including (but not limited to) the model, the positive and negative prompts, the seed, the sampler, the exact image size, any upscaling steps, etc.
|
||||||
|
|
||||||
|
## OSErrors on Windows while installing dependencies
|
||||||
|
|
||||||
**Where can I get more help?**
|
During a zip file installation or an update, installation stops with an error like this:
|
||||||
|
|
||||||
- Create an issue on [GitHub](https://github.com/invoke-ai/InvokeAI/issues) or post in the [#help channel](https://discord.com/channels/1020123559063990373/1149510134058471514) of the InvokeAI Discord
|
{:width="800px"}
|
||||||
|
|
||||||
|
To resolve this, re-install the application as described above.
|
||||||
|
|
||||||
|
## Stable Diffusion XL generation fails after trying to load UNet
|
||||||
|
|
||||||
|
InvokeAI is working in other respects, but when trying to generate
|
||||||
|
images with Stable Diffusion XL you get a "Server Error". The text log
|
||||||
|
in the launch window contains this log line above several more lines of
|
||||||
|
error messages:
|
||||||
|
|
||||||
|
`INFO --> Loading model:D:\LONG\PATH\TO\MODEL, type sdxl:main:unet`
|
||||||
|
|
||||||
|
This failure mode occurs when there is a network glitch during
|
||||||
|
downloading the very large SDXL model.
|
||||||
|
|
||||||
|
To address this, first go to the Model Manager and delete the
|
||||||
|
Stable-Diffusion-XL-base-1.X model. Then, click the HuggingFace tab,
|
||||||
|
paste the Repo ID stabilityai/stable-diffusion-xl-base-1.0 and install
|
||||||
|
the model.
|
||||||
|
|
||||||
|
## Package dependency conflicts during installation or update
|
||||||
|
|
||||||
|
If you have previously installed InvokeAI or another Stable Diffusion
|
||||||
|
package, the installer may occasionally pick up outdated libraries and
|
||||||
|
either the installer or `invoke` will fail with complaints about
|
||||||
|
library conflicts.
|
||||||
|
|
||||||
|
To resolve this, re-install the application as described above.
|
||||||
|
|
||||||
|
## Invalid configuration file
|
||||||
|
|
||||||
|
Everything seems to install ok, you get a `ValidationError` when starting up the app.
|
||||||
|
|
||||||
|
This is caused by an invalid setting in the `invokeai.yaml` configuration file. The error message should tell you what is wrong.
|
||||||
|
|
||||||
|
Check the [configuration docs] for more detail about the settings and how to specify them.
|
||||||
|
|
||||||
|
## Out of Memory Issues
|
||||||
|
|
||||||
|
The models are large, VRAM is expensive, and you may find yourself
|
||||||
|
faced with Out of Memory errors when generating images. Here are some
|
||||||
|
tips to reduce the problem:
|
||||||
|
|
||||||
|
!!! info "Optimizing for GPU VRAM"
|
||||||
|
|
||||||
|
=== "4GB VRAM GPU"
|
||||||
|
|
||||||
|
This should be adequate for 512x512 pixel images using Stable Diffusion 1.5
|
||||||
|
and derived models, provided that you do not use the NSFW checker. It won't be loaded unless you go into the UI settings and turn it on.
|
||||||
|
|
||||||
|
If you are on a CUDA-enabled GPU, we will automatically use xformers or torch-sdp to reduce VRAM requirements, though you can explicitly configure this. See the [configuration docs].
|
||||||
|
|
||||||
|
=== "6GB VRAM GPU"
|
||||||
|
|
||||||
|
This is a border case. Using the SD 1.5 series you should be able to
|
||||||
|
generate images up to 640x640 with the NSFW checker enabled, and up to
|
||||||
|
1024x1024 with it disabled.
|
||||||
|
|
||||||
|
If you run into persistent memory issues there are a series of
|
||||||
|
environment variables that you can set before launching InvokeAI that
|
||||||
|
alter how the PyTorch machine learning library manages memory. See
|
||||||
|
<https://pytorch.org/docs/stable/notes/cuda.html#memory-management> for
|
||||||
|
a list of these tweaks.
|
||||||
|
|
||||||
|
=== "12GB VRAM GPU"
|
||||||
|
|
||||||
|
This should be sufficient to generate larger images up to about 1280x1280.
|
||||||
|
|
||||||
|
## Memory Leak (Linux)
|
||||||
|
|
||||||
|
If you notice a memory leak, it could be caused to memory fragmentation as models are loaded and/or moved from CPU to GPU.
|
||||||
|
|
||||||
|
A workaround is to tune memory allocation with an environment variable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Force blocks >1MB to be allocated with `mmap` so that they are released to the system immediately when they are freed.
|
||||||
|
MALLOC_MMAP_THRESHOLD_=1048576
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! warning "Speed vs Memory Tradeoff"
|
||||||
|
|
||||||
|
Your generations may be slower overall when setting this environment variable.
|
||||||
|
|
||||||
|
!!! info "Possibly dependent on `libc` implementation"
|
||||||
|
|
||||||
|
It's not known if this issue occurs with other `libc` implementations such as `musl`.
|
||||||
|
|
||||||
|
If you encounter this issue and your system uses a different implementation, please try this environment variable and let us know if it fixes the issue.
|
||||||
|
|
||||||
|
<h3>Detailed Discussion</h3>
|
||||||
|
|
||||||
|
Python (and PyTorch) relies on the memory allocator from the C Standard Library (`libc`). On linux, with the GNU C Standard Library implementation (`glibc`), our memory access patterns have been observed to cause severe memory fragmentation.
|
||||||
|
|
||||||
|
This fragmentation results in large amounts of memory that has been freed but can't be released back to the OS. Loading models from disk and moving them between CPU/CUDA seem to be the operations that contribute most to the fragmentation.
|
||||||
|
|
||||||
|
This memory fragmentation issue can result in OOM crashes during frequent model switching, even if `ram` (the max RAM cache size) is set to a reasonable value (e.g. a OOM crash with `ram=16` on a system with 32GB of RAM).
|
||||||
|
|
||||||
|
This problem may also exist on other OSes, and other `libc` implementations. But, at the time of writing, it has only been investigated on linux with `glibc`.
|
||||||
|
|
||||||
|
To better understand how the `glibc` memory allocator works, see these references:
|
||||||
|
|
||||||
|
- Basics: <https://www.gnu.org/software/libc/manual/html_node/The-GNU-Allocator.html>
|
||||||
|
- Details: <https://sourceware.org/glibc/wiki/MallocInternals>
|
||||||
|
|
||||||
|
Note the differences between memory allocated as chunks in an arena vs. memory allocated with `mmap`. Under `glibc`'s default configuration, most model tensors get allocated as chunks in an arena making them vulnerable to the problem of fragmentation.
|
||||||
|
|
||||||
|
[model install docs]: ../installation/050_INSTALLING_MODELS.md
|
||||||
|
[system requirements]: ../installation/INSTALL_REQUIREMENTS.md
|
||||||
|
[latest release]: https://github.com/invoke-ai/InvokeAI/releases/latest
|
||||||
|
[create an issue]: https://github.com/invoke-ai/InvokeAI/issues
|
||||||
|
[discord]: https://discord.gg/ZmtBAhwWhy
|
||||||
|
[configuration docs]: ../features/CONFIGURATION.md
|
||||||
|
|||||||
@ -153,8 +153,7 @@ Mac and Linux machines, and runs on GPU cards with as little as 4 GB of RAM.
|
|||||||
## :material-target: Troubleshooting
|
## :material-target: Troubleshooting
|
||||||
|
|
||||||
Please check out our **[:material-frequently-asked-questions:
|
Please check out our **[:material-frequently-asked-questions:
|
||||||
Troubleshooting
|
FAQ](help/FAQ/)** to
|
||||||
Guide](installation/INSTALL_TROUBLESHOOTING/)** to
|
|
||||||
get solutions for common installation problems and other issues.
|
get solutions for common installation problems and other issues.
|
||||||
|
|
||||||
## :octicons-repo-push-24: Contributing
|
## :octicons-repo-push-24: Contributing
|
||||||
|
|||||||
@ -94,10 +94,10 @@ Updating is exactly the same as installing - download the latest installer, choo
|
|||||||
|
|
||||||
## Installation Issues
|
## Installation Issues
|
||||||
|
|
||||||
If you have installation issues, please review the [installation troubleshooting] docs. You can also [create an issue] or ask for help on [discord].
|
If you have installation issues, please review the [FAQ]. You can also [create an issue] or ask for help on [discord].
|
||||||
|
|
||||||
[installation requirements]: INSTALLATION.md#installation-requirements
|
[installation requirements]: INSTALLATION.md#installation-requirements
|
||||||
[installation troubleshooting]: INSTALL_TROUBLESHOOTING.md
|
[FAQ]: ../help/FAQ.md
|
||||||
[install some models]: 050_INSTALLING_MODELS.md
|
[install some models]: 050_INSTALLING_MODELS.md
|
||||||
[configuration docs]: ../features/CONFIGURATION.md
|
[configuration docs]: ../features/CONFIGURATION.md
|
||||||
[latest release]: https://github.com/invoke-ai/InvokeAI/releases/latest
|
[latest release]: https://github.com/invoke-ai/InvokeAI/releases/latest
|
||||||
|
|||||||
@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
Before installing, review the [installation requirements] to ensure your system is set up properly.
|
Before installing, review the [installation requirements] to ensure your system is set up properly.
|
||||||
|
|
||||||
See the [installation troubleshooting] guide for frequently-encountered installation issues.
|
See the [FAQ] for frequently-encountered installation issues.
|
||||||
|
|
||||||
If you need more help, join our [discord] or [create an issue].
|
If you need more help, join our [discord] or [create an issue].
|
||||||
|
|
||||||
@ -35,7 +35,7 @@ See the [docker installation guide].
|
|||||||
[developer install guide]: INSTALL_DEVELOPMENT.md
|
[developer install guide]: INSTALL_DEVELOPMENT.md
|
||||||
[docker installation guide]: 040_INSTALL_DOCKER.md
|
[docker installation guide]: 040_INSTALL_DOCKER.md
|
||||||
[installation guide]: 010_INSTALL_AUTOMATED.md
|
[installation guide]: 010_INSTALL_AUTOMATED.md
|
||||||
[installation troubleshooting]: INSTALL_TROUBLESHOOTING.md
|
[FAQ]: ../help/FAQ.md
|
||||||
[discord]: discord.gg/invoke-ai
|
[discord]: discord.gg/invoke-ai
|
||||||
[create an issue]: https://github.com/invoke-ai/InvokeAI/issues
|
[create an issue]: https://github.com/invoke-ai/InvokeAI/issues
|
||||||
[installation requirements]: INSTALL_REQUIREMENTS.md
|
[installation requirements]: INSTALL_REQUIREMENTS.md
|
||||||
|
|||||||
@ -1,134 +0,0 @@
|
|||||||
# Installation Troubleshooting
|
|
||||||
|
|
||||||
!!! info "How to Reinstall"
|
|
||||||
|
|
||||||
Many issues can be resolved by re-installing the application. You won't lose any data by re-installing. We suggest downloading the [latest release] and using it to re-install the application.
|
|
||||||
|
|
||||||
When you run the installer, you'll have an option to select the version to install. If you aren't ready to upgrade, you choose the current version to fix a broken install.
|
|
||||||
|
|
||||||
If the troubleshooting steps on this page don't get you up and running, please either [create an issue] or hop on [discord] for help.
|
|
||||||
|
|
||||||
## OSErrors on Windows while installing dependencies
|
|
||||||
|
|
||||||
During a zip file installation or an online update, installation stops
|
|
||||||
with an error like this:
|
|
||||||
|
|
||||||
{:width="800px"}
|
|
||||||
|
|
||||||
To resolve this, re-install the application as described above.
|
|
||||||
|
|
||||||
## Stable Diffusion XL generation fails after trying to load UNet
|
|
||||||
|
|
||||||
InvokeAI is working in other respects, but when trying to generate
|
|
||||||
images with Stable Diffusion XL you get a "Server Error". The text log
|
|
||||||
in the launch window contains this log line above several more lines of
|
|
||||||
error messages:
|
|
||||||
|
|
||||||
`INFO --> Loading model:D:\LONG\PATH\TO\MODEL, type sdxl:main:unet`
|
|
||||||
|
|
||||||
This failure mode occurs when there is a network glitch during
|
|
||||||
downloading the very large SDXL model.
|
|
||||||
|
|
||||||
To address this, first go to the Model Manager and delete the
|
|
||||||
Stable-Diffusion-XL-base-1.X model. Then, click the HuggingFace tab,
|
|
||||||
paste the Repo ID stabilityai/stable-diffusion-xl-base-1.0 and install
|
|
||||||
the model.
|
|
||||||
|
|
||||||
## Package dependency conflicts
|
|
||||||
|
|
||||||
If you have previously installed InvokeAI or another Stable Diffusion
|
|
||||||
package, the installer may occasionally pick up outdated libraries and
|
|
||||||
either the installer or `invoke` will fail with complaints about
|
|
||||||
library conflicts.
|
|
||||||
|
|
||||||
To resolve this, re-install the application as described above.
|
|
||||||
|
|
||||||
## InvokeAI runs extremely slowly on Linux or Windows systems
|
|
||||||
|
|
||||||
The most frequent cause of this problem is when the installation
|
|
||||||
process installed the CPU-only version of the torch machine-learning
|
|
||||||
library, rather than a version that takes advantage of GPU
|
|
||||||
acceleration. To confirm this issue, look at the InvokeAI startup
|
|
||||||
messages. If you see a message saying ">> Using device CPU", then
|
|
||||||
this is what happened.
|
|
||||||
|
|
||||||
To resolve this, re-install the application as described above. Be sure to select the correct GPU device.
|
|
||||||
|
|
||||||
## Invalid configuration file
|
|
||||||
|
|
||||||
Everything seems to install ok, you get a `ValidationError` when starting up the app.
|
|
||||||
|
|
||||||
This is caused by an invalid setting in the `invokeai.yaml` configuration file. The error message should tell you what is wrong.
|
|
||||||
|
|
||||||
Check the [configuration docs] for more detail about the settings and how to specify them.
|
|
||||||
|
|
||||||
## Out of Memory Issues
|
|
||||||
|
|
||||||
The models are large, VRAM is expensive, and you may find yourself
|
|
||||||
faced with Out of Memory errors when generating images. Here are some
|
|
||||||
tips to reduce the problem:
|
|
||||||
|
|
||||||
<h3>4 GB of VRAM</h3>
|
|
||||||
|
|
||||||
This should be adequate for 512x512 pixel images using Stable Diffusion 1.5
|
|
||||||
and derived models, provided that you do not use the NSFW checker. It won't be loaded unless you go into the UI settings and turn it on.
|
|
||||||
|
|
||||||
If you are on a CUDA-enabled GPU, we will automatically use xformers or torch-sdp to reduce VRAM requirements, though you can explicitly configure this. See the [configuration docs].
|
|
||||||
|
|
||||||
<h3>6 GB of VRAM</h3>
|
|
||||||
|
|
||||||
This is a border case. Using the SD 1.5 series you should be able to
|
|
||||||
generate images up to 640x640 with the NSFW checker enabled, and up to
|
|
||||||
1024x1024 with it disabled.
|
|
||||||
|
|
||||||
If you run into persistent memory issues there are a series of
|
|
||||||
environment variables that you can set before launching InvokeAI that
|
|
||||||
alter how the PyTorch machine learning library manages memory. See
|
|
||||||
<https://pytorch.org/docs/stable/notes/cuda.html#memory-management> for
|
|
||||||
a list of these tweaks.
|
|
||||||
|
|
||||||
<h3>12 GB of VRAM</h3>
|
|
||||||
|
|
||||||
This should be sufficient to generate larger images up to about 1280x1280.
|
|
||||||
|
|
||||||
[create an issue]: https://github.com/invoke-ai/InvokeAI/issues
|
|
||||||
[discord]: https://discord.gg/ZmtBAhwWhy
|
|
||||||
[configuration docs]: ../features/CONFIGURATION.md
|
|
||||||
|
|
||||||
## Memory Leak (Linux)
|
|
||||||
|
|
||||||
If you notice a memory leak, it could be caused to memory fragmentation as models are loaded and/or moved from CPU to GPU.
|
|
||||||
|
|
||||||
A workaround is to tune memory allocation with an environment variable:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Force blocks >1MB to be allocated with `mmap` so that they are released to the system immediately when they are freed.
|
|
||||||
MALLOC_MMAP_THRESHOLD_=1048576
|
|
||||||
```
|
|
||||||
|
|
||||||
!!! warning "Speed vs Memory Tradeoff"
|
|
||||||
|
|
||||||
Your generations may be slower overall when setting this environment variable.
|
|
||||||
|
|
||||||
!!! info "Possibly dependent on `libc` implementation"
|
|
||||||
|
|
||||||
It's not known if this issue occurs with other `libc` implementations such as `musl`.
|
|
||||||
|
|
||||||
If you encounter this issue and your system uses a different implementation, please try this environment variable and let us know if it fixes the issue.
|
|
||||||
|
|
||||||
<h3>Detailed Discussion</h3>
|
|
||||||
|
|
||||||
Python (and PyTorch) relies on the memory allocator from the C Standard Library (`libc`). On linux, with the GNU C Standard Library implementation (`glibc`), our memory access patterns have been observed to cause severe memory fragmentation.
|
|
||||||
|
|
||||||
This fragmentation results in large amounts of memory that has been freed but can't be released back to the OS. Loading models from disk and moving them between CPU/CUDA seem to be the operations that contribute most to the fragmentation.
|
|
||||||
|
|
||||||
This memory fragmentation issue can result in OOM crashes during frequent model switching, even if `ram` (the max RAM cache size) is set to a reasonable value (e.g. a OOM crash with `ram=16` on a system with 32GB of RAM).
|
|
||||||
|
|
||||||
This problem may also exist on other OSes, and other `libc` implementations. But, at the time of writing, it has only been investigated on linux with `glibc`.
|
|
||||||
|
|
||||||
To better understand how the `glibc` memory allocator works, see these references:
|
|
||||||
|
|
||||||
- Basics: <https://www.gnu.org/software/libc/manual/html_node/The-GNU-Allocator.html>
|
|
||||||
- Details: <https://sourceware.org/glibc/wiki/MallocInternals>
|
|
||||||
|
|
||||||
Note the differences between memory allocated as chunks in an arena vs. memory allocated with `mmap`. Under `glibc`'s default configuration, most model tensors get allocated as chunks in an arena making them vulnerable to the problem of fragmentation.
|
|
||||||
@ -16,7 +16,7 @@ if "%1" == "use-cache" (
|
|||||||
@rem The version in the next line is replaced by an up to date release number
|
@rem The version in the next line is replaced by an up to date release number
|
||||||
@rem when create_installer.sh is run. Change the release number there.
|
@rem when create_installer.sh is run. Change the release number there.
|
||||||
set INSTRUCTIONS=https://invoke-ai.github.io/InvokeAI/installation/INSTALL_AUTOMATED/
|
set INSTRUCTIONS=https://invoke-ai.github.io/InvokeAI/installation/INSTALL_AUTOMATED/
|
||||||
set TROUBLESHOOTING=https://invoke-ai.github.io/InvokeAI/installation/INSTALL_TROUBLESHOOTING
|
set TROUBLESHOOTING=https://invoke-ai.github.io/InvokeAI/help/FAQ/
|
||||||
set PYTHON_URL=https://www.python.org/downloads/windows/
|
set PYTHON_URL=https://www.python.org/downloads/windows/
|
||||||
set MINIMUM_PYTHON_VERSION=3.10.0
|
set MINIMUM_PYTHON_VERSION=3.10.0
|
||||||
set PYTHON_URL=https://www.python.org/downloads/release/python-3109/
|
set PYTHON_URL=https://www.python.org/downloads/release/python-3109/
|
||||||
|
|||||||
@ -124,7 +124,6 @@ nav:
|
|||||||
- Manual Install: 'installation/020_INSTALL_MANUAL.md'
|
- Manual Install: 'installation/020_INSTALL_MANUAL.md'
|
||||||
- Developer Install: 'installation/INSTALL_DEVELOPMENT.md'
|
- Developer Install: 'installation/INSTALL_DEVELOPMENT.md'
|
||||||
- Docker: 'installation/040_INSTALL_DOCKER.md'
|
- Docker: 'installation/040_INSTALL_DOCKER.md'
|
||||||
- Troubleshooting: 'installation/INSTALL_TROUBLESHOOTING.md'
|
|
||||||
- Installing Models: 'installation/050_INSTALLING_MODELS.md'
|
- Installing Models: 'installation/050_INSTALLING_MODELS.md'
|
||||||
- Installing PyPatchMatch: 'installation/060_INSTALL_PATCHMATCH.md'
|
- Installing PyPatchMatch: 'installation/060_INSTALL_PATCHMATCH.md'
|
||||||
- Workflows & Nodes:
|
- Workflows & Nodes:
|
||||||
@ -182,7 +181,7 @@ nav:
|
|||||||
- Tutorials: 'contributing/contribution_guides/tutorials.md'
|
- Tutorials: 'contributing/contribution_guides/tutorials.md'
|
||||||
- Help:
|
- Help:
|
||||||
- Getting Started: 'help/gettingStartedWithAI.md'
|
- Getting Started: 'help/gettingStartedWithAI.md'
|
||||||
- FAQs: 'help/FAQ.md'
|
- FAQ: 'help/FAQ.md'
|
||||||
- Diffusion Overview: 'help/diffusion.md'
|
- Diffusion Overview: 'help/diffusion.md'
|
||||||
- Sampler Convergence: 'help/SAMPLER_CONVERGENCE.md'
|
- Sampler Convergence: 'help/SAMPLER_CONVERGENCE.md'
|
||||||
- Other:
|
- Other:
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user