Mirrored_Repos/InvokeAI
1
0
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI synced 2024-08-30 20:32:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

update (docker-)build scripts, .dockerignore and add patchmatch (#1970)

Browse Source 
* update build scripts and dockerignore
updates to build and run script:
- read repository name
- include flavor in container name
- read arch via arch command
- use latest tag instead of arch
- don't bindmount `$HOME/.huggingface`
- make sure HUGGINGFACE_TOKEN is set

updates to .dockerignore
- include environment-and-requirements
- exclude binary_installer
- exclude docker-build
- exclude docs

* disable push and pr triggers of cloud image
also disable pushing.

This was decided since:
- it is not multiarch useable
- the default image is already cloud aproved

* integrate patchmatch in container

* pin verisons of recently introduced dependencies

* remove now unecesarry part from build.sh
move huggingface token to run script, so it can download missing models

* move GPU_FLAGS to run script
since not needed at build time

* update env.sh

- read REPOSITORY_NAME from env if available
- add comment to explain the intension of this file
- remove unecesarry exports

* get rid of repository_name_lc

* capitalize variables

* update INSTALL_DOCKER with new variables

* add comments pointing to the docs

Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
This commit is contained in:
Matthias Wild 2022-12-16 13:53:37 +01:00 committed by GitHub
parent 7d09d9da49
commit c0c4d7ca69
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 83 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
.dockerignore
View File

@ -1,16 +1,13 @@
* *
!backend !backend
!environments-and-requirements
!frontend !frontend
!binary_installer
!ldm !ldm
!main.py !main.py
!scripts !scripts
!server !server
!static !static
!setup.py !setup.py
!docker-build
!docs
docker-build/Dockerfile
# Guard against pulling in any models that might exist in the directory tree # Guard against pulling in any models that might exist in the directory tree
**/*.pt* **/*.pt*
@ -19,8 +16,4 @@ docker-build/Dockerfile
!configs !configs
configs/models.yaml configs/models.yaml
# unignore environment dirs/files, but ignore the environment.yml file or symlink in case it exists
!environment*
environment.yml
**/__pycache__ **/__pycache__

20
.github/workflows/build-cloud-img.yml vendored
View File

@ -1,15 +1,15 @@
name: Build and push cloud image name: Build and push cloud image
on: on:
workflow_dispatch: workflow_dispatch:
push: # push:
branches: # branches:
- main # - main
tags: # tags:
- v* # - v*
# we will NOT push the image on pull requests, only test buildability. # # we will NOT push the image on pull requests, only test buildability.
pull_request: # pull_request:
branches: # branches:
- main # - main
permissions: permissions:
contents: read contents: read
@ -82,6 +82,6 @@ jobs:
file: docker-build/Dockerfile.cloud file: docker-build/Dockerfile.cloud
platforms: Linux/${{ matrix.arch }} platforms: Linux/${{ matrix.arch }}
# do not push the image on PRs # do not push the image on PRs
push: ${{ github.event_name != 'pull_request' }} push: false
tags: ${{ steps.meta.outputs.tags }} tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }} labels: ${{ steps.meta.outputs.labels }}

27
docker-build/Dockerfile
View File

@ -14,9 +14,10 @@ RUN apt-get update \
&& apt-get clean \ && apt-get clean \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# set workdir, PATH and copy sources # set WORKDIR, PATH and copy sources
WORKDIR /usr/src/app ARG WORKDIR=/usr/src/app
ENV PATH /usr/src/app/.venv/bin:$PATH WORKDIR ${WORKDIR}
ENV PATH ${WORKDIR}/.venv/bin:$PATH
ARG PIP_REQUIREMENTS=requirements-lin-cuda.txt ARG PIP_REQUIREMENTS=requirements-lin-cuda.txt
COPY . ./environments-and-requirements/${PIP_REQUIREMENTS} ./ COPY . ./environments-and-requirements/${PIP_REQUIREMENTS} ./
@ -38,18 +39,28 @@ FROM python:3.10-slim AS runtime
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y \ && apt-get install -y \
--no-install-recommends \ --no-install-recommends \
build-essential=12.9 \
libgl1-mesa-glx=20.3.* \ libgl1-mesa-glx=20.3.* \
libglib2.0-0=2.66.* \ libglib2.0-0=2.66.* \
libopencv-dev=4.5.* \
python3-opencv=4.5.* \
&& apt-get clean \ && apt-get clean \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /usr/src/app # setup environment
COPY --from=builder /usr/src/app . ARG WORKDIR=/usr/src/app
WORKDIR ${WORKDIR}
# set Environment, Entrypoint and default CMD COPY --from=builder ${WORKDIR} .
ENV PATH=${WORKDIR}/.venv/bin:$PATH
ENV INVOKEAI_ROOT /data ENV INVOKEAI_ROOT /data
ENV INVOKE_MODEL_RECONFIGURE --yes ENV INVOKE_MODEL_RECONFIGURE --yes
ENV PATH=/usr/src/app/.venv/bin:$PATH
# Initialize patchmatch
RUN ln -sf \
/usr/lib/"$(arch)"-linux-gnu/pkgconfig/opencv4.pc \
/usr/lib/"$(arch)"-linux-gnu/pkgconfig/opencv.pc \
&& python3 -c "from patchmatch import patch_match"
# set Entrypoint and default CMD
ENTRYPOINT [ "python3", "scripts/invoke.py" ] ENTRYPOINT [ "python3", "scripts/invoke.py" ]
CMD [ "--web", "--host=0.0.0.0" ] CMD [ "--web", "--host=0.0.0.0" ]

46
docker-build/build.sh
View File

@ -1,49 +1,35 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -e set -e
# IMPORTANT: You need to have a token on huggingface.co to be able to download the checkpoints!!! # How to use: https://invoke-ai.github.io/InvokeAI/installation/INSTALL_DOCKER/#setup
# configure values by using env when executing build.sh f.e. `env ARCH=aarch64 ./build.sh`
source ./docker-build/env.sh \ source ./docker-build/env.sh \
|| echo "please execute docker-build/build.sh from repository root" \ || echo "please execute docker-build/build.sh from repository root" \
|| exit 1 || exit 1
pip_requirements=${PIP_REQUIREMENTS:-requirements-lin-cuda.txt} PIP_REQUIREMENTS=${PIP_REQUIREMENTS:-requirements-lin-cuda.txt}
dockerfile=${INVOKE_DOCKERFILE:-docker-build/Dockerfile} DOCKERFILE=${INVOKE_DOCKERFILE:-docker-build/Dockerfile}
# print the settings # print the settings
echo -e "You are using these values:\n" echo -e "You are using these values:\n"
echo -e "Dockerfile:\t ${dockerfile}" echo -e "Dockerfile:\t ${DOCKERFILE}"
echo -e "requirements:\t ${pip_requirements}" echo -e "Requirements:\t ${PIP_REQUIREMENTS}"
echo -e "volumename:\t ${volumename}" echo -e "Volumename:\t ${VOLUMENAME}"
echo -e "arch:\t\t ${arch}" echo -e "arch:\t\t ${ARCH}"
echo -e "platform:\t ${platform}" echo -e "Platform:\t ${PLATFORM}"
echo -e "invokeai_tag:\t ${invokeai_tag}\n" echo -e "Invokeai_tag:\t ${INVOKEAI_TAG}\n"
if [[ -n "$(docker volume ls -f name="${volumename}" -q)" ]]; then if [[ -n "$(docker volume ls -f name="${VOLUMENAME}" -q)" ]]; then
echo "Volume already exists" echo -e "Volume already exists\n"
echo
else else
echo -n "createing docker volume " echo -n "createing docker volume "
docker volume create "${volumename}" docker volume create "${VOLUMENAME}"
fi fi
# Build Container # Build Container
docker build \ docker build \
--platform="${platform}" \ --platform="${PLATFORM}" \
--tag="${invokeai_tag}" \ --tag="${INVOKEAI_TAG}" \
--build-arg="PIP_REQUIREMENTS=${pip_requirements}" \ --build-arg="PIP_REQUIREMENTS=${PIP_REQUIREMENTS}" \
--file="${dockerfile}" \ --file="${DOCKERFILE}" \
. .
docker run \
--rm \
--platform="$platform" \
--name="$project_name" \
--hostname="$project_name" \
--mount="source=$volumename,target=/data" \
--mount="type=bind,source=$HOME/.huggingface,target=/root/.huggingface" \
--env="HUGGINGFACE_TOKEN=${HUGGINGFACE_TOKEN}" \
--entrypoint="python3" \
"${invokeai_tag}" \
scripts/configure_invokeai.py --yes

20
docker-build/env.sh
View File

@ -1,15 +1,9 @@
#!/usr/bin/env bash #!/usr/bin/env bash
project_name=${PROJECT_NAME:-invokeai} # Variables shared by build.sh and run.sh
volumename=${VOLUMENAME:-${project_name}_data} REPOSITORY_NAME=${REPOSITORY_NAME:-$(basename "$(git rev-parse --show-toplevel)")}
arch=${ARCH:-x86_64} VOLUMENAME=${VOLUMENAME:-${REPOSITORY_NAME,,}_data}
platform=${PLATFORM:-Linux/${arch}} ARCH=${ARCH:-$(arch)}
invokeai_tag=${INVOKEAI_TAG:-${project_name}:${arch}} PLATFORM=${PLATFORM:-Linux/${ARCH}}
gpus=${GPU_FLAGS:+--gpus=${GPU_FLAGS}} CONTAINER_FLAVOR=${CONTAINER_FLAVOR:-cuda}
INVOKEAI_TAG=${REPOSITORY_NAME,,}-${CONTAINER_FLAVOR}:${INVOKEAI_TAG:-latest}
export project_name
export volumename
export arch
export platform
export invokeai_tag
export gpus

28
docker-build/run.sh
View File

@ -1,21 +1,31 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -e set -e
source ./docker-build/env.sh || echo "please run from repository root" || exit 1 # How to use: https://invoke-ai.github.io/InvokeAI/installation/INSTALL_DOCKER/#run-the-container
# IMPORTANT: You need to have a token on huggingface.co to be able to download the checkpoints!!!
source ./docker-build/env.sh \
|| echo "please run from repository root" \
|| exit 1
# check if HUGGINGFACE_TOKEN is available
# You must have accepted the terms of use for required models
HUGGINGFACE_TOKEN=${HUGGINGFACE_TOKEN:?Please set your token for Huggingface as HUGGINGFACE_TOKEN}
echo -e "You are using these values:\n" echo -e "You are using these values:\n"
echo -e "volumename:\t ${volumename}" echo -e "Volumename:\t ${VOLUMENAME}"
echo -e "invokeai_tag:\t ${invokeai_tag}\n" echo -e "Invokeai_tag:\t ${INVOKEAI_TAG}\n"
docker run \ docker run \
--interactive \ --interactive \
--tty \ --tty \
--rm \ --rm \
--platform="$platform" \ --platform="$PLATFORM" \
--name="$project_name" \ --name="${REPOSITORY_NAME,,}" \
--hostname="$project_name" \ --hostname="${REPOSITORY_NAME,,}" \
--mount="source=$volumename,target=/data" \ --mount="source=$VOLUMENAME,target=/data" \
--env="HUGGINGFACE_TOKEN=${HUGGINGFACE_TOKEN}" \
--publish=9090:9090 \ --publish=9090:9090 \
--cap-add=sys_nice \ --cap-add=sys_nice \
$gpus \ ${GPU_FLAGS:+--gpus=${GPU_FLAGS}} \
"$invokeai_tag" ${1:+$@} "$INVOKEAI_TAG" ${1:+$@}

21
docs/installation/040_INSTALL_DOCKER.md
View File

@ -78,15 +78,16 @@ Some Suggestions of variables you may want to change besides the Token:
<figure markdown> <figure markdown>
| Environment-Variable | Default value | Description | | Environment-Variable | Default value | Description |
| -------------------- | ----------------------------- | -------------------------------------------------------------------------------------------- | | -------------------- | ----------------------------- | -------------------------------------------------------------------------------------------- |
| `HUGGINGFACE_TOKEN` | No default, but **required**! | This is the only **required** variable, without it you can't download the huggingface models | | `HUGGINGFACE_TOKEN` | No default, but **required**! | This is the only **required** variable, without it you can't download the huggingface models |
| `PROJECT_NAME` | `invokeai` | affects the project folder, tag- and volume name | | `REPOSITORY_NAME` | The Basename of the Repo folder | This name will used as the container repository/image name |
| `VOLUMENAME` | `${PROJECT_NAME}_data` | Name of the Docker Volume where model files will be stored | | `VOLUMENAME` | `${REPOSITORY_NAME,,}_data` | Name of the Docker Volume where model files will be stored |
| `ARCH` | `x86_64` | can be changed to f.e. aarch64 if you are using a ARM based CPU | | `ARCH` | arch of the build machine | can be changed if you want to build the image for another arch |
| `INVOKEAI_TAG` | `${PROJECT_NAME}:${ARCH}` | the Container Repository / Tag which will be used | | `INVOKEAI_TAG` | latest | the Container Repository / Tag which will be used |
| `PIP_REQUIREMENTS` | `requirements-lin-cuda.txt` | the requirements file to use (from `environments-and-requirements`) | | `PIP_REQUIREMENTS` | `requirements-lin-cuda.txt` | the requirements file to use (from `environments-and-requirements`) |
| `INVOKE_DOCKERFILE` | `docker-build/Dockerfile` | the Dockerfile which should be built, handy for development | | `CONTAINER_FLAVOR` | cuda | the flavor of the image, which can be changed if you build f.e. with amd requirements file. |
| `INVOKE_DOCKERFILE` | `docker-build/Dockerfile` | the Dockerfile which should be built, handy for development |
</figure> </figure>
@ -129,7 +130,7 @@ also do so.
## Running the container on your GPU ## Running the container on your GPU
If you have an Nvidia GPU, you can enable InvokeAI to run on the GPU by running the container with an extra If you have an Nvidia GPU, you can enable InvokeAI to run on the GPU by running the container with an extra
environment variable to enable GPU usage and have the process run much faster: environment variable to enable GPU usage and have the process run much faster:
```bash ```bash