Experimental support for Portainer CE 2.0 & 2.1

Added:
- Test PSU with Portainer CE 2.0.1 API
- Test PSU with Portainer CE 2.1.1 API

Changed:
- Use Docker Compose 1.28.3 instead of Docker Compose 1.26.2
- Use Traefik 2.4 instead of Traefik 2.2 for testing
- Upgrade operating system of Docker based images, with Alpine 3.13

Removed:
- Test PSU with Portainer 1.22.2 API
- Test PSU with Portainer 1.23.2 API
This commit is contained in:
Tortue Torche 2021-02-18 15:31:42 +01:00
parent 498418f816
commit 5275fc78f2
13 changed files with 137 additions and 51 deletions

3
.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
/tests/*-final.json
/tests/dockerfiles/.env
/tests/dockerfiles/*-final.yml

View File

@ -3,8 +3,10 @@ image: $CI_REGISTRY/$CI_PROJECT_NAMESPACE/hub/auto-deploy-image:latest
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
TRAEFIK_VERSION: "2.2"
TRAEFIK_VERSION: "2.4"
PORTAINER_IMAGE: "portainer/portainer-ce"
PORTAINER_VERSION: latest
PORTAINER_COMMAND_OPTIONS: ""
PSU_IMAGE: ${CI_REGISTRY_IMAGE}/builds
PSU_TAG: $$CI_COMMIT_SHA
PSU_TAG_CORE: core-$$CI_COMMIT_SHA
@ -91,15 +93,18 @@ build:debian-core:
- registry_login
- bash scripts/test.sh
test:portainer-1.22.2:
test:portainer-1.24.1:
<<: *test_definition
variables:
PORTAINER_VERSION: 1.22.2
PORTAINER_IMAGE: "portainer/portainer"
PORTAINER_VERSION: 1.24.1
PORTAINER_COMMAND_OPTIONS: " --no-analytics"
test:portainer-1.23.2:
test:portainer-2.0.1:
<<: *test_definition
variables:
PORTAINER_VERSION: 1.23.2
PORTAINER_VERSION: 2.0.1
test:portainer-latest:
<<: *test_definition

View File

@ -6,6 +6,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [1.2.0-alpha] - <abbr title="To Be Announced">TBA</abbr>
### Added
- Test <abbr title="Portainer Stack Utils">PSU</abbr> with Portainer <abbr title="Community Edition">CE</abbr> [2.0.1](https://app.swaggerhub.com/apis/deviantony/Portainer/2.0.1) API
- Test <abbr title="Portainer Stack Utils">PSU</abbr> with Portainer <abbr title="Community Edition">CE</abbr> [2.1.1](https://app.swaggerhub.com/apis/deviantony/Portainer/2.0.1) API
### Changed
- Use Docker [Compose 1.28.3](https://github.com/docker/compose/releases/tag/1.28.3) instead of Docker [Compose 1.26.2](https://github.com/docker/compose/releases/tag/1.26.2)
- Use [Traefik 2.4](https://docs.traefik.io/traefik/) instead of [Traefik 2.2](https://docs.traefik.io/traefik/v2.2/) for testing
- Upgrade operating system of Docker based images, with [Alpine 3.13](https://hub.docker.com/_/alpine)
### Removed
- Test <abbr title="Portainer Stack Utils">PSU</abbr> with Portainer [1.22.2](https://app.swaggerhub.com/apis/deviantony/Portainer/1.22.2) API
- Test <abbr title="Portainer Stack Utils">PSU</abbr> with Portainer [1.23.2](https://app.swaggerhub.com/apis/deviantony/Portainer/1.23.2) API
## [1.1.0] - 2021-02-18
### Changed
- Use [Traefik 2.2](https://docs.traefik.io/v2.2/) instead of [Traefik 2.1](https://docs.traefik.io/v2.1/) for testing
@ -106,7 +120,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Debug mode
- Strict mode
[Unreleased]: https://gitlab.com/psuapp/psu/compare/v1.1.0...1-1-stable
[Unreleased]: https://gitlab.com/psuapp/psu/compare/v1.2.0-alpha...1-2-next
[1.2.0-alpha]: https://gitlab.com/psuapp/psu/-/tags/v1.2.0-alpha
[1.1.0]: https://gitlab.com/psuapp/psu/-/tags/v1.1.0
[1.1.0-alpha]: https://gitlab.com/psuapp/psu/-/tags/v1.1.0-alpha
[1.0.7]: https://gitlab.com/psuapp/psu/-/tags/v1.0.7

View File

@ -1,11 +1,11 @@
FROM alpine:3.12
FROM alpine:3.13
RUN set -e; \
apk add --no-cache \
bash ca-certificates gettext jq \
py3-pip python3-dev libc-dev libffi-dev openssl-dev gcc make musl-dev cargo; \
\
pip3 --no-cache-dir install 'docker-compose>=1.26.2,<1.27.0' 'httpie>=1.0.3,<1.1.0' 'cryptography>=3.3.0,<3.4.0'; \
pip3 --no-cache-dir install 'docker-compose>=1.28.3,<1.29.0' 'httpie>=1.0.3,<1.1.0' 'cryptography>=3.3.2,<3.4.0'; \
\
apk del python3-dev libc-dev libffi-dev openssl-dev gcc make musl-dev cargo; \
rm -rf /tmp/src

View File

@ -1,4 +1,4 @@
FROM alpine:3.12
FROM alpine:3.13
RUN set -e; \
apk add --no-cache \

View File

@ -5,7 +5,7 @@ RUN set -e; \
apt-get install \
ca-certificates gettext-base httpie jq curl -yqq; \
\
curl -fL "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose; \
curl -fL "https://github.com/docker/compose/releases/download/1.28.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose; \
chmod +x /usr/local/bin/docker-compose; \
\
apt-get purge curl -y; \

View File

@ -3,7 +3,7 @@
[![Docker Pulls](https://img.shields.io/docker/pulls/psuapp/psu.svg)](https://hub.docker.com/r/psuapp/psu/)
[![Microbadger](https://images.microbadger.com/badges/image/psuapp/psu.svg)](http://microbadger.com/images/psuapp/psu "Image size")
[![pipeline status](https://gitlab.com/psuapp/psu/badges/1-1-stable/pipeline.svg)](https://gitlab.com/psuapp/psu/commits/1-1-stable)
[![pipeline status](https://gitlab.com/psuapp/psu/badges/1-2-next/pipeline.svg)](https://gitlab.com/psuapp/psu/commits/1-2-next)
Bash script to deploy/update/remove stacks in a [Portainer](https://portainer.io/) instance from a [docker-compose](https://docs.docker.com/compose) [yaml file](https://docs.docker.com/compose/compose-file).
@ -55,7 +55,7 @@ For Debian and similar apt-powered systems: `apt install bash httpie jq`.
If you don't want or can't install `psu` and its dependencies, you can run it with the default [published Docker image](https://hub.docker.com/r/psuapp/psu), like this:
```bash
docker run psuapp/psu:1.1 deploy ...
docker run psuapp/psu:1.2 deploy ...
```
> **Note**: Docker images are also available on [GitLab](https://gitlab.com/psuapp/psu/container_registry).
@ -66,6 +66,7 @@ For detailed instructions, see [How to use](#how-to-use) section.
Published Docker images are [tagged](https://hub.docker.com/r/psuapp/psu/tags) matching [GitLab tags](https://gitlab.com/psuapp/psu/-/tags):
<!-- - `dev` -> [`dev`](https://gitlab.com/psuapp/psu/-/tags/dev) -->
- `1.2.0-alpha` -> [`v1.2.0-alpha`](https://gitlab.com/psuapp/psu/-/tags/v1.2.0-alpha)
- `1`, `1.1`, `1.1.0` -> [`v1.1.0`](https://gitlab.com/psuapp/psu/-/tags/v1.1.0)
- `1.1.0-alpha` -> [`v1.1.0-alpha`](https://gitlab.com/psuapp/psu/-/tags/v1.1.0-alpha)
- `1.0`, `1.0.7` -> [`v1.0.7`](https://gitlab.com/psuapp/psu/-/tags/v1.0.7)
@ -101,10 +102,10 @@ The `debian` and `debian-core` variants use [Debian](https://www.debian.org) ins
For testing/debugging, you can use this Docker image in interactive mode, to run any commands inside the container:
```bash
docker run -v $(pwd)/docker-compose.yml:/docker-compose.yml -it --rm --entrypoint bash psuapp/psu:1.1
docker run -v $(pwd)/docker-compose.yml:/docker-compose.yml -it --rm --entrypoint bash psuapp/psu:1.2
# Run any commands here! E.g.
$ psu --version
Portainer Stack Utils, version 1.1.0
Portainer Stack Utils, version 1.2.0-alpha
License GPLv3: GNU GPL version 3
```
@ -137,7 +138,7 @@ bash ./psu rm --user admin --password password --url https://portainer.local --n
**With Docker:**
```bash
docker run -v $(pwd)/docker-compose.yml:/docker-compose.yml -v $(pwd)/.env:/.env psuapp/psu:1.1 deploy --user admin --password password --url https://portainer.local --name mystack --compose-file docker-compose.yml --env-file .env
docker run -v $(pwd)/docker-compose.yml:/docker-compose.yml -v $(pwd)/.env:/.env psuapp/psu:1.2 deploy --user admin --password password --url https://portainer.local --name mystack --compose-file docker-compose.yml --env-file .env
```
### With flags
@ -165,7 +166,7 @@ bash ./psu rm -u admin -p password -l https://portainer.local -n mystack
**With Docker:**
```bash
docker run -v $(pwd)/docker-compose.yml:/docker-compose.yml -v $(pwd)/.env:/.env psuapp/psu:1.1 deploy -u admin -p password -l https://portainer.local -n mystack -c docker-compose.yml -g .env
docker run -v $(pwd)/docker-compose.yml:/docker-compose.yml -v $(pwd)/.env:/.env psuapp/psu:1.2 deploy -u admin -p password -l https://portainer.local -n mystack -c docker-compose.yml -g .env
```
### With envvars
@ -207,20 +208,20 @@ bash ./psu
**With Docker:**
```bash
docker run -v $(pwd)/docker-compose.yml:/docker-compose.yml -v $(pwd)/.env:/.env -e ACTION="deploy" -e PORTAINER_USER="admin" -e PORTAINER_PASSWORD="password" -e PORTAINER_URL="https://portainer.local" -e PORTAINER_STACK_NAME="mystack" -e DOCKER_COMPOSE_FILE="docker-compose.yml" -e ENVIRONMENT_VARIABLES_FILE=".env" psuapp/psu:1.1
docker run -v $(pwd)/docker-compose.yml:/docker-compose.yml -v $(pwd)/.env:/.env -e ACTION="deploy" -e PORTAINER_USER="admin" -e PORTAINER_PASSWORD="password" -e PORTAINER_URL="https://portainer.local" -e PORTAINER_STACK_NAME="mystack" -e DOCKER_COMPOSE_FILE="docker-compose.yml" -e ENVIRONMENT_VARIABLES_FILE=".env" psuapp/psu:1.2
```
## Documentation
<div class="docsify-hidden">
For advanced usage, see the full <a href="https://psuapp.gitlab.io/psu/1-1-stable"><abbr title="Portainer Stack Utils">PSU</abbr> documentation</a>.
For advanced usage, see the full <a href="https://psuapp.gitlab.io/psu/1-2-next"><abbr title="Portainer Stack Utils">PSU</abbr> documentation</a>.
</div>
For detailed instructions, see the [CLI Commands](docs/README.md) documentation.
## Supported Portainer API
<abbr title="Portainer Stack Utils">PSU</abbr> was created for the latest versions of Portainer API, which at the time of writing are [1.22.2](https://app.swaggerhub.com/apis/deviantony/Portainer/1.22.2), [1.23.2](https://app.swaggerhub.com/apis/deviantony/Portainer/1.23.2) and [1.24.1](https://app.swaggerhub.com/apis/deviantony/Portainer/1.24.1).
<abbr title="Portainer Stack Utils">PSU</abbr> was created for the latest versions of Portainer API, which at the time of writing are [1.24.1](https://app.swaggerhub.com/apis/deviantony/Portainer/1.24.1), [2.0.1](https://app.swaggerhub.com/apis/deviantony/Portainer/2.0.1) and [2.1.1](https://app.swaggerhub.com/apis/deviantony/Portainer/2.0.1).
## License

2
psu
View File

@ -27,7 +27,7 @@ set -e
# None #
############################
main() {
VERSION="1.1.0"
VERSION="1.2.0-alpha"
OPTIONS_TABLE=(
# option_key;flag_text;option_text;description

View File

@ -0,0 +1,27 @@
#TRACE=true
CI_REGISTRY=localhost.com
CI_PROJECT_NAMESPACE=psuapp
CI_REGISTRY_IMAGE=localhost.com/psupapp/psu
CI_REGISTRY_USER=gitlab-ci-token
CI_REGISTRY_PASSWORD=longalfanumstring
CI_COMMIT_SHA=ee7ee6556462d46423840dcc49991be630545f60
#DOCKER_REGISTRY_IMAGE=$DOCKER_REGISTRY/$CI_PROJECT_PATH
PSU_IMAGE=localhost.com/psuapp/psu/builds
PSU_TAG=ee7ee6556462d46423840dcc49991be630545f60
PSU_TAG_CORE=core-ee7ee6556462d46423840dcc49991be630545f60
PSU_PASSWORD=portainerAdminPassword
TRAEFIK_VERSION="2.4"
PORTAINER_IMAGE="portainer/portainer-ce"
#PORTAINER_IMAGE="portainer/portainer"
PORTAINER_VERSION=latest
#PORTAINER_VERSION="2.0.1"
#PORTAINER_VERSION="1.24.1"
PORTAINER_COMMAND_OPTIONS=""
#PORTAINER_COMMAND_OPTIONS=" --no-analytics"
CLUSTER_NAME=localhost
CLUSTER_IP=$(hostname -I | awk '{ print $1 }')

View File

@ -1,12 +1,12 @@
version: '3.6'
version: '3.7'
services:
portainer:
image: portainer/portainer:$PORTAINER_VERSION
command: --admin-password-file '/run/secrets/portainer-password' --no-analytics
image: ${PORTAINER_IMAGE}:${PORTAINER_VERSION}
command: --admin-password-file '/run/secrets/psu-portainer-password'${PORTAINER_COMMAND_OPTIONS}
labels:
- traefik.enable=true
- traefik.docker.network=traefik-net
- traefik.docker.network=psu-traefik-net
# HTTPS route
- "traefik.http.routers.portainer.entrypoints=https"
- "traefik.http.routers.portainer.rule=Host(`portainer.$BASE_DOMAIN`)"
@ -18,7 +18,7 @@ services:
# Service
- traefik.http.services.portainer.loadbalancer.server.port=9000
networks:
- traefik-net
- psu-traefik-net
environment:
- HTTP_PROXY
- HTTPS_PROXY
@ -27,9 +27,9 @@ services:
- NO_PROXY
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- portainer:/data
- psu-portainer:/data
secrets:
- portainer-password
- psu-portainer-password
deploy:
placement:
constraints:
@ -39,12 +39,12 @@ services:
delay: 10s
networks:
traefik-net:
psu-traefik-net:
driver: overlay
external: true
secrets:
portainer-password:
psu-portainer-password:
external: true
volumes:
portainer:
psu-portainer:

View File

@ -1,4 +1,4 @@
version: '3.6'
version: '3.7'
services:
reverse-proxy:
@ -19,7 +19,7 @@ services:
- NO_PROXY
labels:
- "traefik.enable=true"
- traefik.docker.network=traefik-net
- traefik.docker.network=psu-traefik-net
- traefik.http.middlewares.retry-if-fails.retry.attempts=10
- traefik.http.middlewares.https-only.redirectscheme.scheme=https
- traefik.http.middlewares.secured.chain.middlewares=retry-if-fails,https-only
@ -32,7 +32,7 @@ services:
- "80:80"
- 443:443
networks:
- traefik-net
- psu-traefik-net
volumes:
# So that Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
@ -46,6 +46,6 @@ services:
delay: 10s
networks:
traefik-net:
psu-traefik-net:
driver: overlay
name: traefik-net
name: psu-traefik-net

View File

@ -1,4 +1,4 @@
version: '3.6'
version: '3.7'
services:
job:
@ -27,7 +27,7 @@ services:
- https_proxy
- NO_PROXY
volumes:
- php-runner:/var/run/php
- psu-php-runner:/var/run/php
deploy:
update_config:
order: start-first
@ -37,7 +37,7 @@ services:
image: $CI_REGISTRY/$CI_PROJECT_NAMESPACE/hub/testing/psu-apache2:latest
labels:
- traefik.enable=true
- traefik.docker.network=traefik-net
- traefik.docker.network=psu-traefik-net
# HTTPS route
- "traefik.http.routers.web-php-app.entrypoints=https"
- "traefik.http.routers.web-php-app.rule=Host(`$PSU_STACK_NAME.$BASE_DOMAIN`)"
@ -56,17 +56,17 @@ services:
- https_proxy
- NO_PROXY
volumes:
- php-runner:/var/run/php
- psu-php-runner:/var/run/php
deploy:
update_config:
failure_action: rollback
order: start-first
networks:
- traefik-net
- psu-traefik-net
networks:
traefik-net:
psu-traefik-net:
external: true
volumes:
php-runner:
psu-php-runner:

View File

@ -1,13 +1,25 @@
#!/usr/bin/env bash
set -e
# Change working directory to 'tests/'
cd "$(dirname "$0")"
# Using .env file when running tests without GitLab CI
if [[ -f "dockerfiles/.env" ]]; then
set -a
source "dockerfiles/.env"
set +a
fi
[[ "$TRACE" ]] && set -x
CLUSTER_IP=$(getent hosts cluster | awk '{ print $1 }')
CLUSTER_NAME=${CLUSTER_NAME:-cluster}
CLUSTER_IP=${CLUSTER_IP:-$(getent hosts $CLUSTER_NAME | awk '{ print $1 }')}
export BASE_DOMAIN="$CLUSTER_IP.nip.io"
export PSU_STACK_NAME="web-app"
PSU_URL="https://portainer.$BASE_DOMAIN"
PSU_USER="admin"
PSU_PASSWORD="mypassword"
PSU_PASSWORD=${PSU_PASSWORD:-"$(openssl rand -hex 50)"}
PSU_TAG=$(if [ -n "$PSU_TAG" ]; then
eval echo "$PSU_TAG";
@ -23,9 +35,6 @@ PSU_TAG_CORE=$(if [ -n "$PSU_TAG_CORE" ]; then
fi)
export PSU_TAG_CORE
# Change working directory to 'tests/'
cd "$(dirname "$0")"
function psu_wrapper() {
docker run --rm $PSU_IMAGE:$PSU_TAG "$@"
}
@ -53,6 +62,23 @@ function application_exists() {
fi
}
# Source: https://stackoverflow.com/a/24067243
function version {
echo "$@" | awk -F. '{ printf("%03d%03d%03d\n", $1,$2,$3); }';
}
# Leave Docker Swarm, if already set
docker swarm leave --force && sleep 3 | true
# Remove admin password from Docker secrets, if already set
docker secret rm psu-portainer-password | true
# Remove portainer data, if already set
docker volume rm --force portainer_psu-portainer && sleep 2 | true
# Remove web-app data, if already set
docker volume rm --force web-app_psu-php-runner && sleep 2 | true
# Init Docker Swarm
docker swarm init
@ -60,12 +86,15 @@ docker swarm init
# Parse the Docker traefik stack file to deploy
envsubst '$TRAEFIK_VERSION,$BASE_DOMAIN' < dockerfiles/docker-stack-traefik.yml > dockerfiles/docker-stack-traefik-final.yml
docker stack deploy -c dockerfiles/docker-stack-traefik-final.yml traefik --with-registry-auth
bash -c "timeout 20 bash -c 'while ! (echo > /dev/tcp/cluster/443 && curl -fks --max-time 2 https://traefik.$BASE_DOMAIN) >/dev/null 2>&1; do sleep 1; done;'"
bash -c "timeout 20 bash -c 'while ! (echo > /dev/tcp/$CLUSTER_NAME/443 && curl -fks --max-time 2 https://traefik.$BASE_DOMAIN) >/dev/null 2>&1; do sleep 1; done;'"
# Deploy Portainer test
echo -n $PSU_PASSWORD | docker secret create portainer-password -
# Create admin password as a Docker secret
# See: https://documentation.portainer.io/v2.0/deploy/initial/
echo -n $PSU_PASSWORD | docker secret create psu-portainer-password -
# Parse the Docker portainer stack file to deploy
envsubst '$PORTAINER_VERSION,$BASE_DOMAIN' < dockerfiles/docker-stack-portainer.yml > dockerfiles/docker-stack-portainer-final.yml
envsubst '$PORTAINER_IMAGE,$PORTAINER_VERSION,$PORTAINER_COMMAND_OPTIONS,$BASE_DOMAIN' < dockerfiles/docker-stack-portainer.yml > dockerfiles/docker-stack-portainer-final.yml
docker stack deploy -c dockerfiles/docker-stack-portainer-final.yml portainer --with-registry-auth
bash -c "timeout 20 bash -c 'while ! (curl -fkLs --max-time 2 $PSU_URL) >/dev/null 2>&1; do sleep 1; done;'"
@ -86,8 +115,14 @@ PSU_AUTH_TOKEN=$(psu_wrapper login --user $PSU_USER --password $PSU_PASSWORD --u
# Add GitLab Docker registry access to Portainer
envsubst '$CI_REGISTRY_USER,$CI_REGISTRY_PASSWORD' < gitlab-registry.json > gitlab-registry-final.json
http --check-status --ignore-stdin --verify=no --timeout=10 POST "$PSU_URL/api/registries" "Authorization: Bearer $PSU_AUTH_TOKEN" @gitlab-registry-final.json
# Add local endpoint to the Portainer instance
http --check-status --ignore-stdin --verify=no --timeout=10 POST "$PSU_URL/api/endpoints" "Authorization: Bearer $PSU_AUTH_TOKEN" Name==local EndpointType==1
end_point_type_param=EndpointType
if [[ "$PORTAINER_VERSION" == "latest" ]] || [ "$(version $PORTAINER_VERSION)" -ge "$(version 2.0)" ]; then
# See: https://github.com/portainer/portainer/issues/4602#issuecomment-746819197
end_point_type_param=EndpointCreationType
fi
http --check-status --ignore-stdin --verify=no --timeout=10 POST "$PSU_URL/api/endpoints" "Authorization: Bearer $PSU_AUTH_TOKEN" Name==local $end_point_type_param==1
# Docker system info from Portainer test
docker_info=$(psu_wrapper system:info --user $PSU_USER --password $PSU_PASSWORD --url $PSU_URL --insecure --debug false --verbose false)