Commit Graph

103 Commits

Author SHA1 Message Date
Ben Wallis
3dd3e5d5ea Added cargo cmd-doc-gen to auto-generate the table of commands in the new players/commands.md page in the book 2021-08-21 13:36:55 +01:00
Marcel Märtens
9ad423488f implement a new tag system for all veloren runners, including
- project tag:
   - `veloren/veloren`: for the veloren repo only
   - `veloren/*`: for projects thats are in the veloren namespace, e.g. usefull for smaller projects that dont have their own tag
   - `*/*` for projects by 3rd parties, e.g. personal projects
 - trusted tag:
   - `owned`:  The runner is hosted on veloren hardware
   - `trusted`: The runner is hosted by a private person, trusted by the veloren devs
   - NONE: if no special trust is given in a runner
 - check/build tag:
   - `check`: a job only performs a check and NO executable is build for users
   - `build`: the job contains atleast 1 executable that can be run by users
   - NONE: neither a check nor a executable is build, e.g. usefull for pages or meta jobs
 - publish tag:
   - `publish`: this job produces a artifact that is automatically pushed to users
   - NONE: no artifact is pushed automatically to users
 - pin tags:
   - `benchmark`: pin a runner to a specific job for the cause of benchmarks
   - `veloren/*:macos`: runs on a native macos runner ONLY, this needs to be a modified project tag, as otherwise also normal checks would run on this runner.
   - NONE: no runner is pinned
2021-08-19 09:23:31 +02:00
Ben Wallis
6935d0f6cd Added veloren-publish tags and veloren-build tags to CI configuration 2021-08-19 09:08:49 +02:00
Marcel Märtens
3717ca2b8f move translation check to seperate ci step
note: TAGUUID is used to avoid sqlinjections by choosing a branch name
2021-08-02 20:11:02 +02:00
Snowram
49461554e5 Retry release jobs on failure 2021-07-31 00:09:54 +02:00
Marcel Märtens
cfe3a027be remove some reexports 2021-07-29 22:18:34 +02:00
Marcel Märtens
365e23992e execute jobs only if all previous jobs in previous stages succeded, e.g. in case of error in windows job dont push the linux docker container or change the tag yet 2021-07-22 12:05:43 +02:00
Marcel Märtens
5000803966 create a nightly tag that always point to the released nightly version of veloren in the respective schedule so that others can consume it 2021-07-21 08:43:48 +02:00
Marcel Märtens
c3ec385ddb quickfix to avoid kaniko stopping with
"kaniko should only be run inside of a container, run with the --force flag if you are sure you want to continue" error
applied as described here https://github.com/GoogleContainerTools/kaniko/issues/1542

its also done in veloren-docker-cli: c8aa8ac857

We didnt had that problem in veloren repo until now.
2021-07-19 13:33:19 +02:00
Marcel Märtens
464960e95f now fix tarpaulin target cleanup, hopefully 2021-07-19 10:39:22 +02:00
Marcel Märtens
013faa8c6a set CI to also check a MR if it only modifies shell scripts 2021-07-18 18:53:52 +02:00
Marcel Märtens
c6d9fbd147 choose correct path for tarpaulin 2021-07-18 18:52:08 +02:00
Marcel Märtens
6de161a68f Revert "no longer incremental builds in CI"
This reverts commit b3983c5451.
2021-07-17 21:53:32 +02:00
Marcel Märtens
b04ba58040 fix scripts that ended with && and correct the ASSETS variable for tests 2021-07-16 15:32:39 +02:00
Marcel Märtens
ac10410782 only build once, dont build multiple targets. save time
Profiles:
(all are build, all are release, all are --no-default-features --features="default-publish" )

A) -p veloren-voxygen AND -p veloren-server-cli
B) NONE AND -p veloren-server-cli
C) NONE
D) NONE but server-cli/Cargo.toml has a default-publish feature

Timings
A)
> time cargo build -p veloren-voxygen --release --no-default-features --features default-publish
Compiling veloren-client v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/client)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling gfx-auxil v0.9.0 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling gfx-backend-gl v0.8.1 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling wgpu v0.8.0 (https://github.com/gfx-rs/wgpu.git?rev=a92b8549a8e2cb9dac781bafc5ed32828f3caf46#a92b8549)
   Compiling wgpu-profiler v0.4.0 (https://github.com/Imberflur/wgpu-profiler?tag=wgpu-0.8#b156eb14)
    Finished release [optimized] target(s) in 16m 19s
cargo build -p veloren-voxygen --release --no-default-features --features   3091.78s user 71.80s system 322% cpu 16:19.44 total
> time cargo build --release -p veloren-server-cli
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 6m 06s
cargo build --release -p veloren-server-cli  1024.29s user 18.42s system 284% cpu 6:06.62 total

B)
> time cargo build --release --no-default-features --features default-publish
   Compiling veloren-client v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/client)
   Compiling veloren-voxygen-egui v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui)
   Compiling gfx-auxil v0.9.0 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling gfx-backend-gl v0.8.1 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling wgpu v0.8.0 (https://github.com/gfx-rs/wgpu.git?rev=a92b8549a8e2cb9dac781bafc5ed32828f3caf46#a92b8549)
   Compiling wgpu-profiler v0.4.0 (https://github.com/Imberflur/wgpu-profiler?tag=wgpu-0.8#b156eb14)
   Compiling veloren-voxygen-anim-dyn v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim/dyn)
   Compiling veloren-voxygen-egui-dyn v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui/dyn)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 17m 10s
cargo build --release --no-default-features --features default-publish  4415.58s user 91.10s system 437% cpu 17:10.66 total
> time cargo build --release -p veloren-server-cli
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 6m 08s
cargo build --release -p veloren-server-cli  1123.93s user 23.01s system 311% cpu 6:08.14 total

C)
> time cargo build --release --no-default-features --features default-publish
   Compiling prometheus-hyper v0.1.2
   Compiling authc v1.0.0 (https://gitlab.com/veloren/auth.git?rev=fb3dcbc4962b367253f8f2f92760ef44d2679c9a#fb3dcbc4)
   Compiling veloren-common-net v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/net)
   Compiling veloren-plugin-api v0.1.0 (/mnt/nfs/marcel/Entw/Rust/veloren/plugin/api)
   Compiling veloren-voxygen-anim v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim)
   Compiling veloren-plugin-rt v0.1.0 (/mnt/nfs/marcel/Entw/Rust/veloren/plugin/rt)
   Compiling veloren-common-systems v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/systems)
   Compiling veloren-common-state v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/state)
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling veloren-client v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/client)
   Compiling veloren-voxygen-egui v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling gfx-auxil v0.9.0 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling gfx-backend-gl v0.8.1 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling wgpu v0.8.0 (https://github.com/gfx-rs/wgpu.git?rev=a92b8549a8e2cb9dac781bafc5ed32828f3caf46#a92b8549)
   Compiling wgpu-profiler v0.4.0 (https://github.com/Imberflur/wgpu-profiler?tag=wgpu-0.8#b156eb14)
   Compiling veloren-voxygen-anim-dyn v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim/dyn)
   Compiling veloren-voxygen-egui-dyn v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui/dyn)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 16m 41s
cargo build --release --no-default-features --features default-publish  4349.71s user 90.02s system 443% cpu 16:41.58 total

D)
> time cargo build --release --no-default-features --features default-publish
   Compiling prometheus-hyper v0.1.2
   Compiling authc v1.0.0 (https://gitlab.com/veloren/auth.git?rev=fb3dcbc4962b367253f8f2f92760ef44d2679c9a#fb3dcbc4)
   Compiling veloren-common-net v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/net)
   Compiling veloren-plugin-api v0.1.0 (/mnt/nfs/marcel/Entw/Rust/veloren/plugin/api)
   Compiling veloren-voxygen-anim v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim)
   Compiling veloren-plugin-rt v0.1.0 (/mnt/nfs/marcel/Entw/Rust/veloren/plugin/rt)
   Compiling veloren-common-state v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/state)
   Compiling veloren-common-systems v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/systems)
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling veloren-client v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/client)
   Compiling veloren-voxygen-egui v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling gfx-auxil v0.9.0 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling gfx-backend-gl v0.8.1 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling wgpu v0.8.0 (https://github.com/gfx-rs/wgpu.git?rev=a92b8549a8e2cb9dac781bafc5ed32828f3caf46#a92b8549)
   Compiling wgpu-profiler v0.4.0 (https://github.com/Imberflur/wgpu-profiler?tag=wgpu-0.8#b156eb14)
   Compiling veloren-voxygen-egui-dyn v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui/dyn)
   Compiling veloren-voxygen-anim-dyn v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim/dyn)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 17m 30s
cargo build --release --no-default-features --features default-publish  4462.34s user 92.17s system 433% cpu 17:31.00 total

Hashes
A)
veloren-voxygen: e9de69144b4d459dc0da2775b2b87ac9
veloren-server-cli: 8c131c1b1daaa6efd0f83d95d9b5dad9
B)
veloren-voxygen: 415e14ec1c14a6a559d2908b843d88d5
veloren-server-cli: 8c131c1b1daaa6efd0f83d95d9b5dad9
C)
veloren-voxygen: 415e14ec1c14a6a559d2908b843d88d5
veloren-server-cli: 3bf56d0d0a76191a578ed5e71389f512
D)
veloren-voxygen: 415e14ec1c14a6a559d2908b843d88d5
veloren-server-cli: c9dc4fb0bfcbd831150eebac788e2f71

Binary Sizes:
A)
veloren-voxygen: 156667288
veloren-server-cli: 85136736
B)
veloren-voxygen: 162860552
veloren-server-cli: 85136736
C)
veloren-voxygen: 162860552
veloren-server-cli: 95597304
D)
veloren-voxygen: 162860552
veloren-server-cli: 95586360

Total Cache Size:
A) 4.1G
B) 5G
C) 4.1G
D) 4.1G
2021-07-15 10:56:32 +02:00
Marcel Märtens
9379aa929a no longer incremental builds in CI 2021-07-14 21:55:12 +02:00
Marcel Märtens
6cb502c229 extract relevant code in a script in order to remove duplicate coding with docker-ci repo
export VELOREN_ASSETS without pwd to keep caching working
2021-07-14 20:57:08 +02:00
Marcel Märtens
fb8e5efe87 allow optinal releases on master if its no merge_request to allow it on forks, also adjust the workflow rules to run on release branches
ignore Gitlab user also for releases, as all protected branches are synced
2021-07-09 19:06:39 +02:00
Marcel Märtens
81dc34ec68 test on merge event rather than branch name master as a fork could also have a branch master 2021-07-09 11:19:22 +02:00
Marcel Märtens
7bac7a99c7 make sure that all CI jobs run always and not only in branch 2021-07-09 11:19:20 +02:00
juliancoffee
5157b84f2f Run asset_tweak tests separately
+ Make parse_all_ron_files_to_value panic-friendly. Remove unwraps
to unwrap_or_else with panic message.
2021-07-04 19:31:57 +03:00
Avi Weinstock
7a5388b8bb Use new dungeons in dungeon_voxel_export, remove old dungeon sampling function, and add more of world/examples to CI. 2021-06-27 15:35:43 -04:00
Ben Wallis
f8dc0e3a2c Added default-publish feature and updated build scripts. 2021-06-26 17:48:48 +01:00
Marcel
1104dfc983 Merge branch 'aweinstock/recipe-graphviz' into 'master'
Add graphviz diagram generator for crafting recipes.

See merge request veloren/veloren!2401
2021-06-09 09:14:13 +00:00
juliancoffee
1d5c35a48b add asset_tweaks to unittests on CI 2021-06-08 23:00:59 +03:00
Avi Weinstock
e9202024d7 Run clippy for graphviz target in CI and add instructions on how to consume the output. 2021-06-08 15:08:14 -04:00
juliancoffee
8c28e29056 place asset_tweak behind a feature 2021-06-08 13:34:41 +03:00
Marcel Märtens
c339c57fa3 --all-targets does not run targets that needs some features which are not provided. --all-features is NOT possible as we have some features which are conflicting.
The only possibility is to manually add the features we need to check in CI to the clippy query
2021-06-07 12:29:08 +02:00
Imbris
2ffeaab2fe Make docker-nightly job depend on the nightly air-linux job 2021-06-04 03:24:57 -04:00
Imbris
03a89bbfcf Merge branch 'imbris/nightly-ci-improvement' into 'master'
Avoid extra jobs on nightly builds

See merge request veloren/veloren!2367
2021-06-03 20:54:16 +00:00
Imbris
57637f3f0a Merge branch 'imbris/wgpu-master-rebased' into 'master'
wgpu

See merge request veloren/veloren!1947
2021-06-03 19:58:48 +00:00
Imbris
72bc86f5cb Unhack the wgpu branch CI 2021-06-03 14:24:31 -04:00
Marcel Märtens
afeedb412e change cidb settings 2021-06-03 17:26:09 +02:00
Imbris
d98420cd3b Avoid extra jobs on nightly builds 2021-06-03 04:10:22 -04:00
Imbris
0fbad9c961 CI tweaks, make windows build use posix mingw and copy necessary DLL files 2021-06-02 23:59:50 -04:00
Imbris
17a3bebbfe Disable CI that does not need to run for builds 2021-06-02 23:59:50 -04:00
Imbris
857eb98632 Add missing userdata strategy env var to macos builds 2021-06-02 16:50:42 -04:00
Imbris
92c6ea9017 Fix benchmarks by removing the rest of the incremental folder 2021-06-01 17:32:31 -04:00
Imbris
18a064c855 Merge branch 'imbris/fix-pages' into 'master'
Fix pages by specifying a docker image

See merge request veloren/veloren!2363
2021-06-01 21:07:57 +00:00
Imbris
988f2eae25 Fix pages by specifying a docker image 2021-06-01 16:26:41 -04:00
Imbris
82788716e0 Use native macos runner for builds 2021-06-01 16:08:47 -04:00
Imbris
72136103d7 delete all incremental things for clippy CI 2021-06-01 14:15:01 -04:00
Imbris
1cc565a32f Use new multi image cache scheme in the CI and consolidate runner tag specification so that it is easier to modify 2021-06-01 13:19:12 -04:00
juliancoffee
333754b2e6 Extract voxygen/src/i18n.rs into own crate
- New crate (i18n), currently under voxygen directory
- Updated CI to run localization test from i18n, not from voxygen
2021-05-09 13:12:26 +03:00
Marcel Märtens
e157c81671 have a seperate stage for scheduled builds 2021-04-23 17:48:02 +02:00
Marcel
8fb27f3d6d use different docker tags for master and nightly runs,
This allow us to decouple our test-ci from the release-ci and is necessary for multiple release channels in the future.
E.g. we can run a master build without it directly beeing pushed to watchtower and airshipper (config setting requiered on airshipper)
Adjust Tags for server-cli
2021-04-22 21:38:29 +02:00
Marcel
a4e836bf9b Merge branch 'xMAC94x/fix_benchmarks' into 'master'
fix upload of benchmarks

See merge request veloren/veloren!2149
2021-04-18 20:55:05 +00:00
Marcel Märtens
bbaf8280fa fix upload of benchmarks 2021-04-18 20:01:26 +02:00
Sam
2a860d1baf Moved coverage and benchmarks to only run post-merge. 2021-04-17 18:50:31 -04:00
Songtronix
0307cab21f
change(ci): only create artifacts triggered by schedule 2021-03-31 17:02:47 +02:00