Commit Graph

159 Commits

Author SHA1 Message Date
Marcel Märtens
636838f510 update crates 2021-09-20 14:39:01 +02:00
Marcel Märtens
215b20fa92 Change the version number to 0.11 2021-09-10 20:19:17 +02:00
Ben Wallis
85cd510ce4 Fixed server-cli command parsing failure on Windows 2021-09-05 16:46:20 +01:00
Joshua Barretto
2e72d97b5e Multiple color gradient options 2021-08-29 13:57:36 +01:00
Joshua Barretto
432bfa91d9 Added feature flag for terrain persistence 2021-08-12 10:48:00 +01:00
Imbris
9d761d5096 Make admin add/remove command arguments positional again 2021-07-30 01:25:38 -04:00
Dr. Dystopia
d5e2fcca0f Resolve unused '#[allow(clippy::unnecessary_wraps)]' error supressions 2021-07-24 20:14:26 +02:00
Marcel
f10f583a92 Merge branch 'xMAC94x/experiment_ci' into 'master'
fix/cleanup ci

See merge request veloren/veloren!2595
2021-07-16 00:33:35 +00:00
Marcel Märtens
ac10410782 only build once, dont build multiple targets. save time
Profiles:
(all are build, all are release, all are --no-default-features --features="default-publish" )

A) -p veloren-voxygen AND -p veloren-server-cli
B) NONE AND -p veloren-server-cli
C) NONE
D) NONE but server-cli/Cargo.toml has a default-publish feature

Timings
A)
> time cargo build -p veloren-voxygen --release --no-default-features --features default-publish
Compiling veloren-client v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/client)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling gfx-auxil v0.9.0 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling gfx-backend-gl v0.8.1 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling wgpu v0.8.0 (https://github.com/gfx-rs/wgpu.git?rev=a92b8549a8e2cb9dac781bafc5ed32828f3caf46#a92b8549)
   Compiling wgpu-profiler v0.4.0 (https://github.com/Imberflur/wgpu-profiler?tag=wgpu-0.8#b156eb14)
    Finished release [optimized] target(s) in 16m 19s
cargo build -p veloren-voxygen --release --no-default-features --features   3091.78s user 71.80s system 322% cpu 16:19.44 total
> time cargo build --release -p veloren-server-cli
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 6m 06s
cargo build --release -p veloren-server-cli  1024.29s user 18.42s system 284% cpu 6:06.62 total

B)
> time cargo build --release --no-default-features --features default-publish
   Compiling veloren-client v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/client)
   Compiling veloren-voxygen-egui v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui)
   Compiling gfx-auxil v0.9.0 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling gfx-backend-gl v0.8.1 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling wgpu v0.8.0 (https://github.com/gfx-rs/wgpu.git?rev=a92b8549a8e2cb9dac781bafc5ed32828f3caf46#a92b8549)
   Compiling wgpu-profiler v0.4.0 (https://github.com/Imberflur/wgpu-profiler?tag=wgpu-0.8#b156eb14)
   Compiling veloren-voxygen-anim-dyn v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim/dyn)
   Compiling veloren-voxygen-egui-dyn v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui/dyn)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 17m 10s
cargo build --release --no-default-features --features default-publish  4415.58s user 91.10s system 437% cpu 17:10.66 total
> time cargo build --release -p veloren-server-cli
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 6m 08s
cargo build --release -p veloren-server-cli  1123.93s user 23.01s system 311% cpu 6:08.14 total

C)
> time cargo build --release --no-default-features --features default-publish
   Compiling prometheus-hyper v0.1.2
   Compiling authc v1.0.0 (https://gitlab.com/veloren/auth.git?rev=fb3dcbc4962b367253f8f2f92760ef44d2679c9a#fb3dcbc4)
   Compiling veloren-common-net v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/net)
   Compiling veloren-plugin-api v0.1.0 (/mnt/nfs/marcel/Entw/Rust/veloren/plugin/api)
   Compiling veloren-voxygen-anim v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim)
   Compiling veloren-plugin-rt v0.1.0 (/mnt/nfs/marcel/Entw/Rust/veloren/plugin/rt)
   Compiling veloren-common-systems v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/systems)
   Compiling veloren-common-state v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/state)
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling veloren-client v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/client)
   Compiling veloren-voxygen-egui v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling gfx-auxil v0.9.0 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling gfx-backend-gl v0.8.1 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling wgpu v0.8.0 (https://github.com/gfx-rs/wgpu.git?rev=a92b8549a8e2cb9dac781bafc5ed32828f3caf46#a92b8549)
   Compiling wgpu-profiler v0.4.0 (https://github.com/Imberflur/wgpu-profiler?tag=wgpu-0.8#b156eb14)
   Compiling veloren-voxygen-anim-dyn v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim/dyn)
   Compiling veloren-voxygen-egui-dyn v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui/dyn)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 16m 41s
cargo build --release --no-default-features --features default-publish  4349.71s user 90.02s system 443% cpu 16:41.58 total

D)
> time cargo build --release --no-default-features --features default-publish
   Compiling prometheus-hyper v0.1.2
   Compiling authc v1.0.0 (https://gitlab.com/veloren/auth.git?rev=fb3dcbc4962b367253f8f2f92760ef44d2679c9a#fb3dcbc4)
   Compiling veloren-common-net v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/net)
   Compiling veloren-plugin-api v0.1.0 (/mnt/nfs/marcel/Entw/Rust/veloren/plugin/api)
   Compiling veloren-voxygen-anim v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim)
   Compiling veloren-plugin-rt v0.1.0 (/mnt/nfs/marcel/Entw/Rust/veloren/plugin/rt)
   Compiling veloren-common-state v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/state)
   Compiling veloren-common-systems v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/common/systems)
   Compiling veloren-world v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/world)
   Compiling veloren-client v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/client)
   Compiling veloren-voxygen-egui v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui)
   Compiling refinery v0.5.0 (https://gitlab.com/veloren/refinery.git?rev=8ecf4b4772d791e6c8c0a3f9b66a7530fad1af3e#8ecf4b47)
   Compiling veloren-server v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server)
   Compiling gfx-auxil v0.9.0 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling gfx-backend-gl v0.8.1 (https://github.com/gfx-rs/gfx?rev=27a1dae3796d33d23812f2bb8c7e3b5aea18b521#27a1dae3)
   Compiling wgpu v0.8.0 (https://github.com/gfx-rs/wgpu.git?rev=a92b8549a8e2cb9dac781bafc5ed32828f3caf46#a92b8549)
   Compiling wgpu-profiler v0.4.0 (https://github.com/Imberflur/wgpu-profiler?tag=wgpu-0.8#b156eb14)
   Compiling veloren-voxygen-egui-dyn v0.9.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/egui/dyn)
   Compiling veloren-voxygen-anim-dyn v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/voxygen/anim/dyn)
   Compiling veloren-server-cli v0.10.0 (/mnt/nfs/marcel/Entw/Rust/veloren/server-cli)
    Finished release [optimized] target(s) in 17m 30s
cargo build --release --no-default-features --features default-publish  4462.34s user 92.17s system 433% cpu 17:31.00 total

Hashes
A)
veloren-voxygen: e9de69144b4d459dc0da2775b2b87ac9
veloren-server-cli: 8c131c1b1daaa6efd0f83d95d9b5dad9
B)
veloren-voxygen: 415e14ec1c14a6a559d2908b843d88d5
veloren-server-cli: 8c131c1b1daaa6efd0f83d95d9b5dad9
C)
veloren-voxygen: 415e14ec1c14a6a559d2908b843d88d5
veloren-server-cli: 3bf56d0d0a76191a578ed5e71389f512
D)
veloren-voxygen: 415e14ec1c14a6a559d2908b843d88d5
veloren-server-cli: c9dc4fb0bfcbd831150eebac788e2f71

Binary Sizes:
A)
veloren-voxygen: 156667288
veloren-server-cli: 85136736
B)
veloren-voxygen: 162860552
veloren-server-cli: 85136736
C)
veloren-voxygen: 162860552
veloren-server-cli: 95597304
D)
veloren-voxygen: 162860552
veloren-server-cli: 95586360

Total Cache Size:
A) 4.1G
B) 5G
C) 4.1G
D) 4.1G
2021-07-15 10:56:32 +02:00
Ben Wallis
3568fbadcb Changed veloren-server-cli to run in interactive basic mode as default. Removed -b and added -t which enables the tui. 2021-07-14 20:03:16 +01:00
Marcel Märtens
7153e8879d fix clippy warnings 2021-07-12 12:09:09 +02:00
Marcel Märtens
e2e1f15cff update depednencies 2021-07-08 12:21:02 +02:00
Ben Wallis
ed9c619822 Fixed build warnings for server-cli related to non-default plugin feature 2021-06-26 21:56:03 +01:00
Imbris
156e2e86b5 Remove tracy features that are no longer needed now that macros have been fixed 2021-06-19 04:27:50 -04:00
Marcel Märtens
d291a71f79 Change the version number to 0.10 2021-06-12 10:14:07 +02:00
Imbris
df2ad92ad3 Require all commands to succeed in server-cli Dockerfile 2021-06-07 13:31:01 -04:00
Marcel Märtens
fa2b181628 use split_inclusive instead 2021-05-12 18:40:12 +02:00
Marcel Märtens
0f3717d047 enable TUI to actually handle newline character.
but we are not sure if NEWLINEs are wanted in logs as it could cause problems with automated parsing.
2021-05-12 18:40:08 +02:00
Joshua Yanovski
39369e090f More fixes to stdin input.
Adding tty makes sure docker attach won't be accidentally detached
by ctrl-C (there are better ways of doing this but this one works
for now)

shell-words more closely emulates Bash's tokenizer rules (but without
doing things like environment variable expansion) which allows us to use
multiline strings as reasons, etc.  Unfortunately entering newlines
still won't work the way we've written things since shell-words does not
right now give enough information to incrementally build up a valid
string, just says there was a tokenizing error; but maybe in the future
we can fix that.
2021-05-11 04:15:59 -07:00
Joshua Yanovski
99dd89ac60 Open stdin when using Veloren with docker.
This allows entering CLI commands like `help`, `admin add`, and
`shutdown` directly using `docker attach`, rather than needing to run
server-cli commands separately or perform a manual reboot or interrupt.
2021-05-10 11:44:04 -07:00
Joshua Yanovski
3f8d2f95be Added non-admin moderators and timed bans.
The security model has been updated to reflect this change (for example,
moderators cannot revert a ban by an administrator).  Ban history is
also now recorded in the ban file, and much more information about the
ban is stored (whitelists and administrators also have extra
information).

To support the new information without losing important information,
this commit also introduces a new migration path for editable settings
(both from legacy to the new format, and between versions).  Examples
of how to do this correctly, and migrate to new versions of a settings
file, are in the settings/ subdirectory.

As part of this effort, editable settings have been revamped to
guarantee atomic saves (due to the increased amount of information in
each file), some latent bugs in networking were fixed, and server-cli
has been updated to go through StructOpt for both calls through TUI
and argv, greatly simplifying parsing logic.
2021-05-09 21:19:16 -07:00
Marcel Märtens
af7091182e use a constant for recommended_threads 2021-05-05 14:35:06 +02:00
Marcel Märtens
87cf347377 limit tokio threads to 1/4 of num_cpus
make sure that rayon has at least 2 threads even on 1cpu systems to increase stability
2021-05-04 21:06:11 +02:00
Marcel
8fb27f3d6d use different docker tags for master and nightly runs,
This allow us to decouple our test-ci from the release-ci and is necessary for multiple release channels in the future.
E.g. we can run a master build without it directly beeing pushed to watchtower and airshipper (config setting requiered on airshipper)
Adjust Tags for server-cli
2021-04-22 21:38:29 +02:00
Yusuf Bera Ertan
84a7f666c2
build(nix): switch to nix-cargo-integration 2021-04-17 21:37:57 +03:00
Ben Wallis
404aca3e20 Fixed cargo test-server. Removed -Z package-features from .cargo/config as it is no longer required 2021-04-15 19:07:46 +01:00
Ben Wallis
1fbea66a66 * Replaced diesel with rusqlite and refinery
* Added "migration of migrations" to transfer the data from the __diesel_schema_migrations table to the refinery_schema_history table
* Removed all down migrations as refinery does not support down migrations
* Changed all diesel up migrations to refinery naming format
* Added --sql-log-mode parameter to veloren-server-cli to allow SQL tracing and profiling
* Added /disconnect_all_players admin command
* Added disconnectall CLI command
* Fixes for several potential persistence-related race conditions
2021-04-13 22:05:47 +00:00
Marcel Märtens
7afa77a06f move tracing code to own crate 2021-03-30 16:35:36 +02:00
Marcel Märtens
2198fece98 make interactive OPT-OUT isntead so that it's not forgotten when its important. and its more userfriendly for the average user. scripts prob just add a -n or -b 2021-03-26 00:22:46 +01:00
Marcel Märtens
58e557cac7 give more threads a fixed name 2021-03-22 12:37:03 +01:00
Marcel Märtens
13d79625f2 add keepalive in server-cli so we know it still ticks, ~ all 30s 2021-03-21 19:15:40 +01:00
Marcel Märtens
fcb00df6b8 Change the version number to 0.9 2021-03-20 13:07:09 +01:00
Marcel Märtens
0f405626e8 create a wrapper around rayon, slowjobpool, that spawns on rayon but drop feed it 2021-03-16 13:31:04 +01:00
Avi Weinstock
d6909f403e Copy assets/server to the right place in server-cli's Dockerfile. 2021-03-15 09:46:16 -04:00
Imbris
aff3536ac9 Fix tracy not working in certain areas, add span to chunk gen, missing fmt, remove extra span, make voxygen use INFO level instead of TRACE in tracy mode 2021-03-13 13:30:20 -05:00
Imbris
26df88bcdb Add tracy entity count plot, span tweaks 2021-03-13 02:59:58 -05:00
Imbris
87a7d7afd1 Move server-cli commands to separate file, tracy profiling par_join improvements, misc improvements
- remove overwritten logging setting in server-cli
- add server-cli command to load a random area for testing without a client
- make admin add/remove commands modify ingame players instead of needing to reconnect
- add spans to par_join jobs
- added test command that loads up an area of the world
- add tracy-world-server alias
- set debug directives to info for logging
2021-03-13 02:28:59 -05:00
Marcel Märtens
a62efc02d4 the docker container was missing the ca-certificates root store
Its now using ubuntus default root-store for tls operations.
2021-03-12 00:08:34 +01:00
Marcel Märtens
aa2a5a2549 fix tracy 2021-03-11 17:05:17 +01:00
Marcel Märtens
eacb60cfa0 Auth requests are now done async, the register system is fetching it, via a PendingLogin component
The auth server no longer allows the protocol to be specified. we enforce `https` for the auth server, so DO NOT provide a auth url with `https://` but without.
correct is now `auth.veloren.net`
incorrect is: `https://auth.veloren.net`
2021-03-11 17:05:02 +01:00
Marcel Märtens
51cf530877 update dependencies, including removal of some tracy deps as they are get through common/tracy 2021-03-09 20:17:29 +01:00
Marcel Märtens
c0129b0505 Move Specs code to own common_ecs create, put tracy and macros into common_base 2021-03-09 00:54:01 +01:00
Marcel Märtens
def5db54de Update Network Protocol
- now last digit version is compatible 0.6.0 will connect to 0.6.1
 - the TCP DATA Frames no longer contain START field, as it's not needed
 - the TCP OPENSTREAM Frames will now contain the BANDWIDTH field
 - MID is not Protocol internal

Update network
 - update API with Bandwidth

Update veloren
 - introduce better runtime and `async` things that are IO bound.
 - Remove `uvth` and instead use `tokio::runtime::Runtime::spawn_blocking`
 - remove futures_execute from client and server use tokio::runtime::Runtime instead
 - give threads a Name
2021-02-22 17:34:55 +01:00
Marcel Märtens
048fcf88f8 Stabelize Network again:
- completly switch to Bytes, even in api. speed up TCP by fak 2
 - improve benchmarks
 - speed up mpsc metrics
 - gracefully handle shutdown by interpreting Ok(0) as tokio::tcpstream closed now.
 - fix hotloop in participants by adding `Some(n)` to fix endless handing.
 - fix closing bug by closing streams after `recv_mgr` is shutdown even if now shutdown is triggered locally.
 - fix prometheus
 - no longer throw when a `Stream` is dropped while participant still receives a msg for it.
 - fix the bandwith handling, TCP network send speed is up to 1.5GiB/s while recv is 150MiB/s
 - add documentation
 - tmp require rt-multi-threaded in client for tokio, to not fail cargo check

this is prob stable, i tested over 1 hour.
after that some optimisations in priomgr.
and impl. propper bandwith.
Speed is up to 2GB/s write and 150MB/s recv on a single core

sync add documentation
2021-02-17 19:37:48 +01:00
Marcel Märtens
00386b6955 Great improvements to the codebase:
- better logging in network
 - we now notify the send of what happened in recv in participant.
 - works with veloren master servers
 - works in singleplayer, using a actual mid.
 - add `mpsc` in whole stack incl tests
 - speed up internal read/write with `Bytes` crate
 - use `prometheus-hyper` for metrics
 - use a metrics cache
2021-02-17 16:15:00 +01:00
Marcel Märtens
13276a6d00 get rid of async_std::channel
switch to `tokio` and `async_channel` crate.
I wanted to do tokio first, but it doesnt feature Sender::close(), thus i included async_channel
Got rid of `futures` and only need `futures_core` and `futures_util`.

Tokio does not support `Stream` and `StreamExt` so for now i need to use `tokio-stream`, i think this will go in `std` in the future

Created `b2b_close_stream_opened_sender_r` as the shutdown procedure does not need a copy of a Sender, it just need to stop it.

Various adjustments, e.g. for `select!` which now requieres a `&mut` for oneshots.

Future things to do:
 - Use some better signalling than oneshot<()> in some cases.
 - Use a Watch for the Prio propergation (impl. it ofc)
 - Use Bounded Channels in order to improve performance
 - adjust tests coding

bring tests to work
2021-02-17 12:38:53 +01:00
Marcel Märtens
88620c804a Initial switch to tokio for network, minimum working example. 2021-02-17 12:37:59 +01:00
Caelan
86d1b00735 Clean and update dependencies
* Remove tweak feature
 * Remove const-tweaker
 * Update tiny_http
 * Update bitvec to 0.21.0
 * Downgrade euc to avoid conflict with vek 0.12.0
 * Require exactly vek 0.12.0
 * Update all other dependencies automatically based on these changes
 * Update gilrs to latest at the request of Ada Lovegirls
 * Update meshing benchmarks to new criterion API
2021-02-17 01:27:06 -08:00
Ben Wallis
3ab3473230 Follow-up fixes from review of inventory MR 2021-01-10 11:27:09 +00:00
Ben Wallis
8661cb1ffc Refactored loadout to have public functions for each slot instead of requiring callers to use the _INDEX consts 2021-01-08 19:12:09 +00:00