Commit Graph

182 Commits

Author SHA1 Message Date
Marcel Märtens
95b186e29a QuicSink and QuicDrain do work now.
When local SendProtocol is opening a Stream, it will send a empty message to QuicDrain which will then know that its time to open a quic stream.
It will open a QuicStream and send its SID over to remote.
The RecvStream will be send to local QuicSink
RemoteRecv will notice a new BiStream was opened and read its Sid. It will now start listening on it. while remote main will get the information that a stream was opened and will notice the frontend.
in participant remote Recv is synced with remote send (without triggering a empty message!).
RemoteRecv Sink will send the sendstream to RemoteSend Drain and it will be used when a first message is send on this stream.
2021-04-29 15:58:39 +02:00
Marcel Märtens
760c382ed9 protocoladdr change for listen and connect
(remove a loop in quic protocol which wasnt a actual loop)
2021-04-29 15:58:34 +02:00
Marcel Märtens
9f0aceba4c work on getting quic in the network 2021-04-29 15:58:26 +02:00
Marcel Märtens
383482a36e Quic: We had the followuing problem:
- locally we open a stream, our local Drain is sending OpenStream
 - remote Sink will know this and notify remote Drain
 - remote side sends a message
 - local sink does not know about the Stream. as there is (and CANT) be a wat to notify local Sink from local Drain (it could introduce race conditions).

One of the possible solutions was, that the remote drain will copy the OpenStream Msg ON the Quic::stream before first data is send. This would work but is complicated.

Instead we now just mark such streams as "potentially open" and we listen for the first DataHeader to get it's SID.

add support for unreliable messages in quic protocol, benchmarks
2021-04-29 15:58:23 +02:00
Ben Wallis
2e08c2f76f Added client/server version mismatch message when a network error is encountered during client init.
Added warning banner on character select when successfully connected to a server with a mismatched version.
2021-04-24 09:08:30 +01:00
Marcel Märtens
8ee83df600 fixed #1067 2021-04-13 20:39:56 +02:00
Marcel Märtens
9638581d09 ITFrame::read_frame now throws an error when the frame_no is invalid.
This will be catched by the respective protocols, e.g. tcp and cause a violation
2021-04-13 00:13:08 +02:00
Marcel Märtens
4fed9dab83 Have a clear error for when the I/O closes and when some protocol is violated. this should help find the rootcause of a bug.
If its Closed it looks like the TCP connection got dropped/cut off (e.g. OS, Wifi).
If its Violated we for sure know the cause is the messages send/recv in a wrong way
2021-04-13 00:12:20 +02:00
Marcel Märtens
ea5a02d7cd change some Ordering::Relaxed to Ordering::SeqCst when we do not want to have it moved/or taken effects from other threads.
some id increases are kept Relaxed, SeqCst shouldn't be necessary there.
Not sure about the bool checks in api.rs
2021-04-07 23:17:09 +02:00
Marcel Märtens
7ca2f3b9d6 make a panic a error! and improve logging 2021-04-03 19:58:36 +02:00
Marcel Märtens
c77446a014 fix some tracy only and no default features 2021-03-27 18:24:10 +01:00
Marcel Märtens
6a95fb6b74 extend network of incomming TCP metrics and failed handshake metric 2021-03-27 18:24:10 +01:00
Marcel Märtens
aea52d8b54 implement Upload Bandwidth prediction.
Its available to `api` and `metrics` and can be used to slow down msg send in veloren.
It uses a tokio::watch for now, as i plan to have a watch job in the scheduler that recalculates prio on change.
Also cleaning up participant metrics after a disconnect
2021-03-26 08:58:03 +01:00
Marcel Märtens
034d0f0c5d fix a bug that some closes could get lost 2021-03-26 08:57:56 +01:00
Marcel Märtens
8dccc21125 preparation for multiple-channel participants.
When a stream is opened we are searching for the best (currently) available channel.
The stream will then be keept on that channel.
Adjusted the rest of the algorithms that they now respect this rule.
improved a HashMap for Pids to be based on a Vec. Also using this for Sid -> Cid relation which is more performance critical
WARN: our current send()? error handling allows it for some close_stream messages to get lost.
2021-03-26 08:57:50 +01:00
Marcel Märtens
01c82b70ab network scheduler and rawmsg cleanup 2021-03-26 08:57:42 +01:00
Marcel Märtens
6b23101fac update toolchain to nightly-2021-03-22 2021-03-22 16:41:04 +01:00
Marcel Märtens
9084ac48f1 defer some trace, so that we wont spam the log. 2021-03-22 09:16:07 +01:00
Marcel Märtens
18aefa8dbf remove prometheus dos via protocol.
Clean up unused labely to keep prometheus values down
2021-03-20 23:50:38 +01:00
Marcel Märtens
d7df741671 update dependencies, including removal of some tracy deps as they are get through common/tracy 2021-03-09 20:17:29 +01:00
Marcel Märtens
9028578bc8 Change the way Network is dropped.
Instead of keeping Runtime and manually spawn a task on `drop` this task is spawned at start and will wait to be triggered.
The `drop` methods then wait for completion, UNLESS they are in a async context, then they MUST NOT BLOCK (deadlock potential), so they defer it to the Runtime and HOPE for the runtime to exist long enough.
This get rid of the weird `block_in_place` which is only accessable with `rt-multi-threaded` and has some disadvantages.
We also wont requiere the runtime to be active all the time. Though its needed for a clean shutdown
2021-03-03 11:28:40 +01:00
Marcel Märtens
44817870ee Shutdown improvements
- Timeout for Participant::drop, it will stop eventually
 - Detect tokio runtime in Participant::drop and no longer use std::sleep in that case (it could hang the thread that is actually doing the shutdown work and deadlock
 - Parallel Shutdown in Scheduler: Instead of a slow shutdown locking up everything we can now shutdown participants in parallel, this should reduces `WARN` part took long for shutdown dramatically
2021-02-26 10:50:30 +01:00
Marcel Märtens
3f5c64bec0 Client::new can now resolve DNS requests, better networking error messages 2021-02-22 17:35:19 +01:00
Marcel Märtens
1a7c179bbb share tokio Runtime between Client and Server, name rayon Threadpool 2021-02-22 17:35:06 +01:00
Marcel Märtens
514d5db038 Update Network Protocol
- now last digit version is compatible 0.6.0 will connect to 0.6.1
 - the TCP DATA Frames no longer contain START field, as it's not needed
 - the TCP OPENSTREAM Frames will now contain the BANDWIDTH field
 - MID is not Protocol internal

Update network
 - update API with Bandwidth

Update veloren
 - introduce better runtime and `async` things that are IO bound.
 - Remove `uvth` and instead use `tokio::runtime::Runtime::spawn_blocking`
 - remove futures_execute from client and server use tokio::runtime::Runtime instead
 - give threads a Name
2021-02-22 17:34:55 +01:00
Marcel Märtens
5a48bffcb0 fix main thread blocking which was a bad combination of
- a channel was stale and wasn't shut down propertly AS WELL AS
 - the msg ingoing pipe was bounded, so it could fill up

To mitigate this we
 a) unbounded the pipe
 b) stoped spam the log in no channel case
 c) instead of ever reaching "no channel" state we actually shutdown participant
 d) when send_mgr is closed it will no longer be able to SEND on streams
2021-02-18 20:00:07 +01:00
Marcel Märtens
03af9937cf Stabelize Network again:
- completly switch to Bytes, even in api. speed up TCP by fak 2
 - improve benchmarks
 - speed up mpsc metrics
 - gracefully handle shutdown by interpreting Ok(0) as tokio::tcpstream closed now.
 - fix hotloop in participants by adding `Some(n)` to fix endless handing.
 - fix closing bug by closing streams after `recv_mgr` is shutdown even if now shutdown is triggered locally.
 - fix prometheus
 - no longer throw when a `Stream` is dropped while participant still receives a msg for it.
 - fix the bandwith handling, TCP network send speed is up to 1.5GiB/s while recv is 150MiB/s
 - add documentation
 - tmp require rt-multi-threaded in client for tokio, to not fail cargo check

this is prob stable, i tested over 1 hour.
after that some optimisations in priomgr.
and impl. propper bandwith.
Speed is up to 2GB/s write and 150MB/s recv on a single core

sync add documentation
2021-02-17 19:37:48 +01:00
Marcel Märtens
ea8ab1ce7a Great improvements to the codebase:
- better logging in network
 - we now notify the send of what happened in recv in participant.
 - works with veloren master servers
 - works in singleplayer, using a actual mid.
 - add `mpsc` in whole stack incl tests
 - speed up internal read/write with `Bytes` crate
 - use `prometheus-hyper` for metrics
 - use a metrics cache
2021-02-17 16:15:00 +01:00
Marcel Märtens
9884019963 COMPLETE REDESIGN of network crate
- Implementing a async non-io protocol crate
    a) no tokio / no channels
    b) I/O is based on abstraction Sink/Drain
    c) different Protocols can have a different Drain Type
       This allow MPSC to send its content without splitting up messages at all!
       It allows UDP to have internal extra frames to care for security
       It allows better abstraction for tests
       Allows benchmarks on the mpsc variant
       Custom Handshakes to allow sth like Quic protocol easily
 - reduce the participant managers to 4: channel creations, send, recv and shutdown.
   keeping the `mut data` in one manager removes the need for all RwLocks.
   reducing complexity and parallel access problems
 - more strategic participant shutdown. first send. then wait for remote side to notice recv stop, then remote side will stop send, then local side can stop recv.
 - metrics are internally abstracted to fit protocol and network layer
 - in this commit network/protocol tests work and network tests work someway, veloren compiles but does not work
 - handshake compatible to async_std
2021-02-17 12:39:47 +01:00
Marcel Märtens
3f85506761 fix most unittests (not all) by a) dropping network/participant BEFORE runtime and by transfering a expect into a warn! in the protocol 2021-02-17 12:38:58 +01:00
Marcel Märtens
5aa1940ef8 get rid of async_std::channel
switch to `tokio` and `async_channel` crate.
I wanted to do tokio first, but it doesnt feature Sender::close(), thus i included async_channel
Got rid of `futures` and only need `futures_core` and `futures_util`.

Tokio does not support `Stream` and `StreamExt` so for now i need to use `tokio-stream`, i think this will go in `std` in the future

Created `b2b_close_stream_opened_sender_r` as the shutdown procedure does not need a copy of a Sender, it just need to stop it.

Various adjustments, e.g. for `select!` which now requieres a `&mut` for oneshots.

Future things to do:
 - Use some better signalling than oneshot<()> in some cases.
 - Use a Watch for the Prio propergation (impl. it ofc)
 - Use Bounded Channels in order to improve performance
 - adjust tests coding

bring tests to work
2021-02-17 12:38:53 +01:00
Marcel Märtens
1b77b6dc41 Initial switch to tokio for network, minimum working example. 2021-02-17 12:37:59 +01:00
Caelan
dda4931f46 Clean and update dependencies
* Remove tweak feature
 * Remove const-tweaker
 * Update tiny_http
 * Update bitvec to 0.21.0
 * Downgrade euc to avoid conflict with vek 0.12.0
 * Require exactly vek 0.12.0
 * Update all other dependencies automatically based on these changes
 * Update gilrs to latest at the request of Ada Lovegirls
 * Update meshing benchmarks to new criterion API
2021-02-17 01:27:06 -08:00
jiminycrick
8b97199245 Update rand dependency 2021-01-26 20:35:08 -08:00
Acrimon
9f16a946ee
update a few deps 2021-01-20 15:53:58 +01:00
Marcel Märtens
6390e758d4 fix clippy in all examples.
added a ignore for plugins, as we cannot remove the `Result<>` type, it is necessary
2021-01-13 15:06:04 +01:00
Marcel Märtens
26918d10c9 update chrossbeam, tracy, prometheus (and reduce server deps to crossbeam-channel) 2020-12-16 00:51:07 +01:00
Joshua Barretto
fc96fd780b Moved plugin API to Uid instead of usize for entity IDs, hello plugin to an example 2020-12-13 23:08:15 +00:00
Joshua Barretto
f8c8e342e6 Moved common networking code to common/net, clippy fixes 2020-12-13 17:23:45 +00:00
Marcel Märtens
7a7c1f6f50 I would except this to be implcitly done by the drop though it doesn't hurt here, as this channel is dropped anyway a line later.
But i have the feeling that maybe something with the channel is wrong which leads to this behavior (or maybe did i made a copy somewhere, though i dobt this).
Again, not sure if this is a fix, but i think it doesn't hurt
2020-11-27 10:47:01 +01:00
Marcel Märtens
1b18b43fa7 Drop is run only AFTER each statement.
And the guard generated here in the if let Some() will live through the WHOLE statement, which is definitly critical.
The fix is quite easy. We just move it out in a seperate line.
There are still some participants where the dropping will fail, and followup disconnects will NOT get triggered and be queued up till ethernity, but at least we are not blocking new logins
2020-11-25 22:10:10 +01:00
Ben Wallis
639281bc32 Replaced usage of 0.0.0.0 with 127.0.0.1 in network tests to prevent firewall prompt on Windows when running tests 2020-11-15 22:10:31 +00:00
Marcel Märtens
40f5afc2b0 ci cleanup, dependency update 2020-11-06 14:34:42 +01:00
Marcel Märtens
153c6c3b13 Fixing Tarpaulin isn't easy.
So first off all we had to update the toolchain, i think everything in september is okay, but we got with this current version.
Then we had to update several dependencies, which broke:
 - need a specific fix of winit, i think we want to get rid of this with iced, hopefully, because its buggy as hell. update wayland client to 0.27
 - use a updated version of glutin which has wayland-client 0.27 and no longer the broke version 0.23
 - update conrod to use modern copypasta 0.7
 - use `packed_simd_2` instead of `packed_simd` as the owner of the create abandoned the project.
 - adjust all the coding to work with the newer glutin and winit version. that also includes fixing a macro in one of the dependencies that did some crazy conversion from 1 event type to another event type.
   It was called `convert_event`
 - make a `simd` feature which is default and introduce conditional compiling.
   AS I HAVE NO IDEA OF SIMD AND THE CODE. AND I DIDN'T INTRODUCE THE ERROR IN THE FIRST PLACE, WE PANIC FOR NON SIMD CASE FOR NOW. BUT IT WORKS FOR TESTS.
 - tarpaulin doesnt support no-default-features. so we have to `sed` them away. semms fair.
2020-10-26 17:04:20 +01:00
Marcel Märtens
b5f48014a9 Streams no longer panic when recv on a StreamClosed Stream. Panicing is a "feature" of futures::channel
Refactor the `send_raw` and `recv_raw` completly. We now expost `Message` which has a public `serialize` and `deseialize` fn for the first time.
This makes using the `raw` methods of a stream much easier and is a requierement for using "copy_less" sending to multiple streams
2020-10-19 10:23:30 +02:00
Marcel Märtens
572b83e262 add a try_recv fn to Stream which is NOT async 2020-10-19 10:23:27 +02:00
Marcel Märtens
ba1299e670 apparently span doesnt work for async, so i replaced it by an instrument version 2020-10-14 17:54:01 +02:00
Marcel Märtens
e914c29728 FIX for hanging participant deletion.
There is a rare bug that recently got triggered more often with the release of xMAC94x/netfixA
if the bug triggeres, a Participant never gets cleaned up gracefully.

Reason:
When `participant_shutdown_mgr` was called it stopped all managers at once.
Especially stream_close_mgr and send_mgr.
The problem with stream_close_mgr is, it's responsible for gracefully flushing streams when the Participant is dropped locally.
So when it was interupted self.streams where no longer flushed gracefully.
The next problem was with send_mgr.
It is triggering the PrioManager, and the PrioManager is responsible for notifying once a stream is completly flushed.
This lead to the problem, that a stream flush could be requested, but was actually never executed (as send_mgr was already down).

Solution:
1. when stream_close_mgr is stopped it MUST flush all remaining streams
2. wait for stream_close_mgr to finish before shutting down the send_mgr
3. no longer delete streams when closing the API (this also wasn't tracked in metrics so far)

Additionally i added a dependency, so that the network/examples compile again, fixed some spelling.
I created a `delete_stream` fn that basically just moved the code over.
2020-10-14 15:03:49 +02:00
Marcel Märtens
24af657fd5 quickfix for closing participants more reliable 2020-10-13 20:06:20 +02:00
Marcel Märtens
9af2fcfe46 add more tracing and drop lock earlier 2020-10-13 19:01:53 +02:00
Ben Wallis
b3dd8e8a02 Added #![deny(clippy::clone_on_ref_ptr)] to all crates and fixed resulting lint errors 2020-09-27 17:25:33 +01:00
Marcel Märtens
a7b7ae3a2c fix compiling with metrics 2020-08-27 09:35:06 +02:00
Imbris
dcce5641f7 Fix broken features and avoid panic if the client leaves before character data loads 2020-08-26 20:47:39 -04:00
Marcel Märtens
9170622611 reduce load on metrics by ALOT!
- first remove participant AND channel in same metric. this caused a matrix full of 0 values which bloated alot.
 - then did the cid cache to be lazy loading to no longer generate that much 0 values
 - possible would also be no longer keeping metrics for INIT, HANDSHAKE and PARTICIPANTID
2020-08-27 01:55:13 +02:00
Marcel Märtens
8c883c200d Switch veloren_network over to use the official example layout.
adjusted those examples to run again
created a CI TEST to always `check` examples
fixed fmt in examples so that pipeline gets green
2020-08-26 10:07:22 +02:00
Marcel Märtens
144f88f811 Propper Compression support of network.
- Compression is no longer enabled always but can now be enabled per Stream.
   If a Stream is Compression enabled it will compress and decompress all msg (except for `raw` access) before handling them internally.
   You need to handle compression yourself for `raw` fn.
 - added a new feature to the network crate to enable or disable the compression
 - switched to `lz-fear` instead of `lz4-compression`
 - use `bitflags` to represent the `Promises` struct
2020-08-25 23:55:27 +02:00
notoria
2be4202d01 Corrected some spelling errors 2020-08-25 12:21:25 +00:00
Marcel Märtens
5fe7c05d9c Redefine Close behavior:
- When Participant A was closed by remote side. Then a `disconnect` on `A`
   shall return Ok() (instead of ParticipantDisconected) IF:
   A was already flushed and no data needs to be sended any more.
 so a `disconnect` doesnt differ if the other side initiated the disconnect before or not. it tries to clean things up and returns Ok(()) if both sides agree
2020-08-24 16:22:12 +02:00
Marcel Märtens
91d296cda1 Fixed bug in tcp protocol.rs
- It was possible for a end_receiver to be triggered in the moment while a frame was started by not finished.
   This removed bytes from the stream with them getting lost. this almost ever was followed by a RAW frame as the TCP stream was now invalid.
   The TCP stream was then detected by participant or caused one or multiple failures
 - introduces some simplifications, removed a macro, reuse code
2020-08-24 16:22:06 +02:00
Marcel Märtens
d37ca02913 using Locks a more sensitive way.
- replace RwLock by Mutex if it's only accessed for insert/delete
 - use RwLock<HashMap<Mutex>> pattern otherwise in order to allow concurrent `.read()`
 - fixed a deadlock O.o
2020-08-23 21:43:17 +02:00
Marcel Märtens
b9cb79918d hotfix a lock 2020-08-22 12:03:06 +02:00
Marcel Märtens
1eb126736d workaround for impossible RAW msg 2020-08-22 01:09:07 +02:00
Marcel Märtens
34a4c72a73 Fix scheduler not really shutting down when they where listening on a Port. Add a seperate test for this.
- 1000ms sleep isn't enough in tracing anyway, so remove it
2020-08-21 18:00:34 +02:00
Marcel Märtens
926d334082 Fixed the unclean disconnecting of participants.
Till now, we just dropped the TCP connection and registered this as a clean shutdown.
The prodocol reader intereted this and send a Frame::Shutdown frame to it's local processor.
This is ofc wrong.
So now the protocol reader will detect a Frame::Shutdown frame and send it over. if the Tcp connection gets closed it will return an Error up.
The processor will then pick up this error and request a unclear shutdown and notifies the user.
Also when doing a clean shutdown we are sending a Frame::Shutdown now to the remote side to trigger this behavior.

Before we wrongly added the feature of only using a `select` in channel. This is WRONG,
 as it could mean that the write maybe fails, but the read still had some Frames buffered which then get dropped.
Its fixed now by the clean shutdown mechanims defined before.

Also when a channel is closed now inside a participant we are closing the whole participant as a protection.
However, we must not close the recv channel as the `handle_frames_mgr` might still be working on them, so we only stop writing/sending.

Debugging this also let me introduce some smaller fixes:
 - PID in tests are now 0 and 1+1*64+1*64*64+... this makes the traces appear as AAAAAA and BBBBBB instead of ABAAAA and ACAAAA
 - veloren client now better seperates between clean shutdown and unclear shutdown.
 - added a new type: C2pFrame for `(cid, Result<Frame, ()>)`
 - wrong frames inside the handshare are not counted in metrics
 -
2020-08-21 18:00:28 +02:00
Marcel Märtens
42141b3aa3 remove some trace! in network which a) was only spam and b) could be replaced by a metric way better.
added a span for disconnecting on the gameserver side. also added more debug! tracing there
Just keeping a trace! all 10000ms active to have a keep alive feeling.
2020-08-21 18:00:14 +02:00
Marcel Märtens
12b46250f5 protocols no longer send a Close Frame in case the read fails. They just fail, let participant handle this!
Participant will now handle a close in the `create_channel_mgr` rather then the `send` fn. Its the better place, which makes a HashMap better for delete lookup
Since tcp_read now broke but tcp_write didn't and the Participant wasnt updated till both were broke, we changed CHANNEL tcp_read and tcp_write in protocols to be a `select` rather than a `join`
However only do this in the CHANNEL, but in the HANDSHAKE. it fails if you try to. Also the handshake will take care of any failed read or write manually and will handle a clear teardown in this case.
2020-08-21 18:00:07 +02:00
Marcel Märtens
b59fc2ff0c improve tracing and spans in network crate 2020-08-21 18:00:00 +02:00
Joshua Yanovski
2dfd3b13d1 (See sharp/lod-history) LOD, shadows, greedy meshing, new lighting, perf
---

Pretty much a Veloren fork at this point.  Here's a high level overview of the changes (will be added to CHANGELOG just before merge).

At a high level this MR incorporates roughly two groups of changes.

The first group consists of new game features: more flexible map sizes, level of detail terrain, shadow maps, and a new lighting
engine.  This is "feature work" that (mostly)  only adds new things to Veloren, and mostly shouldn't affect old stuff.

The second big group of changes are those addressing the fallout from all the new features.  These include performance fixes of
various sorts: the addition of multiple graphics options and optimization of the cheap ones to avoid work, switching all voxel
models to use some variant of greedy meshing, switching over much of our CPU-side vector math to exploit SIMD instructions
(coinciding with a fork of `vek`), and a rewrite of how the UI handles text rendering (coinciding with updates to our fork
of `conrod`).  Making Veloren's hardcoded colors appear correct under the new lighting engine also required considerably
changes (TODO: Fill in this section when it's complete).

The second category of changes often heavily touches code owned by other people, including frequently modified code "owned" by a
handful of people, so I recommend that this code be reviewed particularly carefully.

---

At a high level (each will be described in more detail below):
- The world map has been refactored.
  - The world size is no longer hardcoded (@zesterer).
  - The map generation code was made generic to allow using it outside of the `world` crate (@zesterer).
  - On world creation, we now compute *horizon maps* (@zesterer).
  - The way we pass the world from the server to the client has been updated (@xMAC94x).
  - Artifacts related to image rotation were fixed (@imbris).
  - Multiflow rivers were enabled (@zesterer).
  - In the process of making changes related to the world map, various incidental fixes and optimizations were required.
- The new *level of detail* feature was added (@zesterer wrote part of this and has checked out the rest).
  - A new LOD terrain rendering step was added to the pipeline.
  - The LOD terrain quality was made configurable via a graphics setting.
  - Horizon maps were used to cast shadows from LOD chunks on both LOD and non-LOD terrain.
  - A "voxelization" effect was incorporated into rendered LOD terrain to make it blend better into the world.
  - In the process of making changes related to LOD, various incidental fixes and optimizations were required.
- Veloren's lighting has been completely overhauled (@zesterer has already checked most of this out).
  - A semi-accurate index of refraction was assigned to our materials.
  - A new, more realistic, physically based approach to lighting was used using the *Ashikhmin Shirley* BRDF.
  - We emulate *atmospheric scattering* using equations designed for measuring solar panel light exposure.
  - We attempt to compute *realistic light attenuation* in water using its real material properties.
  - In the process of making changes related to LOD, various incidental fixes and optimizations were required.
- Point and directional lights now cast realistic shadows, using *shadow mapping.* (@imbris, @zesterer, @Treeco, @YuriMomo)
  - Point light shadow maps were added to the rendering pipeline, using geometry shaders and *seamless cube maps*.
  - Directional light shadows were added to the rendering pipeline, using LISPSM together with disabling *depth clamping*.
  - "Shadow-only" chunks and NPCs were added to prevent shadows from models behind you from disappearing.
  - In the process of making changes related to shadow maps, various incidental fixes and optimizations were required.

The addition of shadow maps, LOD terrain, and the new lighting all led to significant performance degradation, on top of other
changes happening in master.  Therefore, a large number of performance improvements were also needed:

- The graphics options were made much more flexible and configurable, and shaders were optimize.
  - New options were provided for how to render lights and shadows  (@Pfauenauge, @zesterer).
  - Graphic setting storage and configuration were overhauled to make adding new features easier (@Pfauenauge, @imbris).
  - Shaders were rewritten to utilize GLSL's preprocessor to avoid overhead (@zesterer, @YuriMomo).
  - In the process of making changes related to providing additional rendering options, various incidental fixes and optimizations were required.
- Voxel model creation was switched to use *greedy meshing.*
  - A new voxel meshing method, greedy meshing, was added (@imbris).
  - Uses of the older meshing methods were migrated to use greedy meshing (@imbris, @jshipsey, @Pfauenauge).
  - New restrictions were added to terrain, figure, and sprites to future proof them for further optimizations (@jshipsey, @Pfauenauge, @zesterer).
  - Most positions are now relative to either chunk or player position for better precision (@imbris, @zesterer, @scottc).
  - In the process of making changes related to greedy meshing, various incidental fixes and optimizations were required.
- Animation and terrain math were switched to use SIMD where possible.
  - Fixes were made to vek to make its SIMD feature usable for us (@zesterer, @imbris).
  - The interface and types used in bone animation were changed in various ways (@jshipsey, @Snowram, @Pfauenauge).
  - Redundant code generation for body animation is now partly taken care of by a macro (@jshipsey, @Snowram, @Pfauenauge).
  - Animation code was modified to to use vek's SIMD representation where possible (@jshipsey, @Snowram, @Pfauenauge).
  - Terrain meshing code and shadow map math were also modified to use vek's SIMD representation (@imbris).
  - SIMD instruction generation was enabled (@YuriMomo, @jshipsey, @Snowram, @imbris, @Angelonfira, @xMAC94x).
  - In the process of making changes related to greedy meshing, various incidental fixes and optimizations were required.
- The way we cache glyphs was completely refactored, fixed, and optimized.
  - Our fork of `conrod` was optimized in various ways (@imbris).
  - Our fork of `conrod` now exposes whether a widget was updated during the current frame (@imbris).
  - Our use of the glyph cache was rewritten for correctness (@imbris).
  - A *text cache* was introduced that lets us skip remeshing glyphs that have not changed (@imbris).
  - Various changes were made to reduce pressure on the glyph cache, with more planned (@imbris, @Pfauenauge).
  - In the process of making changes related to the glyph cache, various incidental fixes and optimizations were required.
- Colors were changed to keep Veloren's look consistent with master.
  - Some older tree models were brought back (@Pfauenauge).
  - TODO(@Sharp): All hardcoded colors were extracted and made hotloadable.
  - TODO(@Treeco, @Pfauenauge, @imbris, @jshipsey): Hardcoded colors were fixed to conform to Veloren's style.
  - TODO(@Treeco, @Pfauenauge, @imbris, @jshipsey): Color models were fixed to conform to Veloren's style.

A detailed description of the involved changes follows.

---

- The world size is no longer hardcoded.  All functions dependent on world size now take a `WorldSizeLg`, which holds the base 2 logarithm of each actual world dimension and is guaranteed to maintain certain properties (outlined in `common/src/terrain/map.rs`).  Additionally, many utility functions that utilize the world size were moved into `common` as well (mostly `common/src/terrain/mod.rs`).  Finally, the world map format was updated in order to store its size explicitly, with a migration path from the old format that should work whenever the old formatted map was a square (practically always).  See `world/src/sim/mod.rs` for these changes.
- The map generation code was made generic to allow using it outside of the `world` crate.  The parts of the map generating code that do not need to query the world were moved over to `common/src/terrain/map.rs`, allowing them to be used from the client without creating a dependency on `world`.  The rest of it was turned into helper functions in `world/src/sim/map.rs`, which can be passed as closures to the generic map generation code to complete its construction.  This also means that colors are now passed in separately to the map generation function.  See <https://veloren.net/devblog-78/> for more details.
- On world creation, we now compute *horizon maps*.  See the function in `world/src/sim/util.rs`.

  Given a height map and a plane intersecting that height map, our horizon maps allow us to encode enough information to reconstruct shadows for each point on the height map using only the *horizon angle* (the angle at which the sun starts to become visible).  As Veloren's sun only covers one plane, this is sufficient for encoding sun shadows for LOD terrain, by encoding two angles per chunk (one for each 90 degrees the sun covers).  We can also use this for the moon, if we want, since the moon follows the same path.  Additionally, we store the *height* of the furthest occluder, to try to make the shadows volumetric; so this means 4 bytes in total for each chunk.

  Support for horizon maps has been merged into the map functionality in common as well.
- The way we pass the world from the server to the client has been updated.  Rather than passing the prerendered map, we instead pass three maps with values for each chunk; one with the color information, a second with altitude information, and a third with horizon map information.  We then reconstruct the map on the client, together with some additional information we send from the server (like the sea level and maximum height).  See `common/src/msg/server.rs` for a detailed description of the format of `WorldMapMsg`, and `server/src/libr.rs` and `client/src/lib.rs` for details of the map construction and parsing.
- Artifacts related to image rotation were fixed.  See the commit message for commit SHA `cf74d55f2e3d2ae7d25fd68d5c73b01a6afde86e` for a detailed explanation.  This involved changes to shaders, the addition of a new type of graphic (also reflected in the graphic cache) that allows specifying a border color (which automatically makes the associated texture immutable), and some related fixes.  I reproduce the first two paragraphs of the MR description as well:

 ```
 Fix map image artifacts and remove unneeded allocations.

 Specifically, we address three concerns (the image stretching during
 rotation, artifacts around the image due to clamping to the nearest
 border color when the image is drawn to a larger space than the image
 itself takes up, and potential artifacts around a rotated image which
 accidentally ended up in an atlas and didn't have enough extra space to
 guarantee the rotation would work).
 ```
- Multiflow rivers were enabled.
  This does not really need to be part of this MR, and would be easy to revert, but since it seemed to provide a nice improvement it's currently packaged with it.
  We already computed multiple outflows from each chunk for erosion purposes long before this MR.
  However, we never modified river rendering to be able to handle this case (just a single downhill river flow is complex enough!) so this was not exposed when deciding which chunks were rivers.
  Now that
- In the process of making changes related to the world map, various incidental fixes and optimizations were required.  Some examples of fixes include making sure terrain is never lowered to below sea level (to make the shadow maps report correct values), fixing map altitudes and colors to understand things like cliffs and "block level" coloring (that doesn'te xist on the column level), and fixing a crashbug when rendering images for the UI where source pixels are strongly rectangular.  Some examples of related performance fixes include avoiding allocating a fresh vector for all the maps (i.e. copying it over to change the format from `[u32; n]` to `DynamicImage` and then copying again to convert to `RgbaImage`), and instead using the `gfx::memory::slice` function to accomplish the same thing.  These sorts of changes are spread all arond the code.

This includes the additon of a new scene, `voxygen/src/scene/lod.rs`, a new pipeline `voxygen/src/render/pipeline/lod_terrain.rs`, and new shaders `assets/vxygen/shaders/lod-terrain-vert.glsl` and `assets/vxygen/shaders/lod-terrain-frag.glsl`, as well as associated changes to the renderer in `voxygen/src/render/renderer.rs`.

The main idea behind our initial approach to LOD was to take the world data we now get from the server (altitude, color, and horizon mapping).

  - Some previously computed values were turned into shader uniforms for better prediction on weak processors. (@zesterer)
  - Calls to power or trig functions were removed or replaced with multiplications, where possible.
  - After some deliberation
  - To properly handle sprite "waving" for nearby sprites,

We explicitly designed the greedy meshing system with figures and sprites in mind.
In both cases, we want to be able to *efficiently* pack many different models into the same texture, especially in cases where we know
we will either not be removing any of the grouped-together from the models from the texture, or will remove all of them at once (so
they can be packed into some specific subtexture).
For sprites, since we know every model in advance and never intend to deallocate them, we currently pack them all as efficiently as
possible into one giant tetxure atlas.  However, in the future we might opt to pack them slightly less efficiently in exchange for
shrinking the sprite vertex size.
For figures, we pack all the textures for each *model* into the same atlas.
is a global texture atlas used for every sprite, and for figures which is why we have the ability to mesh multiple
models to the same texture area (using the simple texture atlas allocator) without requiring intermediate vector allocations.
This is accomplished by delaying the time when we actually write the color and light data to the texture until *after* all the
model vertices have been meshed; then, we can just allocate the whole color/light array at once, making the atlas we use an
exact fit.  In computer science-y terms, we accomplish this delay by, after we perform the initial greedy meshing (without
texture information), not continuing to create the texture data, but instead constructing a *continuation*--that is, a function
that, when called, will execute the rest of the computation.  We push this continuation (which in Rust terms is a `FnOnce` closure
that takes the `ColLightsInfo` that it is supposed to write to as context) onto a

onto a vector

resizing.  To allow for suspended writes to texture data, Rust pointed out to me that the continuation that would eventually write the color and light data to the texture atlas (the one that is shared by all models sharing the same greedy mesher) would have to *own* whatever data it mshed.  Because we often generate the model data to mesh as a temporary in `voxygen/src/load.rs`, the

  - Matrix multiplications in the shader were reduced for figure data (@zesterer).
  - Vertex "waves" for fluid data were removed.
  - Terrain "bending" near edges was removed.
  - Scaling was fixed to make sure empty space was not introduced in a space previously occupied by a block.  It was also changed to take ownership of its voxel data,
    rather than sharing it, to let it be used with meshing.
  - Rust's nightly version was bumped in order to use the `array_map` function, which lets us reuse more code between the simple map and `FigureModelCache`.

- PositionedGlyph::standalone.

---

I tried to cite sources in many cases[^realtime],[^lloyd],[^lispsm],[^pbrt],[^greedy],[^tjunctions]
where I needed features from elsewhere but I am particularly grateful for the following resources,
esepcially where they have accompanying source code.  I linked all of them that are accessible to the public (those that are not were
obtained through legal means).

[^realtime]: Eisemann, Elmar, Michael Schwarz, Ulf Assarsson, Michael Wimmer. Real-Time Shadows. A K Peters/CRC Press (T&F), 20160419.
[^lloyd]: Lloyd,B. 2007. [Logarithmic perspective shadow maps](http://gamma.cs.unc.edu/papers/documents/dissertations/lloyd07.pdf). PhD thesis, University of North Carolina.
[^lispsm]: Wimmer, M., Scherzer, D., and Purgathofer, W. 2004. [Light space perspective shadow maps](http://gamma.cs.unc.edu/papers/documents/dissertations/lloyd07.pdf). In Proceedings of Eurographics Symposium on Rendering 2004, pp. 143– 152.

[^pbrt]: Pharr, Matt, et al. [http://gamma.cs.unc.edu/papers/documents/dissertations/lloyd07.pdf](Physically Based Rendering: From Theory to Implementation). Third edition, Morgan Kaufmann Publishers/Elsevier, 2017.
[^greedy]: mikolalysenko. “Meshing in a Minecraft Game.” 0 FPS, 30 June 2012, <https://0fps.net/2012/06/30/meshing-in-a-minecraft-game/>.
[^tjunctions]: blackflux. “Meshing in Voxel Engines – Part 1.” Blackflux.Com, 23 Feb. 2014, <https://blackflux.wordpress.com/2014/02/23/meshing-in-voxel-engines-part-1/>.

I am also especially grateful to Khronos, Wikiepdia, and stackoverflow for answering many of my specific questions while writing the MR.

---

Squashed commit of the following:

commit 300505e730
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 18:46:25 2020 +0200

    Fixing cargo doc and typo in CHANGELOG.

commit ec0aeb18e8
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 15:38:50 2020 +0200

    Hopefully final commit for the LOD branch.

commit 5e8ea0b1ea
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 10:14:26 2020 +0200

    Falling back to power as stopgap.

commit e44a1cbf46
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 09:25:41 2020 +0200

    Address imbris feedback.

    Temporarily disables shiny water, lowers max VD.

    These restrictions will be lifted soon after merging.

commit 561e25778a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 08:31:13 2020 +0200

    Tweaking shaders a bit.

commit 7d19259078
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 07:59:43 2020 +0200

    Fix view example as well.

commit 051cd4934e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 07:29:06 2020 +0200

    Fix meshing benchmark.

commit c95e07db3b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 05:46:22 2020 +0200

    Address MR feedback, fix scene clouds.

commit 1bfb816cab
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 04:39:36 2020 +0200

    Incorporating Pfau's figure color changes.

    New eyes and new humanoid colors.

commit 3f9b89a3ac
Merge: e2f5162e4 62c53963a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 04:29:41 2020 +0200

    Merge remote-tracking branch 'origin/sharp/small-fixes' into sharp/small-fixes

commit e2f5162e4f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 20 04:28:38 2020 +0200

    World colors are all hotloadable.

    They live in assets/world/style/colors.ron.

    Only a small handful of hardcoed colors remain in World; they are either
    part of the map, or difficult to disentangle from the rest of the
    computation.  Comments are made where appropriate.

commit 62c53963ab
Author: Marcel Märtens <marcel.cochem@googlemail.com>
Date:   Wed Aug 19 15:59:00 2020 +0200

    replace pretty_env_logger with tracing

commit 5b1625f99d
Merge: d71003acd 4942b5b39
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Aug 19 05:15:56 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit d71003acda
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Aug 19 05:14:34 2020 +0200

    Hotloading colors, part 1: colors in common.

    Currently, this just entails humanoid colors.  There are only three
    colors not handled; the light emitter colors in
    common/src/comp/inventory/item/tool.rs.  These don't seem important
    enough to me to warrant making hotloadable, at least not right now, but
    if it's needed later we can always add them to the file.

commit 63b5e0e553
Merge: c32b337a4 6d2c4b9c1
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Aug 17 13:05:37 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit c32b337a46
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Aug 17 05:52:04 2020 +0200

    Fixing LOD grid, for real.

commit a166ae0360
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Aug 17 05:28:05 2020 +0200

    Addressing imbris's initial feedback.

    Fixes two minor bugs: explosion particles were no longer spawning
    randomly, and LOD grids were not perfectly even.

commit 4cbad004f4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Aug 16 19:27:58 2020 +0200

    Bumping nightly per request.

commit 548680276a
Merge: acc098604 8f8b20c91
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Aug 16 19:26:06 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit acc0986040
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Aug 15 22:28:32 2020 +0200

    Lower resolution due to lying drivers.

commit d3b878de2a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Aug 15 22:15:38 2020 +0200

    Fix issues msh encountered with Intel 4600.

commit 10245e0c1b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Aug 15 21:15:02 2020 +0200

    Merge more models into one mesh than we did previously.

commit 3155c31e66
Merge: 7204cc8a7 3c199280e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 13 22:35:22 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 7204cc8a7a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 13 22:34:43 2020 +0200

    Fix not yet done NPC animations.

    This forces them all to be the idle animation if not specified.

    This fixes issues where you'd have giant NPCs in water.

commit bc83360f2a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 13 19:36:37 2020 +0200

    Try to fix some bugs:

    - Z fighting with LOD terrain and water.
    - Audio SFX not playing.

commit 1fd104aa60
Merge: 862df3c99 7c2c392a3
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 13 12:02:31 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 862df3c997
Merge: 0a4218ed9 75c1d4401
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 13 05:52:56 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 0a4218ed9d
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Aug 12 22:27:14 2020 +0200

    Fix particle depth.

commit f51dfdeb44
Merge: c6251a956 5e6dc0471
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Aug 12 20:19:04 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit c6251a956a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Aug 12 20:15:46 2020 +0200

    Cache figures more intelligently.

    Cache figures for longer, and don't cache character states for the
    player except where they actually affect the rendered model.

commit 0ed801d540
Merge: c11b9bdf0 eea64f78f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Aug 12 16:32:24 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit c11b9bdf0a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Aug 12 11:47:15 2020 +0200

    Remove unneeded Clippy annotation.

commit 16aa9ef40a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Aug 8 00:53:02 2020 +0200

    Fix hotloading and Clippy.

commit 3dc973e0be
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Aug 7 23:50:27 2020 +0200

    Major speedups with SIMD.

commit fba64a7d93
Merge: 76429d00e d1e10b178
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Aug 7 13:23:19 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 76429d00ee
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Aug 7 13:23:10 2020 +0200

    Add clippy.toml.

commit c79f512f84
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Aug 7 11:55:20 2020 +0200

    Fix all clippy issues, clean up Rust code.

commit 6f90e010b3
Merge: 77a8c7c26 5929cfa5c
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Aug 7 06:47:30 2020 +0200

    Merge remote-tracking branch 'origin/sharp/small-fixes' into sharp/small-fixes

commit 77a8c7c267
Merge: b44e44232 44febaabd
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Aug 7 06:47:10 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 5929cfa5c7
Author: jshipsey <jshipsey18@gmail.com>
Date:   Thu Aug 6 20:47:27 2020 -0400

    fixed in-hand arrow bug

commit b44e442325
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Aug 6 13:40:35 2020 +0200

    Miscellaneous performance improvements.

commit be37acf287
Merge: 125d7fc6c c11876547
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Aug 3 05:49:27 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit 125d7fc6c4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Aug 3 04:55:31 2020 +0200

    Abstract over simd vs. repr_c vectors.

    Also some minor improvements to Event size.

commit d4d4956e92
Merge: 5f3b7294a aced5f979
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Aug 2 20:56:54 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit 5f3b7294af
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Aug 2 20:43:52 2020 +0200

    Fix formatting issues I missed before.

commit a428a3ebba
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Aug 2 20:41:51 2020 +0200

    Fix clippy warnings, part 1.

    There aer still a bunch of type too complex and
    function takes too many arguments warnings that I'll fix later
    (or ignore, since in the one case I did fix a function takes too
    many arguments warning I think it made the code *less* readable).

commit ba54307540
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Jul 30 13:22:42 2020 +0200

    Fix light animations so they are removed when the light turns off.

commit 7e0f4bcbf0
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Jul 29 21:10:20 2020 +0200

    Fix crash in edge case for pixel art.

commit 56da06f7a3
Merge: cf74d55f2 9f53a4a19
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Jul 29 18:56:52 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit cf74d55f2e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Jul 29 18:29:52 2020 +0200

    Fix map image artifacts and remove unneeded allocations.

    Specifically, we address three concerns (the image stretching during
    rotation, artifacts around the image due to clamping to the nearest
    border color when the image is drawn to a larger space than the image
    itself takes up, and potential artifacts around a rotated image which
    accidentally ended up in an atlas and didn't have enough extra space to
    guarantee the rotation would work).

    The first concern was addressed by fixing the dimensions of the map
    images drawn from the UI (so that we always use a square source
    rectangle, rather than a rectangular one according to the dimensions of
    the map).  We also fixed the way rotation was done in the fragment
    shader for north-facing sources to make it properly handle aspect ratio
    (this was already done for north-facing targets).  Together, these fix
    rendering issues peculiar to rectangular maps.

    The second and third concerns were jointly addressed by adding an
    optional border color to every 2D image drawn by the UI.  This turns
    out not to waste extra space even though we hold a full f32 color
    (to avoid an extra dependency on gfx's PackedColor), since voxel
    images already take up more space than Optiion<[f32; 4]> requires.
    This is then implemented automatically using the "border color"
    wrapping method in the attached sampler.

    Since this is implemented in graphics hardware, it only works (at
    least naively) if the actual image bounds match the texture bounds.
    Therefore, we altered the way the graphics cache stores images
    with a border color to guarantee that they are always in their own
    texture, whose size exactly matches their extent.  Since the easiest
    currently exposed way to set a border color is to do so for an
    immutable texture, we went a bit further and added a new "immutable"
    texture storage type used for these cases; currently, it is always
    and automatically used only when there is a specified border color,
    but in theory there's no reason we couldn't provide immutable-only
    images that use the default wrapping mdoe (though clamp to border
    is admittedly not a great default).

    To fix the maps case specifically, we set the border color to a
    translucent version of the ocean border color.  This may need
    tweaking going forward, which shouldn't be hard.

    As part of this process, we had to modify graphics replacement to
    make sure immutable images are *removed* when invalidated, rather
    than just having a validity flag unset (this is normally done by
    the UI to try to reuse allocations in place if images are updated
    in benign ways, since the texture atlases used for Ui do not
    support deallocation; currently this is only used for item images,
    so there should be no overlap with immutable image replacement,
    so this was purely precautionary).

    Since we were already touching the relevant code, we also updated
    the image dependency to a newer version that provides more ways
    to avoid allocations, and made a few other changes that should
    hopefully eliminate redundant most of the intermediate buffer
    allocations we were performing for what should be zero-cost
    conversions.  This may slightly improve performance in some
    cases.

commit ad18ce9399
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Jul 28 13:21:09 2020 +0200

    Fix continent scale hack.

commit 36b1cb074f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Jul 28 12:11:40 2020 +0200

    Enable loading different sized maps without a recompile.

    We may want to tweak the effects of the continent_scale_hack.

commit 13b6d4d534
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Jul 28 10:55:48 2020 +0200

    Removing WORLD_SIZE, part 1.

    Erased almost every instance of WORLD_SIZE and replaced it with a local
    power of two, map_size_lg (which respects certain invariants; see
    common/src/terrain/map.rs for more details about MapSizeLg).  This also
    means we can avoid a dependency on the world crate from client, as
    desired.

    Now that the rest of the code is not expecting a fixed WORLD_SIZE, the
    next step is to arrange for maps to store their world size, and to use
    that world size as a basis prior to loading the map (as well, probably,
    as prior to configuring some of the noise functions).

commit 30b1d2c642
Merge: 7d56ba31b 1377b369f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Jul 27 13:16:58 2020 +0200

    Merge remote-tracking branch 'origin/sharp/small-fixes' into sharp/small-fixes

commit 7d56ba31b4
Merge: 2101113b4 598f14b25
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Jul 27 13:16:27 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit 1377b369f6
Author: Monty Marz <m.marzouq@gmx.de>
Date:   Sun Jul 19 23:25:38 2020 +0200

    more saturated pumpkins

commit ae8d50527f
Author: Monty Marz <m.marzouq@gmx.de>
Date:   Sat Jul 18 20:29:56 2020 +0200

    acacia models

commit 2101113b46
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Jul 18 18:55:25 2020 +0200

    Higher detail LOD.

commit add2cfae04
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Jul 16 01:57:39 2020 +0200

    Revert some irrelevant stuff.

commit 2e2ab3dc1e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Jul 15 13:30:49 2020 +0200

    Fixing various things about shadows.

    * Correcting optimal LISPSM parameter.
    * Figure shadows are cast when they're not visible.
    * Chunk shadows stay cast until you look away.
    * Seamless cubemaps for point lights.
    * Etc.

commit 6c31e6b562
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Jul 12 19:50:26 2020 +0200

    Fix shadow creation.

commit 6332cbe006
Merge: be438657c 930e0028b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Jul 12 18:47:00 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit be438657c3
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Jul 12 18:28:08 2020 +0200

    Tweaks to shadows.

    Added shadow map resolution configuration, added seamless cubemaps,
    documented all existing rendering options, and fixed a few Clippy
    errors.

commit 23b4058906
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Jul 8 10:11:19 2020 +0200

    Fix moon, use nonlinear noise for terrain.

    Note that the latter has a bit of performance cost.

commit 7fbe5cbfbb
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Jul 8 02:23:02 2020 +0200

    Address lies about max texture size.

commit bcfc62b5e1
Merge: 75e3626a7 18a08e8fe
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Jul 8 02:22:08 2020 +0200

    Merge remote-tracking branch 'origin/sharp/small-fixes' into sharp/small-fixes

commit 75e3626a78
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Jul 8 02:21:52 2020 +0200

    OpenGL 3.3 minimum.

commit 18a08e8fe2
Author: Monty Marz <m.marzouq@gmx.de>
Date:   Tue Jul 7 23:57:52 2020 +0200

    settings localization

commit 90c5d1ca36
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Jul 7 21:11:48 2020 +0200

    Lower near distance.

commit 0e66f02b25
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Jul 7 20:09:01 2020 +0200

    All sprites sway in the wind now.

commit db1401a691
Merge: 69e508d8c e8b4b29d7
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Jul 7 19:34:17 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 69e508d8c9
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Jul 7 18:41:37 2020 +0200

    Make it easy to switch to SIMD for math.

commit ffe0f5928c
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Jul 4 21:21:12 2020 +0200

    Fix some issues with underwater rendering.

commit bfda6da42f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Jul 4 19:17:59 2020 +0200

    Fix some minor display issues.

commit 0ed752e087
Merge: ccc6a06a8 518edcb85
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Jul 4 18:14:21 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit ccc6a06a8d
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Jul 4 18:04:34 2020 +0200

    Some minor changes.

commit 4e02024670
Merge: 50a64d927 e05c9267a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Jul 4 16:17:40 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit 50a64d927e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Jul 4 13:07:03 2020 +0200

    Fix far plane.

commit 7dd06da34c
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Jul 2 22:25:35 2020 +0200

    Add shadows.glsl.

commit 618a18c998
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Jul 2 22:10:22 2020 +0200

    Adding shadows, greedy meshing, and more.

commit eaea83fe6a
Merge: 267018495 2f89b863e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu May 21 22:47:07 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 2670184954
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu May 21 21:20:01 2020 +0200

    Make civsim and sites deterministic.

    For anything in worldgen where you use a HashMap, *please* think
    carefully about which hasher you are going to use!  This is
    especially true if (for some reason) you are depending on hashmap
    iteration order remaining stable for some aspect of worldgen.

commit f8376fd5dc
Merge: 654f7e049 cdee191dd
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu May 21 17:53:57 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 654f7e0492
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed May 20 21:22:30 2020 +0200

    Correct backface culling.

commit 560501df05
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue May 19 17:22:06 2020 +0200

    Greedy messhing for shadows.

commit a4d87e1875
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun May 17 05:59:00 2020 +0200

    Shadow maps work for lantern.

commit 243d0837b8
Merge: 04382dc28 71dd520cd
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri May 15 14:53:13 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 04382dc286
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri May 15 14:22:17 2020 +0200

    WIP: better graphics config, better LOD, shadow maps.

commit 22ddbad3eb
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat May 2 18:54:09 2020 +0200

    Minor shader fixes.

commit 746a10e8d0
Merge: 0f4a0e763 40ab94673
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat May 2 04:02:09 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 0f4a0e763d
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri May 1 23:03:24 2020 +0200

    Switch back to pop-in terrain.

commit dd74fa7e4a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri May 1 22:58:55 2020 +0200

    LOD shading closer to voxel shading.

commit ef67bd58ba
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Apr 28 20:49:03 2020 +0200

    Experimental underwater lighting.

commit 2c5ad9d076
Merge: 748279835 303967a6f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Apr 27 22:35:24 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 7482798354
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Apr 27 21:59:55 2020 +0200

    Replace discard in figure-frag.

commit d83b4ae69b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Apr 27 18:45:57 2020 +0200

    Fix sprite lighting, HDR from focus_pos.

commit 0594238004
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Apr 27 18:14:10 2020 +0200

    Proper HDR from point lights.

commit 48c93d2b41
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Apr 27 14:01:43 2020 +0200

    Brighter ambiance, darker LOD shadows.

commit e0452e895c
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Apr 27 13:13:23 2020 +0200

    More proper HDR.

commit 4c6da3ed16
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Apr 27 00:13:10 2020 +0200

    Trying LOD noise.

commit 682a3d74c8
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Apr 26 23:11:08 2020 +0200

    Fix LOD heights in towns.

commit cc39e5734e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sun Apr 26 21:01:23 2020 +0200

    More LOD fixes.

commit 8116b21c2e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 25 23:54:43 2020 +0200

    I like this coloring.

commit bc2560ea90
Merge: 14effdd5d e690efe71
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 25 23:48:33 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit 14effdd5db
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 25 22:24:35 2020 +0200

    Re-saturate.

commit 48a643955d
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 25 22:23:57 2020 +0200

    Various fixes.

commit f7b497a0c2
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 25 03:22:49 2020 +0200

    Render figures again.

commit 44e4aad48d
Merge: e6f0a5a98 9ec319a18
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 25 02:01:04 2020 +0200

    Merge remote-tracking branch 'origin/master' into sharp/small-fixes

commit e6f0a5a981
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Apr 24 16:12:20 2020 +0200

    Add atmospheric scattering.

commit f2953087f6
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 23 00:01:20 2020 +0200

    Fix shadowing for specular reflections.

commit ddd4a67a97
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Wed Apr 22 22:56:12 2020 +0200

    HDR fixes.

commit 1015e60dee
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Tue Apr 21 18:25:19 2020 +0200

    More lighting changes.

commit 80c264abd1
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Mon Apr 13 00:29:59 2020 +0200

    Lighting experiments.

commit 8414987e58
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 9 02:38:40 2020 +0200

    WIP -- lighting changes and soft shadows.

commit 9cd2b3fb0d
Merge: c7ea687eb 8b149ad11
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 4 02:33:29 2020 +0200

    Merge branch 'sharp/new-lighting' into sharp/small-fixes

commit c7ea687ebb
Merge: 476441531 22f3319b4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 4 02:33:02 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes

commit 8b149ad11a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Sat Apr 4 02:32:39 2020 +0200

    Trying out a new lighting model.

commit b0ac9f36f7
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Apr 3 07:56:11 2020 +0200

    Use bicubic interpolation for terrain.

commit f6fc9307a1
Merge: 33140a295 22f3319b4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Apr 3 05:01:41 2020 +0200

    Merge remote-tracking branch 'origin/master' into zesterer/lod

commit 4764415312
Merge: ed2d0111d 13388ee6a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Apr 3 04:54:48 2020 +0200

    Merge branch 'sharp/map-colors' into sharp/small-fixes

commit 13388ee6a4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 2 20:30:08 2020 +0200

    Various fixes (to coloring and to soft shadows).

commit fbd084a94a
Merge: 5a089863b 4fdf6896a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 2 18:50:38 2020 +0200

    Merge branch 'master' of gitlab.com:veloren/veloren into sharp/map-colors

commit ed2d0111d9
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 2 06:49:27 2020 +0200

    Combining colors and LOD.

commit 88342640c6
Merge: 33140a295 5a089863b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 2 04:49:20 2020 +0200

    Merge branch 'sharp/map-colors' into sharp/small-fixes

commit 33140a2951
Merge: 4c65a5aed f34d4b379
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 2 04:36:21 2020 +0200

    Merge remote-tracking branch 'origin/master' into zesterer/lod

commit 5a089863be
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 2 03:17:49 2020 +0200

    Making maps brighter.

    This is probably not the right way to do it, but oh well!

commit 32b2c99109
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Thu Apr 2 02:46:36 2020 +0200

    Horizon mapping and "layered" map generation.

    Horizon mapping is a method of shadow mapping specific to height maps.
    It can handle any angle between 0 and 90 degrees from the ground, as
    long as know the horizontal direction in advance, by remembering only a
    single angle (the "horizon angle" of the shadow map).  More is explained
    in common/src/msg/server.rs.  We also remember the approximate height of
    the largest occluder, to try to be able to generate soft shadows and
    create a vertical position where the shadows can't go higher.

    Additionally, map generation has been reworked.  Instead of computing
    everything from explicit samples, we pass in sampling functions that
    return exactly what the map generator needs.  This allows us to cleanly
    separate the way we sample things like altitudes and colors from the map
    generation process.  We exploit this to generate maps *partially* on the
    server (with colors and rivers, but not shading).  We can then send the
    partially completed map to the client, which can combine it with shadow
    information to generate the final map.  This is useful for two reasons:
    first, it makes sure the client can apply shadow information by itself,
    and second, it lets us pass the unshaded map for use with level of
    detail functionality.

    For similar reasons, river generation is split
    out into its own layer, but for now we opt to still generate rivers on
    the server (since the river wire format is more complicated to compress
    and may require some extra work to make sure we have enough precision to
    draw rivers well enough for LoD).

    Finally, the mostly ad-hoc lighting we were performing has been (mostly)
    replaced with explicit Phong reflection shading (including specular
    highlights).  Regularizing this seems useful and helps clarify the
    "meaning" of the various light intensities, and helps us keep a more
    physically plausible basis.  However, its interaction with soft shadows
    is still imperfect, and it's not yet clear to me what we need to do to
    turn this into something useful for LoD.

commit f8926a5737
Merge: a1aee931e 875ae6ced
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Mar 13 13:32:42 2020 +0100

    Merge remote-tracking branch 'origin/master' into sharp/map-colors

commit 4c65a5aed3
Author: Treeco <5021038-Treeco@users.noreply.gitlab.com>
Date:   Mon Feb 24 16:48:05 2020 +0000

    Made LOD setting slider exponential

commit 2fa7b2d20d
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Mon Feb 24 17:49:53 2020 +0000

    Added mist to LoD

commit aab059a450
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Mon Feb 24 15:14:06 2020 +0000

    Added LoD slider

commit 779c36b538
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Mon Feb 24 12:54:55 2020 +0000

    Reduced cost of vertex pushing

commit 9fea150473
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Mon Feb 24 12:38:53 2020 +0000

    Fixed maths, improved LoD resolution

commit 5481df38fe
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Mon Feb 24 11:22:50 2020 +0000

    Dynamically relocate LoD vertices to enhance details

commit a3e36a50ab
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Sun Feb 23 18:13:51 2020 +0000

    Simpler terrain spiral rendering

commit 255f450ae9
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Sun Feb 23 16:53:17 2020 +0000

    Better LoD precision

commit 3d027aebe8
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Sun Feb 23 16:04:03 2020 +0000

    Better falloff

commit be775c9484
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Sun Feb 23 15:30:45 2020 +0000

    Applied good ideas from experimental branch

commit 58587b6854
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Fri Feb 21 16:15:13 2020 +0000

    Minor fixes to LoD merging

commit 7b42aebd70
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Fri Feb 21 15:04:44 2020 +0000

    Capped LoD dragging

commit 8aafc559f8
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Fri Feb 21 14:54:37 2020 +0000

    Better blending between LoD and terrain border

commit edd3455d51
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Fri Feb 21 14:40:19 2020 +0000

    Fixed LoD z depth, added sea level offset

commit b9b0674462
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Fri Feb 21 14:27:43 2020 +0000

    Better LoD smoothing

commit a1aee931e7
Author: Joshua Yanovski <pythonesque@gmail.com>
Date:   Fri Feb 21 14:52:17 2020 +0100

    Adding shadows.

commit 2400786c13
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Fri Feb 21 13:48:40 2020 +0000

    Use world map as LoD source

commit dbf650f504
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Mon Jan 20 00:48:14 2020 +0000

    Better clouds at distance

commit 5e6f81b86c
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Sun Jan 19 23:59:02 2020 +0000

    sync

commit 745e7540dd
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Fri Nov 22 12:40:48 2019 +0000

    Improved cloud falloff mist, faster noise sampling

commit f6a200d0cb
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Fri Nov 22 10:09:00 2019 +0000

    Improved long-range depth precision, removed unnecessary LoD polygons

commit 63d1b2bb22
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Thu Nov 21 20:57:46 2019 +0000

    Working LoD shader

commit f13d98ee3e
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date:   Thu Nov 21 11:03:40 2019 +0000

    LoD first attempt (stack overflow issue)
2020-08-20 20:34:59 +02:00
Marcel Märtens
8687740265 fix clippy warnings in new version 2020-08-17 11:10:18 +02:00
Marcel Märtens
10970841cc fix master / update toolchain to 2020-08-15 2020-08-17 10:28:09 +02:00
Marcel Märtens
e618eeb386 Fix a isse that might occur when a participant is dropped while the remote wants to open a Stream and we get some bad time condition.
increase the slowlorris timeout. for some reason it seems to trigger alot more often since commit: `75c1d440` but i have no idea why.
My guess would be that the initial sync now sends way more data which slows down TCP to be > 10ms and trigger. Note: the fix might cause small lags when slow people try to connect to the server
2020-08-13 12:06:53 +02:00
Marcel Märtens
dd581bc6c0 Participant closure was immeatiatly, even in case a new participant was connected, send a MSG and then dropped immeadiatly.
The remote site should see it connect, be open for 1 single stream and read the message before it's notified that the participant is closed actually.

This caused the faulure in one of our API tests (in lib, with client and server). Where it was possible that all messages were send and one side was dropped before the other side asked for the opened stream

Also introduce better error detection in participant(and scheduler) by removing the std_async::Result and intruduce `Result<(),ParticipantError>` instead
2020-07-22 09:18:15 +02:00
Marcel Märtens
6c59caf8e1 make prometheus optional in network and fix a panic in the server
- an extra interface `new_with_regisitry` was created to make sure the interface doesn't depend on the features
2020-07-15 16:45:49 +02:00
Marcel Märtens
58cb98deaa use type to reduce complexity 2020-07-15 16:45:44 +02:00
Marcel Märtens
c74e5e4b47 Changes requested in rewiew 2020-07-13 23:41:32 +02:00
Marcel Märtens
6db9c6f91b fix a followup bug, after a protocol fail now Participant is closed, including all streams, so we get the stream errors.
We MUST handle them and we are not allowed to act on a stream after it failed, as i am to lazy to change the structure to ensure the client to be imeadiatly dropped i added a AtomicBool to it.
2020-07-13 13:03:35 +02:00
Marcel Märtens
187ec42aa2 fix Participant shutdown
- we had the problem that Participants couldn't shutdown them self, only by scheduler, which was controlled by api.
  it's needed e.g. to handle the Schudown Frame
 - my initial solution did a full shutdown, which was a problem if in parallel a 2nd shutdown was requested, no possibility of getting the error
 - new solution will only deactivate Participant and Stream. and then still functions correctly, till the api closes the participant and calls the scheduler which then calls the bparticipant again
 - i experimented with a Mutex<oneshot> or 2 and a `select` but it didn't prove that well
 - also adjusted the Error messages to now either Disconnected when gracefully shutdown or ProtocolFailed when some msg couldn't be delivered
  (note later might not be 100% returned correctly yet)
2020-07-13 13:03:30 +02:00
Marcel Märtens
df45d35c0e tcp protocol hardening
- make it harder for the server to crash and also kill invalid sessions properly (instead of waiting for them to close)
 - introduce macros to reduce code duplication
 - added tests to check for valid handshake as well as garbage tcp
2020-07-13 13:03:25 +02:00
Marcel Märtens
9d32e3f884 proper voxygen connect and code cleanups:
- voxygen abort when the server has a invalid veloren_network handshake, e.g. by outdated version instead of try again
- rename Network `Address` to `ProtocolAddr` as sugested by zest as it's a combination of Protocol and std::io::Addr
- remove the manual byte arrays in `protocols.rs` with something more nice
2020-07-13 13:03:20 +02:00
Marcel Märtens
041349be48 Switch API to return Participant rather than Arc<Participant>
- API behavior switched!
 - the `Network` no longer holds a copy of participant, thus if the return of `connect` (before `Arc<Participant>, now `Participant`) got dropped, the `Participant::Drop` is triggered!
 - you can close a Participant async via `Particiant::disconnect()`, no more need to know the network at this point
 - the `Network::Drop` will check and drop not yet disconnected Participants.
 - you can compare Participants via PartialEq, if they are true they point to the same endpoint (it checks remote_pid)
   - Note: multiple Participants are only supported in theory, wont work yet

Additionally:
 - fix some `debug!`
 - veloren-client will now drop the participant gracefully on shutdown
 - rename `error` to `debug` when 2 times Bparticipant shutdown is called, as it is to be expected in a async runtime
2020-07-13 13:03:14 +02:00
Marcel Märtens
8fb445b0e8 better lz4 2020-07-13 13:03:04 +02:00
Marcel Märtens
4cefdcefea zests fix - capitalize first letter 2020-07-13 13:03:01 +02:00
Marcel Märtens
5f902b5eab doing a clean shutdown of the BParticipant once the TCP connection is suddenly interrupted 2020-07-13 13:02:55 +02:00
Marcel Märtens
c212de00c2 updated dependencies and fixed stuff
- replace serde_derive by feature of serde
   incl. source code modifications to compile
 - reduce futures-timer to "2.0" to be same as async_std
 - update notify
 - removed mio, bincode and lz4 compress in common as networking is now in own crate
   btw there is a better lz4 compress crate, which is newer than 2017
 - update prometheus to 0.9
 - can't update uvth yet due to usues
 - hashbrown to 7.2 to only need a single version
 - libsqlite3 update
 - image didn't change as there is a problem with `image 0.23`
 - switch old directories with newer directories-next
 - no num upgrade as we still depend on num 0.2 anyways
 - rodio and cpal upgrade
 - const-tewaker update
 - dispatch (untested) update
 - git2 update
 - iterations update
2020-07-07 09:43:49 +02:00
Joshua Barretto
3c1fddfb0b Merge branch 'zesterer/server-fixes' into 'master'
Server bug fixes

See merge request veloren/veloren!1159
2020-07-05 19:41:30 +00:00
Joshua Barretto
dd2a81b1f3 Increased network test timeouts 2020-07-05 19:56:06 +01:00
Marcel Märtens
3a6319f2f6 compress everything 2020-07-05 20:14:47 +02:00
Marcel Märtens
092b1e0d6c small fix 2020-07-05 18:54:34 +02:00
Marcel
2a7a8b05e6 Merge branch 'network-lockless' into 'master'
Network lockless

See merge request veloren/veloren!1153
2020-07-05 09:17:29 +00:00
Marcel Märtens
e7195b57ad extend network with better Error codes for Network 2020-07-04 12:32:52 +02:00
Marcel Märtens
cbfd398035 remove Mutex in server as Stream is now 'Sync' 2020-07-04 12:31:59 +02:00
Marcel Märtens
6de2eadeb0 make crash -> error for now 2020-07-04 10:32:52 +02:00
Marcel Märtens
1da6f15a43 fixing various smaller network party from issue 657 2020-07-04 02:04:33 +02:00
Marcel Märtens
f9895a7800 crossbeam-channel and log spam
- swap out std::mpsc with crossbeam-channel in networking crate
 - remove log spam by only logging when populating a new cache entry and not on every get
2020-07-03 22:35:29 +02:00
Marcel Märtens
e1b27c51f5 fix clippy issues in tests and add it to CI 2020-07-01 00:37:15 +02:00
Marcel Märtens
6535fa5744 fix various clippy issues 2020-07-01 00:37:06 +02:00
Marcel Märtens
11e7b1f922 increase network sleep in order to fix flanky tests 2020-06-29 14:32:20 +02:00
Marcel Märtens
57453291e4 - switched participant to only error!() once a sec instead of every occurence in order to not spam error
- switched the behavior of prio to keep the order of a stream, EVEN if messages of different length are used!
  This needs to be addresses to fulfill PROMISES_ORDERED
  I think we need to make thoughs if PRIO needs to handle this, or we add additional meta info and handle it on receiver site
2020-06-28 22:29:42 +02:00
Ben Wallis
c1c968f479 Globally suppressed clippy lint option_map_unit_fn for #587 2020-06-14 16:48:07 +00:00
Marcel Märtens
1435d8d6be remove unused files 2020-06-12 11:53:59 +02:00