Participant will now handle a close in the `create_channel_mgr` rather then the `send` fn. Its the better place, which makes a HashMap better for delete lookup
Since tcp_read now broke but tcp_write didn't and the Participant wasnt updated till both were broke, we changed CHANNEL tcp_read and tcp_write in protocols to be a `select` rather than a `join`
However only do this in the CHANNEL, but in the HANDSHAKE. it fails if you try to. Also the handshake will take care of any failed read or write manually and will handle a clear teardown in this case.
---
Pretty much a Veloren fork at this point. Here's a high level overview of the changes (will be added to CHANGELOG just before merge).
At a high level this MR incorporates roughly two groups of changes.
The first group consists of new game features: more flexible map sizes, level of detail terrain, shadow maps, and a new lighting
engine. This is "feature work" that (mostly) only adds new things to Veloren, and mostly shouldn't affect old stuff.
The second big group of changes are those addressing the fallout from all the new features. These include performance fixes of
various sorts: the addition of multiple graphics options and optimization of the cheap ones to avoid work, switching all voxel
models to use some variant of greedy meshing, switching over much of our CPU-side vector math to exploit SIMD instructions
(coinciding with a fork of `vek`), and a rewrite of how the UI handles text rendering (coinciding with updates to our fork
of `conrod`). Making Veloren's hardcoded colors appear correct under the new lighting engine also required considerably
changes (TODO: Fill in this section when it's complete).
The second category of changes often heavily touches code owned by other people, including frequently modified code "owned" by a
handful of people, so I recommend that this code be reviewed particularly carefully.
---
At a high level (each will be described in more detail below):
- The world map has been refactored.
- The world size is no longer hardcoded (@zesterer).
- The map generation code was made generic to allow using it outside of the `world` crate (@zesterer).
- On world creation, we now compute *horizon maps* (@zesterer).
- The way we pass the world from the server to the client has been updated (@xMAC94x).
- Artifacts related to image rotation were fixed (@imbris).
- Multiflow rivers were enabled (@zesterer).
- In the process of making changes related to the world map, various incidental fixes and optimizations were required.
- The new *level of detail* feature was added (@zesterer wrote part of this and has checked out the rest).
- A new LOD terrain rendering step was added to the pipeline.
- The LOD terrain quality was made configurable via a graphics setting.
- Horizon maps were used to cast shadows from LOD chunks on both LOD and non-LOD terrain.
- A "voxelization" effect was incorporated into rendered LOD terrain to make it blend better into the world.
- In the process of making changes related to LOD, various incidental fixes and optimizations were required.
- Veloren's lighting has been completely overhauled (@zesterer has already checked most of this out).
- A semi-accurate index of refraction was assigned to our materials.
- A new, more realistic, physically based approach to lighting was used using the *Ashikhmin Shirley* BRDF.
- We emulate *atmospheric scattering* using equations designed for measuring solar panel light exposure.
- We attempt to compute *realistic light attenuation* in water using its real material properties.
- In the process of making changes related to LOD, various incidental fixes and optimizations were required.
- Point and directional lights now cast realistic shadows, using *shadow mapping.* (@imbris, @zesterer, @Treeco, @YuriMomo)
- Point light shadow maps were added to the rendering pipeline, using geometry shaders and *seamless cube maps*.
- Directional light shadows were added to the rendering pipeline, using LISPSM together with disabling *depth clamping*.
- "Shadow-only" chunks and NPCs were added to prevent shadows from models behind you from disappearing.
- In the process of making changes related to shadow maps, various incidental fixes and optimizations were required.
The addition of shadow maps, LOD terrain, and the new lighting all led to significant performance degradation, on top of other
changes happening in master. Therefore, a large number of performance improvements were also needed:
- The graphics options were made much more flexible and configurable, and shaders were optimize.
- New options were provided for how to render lights and shadows (@Pfauenauge, @zesterer).
- Graphic setting storage and configuration were overhauled to make adding new features easier (@Pfauenauge, @imbris).
- Shaders were rewritten to utilize GLSL's preprocessor to avoid overhead (@zesterer, @YuriMomo).
- In the process of making changes related to providing additional rendering options, various incidental fixes and optimizations were required.
- Voxel model creation was switched to use *greedy meshing.*
- A new voxel meshing method, greedy meshing, was added (@imbris).
- Uses of the older meshing methods were migrated to use greedy meshing (@imbris, @jshipsey, @Pfauenauge).
- New restrictions were added to terrain, figure, and sprites to future proof them for further optimizations (@jshipsey, @Pfauenauge, @zesterer).
- Most positions are now relative to either chunk or player position for better precision (@imbris, @zesterer, @scottc).
- In the process of making changes related to greedy meshing, various incidental fixes and optimizations were required.
- Animation and terrain math were switched to use SIMD where possible.
- Fixes were made to vek to make its SIMD feature usable for us (@zesterer, @imbris).
- The interface and types used in bone animation were changed in various ways (@jshipsey, @Snowram, @Pfauenauge).
- Redundant code generation for body animation is now partly taken care of by a macro (@jshipsey, @Snowram, @Pfauenauge).
- Animation code was modified to to use vek's SIMD representation where possible (@jshipsey, @Snowram, @Pfauenauge).
- Terrain meshing code and shadow map math were also modified to use vek's SIMD representation (@imbris).
- SIMD instruction generation was enabled (@YuriMomo, @jshipsey, @Snowram, @imbris, @Angelonfira, @xMAC94x).
- In the process of making changes related to greedy meshing, various incidental fixes and optimizations were required.
- The way we cache glyphs was completely refactored, fixed, and optimized.
- Our fork of `conrod` was optimized in various ways (@imbris).
- Our fork of `conrod` now exposes whether a widget was updated during the current frame (@imbris).
- Our use of the glyph cache was rewritten for correctness (@imbris).
- A *text cache* was introduced that lets us skip remeshing glyphs that have not changed (@imbris).
- Various changes were made to reduce pressure on the glyph cache, with more planned (@imbris, @Pfauenauge).
- In the process of making changes related to the glyph cache, various incidental fixes and optimizations were required.
- Colors were changed to keep Veloren's look consistent with master.
- Some older tree models were brought back (@Pfauenauge).
- TODO(@Sharp): All hardcoded colors were extracted and made hotloadable.
- TODO(@Treeco, @Pfauenauge, @imbris, @jshipsey): Hardcoded colors were fixed to conform to Veloren's style.
- TODO(@Treeco, @Pfauenauge, @imbris, @jshipsey): Color models were fixed to conform to Veloren's style.
A detailed description of the involved changes follows.
---
- The world size is no longer hardcoded. All functions dependent on world size now take a `WorldSizeLg`, which holds the base 2 logarithm of each actual world dimension and is guaranteed to maintain certain properties (outlined in `common/src/terrain/map.rs`). Additionally, many utility functions that utilize the world size were moved into `common` as well (mostly `common/src/terrain/mod.rs`). Finally, the world map format was updated in order to store its size explicitly, with a migration path from the old format that should work whenever the old formatted map was a square (practically always). See `world/src/sim/mod.rs` for these changes.
- The map generation code was made generic to allow using it outside of the `world` crate. The parts of the map generating code that do not need to query the world were moved over to `common/src/terrain/map.rs`, allowing them to be used from the client without creating a dependency on `world`. The rest of it was turned into helper functions in `world/src/sim/map.rs`, which can be passed as closures to the generic map generation code to complete its construction. This also means that colors are now passed in separately to the map generation function. See <https://veloren.net/devblog-78/> for more details.
- On world creation, we now compute *horizon maps*. See the function in `world/src/sim/util.rs`.
Given a height map and a plane intersecting that height map, our horizon maps allow us to encode enough information to reconstruct shadows for each point on the height map using only the *horizon angle* (the angle at which the sun starts to become visible). As Veloren's sun only covers one plane, this is sufficient for encoding sun shadows for LOD terrain, by encoding two angles per chunk (one for each 90 degrees the sun covers). We can also use this for the moon, if we want, since the moon follows the same path. Additionally, we store the *height* of the furthest occluder, to try to make the shadows volumetric; so this means 4 bytes in total for each chunk.
Support for horizon maps has been merged into the map functionality in common as well.
- The way we pass the world from the server to the client has been updated. Rather than passing the prerendered map, we instead pass three maps with values for each chunk; one with the color information, a second with altitude information, and a third with horizon map information. We then reconstruct the map on the client, together with some additional information we send from the server (like the sea level and maximum height). See `common/src/msg/server.rs` for a detailed description of the format of `WorldMapMsg`, and `server/src/libr.rs` and `client/src/lib.rs` for details of the map construction and parsing.
- Artifacts related to image rotation were fixed. See the commit message for commit SHA `cf74d55f2e3d2ae7d25fd68d5c73b01a6afde86e` for a detailed explanation. This involved changes to shaders, the addition of a new type of graphic (also reflected in the graphic cache) that allows specifying a border color (which automatically makes the associated texture immutable), and some related fixes. I reproduce the first two paragraphs of the MR description as well:
```
Fix map image artifacts and remove unneeded allocations.
Specifically, we address three concerns (the image stretching during
rotation, artifacts around the image due to clamping to the nearest
border color when the image is drawn to a larger space than the image
itself takes up, and potential artifacts around a rotated image which
accidentally ended up in an atlas and didn't have enough extra space to
guarantee the rotation would work).
```
- Multiflow rivers were enabled.
This does not really need to be part of this MR, and would be easy to revert, but since it seemed to provide a nice improvement it's currently packaged with it.
We already computed multiple outflows from each chunk for erosion purposes long before this MR.
However, we never modified river rendering to be able to handle this case (just a single downhill river flow is complex enough!) so this was not exposed when deciding which chunks were rivers.
Now that
- In the process of making changes related to the world map, various incidental fixes and optimizations were required. Some examples of fixes include making sure terrain is never lowered to below sea level (to make the shadow maps report correct values), fixing map altitudes and colors to understand things like cliffs and "block level" coloring (that doesn'te xist on the column level), and fixing a crashbug when rendering images for the UI where source pixels are strongly rectangular. Some examples of related performance fixes include avoiding allocating a fresh vector for all the maps (i.e. copying it over to change the format from `[u32; n]` to `DynamicImage` and then copying again to convert to `RgbaImage`), and instead using the `gfx::memory::slice` function to accomplish the same thing. These sorts of changes are spread all arond the code.
This includes the additon of a new scene, `voxygen/src/scene/lod.rs`, a new pipeline `voxygen/src/render/pipeline/lod_terrain.rs`, and new shaders `assets/vxygen/shaders/lod-terrain-vert.glsl` and `assets/vxygen/shaders/lod-terrain-frag.glsl`, as well as associated changes to the renderer in `voxygen/src/render/renderer.rs`.
The main idea behind our initial approach to LOD was to take the world data we now get from the server (altitude, color, and horizon mapping).
- Some previously computed values were turned into shader uniforms for better prediction on weak processors. (@zesterer)
- Calls to power or trig functions were removed or replaced with multiplications, where possible.
- After some deliberation
- To properly handle sprite "waving" for nearby sprites,
We explicitly designed the greedy meshing system with figures and sprites in mind.
In both cases, we want to be able to *efficiently* pack many different models into the same texture, especially in cases where we know
we will either not be removing any of the grouped-together from the models from the texture, or will remove all of them at once (so
they can be packed into some specific subtexture).
For sprites, since we know every model in advance and never intend to deallocate them, we currently pack them all as efficiently as
possible into one giant tetxure atlas. However, in the future we might opt to pack them slightly less efficiently in exchange for
shrinking the sprite vertex size.
For figures, we pack all the textures for each *model* into the same atlas.
is a global texture atlas used for every sprite, and for figures which is why we have the ability to mesh multiple
models to the same texture area (using the simple texture atlas allocator) without requiring intermediate vector allocations.
This is accomplished by delaying the time when we actually write the color and light data to the texture until *after* all the
model vertices have been meshed; then, we can just allocate the whole color/light array at once, making the atlas we use an
exact fit. In computer science-y terms, we accomplish this delay by, after we perform the initial greedy meshing (without
texture information), not continuing to create the texture data, but instead constructing a *continuation*--that is, a function
that, when called, will execute the rest of the computation. We push this continuation (which in Rust terms is a `FnOnce` closure
that takes the `ColLightsInfo` that it is supposed to write to as context) onto a
onto a vector
resizing. To allow for suspended writes to texture data, Rust pointed out to me that the continuation that would eventually write the color and light data to the texture atlas (the one that is shared by all models sharing the same greedy mesher) would have to *own* whatever data it mshed. Because we often generate the model data to mesh as a temporary in `voxygen/src/load.rs`, the
- Matrix multiplications in the shader were reduced for figure data (@zesterer).
- Vertex "waves" for fluid data were removed.
- Terrain "bending" near edges was removed.
- Scaling was fixed to make sure empty space was not introduced in a space previously occupied by a block. It was also changed to take ownership of its voxel data,
rather than sharing it, to let it be used with meshing.
- Rust's nightly version was bumped in order to use the `array_map` function, which lets us reuse more code between the simple map and `FigureModelCache`.
- PositionedGlyph::standalone.
---
I tried to cite sources in many cases[^realtime],[^lloyd],[^lispsm],[^pbrt],[^greedy],[^tjunctions]
where I needed features from elsewhere but I am particularly grateful for the following resources,
esepcially where they have accompanying source code. I linked all of them that are accessible to the public (those that are not were
obtained through legal means).
[^realtime]: Eisemann, Elmar, Michael Schwarz, Ulf Assarsson, Michael Wimmer. Real-Time Shadows. A K Peters/CRC Press (T&F), 20160419.
[^lloyd]: Lloyd,B. 2007. [Logarithmic perspective shadow maps](http://gamma.cs.unc.edu/papers/documents/dissertations/lloyd07.pdf). PhD thesis, University of North Carolina.
[^lispsm]: Wimmer, M., Scherzer, D., and Purgathofer, W. 2004. [Light space perspective shadow maps](http://gamma.cs.unc.edu/papers/documents/dissertations/lloyd07.pdf). In Proceedings of Eurographics Symposium on Rendering 2004, pp. 143– 152.
[^pbrt]: Pharr, Matt, et al. [http://gamma.cs.unc.edu/papers/documents/dissertations/lloyd07.pdf](Physically Based Rendering: From Theory to Implementation). Third edition, Morgan Kaufmann Publishers/Elsevier, 2017.
[^greedy]: mikolalysenko. “Meshing in a Minecraft Game.” 0 FPS, 30 June 2012, <https://0fps.net/2012/06/30/meshing-in-a-minecraft-game/>.
[^tjunctions]: blackflux. “Meshing in Voxel Engines – Part 1.” Blackflux.Com, 23 Feb. 2014, <https://blackflux.wordpress.com/2014/02/23/meshing-in-voxel-engines-part-1/>.
I am also especially grateful to Khronos, Wikiepdia, and stackoverflow for answering many of my specific questions while writing the MR.
---
Squashed commit of the following:
commit 300505e730
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 18:46:25 2020 +0200
Fixing cargo doc and typo in CHANGELOG.
commit ec0aeb18e8
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 15:38:50 2020 +0200
Hopefully final commit for the LOD branch.
commit 5e8ea0b1ea
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 10:14:26 2020 +0200
Falling back to power as stopgap.
commit e44a1cbf46
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 09:25:41 2020 +0200
Address imbris feedback.
Temporarily disables shiny water, lowers max VD.
These restrictions will be lifted soon after merging.
commit 561e25778a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 08:31:13 2020 +0200
Tweaking shaders a bit.
commit 7d19259078
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 07:59:43 2020 +0200
Fix view example as well.
commit 051cd4934e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 07:29:06 2020 +0200
Fix meshing benchmark.
commit c95e07db3b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 05:46:22 2020 +0200
Address MR feedback, fix scene clouds.
commit 1bfb816cab
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 04:39:36 2020 +0200
Incorporating Pfau's figure color changes.
New eyes and new humanoid colors.
commit 3f9b89a3ac
Merge: e2f5162e462c53963a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 04:29:41 2020 +0200
Merge remote-tracking branch 'origin/sharp/small-fixes' into sharp/small-fixes
commit e2f5162e4f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 20 04:28:38 2020 +0200
World colors are all hotloadable.
They live in assets/world/style/colors.ron.
Only a small handful of hardcoed colors remain in World; they are either
part of the map, or difficult to disentangle from the rest of the
computation. Comments are made where appropriate.
commit 62c53963ab
Author: Marcel Märtens <marcel.cochem@googlemail.com>
Date: Wed Aug 19 15:59:00 2020 +0200
replace pretty_env_logger with tracing
commit 5b1625f99d
Merge: d71003acd4942b5b39
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Aug 19 05:15:56 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit d71003acda
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Aug 19 05:14:34 2020 +0200
Hotloading colors, part 1: colors in common.
Currently, this just entails humanoid colors. There are only three
colors not handled; the light emitter colors in
common/src/comp/inventory/item/tool.rs. These don't seem important
enough to me to warrant making hotloadable, at least not right now, but
if it's needed later we can always add them to the file.
commit 63b5e0e553
Merge: c32b337a46d2c4b9c1
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Aug 17 13:05:37 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit c32b337a46
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Aug 17 05:52:04 2020 +0200
Fixing LOD grid, for real.
commit a166ae0360
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Aug 17 05:28:05 2020 +0200
Addressing imbris's initial feedback.
Fixes two minor bugs: explosion particles were no longer spawning
randomly, and LOD grids were not perfectly even.
commit 4cbad004f4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Aug 16 19:27:58 2020 +0200
Bumping nightly per request.
commit 548680276a
Merge: acc0986048f8b20c91
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Aug 16 19:26:06 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit acc0986040
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Aug 15 22:28:32 2020 +0200
Lower resolution due to lying drivers.
commit d3b878de2a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Aug 15 22:15:38 2020 +0200
Fix issues msh encountered with Intel 4600.
commit 10245e0c1b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Aug 15 21:15:02 2020 +0200
Merge more models into one mesh than we did previously.
commit 3155c31e66
Merge: 7204cc8a73c199280e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 13 22:35:22 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 7204cc8a7a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 13 22:34:43 2020 +0200
Fix not yet done NPC animations.
This forces them all to be the idle animation if not specified.
This fixes issues where you'd have giant NPCs in water.
commit bc83360f2a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 13 19:36:37 2020 +0200
Try to fix some bugs:
- Z fighting with LOD terrain and water.
- Audio SFX not playing.
commit 1fd104aa60
Merge: 862df3c997c2c392a3
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 13 12:02:31 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 862df3c997
Merge: 0a4218ed975c1d4401
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 13 05:52:56 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 0a4218ed9d
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Aug 12 22:27:14 2020 +0200
Fix particle depth.
commit f51dfdeb44
Merge: c6251a9565e6dc0471
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Aug 12 20:19:04 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit c6251a956a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Aug 12 20:15:46 2020 +0200
Cache figures more intelligently.
Cache figures for longer, and don't cache character states for the
player except where they actually affect the rendered model.
commit 0ed801d540
Merge: c11b9bdf0eea64f78f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Aug 12 16:32:24 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit c11b9bdf0a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Aug 12 11:47:15 2020 +0200
Remove unneeded Clippy annotation.
commit 16aa9ef40a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Aug 8 00:53:02 2020 +0200
Fix hotloading and Clippy.
commit 3dc973e0be
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Aug 7 23:50:27 2020 +0200
Major speedups with SIMD.
commit fba64a7d93
Merge: 76429d00ed1e10b178
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Aug 7 13:23:19 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 76429d00ee
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Aug 7 13:23:10 2020 +0200
Add clippy.toml.
commit c79f512f84
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Aug 7 11:55:20 2020 +0200
Fix all clippy issues, clean up Rust code.
commit 6f90e010b3
Merge: 77a8c7c265929cfa5c
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Aug 7 06:47:30 2020 +0200
Merge remote-tracking branch 'origin/sharp/small-fixes' into sharp/small-fixes
commit 77a8c7c267
Merge: b44e4423244febaabd
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Aug 7 06:47:10 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 5929cfa5c7
Author: jshipsey <jshipsey18@gmail.com>
Date: Thu Aug 6 20:47:27 2020 -0400
fixed in-hand arrow bug
commit b44e442325
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Aug 6 13:40:35 2020 +0200
Miscellaneous performance improvements.
commit be37acf287
Merge: 125d7fc6cc11876547
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Aug 3 05:49:27 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit 125d7fc6c4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Aug 3 04:55:31 2020 +0200
Abstract over simd vs. repr_c vectors.
Also some minor improvements to Event size.
commit d4d4956e92
Merge: 5f3b7294aaced5f979
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Aug 2 20:56:54 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit 5f3b7294af
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Aug 2 20:43:52 2020 +0200
Fix formatting issues I missed before.
commit a428a3ebba
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Aug 2 20:41:51 2020 +0200
Fix clippy warnings, part 1.
There aer still a bunch of type too complex and
function takes too many arguments warnings that I'll fix later
(or ignore, since in the one case I did fix a function takes too
many arguments warning I think it made the code *less* readable).
commit ba54307540
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Jul 30 13:22:42 2020 +0200
Fix light animations so they are removed when the light turns off.
commit 7e0f4bcbf0
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Jul 29 21:10:20 2020 +0200
Fix crash in edge case for pixel art.
commit 56da06f7a3
Merge: cf74d55f29f53a4a19
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Jul 29 18:56:52 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit cf74d55f2e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Jul 29 18:29:52 2020 +0200
Fix map image artifacts and remove unneeded allocations.
Specifically, we address three concerns (the image stretching during
rotation, artifacts around the image due to clamping to the nearest
border color when the image is drawn to a larger space than the image
itself takes up, and potential artifacts around a rotated image which
accidentally ended up in an atlas and didn't have enough extra space to
guarantee the rotation would work).
The first concern was addressed by fixing the dimensions of the map
images drawn from the UI (so that we always use a square source
rectangle, rather than a rectangular one according to the dimensions of
the map). We also fixed the way rotation was done in the fragment
shader for north-facing sources to make it properly handle aspect ratio
(this was already done for north-facing targets). Together, these fix
rendering issues peculiar to rectangular maps.
The second and third concerns were jointly addressed by adding an
optional border color to every 2D image drawn by the UI. This turns
out not to waste extra space even though we hold a full f32 color
(to avoid an extra dependency on gfx's PackedColor), since voxel
images already take up more space than Optiion<[f32; 4]> requires.
This is then implemented automatically using the "border color"
wrapping method in the attached sampler.
Since this is implemented in graphics hardware, it only works (at
least naively) if the actual image bounds match the texture bounds.
Therefore, we altered the way the graphics cache stores images
with a border color to guarantee that they are always in their own
texture, whose size exactly matches their extent. Since the easiest
currently exposed way to set a border color is to do so for an
immutable texture, we went a bit further and added a new "immutable"
texture storage type used for these cases; currently, it is always
and automatically used only when there is a specified border color,
but in theory there's no reason we couldn't provide immutable-only
images that use the default wrapping mdoe (though clamp to border
is admittedly not a great default).
To fix the maps case specifically, we set the border color to a
translucent version of the ocean border color. This may need
tweaking going forward, which shouldn't be hard.
As part of this process, we had to modify graphics replacement to
make sure immutable images are *removed* when invalidated, rather
than just having a validity flag unset (this is normally done by
the UI to try to reuse allocations in place if images are updated
in benign ways, since the texture atlases used for Ui do not
support deallocation; currently this is only used for item images,
so there should be no overlap with immutable image replacement,
so this was purely precautionary).
Since we were already touching the relevant code, we also updated
the image dependency to a newer version that provides more ways
to avoid allocations, and made a few other changes that should
hopefully eliminate redundant most of the intermediate buffer
allocations we were performing for what should be zero-cost
conversions. This may slightly improve performance in some
cases.
commit ad18ce9399
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Jul 28 13:21:09 2020 +0200
Fix continent scale hack.
commit 36b1cb074f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Jul 28 12:11:40 2020 +0200
Enable loading different sized maps without a recompile.
We may want to tweak the effects of the continent_scale_hack.
commit 13b6d4d534
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Jul 28 10:55:48 2020 +0200
Removing WORLD_SIZE, part 1.
Erased almost every instance of WORLD_SIZE and replaced it with a local
power of two, map_size_lg (which respects certain invariants; see
common/src/terrain/map.rs for more details about MapSizeLg). This also
means we can avoid a dependency on the world crate from client, as
desired.
Now that the rest of the code is not expecting a fixed WORLD_SIZE, the
next step is to arrange for maps to store their world size, and to use
that world size as a basis prior to loading the map (as well, probably,
as prior to configuring some of the noise functions).
commit 30b1d2c642
Merge: 7d56ba31b1377b369f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Jul 27 13:16:58 2020 +0200
Merge remote-tracking branch 'origin/sharp/small-fixes' into sharp/small-fixes
commit 7d56ba31b4
Merge: 2101113b4598f14b25
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Jul 27 13:16:27 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit 1377b369f6
Author: Monty Marz <m.marzouq@gmx.de>
Date: Sun Jul 19 23:25:38 2020 +0200
more saturated pumpkins
commit ae8d50527f
Author: Monty Marz <m.marzouq@gmx.de>
Date: Sat Jul 18 20:29:56 2020 +0200
acacia models
commit 2101113b46
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Jul 18 18:55:25 2020 +0200
Higher detail LOD.
commit add2cfae04
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Jul 16 01:57:39 2020 +0200
Revert some irrelevant stuff.
commit 2e2ab3dc1e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Jul 15 13:30:49 2020 +0200
Fixing various things about shadows.
* Correcting optimal LISPSM parameter.
* Figure shadows are cast when they're not visible.
* Chunk shadows stay cast until you look away.
* Seamless cubemaps for point lights.
* Etc.
commit 6c31e6b562
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Jul 12 19:50:26 2020 +0200
Fix shadow creation.
commit 6332cbe006
Merge: be438657c930e0028b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Jul 12 18:47:00 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit be438657c3
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Jul 12 18:28:08 2020 +0200
Tweaks to shadows.
Added shadow map resolution configuration, added seamless cubemaps,
documented all existing rendering options, and fixed a few Clippy
errors.
commit 23b4058906
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Jul 8 10:11:19 2020 +0200
Fix moon, use nonlinear noise for terrain.
Note that the latter has a bit of performance cost.
commit 7fbe5cbfbb
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Jul 8 02:23:02 2020 +0200
Address lies about max texture size.
commit bcfc62b5e1
Merge: 75e3626a718a08e8fe
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Jul 8 02:22:08 2020 +0200
Merge remote-tracking branch 'origin/sharp/small-fixes' into sharp/small-fixes
commit 75e3626a78
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Jul 8 02:21:52 2020 +0200
OpenGL 3.3 minimum.
commit 18a08e8fe2
Author: Monty Marz <m.marzouq@gmx.de>
Date: Tue Jul 7 23:57:52 2020 +0200
settings localization
commit 90c5d1ca36
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Jul 7 21:11:48 2020 +0200
Lower near distance.
commit 0e66f02b25
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Jul 7 20:09:01 2020 +0200
All sprites sway in the wind now.
commit db1401a691
Merge: 69e508d8ce8b4b29d7
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Jul 7 19:34:17 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 69e508d8c9
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Jul 7 18:41:37 2020 +0200
Make it easy to switch to SIMD for math.
commit ffe0f5928c
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Jul 4 21:21:12 2020 +0200
Fix some issues with underwater rendering.
commit bfda6da42f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Jul 4 19:17:59 2020 +0200
Fix some minor display issues.
commit 0ed752e087
Merge: ccc6a06a8518edcb85
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Jul 4 18:14:21 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit ccc6a06a8d
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Jul 4 18:04:34 2020 +0200
Some minor changes.
commit 4e02024670
Merge: 50a64d927e05c9267a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Jul 4 16:17:40 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit 50a64d927e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Jul 4 13:07:03 2020 +0200
Fix far plane.
commit 7dd06da34c
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Jul 2 22:25:35 2020 +0200
Add shadows.glsl.
commit 618a18c998
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Jul 2 22:10:22 2020 +0200
Adding shadows, greedy meshing, and more.
commit eaea83fe6a
Merge: 2670184952f89b863e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu May 21 22:47:07 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 2670184954
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu May 21 21:20:01 2020 +0200
Make civsim and sites deterministic.
For anything in worldgen where you use a HashMap, *please* think
carefully about which hasher you are going to use! This is
especially true if (for some reason) you are depending on hashmap
iteration order remaining stable for some aspect of worldgen.
commit f8376fd5dc
Merge: 654f7e049cdee191dd
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu May 21 17:53:57 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 654f7e0492
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed May 20 21:22:30 2020 +0200
Correct backface culling.
commit 560501df05
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue May 19 17:22:06 2020 +0200
Greedy messhing for shadows.
commit a4d87e1875
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun May 17 05:59:00 2020 +0200
Shadow maps work for lantern.
commit 243d0837b8
Merge: 04382dc2871dd520cd
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri May 15 14:53:13 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 04382dc286
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri May 15 14:22:17 2020 +0200
WIP: better graphics config, better LOD, shadow maps.
commit 22ddbad3eb
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat May 2 18:54:09 2020 +0200
Minor shader fixes.
commit 746a10e8d0
Merge: 0f4a0e76340ab94673
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat May 2 04:02:09 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 0f4a0e763d
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri May 1 23:03:24 2020 +0200
Switch back to pop-in terrain.
commit dd74fa7e4a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri May 1 22:58:55 2020 +0200
LOD shading closer to voxel shading.
commit ef67bd58ba
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Apr 28 20:49:03 2020 +0200
Experimental underwater lighting.
commit 2c5ad9d076
Merge: 748279835303967a6f
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Apr 27 22:35:24 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 7482798354
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Apr 27 21:59:55 2020 +0200
Replace discard in figure-frag.
commit d83b4ae69b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Apr 27 18:45:57 2020 +0200
Fix sprite lighting, HDR from focus_pos.
commit 0594238004
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Apr 27 18:14:10 2020 +0200
Proper HDR from point lights.
commit 48c93d2b41
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Apr 27 14:01:43 2020 +0200
Brighter ambiance, darker LOD shadows.
commit e0452e895c
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Apr 27 13:13:23 2020 +0200
More proper HDR.
commit 4c6da3ed16
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Apr 27 00:13:10 2020 +0200
Trying LOD noise.
commit 682a3d74c8
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Apr 26 23:11:08 2020 +0200
Fix LOD heights in towns.
commit cc39e5734e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sun Apr 26 21:01:23 2020 +0200
More LOD fixes.
commit 8116b21c2e
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 25 23:54:43 2020 +0200
I like this coloring.
commit bc2560ea90
Merge: 14effdd5de690efe71
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 25 23:48:33 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit 14effdd5db
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 25 22:24:35 2020 +0200
Re-saturate.
commit 48a643955d
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 25 22:23:57 2020 +0200
Various fixes.
commit f7b497a0c2
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 25 03:22:49 2020 +0200
Render figures again.
commit 44e4aad48d
Merge: e6f0a5a989ec319a18
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 25 02:01:04 2020 +0200
Merge remote-tracking branch 'origin/master' into sharp/small-fixes
commit e6f0a5a981
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Apr 24 16:12:20 2020 +0200
Add atmospheric scattering.
commit f2953087f6
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 23 00:01:20 2020 +0200
Fix shadowing for specular reflections.
commit ddd4a67a97
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Wed Apr 22 22:56:12 2020 +0200
HDR fixes.
commit 1015e60dee
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Tue Apr 21 18:25:19 2020 +0200
More lighting changes.
commit 80c264abd1
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Mon Apr 13 00:29:59 2020 +0200
Lighting experiments.
commit 8414987e58
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 9 02:38:40 2020 +0200
WIP -- lighting changes and soft shadows.
commit 9cd2b3fb0d
Merge: c7ea687eb8b149ad11
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 4 02:33:29 2020 +0200
Merge branch 'sharp/new-lighting' into sharp/small-fixes
commit c7ea687ebb
Merge: 47644153122f3319b4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 4 02:33:02 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/small-fixes
commit 8b149ad11a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Sat Apr 4 02:32:39 2020 +0200
Trying out a new lighting model.
commit b0ac9f36f7
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Apr 3 07:56:11 2020 +0200
Use bicubic interpolation for terrain.
commit f6fc9307a1
Merge: 33140a29522f3319b4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Apr 3 05:01:41 2020 +0200
Merge remote-tracking branch 'origin/master' into zesterer/lod
commit 4764415312
Merge: ed2d0111d13388ee6a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Apr 3 04:54:48 2020 +0200
Merge branch 'sharp/map-colors' into sharp/small-fixes
commit 13388ee6a4
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 2 20:30:08 2020 +0200
Various fixes (to coloring and to soft shadows).
commit fbd084a94a
Merge: 5a089863b4fdf6896a
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 2 18:50:38 2020 +0200
Merge branch 'master' of gitlab.com:veloren/veloren into sharp/map-colors
commit ed2d0111d9
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 2 06:49:27 2020 +0200
Combining colors and LOD.
commit 88342640c6
Merge: 33140a2955a089863b
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 2 04:49:20 2020 +0200
Merge branch 'sharp/map-colors' into sharp/small-fixes
commit 33140a2951
Merge: 4c65a5aedf34d4b379
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 2 04:36:21 2020 +0200
Merge remote-tracking branch 'origin/master' into zesterer/lod
commit 5a089863be
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 2 03:17:49 2020 +0200
Making maps brighter.
This is probably not the right way to do it, but oh well!
commit 32b2c99109
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Thu Apr 2 02:46:36 2020 +0200
Horizon mapping and "layered" map generation.
Horizon mapping is a method of shadow mapping specific to height maps.
It can handle any angle between 0 and 90 degrees from the ground, as
long as know the horizontal direction in advance, by remembering only a
single angle (the "horizon angle" of the shadow map). More is explained
in common/src/msg/server.rs. We also remember the approximate height of
the largest occluder, to try to be able to generate soft shadows and
create a vertical position where the shadows can't go higher.
Additionally, map generation has been reworked. Instead of computing
everything from explicit samples, we pass in sampling functions that
return exactly what the map generator needs. This allows us to cleanly
separate the way we sample things like altitudes and colors from the map
generation process. We exploit this to generate maps *partially* on the
server (with colors and rivers, but not shading). We can then send the
partially completed map to the client, which can combine it with shadow
information to generate the final map. This is useful for two reasons:
first, it makes sure the client can apply shadow information by itself,
and second, it lets us pass the unshaded map for use with level of
detail functionality.
For similar reasons, river generation is split
out into its own layer, but for now we opt to still generate rivers on
the server (since the river wire format is more complicated to compress
and may require some extra work to make sure we have enough precision to
draw rivers well enough for LoD).
Finally, the mostly ad-hoc lighting we were performing has been (mostly)
replaced with explicit Phong reflection shading (including specular
highlights). Regularizing this seems useful and helps clarify the
"meaning" of the various light intensities, and helps us keep a more
physically plausible basis. However, its interaction with soft shadows
is still imperfect, and it's not yet clear to me what we need to do to
turn this into something useful for LoD.
commit f8926a5737
Merge: a1aee931e875ae6ced
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Mar 13 13:32:42 2020 +0100
Merge remote-tracking branch 'origin/master' into sharp/map-colors
commit 4c65a5aed3
Author: Treeco <5021038-Treeco@users.noreply.gitlab.com>
Date: Mon Feb 24 16:48:05 2020 +0000
Made LOD setting slider exponential
commit 2fa7b2d20d
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Mon Feb 24 17:49:53 2020 +0000
Added mist to LoD
commit aab059a450
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Mon Feb 24 15:14:06 2020 +0000
Added LoD slider
commit 779c36b538
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Mon Feb 24 12:54:55 2020 +0000
Reduced cost of vertex pushing
commit 9fea150473
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Mon Feb 24 12:38:53 2020 +0000
Fixed maths, improved LoD resolution
commit 5481df38fe
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Mon Feb 24 11:22:50 2020 +0000
Dynamically relocate LoD vertices to enhance details
commit a3e36a50ab
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Sun Feb 23 18:13:51 2020 +0000
Simpler terrain spiral rendering
commit 255f450ae9
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Sun Feb 23 16:53:17 2020 +0000
Better LoD precision
commit 3d027aebe8
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Sun Feb 23 16:04:03 2020 +0000
Better falloff
commit be775c9484
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Sun Feb 23 15:30:45 2020 +0000
Applied good ideas from experimental branch
commit 58587b6854
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Fri Feb 21 16:15:13 2020 +0000
Minor fixes to LoD merging
commit 7b42aebd70
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Fri Feb 21 15:04:44 2020 +0000
Capped LoD dragging
commit 8aafc559f8
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Fri Feb 21 14:54:37 2020 +0000
Better blending between LoD and terrain border
commit edd3455d51
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Fri Feb 21 14:40:19 2020 +0000
Fixed LoD z depth, added sea level offset
commit b9b0674462
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Fri Feb 21 14:27:43 2020 +0000
Better LoD smoothing
commit a1aee931e7
Author: Joshua Yanovski <pythonesque@gmail.com>
Date: Fri Feb 21 14:52:17 2020 +0100
Adding shadows.
commit 2400786c13
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Fri Feb 21 13:48:40 2020 +0000
Use world map as LoD source
commit dbf650f504
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Mon Jan 20 00:48:14 2020 +0000
Better clouds at distance
commit 5e6f81b86c
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Sun Jan 19 23:59:02 2020 +0000
sync
commit 745e7540dd
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Fri Nov 22 12:40:48 2019 +0000
Improved cloud falloff mist, faster noise sampling
commit f6a200d0cb
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Fri Nov 22 10:09:00 2019 +0000
Improved long-range depth precision, removed unnecessary LoD polygons
commit 63d1b2bb22
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Thu Nov 21 20:57:46 2019 +0000
Working LoD shader
commit f13d98ee3e
Author: Joshua Barretto <joshua.s.barretto@gmail.com>
Date: Thu Nov 21 11:03:40 2019 +0000
LoD first attempt (stack overflow issue)
increase the slowlorris timeout. for some reason it seems to trigger alot more often since commit: `75c1d440` but i have no idea why.
My guess would be that the initial sync now sends way more data which slows down TCP to be > 10ms and trigger. Note: the fix might cause small lags when slow people try to connect to the server
The remote site should see it connect, be open for 1 single stream and read the message before it's notified that the participant is closed actually.
This caused the faulure in one of our API tests (in lib, with client and server). Where it was possible that all messages were send and one side was dropped before the other side asked for the opened stream
Also introduce better error detection in participant(and scheduler) by removing the std_async::Result and intruduce `Result<(),ParticipantError>` instead
We MUST handle them and we are not allowed to act on a stream after it failed, as i am to lazy to change the structure to ensure the client to be imeadiatly dropped i added a AtomicBool to it.
- we had the problem that Participants couldn't shutdown them self, only by scheduler, which was controlled by api.
it's needed e.g. to handle the Schudown Frame
- my initial solution did a full shutdown, which was a problem if in parallel a 2nd shutdown was requested, no possibility of getting the error
- new solution will only deactivate Participant and Stream. and then still functions correctly, till the api closes the participant and calls the scheduler which then calls the bparticipant again
- i experimented with a Mutex<oneshot> or 2 and a `select` but it didn't prove that well
- also adjusted the Error messages to now either Disconnected when gracefully shutdown or ProtocolFailed when some msg couldn't be delivered
(note later might not be 100% returned correctly yet)
- make it harder for the server to crash and also kill invalid sessions properly (instead of waiting for them to close)
- introduce macros to reduce code duplication
- added tests to check for valid handshake as well as garbage tcp
- voxygen abort when the server has a invalid veloren_network handshake, e.g. by outdated version instead of try again
- rename Network `Address` to `ProtocolAddr` as sugested by zest as it's a combination of Protocol and std::io::Addr
- remove the manual byte arrays in `protocols.rs` with something more nice
- API behavior switched!
- the `Network` no longer holds a copy of participant, thus if the return of `connect` (before `Arc<Participant>, now `Participant`) got dropped, the `Participant::Drop` is triggered!
- you can close a Participant async via `Particiant::disconnect()`, no more need to know the network at this point
- the `Network::Drop` will check and drop not yet disconnected Participants.
- you can compare Participants via PartialEq, if they are true they point to the same endpoint (it checks remote_pid)
- Note: multiple Participants are only supported in theory, wont work yet
Additionally:
- fix some `debug!`
- veloren-client will now drop the participant gracefully on shutdown
- rename `error` to `debug` when 2 times Bparticipant shutdown is called, as it is to be expected in a async runtime
- replace serde_derive by feature of serde
incl. source code modifications to compile
- reduce futures-timer to "2.0" to be same as async_std
- update notify
- removed mio, bincode and lz4 compress in common as networking is now in own crate
btw there is a better lz4 compress crate, which is newer than 2017
- update prometheus to 0.9
- can't update uvth yet due to usues
- hashbrown to 7.2 to only need a single version
- libsqlite3 update
- image didn't change as there is a problem with `image 0.23`
- switch old directories with newer directories-next
- no num upgrade as we still depend on num 0.2 anyways
- rodio and cpal upgrade
- const-tewaker update
- dispatch (untested) update
- git2 update
- iterations update
- swap out std::mpsc with crossbeam-channel in networking crate
- remove log spam by only logging when populating a new cache entry and not on every get
- switched the behavior of prio to keep the order of a stream, EVEN if messages of different length are used!
This needs to be addresses to fulfill PROMISES_ORDERED
I think we need to make thoughs if PRIO needs to handle this, or we add additional meta info and handle it on receiver site
- authc no longer uses reqwest
- image only supports PNG
- replace routille with tiny_http
- several other dependencies
- cargo upgrade
- following improvement was measured on R7 1700X:
before:
- cargo build: 3076.73s user / 4:45 total / 589 dependencies
- cargo test: 6118.38s user / 7:30 total / 959 dependencies
after:
- cargo build: 2680.54s user / 4:05 total / 480 dependencies
- cargo test: 5351.81s user / 7:04 total / 791 dependencies
- added xMAC94x to CODEOWNERS for Cargo.toml, he will protect them from now on and hit people with evil looks ;)
- added PartialEq to StreamError for test purposes (only yet!)
- removed async_recv example as it's no longer for any use.
It was created before the COMPLETE REWRITE in order to verify that my own async interface on top of mio works.
However it's now guaranteed by async-std and futures. no need for a special test
- remove uvth from dependencies and replace it with a `FnOnce`
- fix ALL clippy (network) lints
- basic fix for a channel drop scenario:
TODO: this needs some further fixes
up to know only destruction of participant by api was covered correctly.
we had an issue when the underlying channels got dropped. So now we have a participant without channels.
We need to buffer the requests and try to reopen a channel ASAP!
If no channel could be reopened we need to close the Participant, while
a) leaving the BParticipant in takt, knowing that it only waits for a propper close by scheduler
b) close the BParticipant gracefully. Notifying the scheduler to remove its stuff (either scheduler schould detect a stopped BParticipant or BParticipant will send Scheduler it's own destruction, and then Scheduler just does the same like when API forces a close)
Keep the Participant alive and wait for the api to acces BParticipant to notice it's closed and then wait for a disconnect which isn't doing anything as it was already cleaned up in the background
- this bug was initially called imbris bug, as it happened on his runners and i couldn't reproduce it locally at fist :)
- When in a Handshake a seperate mpsc::Channel was created for (Cid, Frame) transport
however the protocol could already catch non handshake data any more and push in into this
mpsc::Channel.
Then this channel got dropped and a fresh one was created for the network::Channel.
These droped Frames are ofc a BUG!
I tried multiple things to solve this:
- dont create a new mpsc::Channel, but instead bind it to the Protocol itself and always use 1.
This would work theoretically, but in bParticipant side we are using 1 mpsc::Channel<(Cid, Frame)>
to handle ALL the network::channel.
If now ever Protocol would have it's own, and with that every network::Channel had it's own it would no longer work out
Bad Idea...
- using the first method but creating the mpsc::Channel inside the scheduler instead protocol neither works, as the
scheduler doesnt know the remote_pid yet
- i dont want a hack to say the protocol only listen to 2 messages and then stop no matter what
So i switched over to the simply method now:
- Do everything like before with 2 mpsc::Channels
- after the handshake. close the receiver and listen for all remaining (cid, frame) combinations
- when starting the channel, reapply them to the new sender/listener combination
- added tracing
- switched Protocol RwLock to Mutex, as it's only ever 1
- Additionally changed the layout and introduces the c2w_frame_s and w2s_cid_frame_s name schema
- Fixed a bug in scheduler which WOULD cause a DEADLOCK if handshake would fail
- fixd a but in api_send_send_main, i need to store the stream_p otherwise it's immeadiatly closed and a stream_a.send() isn't guaranteed
- add extra test to verify that a send message is received even if the Stream is already closed
- changed OutGoing to Outgoing
- fixed a bug that `metrics.tick()` was never called
- removed 2 unused nightly features and added `deny_code`
fix async_recv and double block_on panic on Network::drop and participant::drop
include Cargo.lock from all examples
Found a bug on imbris runners with doc tests of `stream::send` and `stream::recv`
As neither a backtrace, nor tracing on runners in the doc tests seems to help, i disable them and add them as unit tests
however i need to coordinate the prio adjustments in scheduler from now on, so that ParticipantA doesn't get all the network bandwith and ParticipantB nothing
- https://github.com/tikv/rust-prometheus/issues/321
- split up channel into a hanshake part and channel part.
The handshake part is non endless and ends when its either done or aborted.
If its okay i will send a request to the BParticipant which then opens a channel on the existing TCP or UDP connection.
this streamlines the command chain alot. also the channel is almost empty now, thinking about removing it completly.
isnt perfect, as shutdown and udp doesnt work yet
- make PID to print as Base64
- replace rouille with tiny_http
- removing async_serde as it seems to be not usefull
the idea was because deserialising is slow parallising it could speed up.
Whoever we need to keep the order of frames, (at least for controlframes) so serialising in threads would be quite complicated.
Also serialisation is quite fast, about 1 Gbit/s such speed is enough for messaging, it's more important to serve parallel streams better.
Thats why i am removing async serde coding for now
- frames are no longer serialized by serde, by byte by byte manually, increadible speed upgrade
- more metrics
- switch channel_creator into for_each_concurrent
- removing some pool.spwan_ok() as they dont allow me to use self
- reduce features needed
- switch `listen` to async in oder to verify if the bind was successful
- Introduce the following examples
- network speed
- chat
- fileshare
- add additional tests
- fix dropping stream before last messages can be handled bug, when dropping a stream, BParticipant will wait for prio to be empty before dropping the stream and sending the signal
- correct closing of stream and participant
- move tcp to protocols and create udp front and backend
- tracing and fixing a bug that is caused by not waiting for configuration after receiving a frame
- fix a bug in network-speed, but there is still a bug if trace=warn after 2.000.000 messages the server doesnt get that client has shut down and seems to lock somewhere. hard to reproduce
open tasks
[ ] verify UDP works correctly, especcially the connect!
[ ] implements UDP shutdown correctly, the one created in connect!
[ ] unify logging
[ ] fill metrics
[ ] fix dropping stream before last messages can be handled bug
[ ] add documentation
[ ] add benchmarks
[ ] remove async_serde???
[ ] add mpsc
- We can now get rid of most sleep and get true remote part and stream working, however there seems to be a deadlock after registered new handle trace with 10% spawn chance
- removal of the events trait, as we use channels
- streams now directly communicate with each other for performance reasons, somewhere are still deadlocks, oonce directly at listening somehow and after the first message has read, but i also got it to run perfectly through at this state without code change, maybe a sleep or more detailed rust-dgb session would help here!
It should compile and tests run fine now.
If not, the 2nd last squashed commit message said it currently only send frames but not incomming messages, also recv would only handle frames. The last one said i added internal messages and a reverse path (prob for .recv)
- introduce a loadtest, for tcp messages
- cleanup api
- added a unittest
- prepared a handshake message, which will in next commits get removed again
- experimental mio worker merges
- using uuid for participant id
