Mirrored_Repos/veloren
1
0
Fork 0
mirror of https://gitlab.com/veloren/veloren.git synced 2024-08-30 18:12:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
198b1970a7
veloren/network/Cargo.toml

44 lines
1.5 KiB
TOML
Raw Normal View History

initial implementation of a network api
2019-12-20 13:56:01 +00:00
[package]
shutdown and udp/mpsc - theorectically closing of streams and shutdown - mpsc and udp preparations - cleanup and build better tests
2020-03-10 00:07:36 +00:00
name = "veloren_network"
Propper Compression support of network. - Compression is no longer enabled always but can now be enabled per Stream. If a Stream is Compression enabled it will compress and decompress all msg (except for `raw` access) before handling them internally. You need to handle compression yourself for `raw` fn. - added a new feature to the network crate to enable or disable the compression - switched to `lz-fear` instead of `lz4-compression` - use `bitflags` to represent the `Promises` struct
2020-08-25 13:32:42 +00:00
version = "0.2.0"
initial implementation of a network api
2019-12-20 13:56:01 +00:00
authors = ["Marcel Märtens <marcel.cochem@googlemail.com>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
make `prometheus` optional in network and fix a panic in the server - an extra interface `new_with_regisitry` was created to make sure the interface doesn't depend on the features
2020-07-14 23:34:41 +00:00
[features]
metrics = ["prometheus"]
Propper Compression support of network. - Compression is no longer enabled always but can now be enabled per Stream. If a Stream is Compression enabled it will compress and decompress all msg (except for `raw` access) before handling them internally. You need to handle compression yourself for `raw` fn. - added a new feature to the network crate to enable or disable the compression - switched to `lz-fear` instead of `lz4-compression` - use `bitflags` to represent the `Promises` struct
2020-08-25 13:32:42 +00:00
compression = ["lz-fear"]
make `prometheus` optional in network and fix a panic in the server - an extra interface `new_with_regisitry` was created to make sure the interface doesn't depend on the features
2020-07-14 23:34:41 +00:00
Propper Compression support of network. - Compression is no longer enabled always but can now be enabled per Stream. If a Stream is Compression enabled it will compress and decompress all msg (except for `raw` access) before handling them internally. You need to handle compression yourself for `raw` fn. - added a new feature to the network crate to enable or disable the compression - switched to `lz-fear` instead of `lz4-compression` - use `bitflags` to represent the `Promises` struct
2020-08-25 13:32:42 +00:00
default = ["metrics","compression"]
make `prometheus` optional in network and fix a panic in the server - an extra interface `new_with_regisitry` was created to make sure the interface doesn't depend on the features
2020-07-14 23:34:41 +00:00
initial implementation of a network api
2019-12-20 13:56:01 +00:00
[dependencies]
Code/Dependency Cleanup
2020-03-04 15:52:30 +00:00
#serialisation
initial implementation of a network api
2019-12-20 13:56:01 +00:00
bincode = "1.2"
switch from serde to manually for speed, remove async_serde - removing async_serde as it seems to be not usefull the idea was because deserialising is slow parallising it could speed up. Whoever we need to keep the order of frames, (at least for controlframes) so serialising in threads would be quite complicated. Also serialisation is quite fast, about 1 Gbit/s such speed is enough for messaging, it's more important to serve parallel streams better. Thats why i am removing async serde coding for now - frames are no longer serialized by serde, by byte by byte manually, increadible speed upgrade - more metrics - switch channel_creator into for_each_concurrent - removing some pool.spwan_ok() as they dont allow me to use self - reduce features needed
2020-04-24 10:56:04 +00:00
serde = { version = "1.0" }
Code/Dependency Cleanup
2020-03-04 15:52:30 +00:00
#sending
crossbeam-channel and log spam - swap out std::mpsc with crossbeam-channel in networking crate - remove log spam by only logging when populating a new cache entry and not on every get
2020-07-03 19:55:00 +00:00
crossbeam-channel = "0.4.2"
updated dependencies and fixed stuff - replace serde_derive by feature of serde incl. source code modifications to compile - reduce futures-timer to "2.0" to be same as async_std - update notify - removed mio, bincode and lz4 compress in common as networking is now in own crate btw there is a better lz4 compress crate, which is newer than 2017 - update prometheus to 0.9 - can't update uvth yet due to usues - hashbrown to 7.2 to only need a single version - libsqlite3 update - image didn't change as there is a problem with `image 0.23` - switch old directories with newer directories-next - no num upgrade as we still depend on num 0.2 anyways - rodio and cpal upgrade - const-tewaker update - dispatch (untested) update - git2 update - iterations update
2020-07-06 14:23:08 +00:00
# NOTE: Upgrading async-std can trigger spontanious crashes for `network`ing. Consider elaborate tests before upgrading
dependency reduction: - authc no longer uses reqwest - image only supports PNG - replace routille with tiny_http - several other dependencies - cargo upgrade - following improvement was measured on R7 1700X: before: - cargo build: 3076.73s user / 4:45 total / 589 dependencies - cargo test: 6118.38s user / 7:30 total / 959 dependencies after: - cargo build: 2680.54s user / 4:05 total / 480 dependencies - cargo test: 5351.81s user / 7:04 total / 791 dependencies - added xMAC94x to CODEOWNERS for Cargo.toml, he will protect them from now on and hit people with evil looks ;)
2020-06-01 00:13:35 +00:00
async-std = { version = "~1.5", default-features = false, features = ["std", "async-task", "default"] }
Code/Dependency Cleanup
2020-03-04 15:52:30 +00:00
#tracing and metrics
switch from serde to manually for speed, remove async_serde - removing async_serde as it seems to be not usefull the idea was because deserialising is slow parallising it could speed up. Whoever we need to keep the order of frames, (at least for controlframes) so serialising in threads would be quite complicated. Also serialisation is quite fast, about 1 Gbit/s such speed is enough for messaging, it's more important to serve parallel streams better. Thats why i am removing async serde coding for now - frames are no longer serialized by serde, by byte by byte manually, increadible speed upgrade - more metrics - switch channel_creator into for_each_concurrent - removing some pool.spwan_ok() as they dont allow me to use self - reduce features needed
2020-04-24 10:56:04 +00:00
tracing = { version = "0.1", default-features = false }
COMPLETE REWRITE - use async_std and implement a async serialisaition - new participant, stream and drop on the participant - sending and receiving on streams
2020-03-22 13:47:21 +00:00
tracing-futures = "0.2"
ci cleanup, dependency update
2020-11-06 13:14:54 +00:00
prometheus = { version = "0.10", default-features = false, optional = true }
Code/Dependency Cleanup
2020-03-04 15:52:30 +00:00
#async
COMPLETE REWRITE - use async_std and implement a async serialisaition - new participant, stream and drop on the participant - sending and receiving on streams
2020-03-22 13:47:21 +00:00
futures = { version = "0.3", features = ["thread-pool"] }
shutdown and udp/mpsc - theorectically closing of streams and shutdown - mpsc and udp preparations - cleanup and build better tests
2020-03-10 00:07:36 +00:00
#mpsc channel registry
switch from serde to manually for speed, remove async_serde - removing async_serde as it seems to be not usefull the idea was because deserialising is slow parallising it could speed up. Whoever we need to keep the order of frames, (at least for controlframes) so serialising in threads would be quite complicated. Also serialisation is quite fast, about 1 Gbit/s such speed is enough for messaging, it's more important to serve parallel streams better. Thats why i am removing async serde coding for now - frames are no longer serialized by serde, by byte by byte manually, increadible speed upgrade - more metrics - switch channel_creator into for_each_concurrent - removing some pool.spwan_ok() as they dont allow me to use self - reduce features needed
2020-04-24 10:56:04 +00:00
lazy_static = { version = "1.4", default-features = false }
rand = { version = "0.7" }
Propper Compression support of network. - Compression is no longer enabled always but can now be enabled per Stream. If a Stream is Compression enabled it will compress and decompress all msg (except for `raw` access) before handling them internally. You need to handle compression yourself for `raw` fn. - added a new feature to the network crate to enable or disable the compression - switched to `lz-fear` instead of `lz4-compression` - use `bitflags` to represent the `Promises` struct
2020-08-25 13:32:42 +00:00
#stream flags
bitflags = "1.2.1"
lz-fear = { version = "0.1.1", optional = true }
Add a speedtest program to benchmark networking
2020-02-25 18:30:50 +00:00
[dev-dependencies]
StreamError::Deserialize is now triggered when `recv` fails because of wrong type - added PartialEq to StreamError for test purposes (only yet!) - removed async_recv example as it's no longer for any use. It was created before the COMPLETE REWRITE in order to verify that my own async interface on top of mio works. However it's now guaranteed by async-std and futures. no need for a special test - remove uvth from dependencies and replace it with a `FnOnce` - fix ALL clippy (network) lints - basic fix for a channel drop scenario: TODO: this needs some further fixes up to know only destruction of participant by api was covered correctly. we had an issue when the underlying channels got dropped. So now we have a participant without channels. We need to buffer the requests and try to reopen a channel ASAP! If no channel could be reopened we need to close the Participant, while a) leaving the BParticipant in takt, knowing that it only waits for a propper close by scheduler b) close the BParticipant gracefully. Notifying the scheduler to remove its stuff (either scheduler schould detect a stopped BParticipant or BParticipant will send Scheduler it's own destruction, and then Scheduler just does the same like when API forces a close) Keep the Participant alive and wait for the api to acces BParticipant to notice it's closed and then wait for a disconnect which isn't doing anything as it was already cleaned up in the background
2020-06-08 09:47:39 +00:00
tracing-subscriber = { version = "0.2.3", default-features = false, features = ["env-filter", "fmt", "chrono", "ansi", "smallvec"] }
updated dependencies and fixed stuff - replace serde_derive by feature of serde incl. source code modifications to compile - reduce futures-timer to "2.0" to be same as async_std - update notify - removed mio, bincode and lz4 compress in common as networking is now in own crate btw there is a better lz4 compress crate, which is newer than 2017 - update prometheus to 0.9 - can't update uvth yet due to usues - hashbrown to 7.2 to only need a single version - libsqlite3 update - image didn't change as there is a problem with `image 0.23` - switch old directories with newer directories-next - no num upgrade as we still depend on num 0.2 anyways - rodio and cpal upgrade - const-tewaker update - dispatch (untested) update - git2 update - iterations update
2020-07-06 14:23:08 +00:00
# `uvth` needed for doc tests
Switch veloren_network over to use the official example layout. adjusted those examples to run again created a CI TEST to always `check` examples fixed fmt in examples so that pipeline gets green
2020-08-26 08:07:22 +00:00
uvth = { version = ">= 3.0, <= 4.0", default-features = false }
clap = { version = "2.33", default-features = false }
shellexpand = "2.0.0"
FIX for hanging participant deletion. There is a rare bug that recently got triggered more often with the release of xMAC94x/netfixA if the bug triggeres, a Participant never gets cleaned up gracefully. Reason: When `participant_shutdown_mgr` was called it stopped all managers at once. Especially stream_close_mgr and send_mgr. The problem with stream_close_mgr is, it's responsible for gracefully flushing streams when the Participant is dropped locally. So when it was interupted self.streams where no longer flushed gracefully. The next problem was with send_mgr. It is triggering the PrioManager, and the PrioManager is responsible for notifying once a stream is completly flushed. This lead to the problem, that a stream flush could be requested, but was actually never executed (as send_mgr was already down). Solution: 1. when stream_close_mgr is stopped it MUST flush all remaining streams 2. wait for stream_close_mgr to finish before shutting down the send_mgr 3. no longer delete streams when closing the API (this also wasn't tracked in metrics so far) Additionally i added a dependency, so that the network/examples compile again, fixed some spelling. I created a `delete_stream` fn that basically just moved the code over.
2020-10-14 12:29:39 +00:00
tiny_http = "0.7.0"
serde = { version = "1.0", features = ["derive"] }
Reference in New Issue Copy Permalink