Skip to content

Commit 55e6901

Browse files
authored
Swap fxhash for foldhash due to hash quality issues (#3111)
As the title notes, this commit replaces fxhash for foldhash as used in the cache. dashmap, due to it's sharding, has to share entropy with what's handed down to internal maps. Since `hashbrown` and by extension `std` use various sections of the high bit range for special grouping & sorting, dashmap is left with the only option to shard on low bits. This, however, presents problems, because fxhash outputs hashes of very bad quality, with only the high bits having any real entropy. This was probably a solid choice back in 2018 when we lacked other good fast alternatives. But since then `ahash` matured and we've had significant research and development in "good enough" hashing for datastructures with short keys, [the most recent step forward coming from a rather well known face][foldhash]. This improves shard selection quite a bit and reduces contention significantly. Using fxhash in a dashmap specific benchmark causes contention to go up by 3-8x when keys are k-sortable with time (Discord snowflakes) on an M1 Pro. [foldhash]: https://github.com/orlp/foldhash
1 parent f160b50 commit 55e6901

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

Cargo.toml

+2-2
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ extract_map = { version = "0.1.0", features = ["serde", "iter_mut"] }
5050
aformat = "0.1.3"
5151
bytes = "1.5.0"
5252
# Optional dependencies
53-
fxhash = { version = "0.2.1", optional = true }
53+
foldhash = { version = "0.1.4", optional = true }
5454
chrono = { version = "0.4.31", default-features = false, features = ["clock", "serde"], optional = true }
5555
flate2 = { version = "1.0.28", optional = true }
5656
zstd-safe = { version = "7.2.1", optional = true }
@@ -90,7 +90,7 @@ default_no_backend = [
9090
builder = ["tokio/fs"]
9191
# Enables the cache, which stores the data received from Discord gateway to provide access to
9292
# complete guild data, channels, users and more without needing HTTP requests.
93-
cache = ["fxhash", "dashmap"]
93+
cache = ["foldhash", "dashmap"]
9494
# Enables collectors, a utility feature that lets you await interaction events in code with
9595
# zero setup, without needing to setup an InteractionCreate event listener.
9696
collector = ["gateway"]

src/cache/wrappers.rs

+2-2
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@ impl<K: Eq + Hash, V> ReadOnlyMapRef<'_, K, V> {
8383
self.0.map_or(0, DashMap::len)
8484
}
8585
}
86-
pub struct Hasher(fxhash::FxHasher);
86+
pub struct Hasher(foldhash::fast::FoldHasher);
8787
impl std::hash::Hasher for Hasher {
8888
fn finish(&self) -> u64 {
8989
self.0.finish()
@@ -98,7 +98,7 @@ impl std::hash::Hasher for Hasher {
9898
impl typesize::TypeSize for Hasher {}
9999

100100
#[derive(Clone, Default)]
101-
pub struct BuildHasher(fxhash::FxBuildHasher);
101+
pub struct BuildHasher(foldhash::fast::RandomState);
102102
impl std::hash::BuildHasher for BuildHasher {
103103
type Hasher = Hasher;
104104

0 commit comments

Comments
 (0)