Projects Being Developed

The page lists major projects being actively developed, or has been planned for future development.

RocksDB on Remote Storage

User Defined Timestamps

BlobDB

BlobDB is RocksDB's implementation of key-value separation, originally inspired by the WiscKey paper. Large values (blobs) are stored in separate blob files, and only references to them are stored in RocksDB's LSM tree. By separating value storage from the LSM tree, BlobDB provides an alternative way of reducing write amplification, instead of tuning compactions. BlobDB is used in production at Facebook.

Does this belong here? What's in development?

File Checksums

See https://github.com/facebook/rocksdb/wiki/Full-File-Checksum#the-next-step

Per Key/Value Checksum

Encryption at Rest

MultiGet()

Bloom Filter Improvements

First phase complete, including

• Fixed flaws in old Bloom implementation for Block-based table with a new implementation (incl Issue 4120 and 5857), enabled using format_version=5. More detail on wiki and in the main PR.
• Allow non-integer bits / key settings for finer granularity in control
• Expose details like LSM level to custom FilterPolicy, for experimentation

Planned:

• Minimize memory internal fragmentation on generated filters (https://github.com/facebook/rocksdb/pull/6427)
• Investigate use of different bits/key for different levels (as in Monkey)
• Investigate use of alternative data structures, most likely based on perfect hashing static functions. See Xor filter, modified with "fuse graph" construction. Or even sgauss. We don't expect much difference in query times, but the primary trade-off to be between construction time and memory footprint for a given false positive rate. It's likely that L0 will continue to construct Bloom filters (fast memtable flushes) while compaction will spend more time to generate more compact structures.
• Re-vamp how filters are configured (based on above developments), probably moving away from bits/key as a proxy for accuracy.

Improving Testing

Adaptive Compaction

Improving RocksDB Backups

Including https://github.com/facebook/rocksdb/issues/6521

User-defined timestamp

https://docs.google.com/document/d/1FcDjOM8-pJzCajCa9waQkox6DKJIWZAHm36buK4wfqs/edit#heading=h.uxub5284i1ti