zns kv

WAL designs

• https://db.in.tum.de/~leis/papers/rethinkingLogging.pdf (see https://github.com/leanstore/leanstore)
• FoundationDB https://dl.acm.org/doi/abs/10.1145/3542700.3542707
• Append is Near, https://www.ssrc.ucsc.edu/media/pubs/8698b15f3152427d1285a995af615fbe7be26c7b.pdf
• Black-box Concurrent Data Structures for NUMA Architectures https://cs.brown.edu/people/irina/papers/asplos2017-final.pdf
• [MongoDB, uses B+ Tree] https://www.mongodb.com/blog/post/breaking-wired-tiger-logjam-write-ahead-log and https://www.mongodb.com/blog/post/breaking-wired-tiger-logjam-wait-free-solution

Open questions

• https://dl.acm.org/doi/full/10.1145/3502846 - how does it differ from ZoneFS?
• How does the design differ from ZenFS use as a simple WAL, how does ZenFS uses append?
• SILK is open source, can we use that? https://github.com/theoanab/SILK-USENIXATC2019 (looks a bit outdated)
• "Storing keys out of order on disk especially benefits write operations, and helps to achieve low tail latencies" - we should be able to show this, https://github.com/BLepers/KVell/blob/master/sosp19-final40.pdf (uses YCSB vs db_bench discussion)
• PebbleDB code is also available, https://www.cs.utexas.edu/~rak/papers/sosp17-pebblesdb.pdf

Evaluation options

• what is the micro-benchmark performance of read, write, append, and resets
• what is the WAL performance on a single page writes (with flush!?) outside the application
  • what is the typical load generated in RocksDB for WAL
  • risk: limited by the hardware performance - all will behave the same with flush latencies
• what the WAL replay time outside the application
• 6.5.2 force tale latency experiment
• 6.5.3 WA factor, is comparable so that is good.
• 6.5.4 investigate reset calls? strange behavior
• 6.5.5 heat-maps are cool but there is discrepancy - is it because of eager vs lazy zone reset management
• concurrency performance
• YCSB vs db_bench - one is only for RocksDB and other more general.

TODO

Animesh: give access to 2 node ZNS device and root access and compile 5.19. Reboot machine.

Krijn: NUMA setting clean. Check where interrupt ZNS device. Memory allocation NUMA. Check NUMA node of ZNS. Setup SPDK natively.