Tiered Storage (Experimental)

How to use

RocksDB Tiered storage feature can now assign the data to various types of storage media based on the temperature of the data (how hot the data is) within the same db column family. For example, the user can set the temperate of the last level to cold:

AdvancedColumnFamilyOptions.last_level_temperature = Temperature::kCold

Then the temperature information will be passed to the FileSystem APIs like NewRandomAccessFile(), NewWritableFile(), etc. It's up to the user to place the file in its corresponding storage with the implementation of its own FileSystem. Also use the temperature information to find the file in corresponding storage. In general, the high levels data are written most recently and more likely to be hot. Also high level data is much more likely to go though compaction, having them in a faster storage media can improve the compaction process. Currently, only the last level temperature can be specified. Which has its limitation, for example for a skewed data set, the hot data set may be compacted frequently and compacted to the last level. To prevent that, a per-key based hot/cold data splitting compaction is introduced.

Tiered Storage Compaction

User can specify the hot data

AdvancedColumnFamilyOptions.preclude_last_level_data_seconds = 259200 // 3 days

Limitations