IO Tracer and Parser

[In Progress]

On Posix systems we can use existing tracing mechanism(strace etc.). But on storage systems where we cannot use existing tracing tools, we added a mechanism to trace IO operations to understand IO behavior of RocksDB while accessing data on the storage.

Table of Contents

• Quick Start
  
• IO Trace Format
  
• Usage
  
• IO Tracer Parser
  

IO Trace Format

IO trace record contains following information:

Required for all records:

Column Name	Values	Comment
Access timestamp in microseconds	unsigned long
File Operation	string	type of operation (Append, Read,...).
Latency	unsigned long
IO Status		IO Status of the file operation returned.

Based on File Operation:

Column Name	Values	Comment
File Name	string
Length	unsigned long
Offset	unsigned long
File Size	unsigned long

Usage

An example to start IO tracing:

Env* env = rocksdb::Env::Default();
EnvOptions env_options;
std::string trace_path = "/tmp/binary_trace_test_example”;
std::unique_ptr<TraceWriter> trace_writer;
DB* db = nullptr;
std::string db_name = "/tmp/rocksdb”;

/*Create the trace file writer*/
NewFileTraceWriter(env, env_options, trace_path, &trace_writer);

DB::Open(options, dbname);

/*Start IO tracing*/
db->StartIOTrace(env, trace_opt, std::move(trace_writer));

/*Your call of RocksDB APIs */

/*End IO tracing*/
db->EndIOTrace();

IO Tracer Parser

The trace file generated from IO tracing is in binary format. So parser can be used to read that binary trace file

./io_tracer_parser -io_trace_file trace_file

Implementation details can be found in https://github.com/facebook/rocksdb/tree/master/tools/io_tracer_parser_tool.h