UCP: Move rocksdb info log to store and rotate as tikv log

[yiwu-arbug]

Description

Currently rocksdb log are stored under rocksdb directory, and the rotation logic is different to tikv log. Rocksdb rotate the log on every restart, and purge old logs except for the last 10 files. This make rocksdb log purge away if tikv fall in a restart loop. We can implement the Logger interface of rocksdb, which reuse tikv log logic to store rocksdb log at the same directory of tikv log, and use the same rotation logic. It will also make rocksdb log easier to find.

It is to solve the problem where tikv restart make rocksdb log rotate away, making rocksdb issues hard to investigate. Related issue #6495

Task breakdown:

• In https://github.com/tikv/rust-rocksdb repo, add interface corresponds to the Logger interface in rocksdb (https://github.com/facebook/rocksdb/blob/master/include/rocksdb/env.h#L976) to allow we implement the interface in Rust. Take src/eventlistener.rs for example to see how it able to allow implementing the EventListener c++ interface in Rust. Add set_info_log method to DBOptions in src/rocksdb_options.rs correspondingly.
• In https://github.com/tikv/tikv repo, implement Logger in components/engine_rocks/src/logger.rs (create a new file). The implementation should reuse logic in https://github.com/tikv/tikv/blob/master/components/tikv_util/src/logger/mod.rs which is used to store and rotate tikv log. In src/config.rs, set the new logger implementation to rocksdb options.
• Test your change manually. RocksDB log (for both raftdb and kvdb) should now appear under tikv.log's directory. Restart TiKV, and see rocksdb log is appending to the same file as last time (instead of opening a new file). The log file should rotate daily, with file name rename to corresponding date.

Difficulty

• Medium

Score

• 1700

Mentor(s)

• @yiwu-arbug

Recommended Skills

• C++
• Rust

[hawkingrei]

/pick-up-challenge

[sre-bot]

@hawkingrei pick up issue success

[sre-bot]

This pick has been automatically canceled after more than a week.

[yiwu-arbug]

@you06 the task is being worked on actively. can this task being re-picked?

[you06]

@yiwu-arbug It's an accident and the pick has been recovered.

[yiwu-arbug]

@hawkingrei thank you for the hard work to finish the task.