Skip to content

Write child compaction wiki #6

@rjl493456442

Description

@rjl493456442

Child compaction is another killer feature of go-leveldb project.

Although the concurrent compaction is already enabled by default
which speeds up the compaction efficiency a lot, but the write stall
is still not fixed.

What's write stall

Write stall refers to the phenomenon that the system write is disabled
temporarily because of the SLOW COMPACTION. The root issue is
the level0 compaction is not fast enough.

But currently the concurrent level0 compaction is not feasible because
the files in level0 are overlapped with each other. In this case we need
some additional mechanism to improve the efficiency of level0 compaction.
I would say the efficiency of level0 compaction is the KEY of the leveldb
write performance.

Pebble has the idea called SubLevel0Compaction. The idea is quite
straightforward: create indexes for level0 files whenver the level0 content
is changed, try to schedule concurrent level0 compactions for these
"non-overlapped" files. However in practice, it's not so easy to find such
files.

So we had this idea called ChildCompaction. Let's write it down for further
discussion.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions