Compaction without tombstone

[sebheitzmann]

Pull request to merge sstables without tombstone.

Todo list :

• create a test in compaction_test to check the removal of tombstoned value
• correct compation
• add a mechanism to garbage collect outside of compaction ( or force a compact )

[thomasjungblut]

great start @sebheitzmann

[thomasjungblut]

do you need anything from me? feel free to ping me once you're ready :)

[sebheitzmann]

I think that it's ok for me. The Tombstone are correctely merge. But only if we do a "full" compaction. If not, we cannot be sure that there is no previous version in the sst not selected for merge. I think that to go further we need some sort of layer merging that is not implemented for the moment. Do you think that this can be merge for the moment ?

[thomasjungblut]

Yes, let me give it one last look. Then we can merge.

[sebheitzmann]

For my usage i will certainely do a merge with layer calculated by an key préfix. In my software the key are already a Hash ( sha1 ). So y can do some folder based on the first 4 caracters of the hash for exemple and compact the layer 0 on this subfolder. After that the layers 1 can be full compacted without risk.

But my usage is not really standard

[thomasjungblut]

code looks good, let's see if there are any failures on CI

After that the layers 1 can be full compacted without risk.

Yeah I didn't get to implement a full "LSM Tree" kind of merge layer. Glad you found some use of it though and definitely thank you tons for contributing :)

[thomasjungblut]

off we go! Thank you very much @sebheitzmann - I'll briefly need to log into my other machine to prep a new minor release, will get that out asap.

[thomasjungblut]

Done!