compaction_algo.txt

Compaction algorithm (Adopted from Apache Cassandra Sized Tier Compaction Stratedgy STCS)

  - For exisitng buckets that does not meet treshold, just skip them

  - For buckets that contains more than the treshold, collect treshold size for compaction and leave the rest
    NOTE1: We also collect the Bucket ID and SSTable filepaths of the tables selected for compaction so that we can delete them from previous bucket where they existed 
    NOTE2: You cannot delete the sstbales until they have been compacted and written to disk to prevent data loss in case of outage

  - Compact the SSTables and write them to buckets

  - Once write is complete, you can delete the sstables from the bucket it once exisited and recalculate the average size of this bucket for the future
    NOTE: The deletion should include both from the disk and in-memory buckets