Skip to content

Concurrent GC (eliminate stop-the-world) #222

Description

@timsehn

Problem

dolt_gc() rewrites the entire chunk store file in a single atomic operation, blocking all access. Sweep phase buffers all surviving chunks in RAM.

Current behavior

  • Mark: BFS from all roots, builds in-memory hash set
  • Sweep: loads ALL surviving chunks into buffer, writes temp file, atomic rename
  • Blocks all readers and writers during entire process

Expected behavior

  • Generational GC: separate old vs new chunks
  • Concurrent collection: readers continue on old snapshot while GC builds new one
  • Incremental marking: don't buffer entire file in RAM

Impact

Repos >10GB can't GC within typical maintenance windows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions