Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replacing Badger with Pebble DB #6515

Open
5 tasks
j1010001 opened this issue Oct 1, 2024 · 5 comments
Open
5 tasks

Replacing Badger with Pebble DB #6515

j1010001 opened this issue Oct 1, 2024 · 5 comments
Labels

Comments

@j1010001
Copy link
Member

j1010001 commented Oct 1, 2024

Why

  1. Badger DB is a project that is not maintained anymore. At some point it will not be compatible with a new version of Go and will block the upgrade of flow-go to newer version of Go language.
  2. Badger has caused memory spikes and performance issues in the past (so far mitigated with more aggressive GC GoMemLimit setting) and as the load on the DB grows these issues are likely to reoccur in the future.
  3. Badger DB does not support data pruning, so on long-running network removing unused data requires node downtime.

We have identified 4 project phases that complete the work of removing flow-go dependency on Badger DB.

Priority/phase 1

  1. 1 of 18
    Execution
    j1010001 zhangchiqing

Priority/Phase 2

  1. 0 of 3
    Execution

Tasks

  1. 0 of 2

Phase 4

@fxamacker
Copy link
Member

Badger DB is a project that is not maintained anymore.

@j1010001 did you mean that BadgerDB v2 and v3 are not maintained anymore (instead of entire BadgerDB project)?

BadgerDB released v4.0 in Feb 2023 and v4.3 in Aug 2024.

  • v4.3.0 (Aug 28, 2024)
  • v3.2103.5 (Dec 15, 2022) is the last v3.
  • v2.2007.4 (Aug 25, 2021) is the last v2 and version currently used by flow-go (go.mod).

More details and other releases at BadgerDB releases.

@bluesign
Copy link
Contributor

bluesign commented Oct 9, 2024

did you have any preliminary results with running nodes with pebble instead of badger? I am using pebble and pretty happy so far, but was thinking to check badger for faster writes, though I am scared on memory usage etc a bit.

@zhangchiqing
Copy link
Member

Unfortunately , we didn’t gather metrics for the proof-of-concept benchmark, as our focus was on ensuring execution correctness. Once this issue is completed, I will collect metrics for comparison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants