Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node does not start after backwards incompatible badger upgrade #4380

Closed
haikoschol opened this issue Dec 5, 2024 · 1 comment
Closed

Node does not start after backwards incompatible badger upgrade #4380

haikoschol opened this issue Dec 5, 2024 · 1 comment
Assignees
Labels
A-debug issue requires detective debug work to figure out what's going wrong. C-chaotic Unpredictable nature of this task/changes makes its chaotic. P-low this is mostly nice to have. S-network issues related to the dot/network package. T-bug this issue covers unexpected and/or wrong behaviour.

Comments

@haikoschol
Copy link
Contributor

haikoschol commented Dec 5, 2024

Describe the bug

Badger is used for a datastore in libp2p. Badger has the concept of a versioned manifest in its database files. When Badger is upgraded to a new version, the manifest version is incremented as well sometimes. In those cases, the node fails to start when trying to load a previously written libp2p datastore.

Expected Behavior

The node starts normally, maybe emitting a warning about an incompatible libp2p datastore.

Current Behavior

The node crashes with the log output given below.

Possible Solution

On startup, detect the error and delete the libp2p datastore. Probably here.

To Reproduce

Steps to reproduce the behaviour:

  1. Build a binary from commit 82433af350d2295a7f9e6e4cd1688171e19e1167.
  2. Run it and let it sync for a while (to make sure it has peers).
  3. Stop the node.
  4. Build a binary from commit a46ad52411cb95dad9e706c8c725f00ff940f36d (after this Badger upgrade)
  5. Look at the logs.

Log output

Log Output
2024-12-05 18:19:09.205	
  github.com/ChainSafe/gossamer/cmd/gossamer/main.go:33 +0x256


2024-12-05 18:19:09.205	main.main()
2024-12-05 18:19:09.205	goroutine 1 [running]:
2024-12-05 18:19:09.205		Please see https://dgraph.io/docs/badger/faq/#i-see-manifest-has-unsupported-version-x-we-support-y-error on how to fix this.
2024-12-05 18:19:09.205	panic: failed to create node services: failed to create network service: failed to create host: failed to create libp2p datastore: unsupported badger version, use github.com/ipfs/badgerds-upgrade to upgrade: manifest has unsupported version: 7 (we support 8).

Specification

  • go version: 1.23.2
  • gossamer version: development
  • gossamer commit tag: NA
  • gossamer commit hash: 82433af350d2295a7f9e6e4cd1688171e19e1167 & a46ad52411cb95dad9e706c8c725f00ff940f36d
  • operating system: any
  • additional links: NA
@haikoschol haikoschol added C-chaotic Unpredictable nature of this task/changes makes its chaotic. P-low this is mostly nice to have. S-network issues related to the dot/network package. T-bug this issue covers unexpected and/or wrong behaviour. labels Dec 5, 2024
@haikoschol haikoschol self-assigned this Dec 5, 2024
@haikoschol
Copy link
Contributor Author

closed in #4382

@P1sar P1sar added the A-debug issue requires detective debug work to figure out what's going wrong. label Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-debug issue requires detective debug work to figure out what's going wrong. C-chaotic Unpredictable nature of this task/changes makes its chaotic. P-low this is mostly nice to have. S-network issues related to the dot/network package. T-bug this issue covers unexpected and/or wrong behaviour.
Projects
None yet
Development

No branches or pull requests

2 participants