Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update badgerDb version #52

Closed
AndrewSisley opened this issue Nov 22, 2021 · 6 comments · Fixed by #56
Closed

Update badgerDb version #52

AndrewSisley opened this issue Nov 22, 2021 · 6 comments · Fixed by #56
Assignees
Labels
area/datastore Related to the datastore / storage engine system refactor This issue specific to or requires *notable* refactoring of existing codebases and components

Comments

@AndrewSisley
Copy link
Contributor

AndrewSisley commented Nov 22, 2021

Check for existence of IM mode first https://dgraph.io/docs/badger/get-started/#in-memory-mode-diskless-mode

IPFS has a wrapper for v2, but might be worth wrapping the newer v3 ourselves - have a think, and if leaning towards v2 make sure it has the IM option.

Badger link

@AndrewSisley AndrewSisley added the area/datastore Related to the datastore / storage engine system label Nov 22, 2021
@AndrewSisley AndrewSisley self-assigned this Nov 22, 2021
@jsimnz
Copy link
Member

jsimnz commented Nov 22, 2021

Check for existence of IM mode first https://dgraph.io/docs/badger/get-started/#in-memory-mode-diskless-mode

IPFS has a wrapper for v2, but might be worth wrapping the newer v3 ourselves - have a think, and if leaning towards v2 make sure it has the IM option.

Badger link

🦡 😂 got me with that one

Seems like it is prob best to create a v3 wrapper. Or at least open up an issue and see what the IPFS team thinks about the current strategy is w.r.t Badger versions and their wrappers. I can't think they want to stick with this approach they have for every released Badger version.

@jsimnz
Copy link
Member

jsimnz commented Nov 22, 2021

I havent fully grokked the in-memory implementation yet, however Badger V2 does support the in-memory mode, so we can at least start with the v2 IPFS version for now.

@AndrewSisley
Copy link
Contributor Author

:D wasn't sure anyone would find it lol (or if the email would plain-text it)

Ah looks like they have had a plan to change the approach since Jan, I guess they just haven't gotten round to it: ipfs/go-ds-badger#109

You'd rather go v2 then v3 and hit our users twice (if we don't get to v3 before the next release)?

@jsimnz
Copy link
Member

jsimnz commented Nov 22, 2021

I just meant try out the v2 mode, to test the in-mem version of Badger, and if all looks good, we can either ping the IPFS team on that issue, and see if they have some idea of timeline, or we:

  1. Create our own v3 wrapper
  2. Fork the existing wrapper, and implement the structure/plan they are already talking about, and see if we can merge it upstream.

@AndrewSisley
Copy link
Contributor Author

AndrewSisley commented Nov 23, 2021

I pushed very WIPy v2 here: https://github.com/sourcenetwork/defradb/pull/new/sisley/chore/I52-badger-2
v3 is here: https://github.com/sourcenetwork/defradb/pull/new/sisley/chore/I52-badger-3

There are a couple of unit test failures, which might actually be an uncaught production bug to do with transaction disposal

Also, they haven't released v2 with context yet - I had to go-get the master branch of the IPFS branch, so I guess that rules out using v2 in the short term

@jsimnz jsimnz added the refactor This issue specific to or requires *notable* refactoring of existing codebases and components label Nov 23, 2021
@jsimnz
Copy link
Member

jsimnz commented Dec 2, 2021

fixed in #56

@jsimnz jsimnz closed this as completed Dec 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/datastore Related to the datastore / storage engine system refactor This issue specific to or requires *notable* refactoring of existing codebases and components
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants