Use a single badgerdb instance

[liamsi]

Currently, we are using two stores: one for tendermint related data and one for IPLD related data.

original discussion: #211 (comment)

ref: #182

[tzdybal]

To be honest, I have no experience using multiple badger instances in single process.
I would expect that it may impact memory usage negatively.

On the other hand, with single instance we will need to double-check that keys are not conflicting.

[tac0turtle]

We do use multiple DBs within ll-core on its own. We should look at cleaning up ll-core before merging the ipfs and state.

[Wondertan]

@marbar3778, can you pls point to the places where we use multiple DBs in ll-core? Optionally with short descriptions of their purposes.

[Wondertan]

@tzdybal, I had an experience using multiple Badgers at the same time. I needed kinda independent namespaced KV storages, but that was a lot of overhead. Badger spawned goroutines ate CPU, RAM was bloated, also every instance did a disk footprint, so overhead is on every dimension. Unfortunately, I don't have any real number logged.

[tac0turtle]

Ah I was thinking about the light client and the node. Within a single node it is a sinlge instance of badger

[liamsi]

IMO, we should also consider settling on the same version as IPFS. Currently, ipfs uses v1. There are plans to switch to v2: ipfs/kubo#6818
and there is a badgerv2 store: https://github.com/ipfs/go-ds-badger2

We could of course also write a v3 ds (if it does not already exist) and use it across the board. It is not entirely clear to me why we picked v3 tbh and if it is some kind of official release or rather a development version. The badger docs on how to choose a version do not even mention v3 (currrently): https://github.com/dgraph-io/badger#choosing-a-version

Either way, this does not have high priority right now but it's something to keep an eye on.

[tzdybal]

Out of curiosity, did some small research.
DGraph is using badger v3 in official releases (see go.mod in latest release).

go-ds-badger2 can be easily switched into v3 - only some options were removed from badger (see dgraph-io/badger#1574, dgraph-io/badger#1555). See tzdybal/go-ds-badger3#1

[Wondertan]

Yeah, in our case migration to v3 should be easy. AFAIK, the badger changes major version only in case of on-disk data format change, so for existing users that would require some special migration. In our case, we don't have any so we are good to go.

[liamsi]

we will move the ipfs storage story to a separate repo. So there is no need inside of this repo to deal with this. If anything we should make pruning after the unbonding period the default (if that is not the case already).