Pinecone experimental support

[olanod]

From https://matrix.org/blog/2021/05/06/introducing-the-pinecone-overlay-network

If Pinecone works out, our intention is to collaborate with the libp2p and IPFS team to incorporate Pinecone routing into libp2p (if they'll have us!) while incorporating their gossipsub routing to improve Matrix federation... and get the best of both worlds :)

I guess they would first try to integrate in the go-libp2p repo as they would use it for dendrite but as someone working with a rust-libp2p based and Matrix dependent project, it would be awesome to start seeing interest and perhaps an early implementation of pinecone in the Rust implementation which could already benefit projects depending on this library, hoping that someday there will be a Rust based P2P Matrix home server that I can embed with the existing rust-libp2p project and have both pieces share the same networking.

[thomaseizinger]

Wow, this is super interesting!

It might be a challenge though for the current abstractions in (rust) libp2p. As the README explains, libp2p is primarily designed around the abstraction of direct connectivity. Routing traffic through other peers kind of breaks this assumption. For example, when would we emit events such as "Connected" and "Disconnected"?

I guess if you treat it as a block-box for sending data from A to B, it might work 🤔

[HaosGames]

I made an attempt at integrating pinecone into libp2p through implementing the NetworkBehaviour Trait, see here. It doesn't work yet but I have a standalone rust port of pinecone that kinda works, see here.

Now that I am trying to get the integration with libp2p to work I am not sure if I should rather do it by implementing the Transport Trait and using the connection abstraction as "sessions" with other nodes in the overlay network. Maybe use UDP instead of TCP connections for peerings and make a TCP layer ontop of pinecone to try to mitigate the uncertainty of the network and the effects of layering TCP ontop of TCP.

[MarcoPolo]

Maybe use UDP instead of TCP connections for peerings and make a TCP layer ontop of pinecone to try to mitigate the uncertainty of the network and the effects of layering TCP ontop of TCP.

Good idea! (more context http://sites.inka.de/~W1011/devel/tcp-tcp.html)

I think the transport makes sense if you want to allow existing behaviors to use pinecone (mostly) seamlessly. And behavior makes sense if you want to expose pinecone as component for the user and/or other behaviors to use directly.

A use case I can see with the behaviour version is that you can use pinecone as another way to do DCuTR. Treating the pinecone overlay as a relay essentially and getting direct connections out of it.

As a transport it could be a great fallback when all else fails.

I'm sure there are other use cases too (and I'd be interested in hearing them!).