⚗️Libp2p to replace network layer?

[D4nte]

Investigate what elements of libp2p could be used to replace some of the exiting/planned features (id, connection pool, encryption, etc).

Provide list of what would be replace and how it would look like.

[thomaseizinger]

State of rust-libp2p

The following list is obviously not exhaustive and only contains stuff which is interesting for us.

What is supported

• noise for transport encryption
• generation of node identifiers from several curves including ed25519
• TCP transport
• Websocket transport

What is not supported

• XK noise handshake

Other interesting stuff

• rust-libp2p is built with WASM in mind, that is, there are feature-flags all over the library to swap out implementations when the target is WASM or emscripten

What we can replace

Basically our whole networking stack, that is

• Manual listening on TCP sockets
• Connection pooling
• ID handling (done by libp2p)

What we would have to do

• Probably change how bam works since we are no longer in control of the actual socket. Instead, we would have to register our own protocol (like /bam/0.1.0) and make it work with the abstractions they give us which is something the call a NetworkBehaviour.
• Implement the XK handshake for our protocol.

Things to think about down the line

If we would use libp2p as the network layer, we could get around to only requiring the other node's ID (COMIT nodes are very likely to have a stable ID as we would generate it once and re-use it.) in the HTTP endpoint because we can utilize find_peer to get their current address (which in turn will use whichever discovery mechanism we want, like DHT or webRTC star, etc).

How do multi-addresses fit into the picture?
They would be almost invisible to us as COMIT developers as well as to users of a COMIT node. Reason being is that you only provide the COMIT node's ID as connection information and let libp2p figure out an address of that ID through any of its discovery mechanisms. The result of such a discovery will be a multi-address but we don't really care since all we would be doing is feed it back to libp2p by saying: dialProtocol(address, /bam/0.1.0) protocol.

What I suggest that we should do:

Go ahead and libp2p.
I gives us many features out-of-the-box. It is actively worked which means other people will help us in maintaining and building the functionality we need :)

[D4nte]

Wouldn't adding the support of XK to their noise implementation a thing we would have to do?
How is their noise implementation? Is ours better?
You did not mention the multi-format addresses. Can you tell us a bit more about the gains there?

[thomaseizinger]

Wouldn't adding the support of XK to their noise implementation a thing we would have to do?

Yes. I thought that is implied by the fact that it is not there :)

How is their noise implementation? Is ours better?

They also depend on snow, so it is using the same noise implementation under the hood. Regarding the handshake, they took a different approach which I don't quite understand yet but that may be because I haven't fully understood the architecture of rust-libp2p.

You did not mention the multi-format addresses. Can you tell us a bit more about the gains there?

Will add information in regards to that.

[bonomat]

Sounds reasonable to me but also quite time intensive.
It also gives me the feeling we took the approach: Oh look, a new library we don't know yet... let's use it.
I'm not saying no to libp2p per se but I would like to see a better summary ala:

• What are the problems we are facing?
• Have we solved them already only partly and they do it better?
• What consequences will this have in regards of our RFCs and the rest of the code base?
• Are any other solutions out there?

Meaning, this will lead us back to a situation where we should do a propper evaluation.
We can then write this decision down using MADR

[thomaseizinger]

What are the problems we are facing?

I partly addresses this in What we can replace. It lists stuff that we would no longer have to do if we adopt libp2p. Meaning, we won't have to face the problems that we are solving there. Quoting from above:

• Manual listening on TCP sockets
• Connection pooling
• ID handling (done by libp2p)

In addition, the 2nd paragraph of "Things to think about down the line" mentions something interesting in regards to handling connections altogether:

How do multi-addresses fit into the picture?
They would be almost invisible to us as COMIT developers as well as to users of a COMIT node. Reason being is that you only provide the COMIT node's ID as connection information and let libp2p figure out an address of that ID through any of its discovery mechanisms. The result of such a discovery will be a multi-address but we don't really care since all we would be doing is feed it back to libp2p by saying: dialProtocol(address, /bam/0.1.0) protocol.

As an alternative, we have to pass actual connection information (an IPv4 f.e.) which limits is in the way we can connect to other peers. (f.e. NAT)

Have we solved them already only partly and they do it better?

Yes. I would say they solve it better because building a network stack is the sole purpose of libp2p whereas we just need it to work. Our domain is then on the application level.

What consequences will this have in regards of our RFCs and the rest of the code base?

Regarding the RFCs. Those are only concerned with the message design and the communication that happens once a connection is established. They don't say anything about how connections are established. However IF we were to adopt libp2p, I think we would create an RFC for specifying how the COMIT protocol can be used in libp2p which would basically define a protocol string (like /bam/).

Regarding the rest of the code base: I tried to address this in What we would have to do and Things to think about down the line.

Probably change how bam works since we are no longer in control of the actual socket. Instead, we would have to register our own protocol (like /bam/0.1.0) and make it work with the abstractions they give us which is something the call a NetworkBehaviour.

This is what we would have to specify in above mentioned RFC.

Are any other solutions out there?

Not that I am aware of. However, answering such question is a bit out of scope of the spike. The goal is to evaluate whether libp2p can replace our network layer.

[D4nte]

I think we can close this spike as it is done. Evaluation is done, thank you and well done @thomaseizinger !

The aim was to understand what libp2p can bring to comit-rs. Now that we know what we could use, I see 2 non-exclusive ways to move forward:

1. We create issues to start replacing network components of comit-rs with libp2p ones as discussed above
2. We evaluate other solutions for the replacement of such components.

My proposal: we do 1. and as part of each created issue do a pre-work: "check if there are worthy alternative solutions to this problematic".

@comit-network/rust-devs any objections? Please use your emojis 👍👎😕

[bonomat]

Sounds good. Should we start using MADR for this?

[D4nte]

coblox/docs#6 tracks the investigation of various solutions (including MADR) to track decision taking. Let's be sure we look at our problem and choose the proper solution for knowledge keep-safing/sharing as you explained in this ticket.

[D4nte]

Follow-up with #779