feat!: Address agnostic p2p

[dima74]

Context

Fixes #5117

How things currently work:

• Topology of the peers (public keys and addresses) are stored on-chain.
• There are (Un)RegisterPeer instructions for changing topology.
• Initial topology is provided in the genesis block (via RegisterPeer instructions)
• In case genesis block is not provided to the peer, there is sumeragi.trusted_peers config parameter with addresses of initial peers

Solution

Planned changes:

• Peer addresses will not be stored on-chain (removed from topology)
• Each peer will store current peer addresses off-chain and do periodic gossiping
• Peer A will accept connection from peer B if peer B public key is in topology
• Peer public address will be added to the config.
  • Public address will be sent when connecting to other peers - so that other peers will know our peer's external address and can broadcast this address to other peers (more details in "Use case: New peer added")
  • Note that peer internal address is already in config - network.address
  • Public port can be different from internal port e.g. in case when peer is in docker or behind nginx

Use case: Peer changes address

• Initially there are three peers: A, B, C
• C changes its address
• C knows addresses of other peers and tries to establish connection with them
• Other peers accept connection because C public key is in topology

Use case: New peer added

• Initially there are peers A and B with addresses iroha1:8000 and iroha2:8000
• We want to add peer C with address iroha3:8000
• First, we invoke RegisterPeer instruction to add C public key to topology
• Then peer C is started with the following config:
  • sumeragi.trusted_peers - only peer A
  • network.public_address - iroha3:8000
• C tries to connect to A
  • A receives connection from 9.9.9.9:52143 (52143 is some random port chosen by C for outcoming connection, and 9.9.9.9 is some IP address, e.g. local IP address in kubernetes cluster)
  • A accepts such connection because C public key is in topology
  • C sends its public address (iroha3:8000) to A
  • A gossip address of C to B
  • B receives address of C and connects to C

Migration Guide

• New parameter was added (network.public_address in the config, or P2P_PUBLIC_ADDRESS env var). This parameter should contain external address of the peer used for p2p (as seen by other peers)
• genesis.json — changed topology:

Before:

"topology": [
    {
        "address": "irohad0:1001",
        "id": "ed0120A88F67F9D5D94F63DC57661D0B15054298BBB66B39E601FED71E97CD593A7FDB"
    },
    {
        "address": "irohad0:1002",
        "id": "ed0120647E9781F012EF44281A13D673178767465E1622E7BB1094C4C375B98133DBD1"
    },
    ...
]

After:

"topology": [
    "ed0120A88F67F9D5D94F63DC57661D0B15054298BBB66B39E601FED71E97CD593A7FDB",
    "ed0120647E9781F012EF44281A13D673178767465E1622E7BB1094C4C375B98133DBD1",
    ...
]

Review notes

Most meaningful changes are:

• crates/iroha_core/src/peers_gossiper.rs - logic related to peers gossiping
• crates/iroha_p2p/src/network.rs
  • Now peer connects to addresses received from PeersGossiper service. (Previously peer connects to addresses of peers from topology)
  • In each pair of peers, both try to establish initial connection (one of two connections will be dropped based on disambiguator rule). This is needed since when adding new peer, other peers might not know its address
• crates/iroha_p2p/src/peer.rs - after establishing connection, peer now sends its public_address
• crates/iroha_data_model/src/peer.rs - moved address from PeerId to Peer

Other changes are mostly related to Peer/PeerId

---

TODO:

• Fix tests (in particular multiple_networks)
• Add new tests
• Fix docker compose

Checklist

• I've read CONTRIBUTING.md.
• (optional) I've written unit tests for the code changes.
• All review comments have been resolved.
• All CI checks pass.

[github-actions]

@BAStos525

[SamHSmith]

LGTM

[dima74]

Update:

• Use full public_address (instead of external_port)
• Store gossip address from each peer independently and use majority rule when choosing address for specific peer
• Fixed tests

[0x009922]

LGTM

[mversic]

I think we need some eviction policy. This can be done separately

[dima74]

Do you mean to periodically drop entries which were received long time ago?

Currently we have eviction based on topology

[mversic]

Currently we have eviction based on topology

I see, I think then it's ok. Can a malicious actor overwhelm other peer's gossip handle?

[dima74]

Can a malicious actor overwhelm other peer's gossip handle?

Single malicious actor can't, since we choose address using majority rule

[mversic]

I get that, but a single malicious actor could send lots of different false addresses, couldn't it?

[dima74]

Only last entry is kept in the map.

---

E.g. consider we have three good peers PeerA, PeerB and PeerC and one malicious PeerD. Let's observe how PeerA will determine address of some PeerX. PeerB and PeerC will send correct address GoodPeerX.com:8000 to PeerA, and PeerD will send Bad1PeerX.com:8000. So gossip_peers for PeerA will look like this:

{
  "PeerX": {
    "PeerB": "GoodPeerX.com:8000",
    "PeerC": "GoodPeerX.com:8000",
    "PeerD": "Bad1PeerX.com:8000"
  }
}

Based on majority rule, PeerA will use GoodPeerX.com:8000 as address for PeerX. Next, consider PeerD again sends another bad address Bad2PeerX.com:8000 to PeerA. Now map will change to:

{
  "PeerX": {
    "PeerB": "GoodPeerX.com:8000",
    "PeerC": "GoodPeerX.com:8000",
    "PeerD": "Bad2PeerX.com:8000"
  }
}

Again using majority rule, GoodPeerX.com:8000 will be selected.

[mversic]

really well documented