feat: seed-based automatic peering #111
Conversation
hacdias
force-pushed
the
peering-cache
branch
4 times, most recently
from
b81f0cc to
493f7d8
Compare
hacdias
changed the title
hacdias
force-pushed
the
peering-cache
branch
2 times, most recently
from
347dedf to
21cf001
Compare
lidel
changed the title
hacdias
changed the title
hacdias
force-pushed
the
peering-cache
branch
2 times, most recently
from
90369b4 to
430702d
Compare
hacdias
force-pushed
the
peering-cache
branch
2 times, most recently
from
c100c2f to
3b34fc8
Compare
setup.go
Outdated
| pr, err = dht.New(ctx, h, | ||
| dht.Datastore(ds), | ||
| dht.BootstrapPeers(dht.GetDefaultBootstrapPeerAddrInfos()...), | ||
| dht.Mode(dht.ModeClient), | ||
| ) |
There was a problem hiding this comment.
Maybe there's other options that would be useful here to make this as light as possible, but I haven't found anything that would seem useful. Do you have any idea? @aschmahmann @lidel
|
@lidel @aschmahmann I have reworked this PR on top of #114. In addition, we now run a parallel DHT just for peer routing if DHT is disabled and the host is not shared. I also extracted some parts of the setup as their own functions - this will simplify #88. |
hacdias
force-pushed
the
feat/shared-cache
branch
from
c011630 to
7d7a783
Compare
hacdias
force-pushed
the
peering-cache
branch
2 times, most recently
from
5461d81 to
ea4b238
Compare
hacdias
force-pushed
the
feat/shared-cache
branch
from
7d7a783 to
e259336
Compare
hacdias
force-pushed
the
feat/shared-cache
branch
from
ae6d7cc to
f6bd4ce
Compare
hacdias
force-pushed
the
peering-cache
branch
2 times, most recently
from
ec9b54f to
17cb3e8
Compare
There was a problem hiding this comment.
Thank you for doing the cleanup while at it!
Lgtm, I think we can ship this as v1.2 (moved remote backends to v1.3).
This is opt-in experimental feature – we will see how connectivity behaves in our infra, and refine if needed.
(pushed some docs for all involved envs)
| Instead, it will set up peering with peers that share the same seed (requires `RAINBOW_SEED_INDEX` to be set up). | ||
|
|
||
| > [!NOTE] | ||
| > Runs a separate light DHT for peer routing with the main host if DHT routing is disabled. |
There was a problem hiding this comment.
What do light DHT and main host mean?
Is the light DHT a DHT instance that will only publish the peer records of the rainbow instance?
There was a problem hiding this comment.
By default, Rainbow runs with two DHT hosts:
- The "main host" which gives the identity of the Rainbow peer.
- The "dht host" that is only used for DHT lookups.
I don't have the full context on why this was decided but maybe @aschmahmann can give more information (he made #5 after all).
If the DHT shared host is enabled, the main host is used for the DHT.
To find peers using the seed-based peering, we need the DHT. This DHT instance must run with the main host, since this is the one that really identifies the Rainbow peer. If DHT shared host is enabled, we can just use that DHT. Otherwise, we need to run a DHT instance in parallel with the DHT host just for peer routing.
Closes #109. This PR adds seed-based automatic peering. This works by either leveraging the already-running DHT (if DHT is enabled with shared host), or running a parallel DHT with the main host only for peer routing.
I've added tests that you can run on your machine, but disabled on the CI. The time it takes to connect can vary and I thought it could be a bit flaky to run in the CI. In addition, it would maybe pollute the Amino DHT with all ephemeral peers that are created in each test.
Depends on #114.
Closes #27