Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add payload threshold for using uTP, and show work #339

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
fixup: switch to python & use fluffy-derived packet size
  • Loading branch information
carver committed Sep 28, 2024
commit 822729885e16ac89a417b72aa3d283c55d29ba28
69 changes: 37 additions & 32 deletions portal-wire-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,38 +68,43 @@ The transmission of data that is too large to fit a single packet is done using

> The Portal wire protocol currently implements uTP over the `TALKREQ/TALKRESP` messages. Future plans are to move to the [sub-protocol data transmission](https://github.com/ethereum/devp2p/issues/229) in order to use a protocol native mechanism for establishing packet streams between clients.

Currently, the standard is to switch to uTP when the payload exceeds 1165 bytes. This may change over time, because it depends on a number of other variables. See an example derivation in rust:
```rs
/// The maximum size of a Discv5 packet.
const MAX_DISCV5_PACKET_SIZE: usize = 1280;

/// The maximum size of a Discv5 talk request payload.
///
/// Discv5 talk request overhead:
/// * masking IV length: 16
/// * static header (protocol ID || version || flag || nonce || authdata-size) length: 23
/// * authdata length: 32
/// * HMAC length: 16
/// * (max) talk request ID length: 8
/// * (max assumed) talk request protocol length: 8
/// * RLP byte array overhead: 6
const MAX_DISCV5_TALK_REQ_PAYLOAD_SIZE: usize =
MAX_DISCV5_PACKET_SIZE - 16 - 23 - 32 - 16 - 8 - 8 - 6;

// NOTE: The wire constant below relies on the following SSZ constants:
// * `ssz::BYTES_PER_UNION_SELECTOR`: 1
// * `ssz::BYTES_PER_LENGTH_OFFSET`: 4

/// The maximum size of a portal CONTENT payload. At the time of writing, this payload either
/// corresponds to a `connection_id`, `enrs`, or `content` payload.
///
/// Portal wire overhead:
/// * portal message SSZ union selector
/// * CONTENT SSZ union selector
/// * CONTENT SSZ length offset for List `enrs` or `content`
const MAX_PORTAL_CONTENT_PAYLOAD_SIZE: usize = MAX_DISCV5_TALK_REQ_PAYLOAD_SIZE
- (ssz::BYTES_PER_UNION_SELECTOR * 2)
- ssz::BYTES_PER_LENGTH_OFFSET;
Currently, the standard is to switch to uTP when the payload exceeds 1177 bytes. This is the full encoded payload size that includes the message ID, etc. The max payload size may change over time, because it depends on a number of other variables. See an example derivation in python:
```py
# The maximum size of a Discv5 packet.
MAX_DISCV5_PACKET_SIZE = 1280

# Discv5 ordinary packet overhead
DISCV5_PACKET_OVERHEAD = ( # sum: 87
# masking IV length
16 +
# static header (protocol ID || version || flag || nonce || authdata-size) length
23 +
# authdata length
32 +
# HMAC length
16
)

# Discv5 talk request overhead
DISCV5_TALK_REQ_OVERHEAD = ( # sum: 16
# talkResp msg id
1 +
# rlp encoding outer list, max length will be encoded in 2 bytes
3 +
# request id (max = 8) + 1 byte from rlp encoding byte string
9 +
# rlp encoding response byte string, max length in 2 bytes
3
)

# The maximum size of a portal CONTENT payload. At the time of writing, this payload either
# corresponds to a `connection_id`, `enrs`, or `content` payload.
Comment on lines +100 to +101
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# The maximum size of a portal CONTENT payload. At the time of writing, this payload either
# corresponds to a `connection_id`, `enrs`, or `content` payload.
# The maximum size of a portal wire protocol message payload.

Considering the 2 bytes for the portal wire msg id and the Union selector (of Content response) are ignored here, we probably should phrase it like above suggestion? Which would apply then to all message requests & responses.


MAX_PORTAL_CONTENT_PAYLOAD_SIZE = ( # 1177
MAX_DISCV5_PACKET_SIZE
- DISCV5_PACKET_OVERHEAD
- DISCV5_TALK_REQ_OVERHEAD
)
```

## Request - Response Messages
Expand Down
Loading