Skip to content

Commit

Permalink
Merge pull request #201 from xmtp/rich/change-nodes
Browse files Browse the repository at this point in the history
1. Define a 'vector clock' type with clearly defined rules.
2. Allow a vector clock to be specified as the cursor, for clients that wish to change gateway nodes. Clients will be tracking this information regardless, and this allows us to avoid any heuristics such as a 'lookback period'. The client's first query to the node will specify a vector clock as the cursor, with subsequent queries free to fall back to using the gateway SID again.

A consideration to be made here is whether we should dispense with gateway sids completely, and use the vector clock for all queries. What holds me back is whether we need to worry about the additional message size for queries, and whether the SQL query will be slower (it'll have one `OR originator_node_id = ... AND originator_sequence_id > ...` for each entry in the vector clock). Getting rid of gateway sids would simplify a lot though.
  • Loading branch information
richardhuaaa authored Sep 3, 2024
2 parents 4848a35 + 682d801 commit 832e0a6
Showing 1 changed file with 11 additions and 15 deletions.
26 changes: 11 additions & 15 deletions proto/xmtpv4/message_api/message_api.proto
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,17 @@ import "mls/api/v1/mls.proto";

option go_package = "github.com/xmtp/proto/v3/go/xmtpv4/message_api";

// The last seen entry per originator. Originators that have not been seen are omitted.
// Entries MUST be sorted in ascending order, so that smaller node ID's appear first.
message VectorClock {
repeated uint64 originator_sids = 1;
}

// Data visible to the server that has been authenticated by the client.
message AuthenticatedData {
uint32 target_originator = 1;
bytes target_topic = 2;
repeated uint64 last_originator_sids = 3;
VectorClock last_seen = 3;
}

message ClientEnvelope {
Expand Down Expand Up @@ -56,12 +62,6 @@ message OriginatorEnvelope {
}
}

// Wraps originator envelope with a sequence ID assigned by the gateway
message GatewayEnvelope {
uint64 gateway_sid = 1;
OriginatorEnvelope originator_envelope = 2;
}

// Misbehavior types
enum Misbehavior {
MISBEHAVIOR_UNSPECIFIED = 0;
Expand All @@ -83,13 +83,9 @@ message EnvelopesQuery {
// Client queries
bytes topic = 1;
// Node queries
uint32 originator_id = 2;
}

oneof last_seen {
uint64 originator_sid = 3;
uint64 gateway_sid = 4;
uint32 originator_node_id = 2;
}
VectorClock last_seen = 3;
}

// Batch subscribe to envelopes
Expand All @@ -103,7 +99,7 @@ message BatchSubscribeEnvelopesRequest {

// Streamed response for batch subscribe - can be multiple envelopes at once
message BatchSubscribeEnvelopesResponse {
repeated GatewayEnvelope envelopes = 1;
repeated OriginatorEnvelope envelopes = 1;
}

// Query envelopes request
Expand All @@ -114,7 +110,7 @@ message QueryEnvelopesRequest {

// Query envelopes response
message QueryEnvelopesResponse {
repeated GatewayEnvelope envelopes = 1;
repeated OriginatorEnvelope envelopes = 1;
}

message PublishEnvelopeRequest {
Expand Down

0 comments on commit 832e0a6

Please sign in to comment.