Skip to content

Commit

Permalink
Merge pull request #206 from xmtp/rich/vector-clock-map
Browse files Browse the repository at this point in the history
It's slowly becoming more apparent that the `sid` concept isn't very useful compared to having separate `node_id` and `sequence_id` fields:
1. We've removed gateway sequence ID's, so there's much less potential for confusion between local and remote sequence ID's
2. We're using a separate `uint32` node ID field in various places already, including `target_originator`, `publisher_node_id` and inside `EnvelopesQuery`
3. Updating and manipulating vector clocks are now a lot more important on both server and client, and using a `map<node_id, sequence_id>` type is more ergonomic compared to a list of sids.

Protobufs use a `varint` under the hood, so there shouldn't be much size difference, exactly as @neekolas pointed out originally. This should also make the spec easier to understand and explain, and gives us more bits to work with for both node ID's and sequence ID's.

One con:
- With the current server implementation using postgres, which only understands int32 and int64, there is some potential for error converting between signed and unsigned types. I'll make sure to add server code to check for overflows, with the idea that we would be migrating long before we use up the entire bit-space.
  • Loading branch information
richardhuaaa authored Sep 10, 2024
2 parents bdb0e67 + db28f6d commit 9fffa2b
Showing 1 changed file with 6 additions and 5 deletions.
11 changes: 6 additions & 5 deletions proto/xmtpv4/message_api/message_api.proto
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ option go_package = "github.com/xmtp/proto/v3/go/xmtpv4/message_api";
// The last seen entry per originator. Originators that have not been seen are omitted.
// Entries MUST be sorted in ascending order, so that smaller node ID's appear first.
message VectorClock {
repeated uint64 originator_sids = 1;
map<uint32, uint64> node_id_to_sequence_id = 1;
}

// Data visible to the server that has been authenticated by the client.
Expand Down Expand Up @@ -42,15 +42,16 @@ message PayerEnvelope {
// For blockchain envelopes, the originator_sid is set by the smart contract,
// but the originator_ns is set by the publishing node
message UnsignedOriginatorEnvelope {
uint64 originator_sid = 1;
int64 originator_ns = 2;
PayerEnvelope payer_envelope = 3;
uint32 originator_node_id = 1;
uint64 originator_sequence_id = 2;
int64 originator_ns = 3;
PayerEnvelope payer_envelope = 4;
}

// An alternative to a signature for blockchain payloads
message BlockchainProof {
uint64 block_number = 1;
uint32 publisher_id = 2;
uint32 publisher_node_id = 2;
}

// Signed originator envelope
Expand Down

0 comments on commit 9fffa2b

Please sign in to comment.