Issues: monorepo
Pull requests: monorepo
User docs:
Specs:
op-supervisor is a service to monitor chains, and quickly determine
cross-chain message safety, for native interoperability.
The op-supervisor functions as a superchain backend, implementing the interop specs.
Warning: this implementation is a work in progress, in active development.
make op-supervisor
# Key configurables:
# datadir: where to store indexed interop data
# dependency-set: where to find chain dependencies (this format is changing, and may be fully onchain in a later iteration)
# l2-rpcs: L2 RPC endpoints to fetch data from (optional, can also be added using the `admin_addL2RPC in the admin-RPC)
./bin/op-supervisor \
--datadir="./op-supervisor-data" \
--dependency-set="./my-network-configs/dependency-set.json" \
--l2-rpcs="ws://example1:8545,ws://example2:8545" \
--rpc.enable-admin \
--rpc.port=8545# from op-supervisor dir:
make op-supervisor
./bin/op-supervisor --help# from op-supervisor dir:
go run ./cmd --helpSee op-supervisor docker-bake target.
There are 3 stages of block safety:
unsafe: optimistically processed blockssafe: blocks reproducible from valid dependenciesfinalized: blocks reproducible from irreversibly valid dependencies
Pre-interop, the only dependency is DA (data availability), i.e. the batch data to derive the chain from. Post-interop, other L2s may be a dependency also. The op-supervisor tracks these dependencies, to maintain a global view of cross-chain message safety.
New blocks are considered local unsafe: sufficient to process the block locally, without guarantees.
Once the L2 dependencies are met we consider it cross unsafe: still missing DA, but forming a valid messaging graph.
Once the DA dependency is met, we consider it local safe:
enough to reproduce the local L2 chain content, but not to reason about cross-L2 interactions.
Once both L2 and DA dependencies are met, we consider it cross safe.
A cross-safe block may be "derived from" a L1 block that confirms all L2 data to reproduce
the local chain as well as the cross-L2 dependencies.
Hence this may take additional L1 data, beyond what a local safe block is derived from.
And once the dependencies become irreversibly valid, we consider it finalized.
We can thus look at what cross-safe has been derived from, and verify against the
flowchart TD
LocalUnsafe[Local Unsafe]--Pass cross-L2 checks-->CrossUnsafe[Cross Unsafe]
LocalUnsafe--Individually reproducible from L1-->LocalSafe
LocalSafe[Local Safe]--All cross-L2 dependencies checked<br/> and reproducible from L1-->CrossSafe[Cross Safe]
CrossSafe--Dependencies are irreversible-->Finalized
Warning: the data flow design is actively changing, see design-doc 171.
Op-nodes, or any compatible consensus-layer L2 node, interact with the op-supervisor, to:
- share the "local" data with the supervisor
- view the "cross" safety once the supervisor has sufficient information
sequenceDiagram
autonumber
participant opgethA as op-geth A
participant opnodeA as op-node A
participant opsup as op-supervisor
participant opnodeB as op-node B
Note over opnodeA: on new block
opnodeA ->> opgethA: engine process unsafe block
opgethA -->> opnodeA: engine processed unsafe block
opnodeA ->> opsup: update Local unsafe
opnodeB ->> opsup: update Local unsafe (maybe)
opsup ->> opgethA: Fetch receipts
opgethA -->> opsup: receipts
opsup ->> opsup: cross-unsafe worker
Note left of opnodeA: (changing - delay unsafeView call)
opnodeA ->> opsup: unsafeView
opsup -->> opnodeA: cross unsafe
opnodeA ->> opnodeA: reorg if we need to
opnodeA ->> opnodeA: backtrack unsafe if we need to
Note over opnodeA: on derived block
opnodeA ->> opsup: update Local safe
opnodeB ->> opsup: update Local safe (maybe)
opsup ->> opsup: cross-safe worker
Note left of opnodeA: (changing - delay safeView call)
opnodeA ->> opsup: safeView
opsup -->> opnodeA: cross safe
opnodeA ->> opnodeA: reorg if we need to
opnodeA ->> opnodeA: backtrack safe if we need to
opnodeA->>opgethA: engine forkchoice-update of safe block
Note over opnodeA: on finalized L1
opnodeA->>opsup: finalized L1
opsup-->>opnodeA: finalized L2
opnodeA->>opgethA: engine forkchoice-update of finalized block
Implementers note: the op-supervisor needs "local" data from the chains before being able to provide "cross" verified updated views. The op-node is not currently notified when the "cross" verified view changes, and thus relies on a revisit of the op-supervisor to determine change.
The op-supervisor maintains a few databases:
- Log database (
eventskind): per chain, we maintain a running list of log-events, separated by block-seals. I.e. this persists the cross-L2 dependency information. local safe(fromdakind): per chain, we store which L2 block was locally derived from which L1 block. I.e. this persists the DA dependency information.cross safe(fromdakind): per chain, we store which L2 block became cross-safe, given all the L2 data available, at which L1 block. I.e. this persists the merged results of verifying both DA and cross-L2 dependencies.
Additionally, the op-supervisor tracks cross unsafe in memory, not persisting it to a database:
it can quickly reproduce this after data-loss by verifying if cross-L2 dependencies
are met by unsafe data, starting from the latest known cross safe block.
The latest L1 finalized block is tracked ephemerally as well:
the L2 finalized block is determined dynamically,
given what was cross safe at this finalized point in L1.
For both the events and fromda DB kinds an append-only format was chosen
to make the database efficient and robust:
data can be read in parallel, does not require compaction (a known problem with execution-layer databases),
and data can always be rewound to a previous consistent state by truncating to a checkpoint.
The database can be searched with binary lookups, and written with O(1) appends.
flowchart TD
user-->opnode
user-->opgeth
opnode[op-node]==block checks==>frontend[frontend RPC]
opgeth[op-geth]==tx-pool checks==>frontend
frontend<==>backend
backend--local unsafe updates-->chainprocessor
backend--local safe updates-->localFromDA
chainsDB--query results-->backend
crossunsafeworker[Cross unsafe worker<br/><i>per chain]
crosssafeworker[Cross safe worker<br/><i>per chain]
subgraph chainsDB[Chains DB]
logDB[Event Log DB<br/><i>per chain]
localFromDA[Local-safe DB<br/><i>per chain]
crossFromDA[Cross-safe DB<br/><i>per chain]
crossunsafe[Cross-unsafe<br/><i>per chain]
finalizedL1[Finalized L1]
end
chainprocessor[Chain processor<br/><i>per chain]
opgeth--blocks/receipts-->chainprocessor
chainprocessor--block-seal and log entries-->logDB
logDB--candidate<br/>unsafe blocks-->crossunsafeworker
logDB--msg reads-->crossunsafeworker
crossunsafeworker -- cross-unsafe<br/>updates --> crossunsafe
localFromDA--candidate<br/>safe blocks-->crosssafeworker
logDB--msg reads-->crosssafeworker
crosssafeworker--cross-safe<br/>updates-->crossFromDA
crossFromDA--known<br/>cross-safe-->crosssafeworker
Main components:
frontend: public API surfacebackend: implements the API (updates, queries, reorgs)ChainsDB: hosts the databases, one of each kind, per chainChain processor: indexes blocks/events, including unsafe blocksCross-unsafe worker: updates cross-unsafe, by cross-verifying unsafe dataCross-safe worker: updates cross-safe, by cross-verifying safe data within a L1 view
Note that the cross-unsafe worker operates on any available L2 dependency data,
whereas the cross-safe worker incrementally expands the L1 scope,
to capture the cross-safe state relative to each L1 block.
Most supervisor branching logic deals with the edge-cases that come with syncing dependency data, and updating the safety views as the dependencies change. This is where the service differs most from interop development simulations: dependency verification is critical to safety, and requires dependencies on DA to be consolidated with the dependencies on cross-chain messaging.
The op-supervisor implementation optimizes safe determination of cross-chain message safety,
with fast feedback to readers.
Data is indexed fast and optimistically to have a minimum level of feedback about a message or block. Indexing changes are then propagated, allowing the safety-checks to quickly follow up with asynchronous full verification of the safety.
The op-supervisor is actively changing.
The most immediate changes are that to the architecture and data flow, as outlined in design-doc 171.
Full support for chain reorgs (detecting them, and resolving them) is the next priority after the above architecture and data changes.
Further background on the design-choices of op-supervisor can be found in the superchain backend desgin-doc.
- Each indexing or safety kind of change is encapsulated in its own asynchronous job.
- Increments in indexing and safety are propagated, such that other follow-up work can be triggered without delay.
- A read-only subset of the API is served, sufficient for nodes to stay in sync, assuming a healthy op-supervisor.
- Databases are rewound trivially by dropping trailing information.
- Databases can be copied at any time, for convenient snapshots.
See design-doc 171 for discussion of missing data and syncing related failure modes.
Generally the supervisor aims to provide existing static data in the case of disruption of cross-chain verification, such that a chain which does not take on new interop dependencies, can continue to be extended with safe blocks.
I.e. safety must be guaranteed at all times, but a minimal level of liveness can be maintained by holding off on cross-chain message acceptance while allowing regular single-chain functionality to proceed.
op-e2e/interop: Go interop system-tests, focused on offchain aspects of services to run end to end.op-e2e/actions/interop: Go interop action-tests, focused on onchain aspects such as safety and state-transition.interop-devnet: docker-compose to run interoperable chains locally.