From da43ef0818d7a8aced83c278218f2a32cdf04b3e Mon Sep 17 00:00:00 2001 From: kdeme Date: Wed, 17 Apr 2024 17:49:36 +0200 Subject: [PATCH 1/5] Add HistoricalRootsBlockProof for merge till capella blocks --- history/history-network.md | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/history/history-network.md b/history/history-network.md index 375b4d2..268c31e 100644 --- a/history/history-network.md +++ b/history/history-network.md @@ -155,9 +155,25 @@ each receipt/transaction and re-rlp-encode it, but only if it is a legacy transa ```python # Content types +# Proof for EL BlockHeader before TheMerge / Paris HistoricalHashesAccumulatorProof = Vector[Bytes32, 15] -BlockHeaderProof = Union[None, HistoricalHashesAccumulatorProof] +# Proof that EL block_hash is in BeaconBlock -> BeaconBlockBody -> ExecutionPayload +BeaconBlockProof = Vector[Bytes32, 11] + +# Proof that BeaconBlock root is part of HistoricalRoots and thus canonical +# For after TheMerge until Capella -> Bellatrix fork. +HistoricalRootsProof = Vector[Bytes32, 14] + +# Proof for EL BlockHeader after TheMerge until Capella +HistoricalRootsBlockProof = Container[ + beaconBlockProof: BeaconBlockProof, + beaconBlocRoot: Bytes32, + historicalRootsProof: HistoricalRootsProof, + slot: Slot +] + +BlockHeaderProof = Union[None, HistoricalHashesAccumulatorProof, HistoricalRootsBlockProof] BlockHeaderWithProof = Container( header: ByteList[MAX_HEADER_LENGTH], # RLP encoded header in SSZ ByteList @@ -168,7 +184,9 @@ BlockHeaderWithProof = Container( > **_Note:_** The `BlockHeaderProof` allows to provide headers without a proof (`None`). For pre-merge headers, clients SHOULD NOT accept headers without a proof as there is the `HistoricalHashesAccumulatorProof` solution available. -For post-merge headers, there is currently no proof solution and clients MAY +For post-merge until Capella headers, clients SHOULD NOT accept headers without a proof as there is the `HistoricalRootsBlockProof` solution available. + +For post Capella headers, there is currently no proof solution and clients SHOULD accept headers without a proof. ##### Block Header by Hash @@ -347,5 +365,13 @@ Merkle proof with the `BlockHeader`'s block hash as leave and the `EpochRecord` digest as root. This digest is available in the `HistoricalHashesAccumulator`. -As the `HistoricalHashesAccumulator` only accounts for blocks pre-merge, this proof can -only be used to verify blocks pre-merge. +As the `HistoricalHashesAccumulator` only accounts for blocks pre-merge, this proof MUST be used to verify blocks pre-merge (pre Paris fork). + +#### HistoricalRootsBlockProof + +The `HistoricalRootsBlockProof` is an SSZ container which holds two Merkle proofs as specified in the [SSZ Merke proofs specification](https://github.com/ethereum/consensus-specs/blob/dev/ssz/merkle-proofs.md#merkle-multiproofs). + +The container holds a chain of 2 proofs. This chain of proofs allows for verifying that an EL `BlockHeader` is part of the canonical chain. The only requirement is having access to the beacon chain `historical_roots`. +The `historical_roots` is a [`BeaconState` field](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#beaconstate) that is frozen since the Capella fork. The `HistoricalRootsBlockProof` MUST be used to verify blocks from TheMerge/Paris until the Capella fork. + +The Portal network does not provide a mechanism to acquire the `historical_roots` over the network. Clients are encouraged to solve this however they choose, with the suggestion that they can include a frozen copy of the `historical_roots` within their client code, and provide a mechanism for users to override this value if they choose so. From 6d7a5227479bd1e8959756dbb4dcb3df3698e461 Mon Sep 17 00:00:00 2001 From: kdeme Date: Tue, 17 Sep 2024 13:13:20 +0200 Subject: [PATCH 2/5] Change naming of the proofs to clarify what they are proving --- history/history-network.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/history/history-network.md b/history/history-network.md index 268c31e..896e37a 100644 --- a/history/history-network.md +++ b/history/history-network.md @@ -156,24 +156,24 @@ each receipt/transaction and re-rlp-encode it, but only if it is a legacy transa # Content types # Proof for EL BlockHeader before TheMerge / Paris -HistoricalHashesAccumulatorProof = Vector[Bytes32, 15] +BlockProofHistoricalHashesAccumulator = Vector[Bytes32, 15] -# Proof that EL block_hash is in BeaconBlock -> BeaconBlockBody -> ExecutionPayload -BeaconBlockProof = Vector[Bytes32, 11] +# Proof that EL block_hash is in BeaconBlock -> BeaconBlockBody -> ExecutionPayload +ExecutionBlockProof = Vector[Bytes32, 11] -# Proof that BeaconBlock root is part of HistoricalRoots and thus canonical -# For after TheMerge until Capella -> Bellatrix fork. -HistoricalRootsProof = Vector[Bytes32, 14] +# Proof that BeaconBlock root is part of historical_roots and thus canonical +# From TheMerge until Capella -> Bellatrix fork. +BeaconBlockProofHistoricalRoots = Vector[Bytes32, 14] -# Proof for EL BlockHeader after TheMerge until Capella -HistoricalRootsBlockProof = Container[ - beaconBlockProof: BeaconBlockProof, - beaconBlocRoot: Bytes32, - historicalRootsProof: HistoricalRootsProof, +# Proof for EL BlockHeader from TheMerge until Capella +BlockProofHistoricalRoots = Container[ + beaconBlockProof: BeaconBlockProofHistoricalRoots, # Proof that the BeaconBlock is part of the historical_roots and thus part of the canonical chain + beaconBlockRoot: Bytes32, + executionBlockProof: ExecutionBlockProof, # Proof that EL BlockHash is part of the BeaconBlock slot: Slot ] -BlockHeaderProof = Union[None, HistoricalHashesAccumulatorProof, HistoricalRootsBlockProof] +BlockHeaderProof = Union[None, BlockProofHistoricalHashesAccumulator, BlockProofHistoricalRoots] BlockHeaderWithProof = Container( header: ByteList[MAX_HEADER_LENGTH], # RLP encoded header in SSZ ByteList @@ -183,8 +183,8 @@ BlockHeaderWithProof = Container( > **_Note:_** The `BlockHeaderProof` allows to provide headers without a proof (`None`). For pre-merge headers, clients SHOULD NOT accept headers without a proof -as there is the `HistoricalHashesAccumulatorProof` solution available. -For post-merge until Capella headers, clients SHOULD NOT accept headers without a proof as there is the `HistoricalRootsBlockProof` solution available. +as there is the `BlockProofHistoricalHashesAccumulator` solution available. +For post-merge until Capella headers, clients SHOULD NOT accept headers without a proof as there is the `BlockProofHistoricalRoots` solution available. For post Capella headers, there is currently no proof solution and clients SHOULD accept headers without a proof. @@ -348,9 +348,9 @@ The `HistoricalHashesAccumulator` is fully build and frozen when the last block The network provides no mechanism for acquiring the fully build `HistoricalHashesAccumulator`. Clients are encouraged to solve this however they choose, with the suggestion that they include a frozen copy of the accumulator at the point of TheMerge within their client code, and provide a mechanism for users to override this value if they so choose. The `hash_tree_root` of the `HistoricalHashesAccumulator` is defined in [EIP-7643](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-7643.md). -#### HistoricalHashesAccumulatorProof +#### BlockProofHistoricalHashesAccumulator -The `HistoricalHashesAccumulatorProof` is a Merkle proof as specified in the +The `BlockProofHistoricalHashesAccumulator` is a Merkle proof as specified in the [SSZ Merke proofs specification](https://github.com/ethereum/consensus-specs/blob/dev/ssz/merkle-proofs.md#merkle-multiproofs). It is a Merkle proof for the `BlockHeader`'s block hash on the relevant @@ -359,7 +359,7 @@ the `BlockHeader`'s block hash is part of. The `GeneralizedIndex` selected must match the leave of the `EpochRecord` merkle tree which holds the `BlockHeader`'s block hash. -An `HistoricalHashesAccumulatorProof` for a specific `BlockHeader` can be used to verify that +An `BlockProofHistoricalHashesAccumulator` for a specific `BlockHeader` can be used to verify that this `BlockHeader` is part of the canonical chain. This is done by verifying the Merkle proof with the `BlockHeader`'s block hash as leave and the `EpochRecord` digest as root. This digest is available in the @@ -367,9 +367,9 @@ Merkle proof with the `BlockHeader`'s block hash as leave and the As the `HistoricalHashesAccumulator` only accounts for blocks pre-merge, this proof MUST be used to verify blocks pre-merge (pre Paris fork). -#### HistoricalRootsBlockProof +#### BlockProofHistoricalRoots -The `HistoricalRootsBlockProof` is an SSZ container which holds two Merkle proofs as specified in the [SSZ Merke proofs specification](https://github.com/ethereum/consensus-specs/blob/dev/ssz/merkle-proofs.md#merkle-multiproofs). +The `BlockProofHistoricalRoots` is an SSZ container which holds two Merkle proofs as specified in the [SSZ Merke proofs specification](https://github.com/ethereum/consensus-specs/blob/dev/ssz/merkle-proofs.md#merkle-multiproofs). The container holds a chain of 2 proofs. This chain of proofs allows for verifying that an EL `BlockHeader` is part of the canonical chain. The only requirement is having access to the beacon chain `historical_roots`. The `historical_roots` is a [`BeaconState` field](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#beaconstate) that is frozen since the Capella fork. The `HistoricalRootsBlockProof` MUST be used to verify blocks from TheMerge/Paris until the Capella fork. From fbb2a434e44036d1981d4204c2cb48e1665c0e11 Mon Sep 17 00:00:00 2001 From: kdeme Date: Tue, 17 Sep 2024 14:36:03 +0200 Subject: [PATCH 3/5] Add more explanation on how the BlockProofHistoricalRoots works --- history/history-network.md | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/history/history-network.md b/history/history-network.md index 896e37a..00e8e46 100644 --- a/history/history-network.md +++ b/history/history-network.md @@ -369,9 +369,22 @@ As the `HistoricalHashesAccumulator` only accounts for blocks pre-merge, this pr #### BlockProofHistoricalRoots -The `BlockProofHistoricalRoots` is an SSZ container which holds two Merkle proofs as specified in the [SSZ Merke proofs specification](https://github.com/ethereum/consensus-specs/blob/dev/ssz/merkle-proofs.md#merkle-multiproofs). +The `BlockProofHistoricalRoots` is an SSZ container which holds two Merkle proofs as specified in the [SSZ Merke proofs specification](https://github.com/ethereum/consensus-specs/blob/dev/ssz/merkle-proofs.md#merkle-multiproofs): +- `BeaconBlockProofHistoricalRoots` +- `ExecutionBlockProof` -The container holds a chain of 2 proofs. This chain of proofs allows for verifying that an EL `BlockHeader` is part of the canonical chain. The only requirement is having access to the beacon chain `historical_roots`. -The `historical_roots` is a [`BeaconState` field](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#beaconstate) that is frozen since the Capella fork. The `HistoricalRootsBlockProof` MUST be used to verify blocks from TheMerge/Paris until the Capella fork. +Additionally the SSZ container holds a `BeaconBlock` root and a slot. + +The chain of the two proofs allows for verifying that an EL `BlockHeader` is part of the canonical chain. +The only requirement is having access to the beacon chain `historical_roots`. +The `historical_roots` is a [`BeaconState` field](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#beaconstate) that is frozen since the Capella fork. The `BlockProofHistoricalRoots` MUST be used to verify blocks from TheMerge/Paris until the Capella fork. The Portal network does not provide a mechanism to acquire the `historical_roots` over the network. Clients are encouraged to solve this however they choose, with the suggestion that they can include a frozen copy of the `historical_roots` within their client code, and provide a mechanism for users to override this value if they choose so. + +The first proof, the `BeaconBlockProofHistoricalRoots`, is to prove that the `BeaconBlock` is part of the `historical_roots` and thus part of the canonical chain. + +In order to verify this proof, the `BeaconBlock` root from the container is provided as leaf and the matching historical root is provided as root. The matching historical root index can be calculated from the slot that is provided and the index can then be used to lookup the root from the `historical_roots`. + +The second proof, the `ExecutionBlockProof`, is to prove that the EL block hash is part of the `BeaconBlock`. + +In order to verify this part of the proof, the EL block hash is provided as leaf and the `BeaconBlock` root as root. From af07120aa9485b93719ccd578827db6dd2b38139 Mon Sep 17 00:00:00 2001 From: kdeme Date: Tue, 17 Sep 2024 17:35:03 +0200 Subject: [PATCH 4/5] Add some mermaid flowcharts in an attempt to explain better --- history/history-network.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/history/history-network.md b/history/history-network.md index 00e8e46..3bfb724 100644 --- a/history/history-network.md +++ b/history/history-network.md @@ -381,6 +381,14 @@ The `historical_roots` is a [`BeaconState` field](https://github.com/ethereum/co The Portal network does not provide a mechanism to acquire the `historical_roots` over the network. Clients are encouraged to solve this however they choose, with the suggestion that they can include a frozen copy of the `historical_roots` within their client code, and provide a mechanism for users to override this value if they choose so. +The relationship of the beacon chain structures that are used for the `BlockProofHistoricalRoots` can be seen below: +```mermaid +flowchart LR + BeaconBlock -- contains --> BeaconBlockBody -- contains --> ExecutionPayload -- contains --> block_hash + HistoricalBatch -- hash_tree_root--> historical_roots + BeaconBlock -- hash_tree_root --> HistoricalBatch +``` + The first proof, the `BeaconBlockProofHistoricalRoots`, is to prove that the `BeaconBlock` is part of the `historical_roots` and thus part of the canonical chain. In order to verify this proof, the `BeaconBlock` root from the container is provided as leaf and the matching historical root is provided as root. The matching historical root index can be calculated from the slot that is provided and the index can then be used to lookup the root from the `historical_roots`. @@ -388,3 +396,19 @@ In order to verify this proof, the `BeaconBlock` root from the container is prov The second proof, the `ExecutionBlockProof`, is to prove that the EL block hash is part of the `BeaconBlock`. In order to verify this part of the proof, the EL block hash is provided as leaf and the `BeaconBlock` root as root. + +Relationship of proof building can be seen here: +```mermaid +flowchart LR + BeaconBlock -- build_proof --> ExecutionBlockProof + HistoricalBatch -- build_proof --> BeaconBlockProofHistoricalRoots +``` + +And the verification path: +```mermaid +flowchart LR + BeaconBlockProofHistoricalRoots --> Proof1([verify_merkle_multiproof]) + root(selected historical root) --> Proof1 --> beaconBlockRoot + ExecutionBlockProof --> Proof2([verify_merkle_multiproof]) + beaconBlockRoot --> Proof2 --> block_hash +``` From 4aef56bf584992f07adbf1efa0ba45ddc5cd700d Mon Sep 17 00:00:00 2001 From: kdeme Date: Wed, 18 Sep 2024 09:50:12 +0200 Subject: [PATCH 5/5] Extra comments for explaining the proof fields --- history/history-network.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/history/history-network.md b/history/history-network.md index 3bfb724..b592d7e 100644 --- a/history/history-network.md +++ b/history/history-network.md @@ -168,9 +168,9 @@ BeaconBlockProofHistoricalRoots = Vector[Bytes32, 14] # Proof for EL BlockHeader from TheMerge until Capella BlockProofHistoricalRoots = Container[ beaconBlockProof: BeaconBlockProofHistoricalRoots, # Proof that the BeaconBlock is part of the historical_roots and thus part of the canonical chain - beaconBlockRoot: Bytes32, + beaconBlockRoot: Bytes32, # hash_tree_root of BeaconBlock used to verify the proofs executionBlockProof: ExecutionBlockProof, # Proof that EL BlockHash is part of the BeaconBlock - slot: Slot + slot: Slot # Slot of BeaconBlock, used to calculate the historical_roots index ] BlockHeaderProof = Union[None, BlockProofHistoricalHashesAccumulator, BlockProofHistoricalRoots] @@ -373,7 +373,7 @@ The `BlockProofHistoricalRoots` is an SSZ container which holds two Merkle proof - `BeaconBlockProofHistoricalRoots` - `ExecutionBlockProof` -Additionally the SSZ container holds a `BeaconBlock` root and a slot. +Additionally the SSZ container holds a `BeaconBlock` hash_tree_root and a slot. The chain of the two proofs allows for verifying that an EL `BlockHeader` is part of the canonical chain. The only requirement is having access to the beacon chain `historical_roots`.