feat: Docker startup logic, op-proposer fixes, docs (#102)

* add * docs * docs * docs * add * feat: add docker startup logic (#103) * add docker startup logic * add * fix: delete the span batch server * fix * spin up and spin down * add * add * add * fix: fetch proofs that failed on server correctly (#104) * rm outdated env * wip * docs: update * add docs * docs
succinctlabs · Sep 5, 2024 · f9795be · f9795be
1 parent e0bd72b
commit f9795be
Show file tree

Hide file tree

Showing 16 changed files with 386 additions and 94 deletions.
diff --git a/.env.server.example b/.env.server.example
@@ -4,12 +4,6 @@ L1_BEACON_RPC=
 L2_RPC=
 L2_NODE_RPC=
 
-# Note: These are used because of how the OP Proposer loads the txMgr config in the Optimism monorepo.
-L1_ETH_RPC=
-BEACON_RPC=
-ROLLUP_RPC=
-
-
 # op-proposer configuration
 POLL_INTERVAL=
 L2OO_ADDRESS=

diff --git a/book/SUMMARY.md b/book/SUMMARY.md
@@ -7,4 +7,5 @@
   - [L2 Output Oracle](./getting-started/l2-output-oracle.md)
   - [Proposer](./getting-started/proposer.md)
   - [Configuration](./getting-started/configuration.md)
-- [Cost Estimator CLI Tool](./cost-estimator.md)
+- [Cost Estimator CLI Tool](./cost-estimator.md)
+- [L2 Node Setup](./node-setup.md)
diff --git a/book/cost-estimator.md b/book/cost-estimator.md
@@ -1,4 +1,4 @@
-# Cycle Counts [Cost Estimator]
+# Cost Estimator
 
 We provide a convenient CLI tool to estimate the RISC-V cycle counts (and cost) for generating ZKPs for a range of blocks for a given rollup.
 
@@ -19,44 +19,52 @@ It is required that the L2 RPC is an archival node for your OP stack rollup, wit
 
 Then run the following command:
 ```shell
-RUST_LOG=info just run-multi <start_l2_block> <end_l2_block>
+RUST_LOG=info just cost-estimator <start_l2_block> <end_l2_block>
 ```
 
-This will fetch the required data for generating a ZKP for the given block range, "execute" the
-corresponding SP1 program, and return the cycle count. Then, you can extrapolate the cycle count
-to a cost based on the cost per billion cycles.
+This command will execute `op-succinct` as if it's in production. First, it will divide the entire block range
+into smaller ranges optimized along the span batch boundaries. Then it will fetch the required data for generating the ZKP for each of these ranges, and execute the SP1 `span` program. Once each program finishes, it will collect the statistics and output the aggregate statistics
+for the entire block range. From this data, you can extrapolate the cycle count to a cost based on the cost per billion cycles.
 
-Proofs over a span batch are split into several "span proofs", which prove the validity of a section of the span batch. Then, these span proofs are aggregated into a single proof, which is submitted to the L1.
+## Example
 
-## Example Block Range
-
-On OP Sepolia, generating a proof from 15840000 to 15840050 (50 blocks) takes ~1.5B cycles and takes
+On Optimism Sepolia, proving the block range 15840000 to 15840050 (50 blocks) generates 4 span proofs, takes ~1.8B cycles and
 ~2 minutes to execute.
 
 ```bash
-RUST_LOG=info just run-multi 15840000 15840050
+RUST_LOG=info just cost-estimator 15840000 15840050
 
 ...Execution Logs...
 
 +--------------------------------+---------------------------+
 | Metric                         | Value                     |
 +--------------------------------+---------------------------+
-| Total Cycles                   |             1,502,329,547 |
-| Block Execution Cycles         |             1,009,112,508 |
-| Total Blocks                   |                        51 |
-| Total Transactions             |                       202 |
-| Cycles per Block               |                19,786,519 |
-| Cycles per Transaction         |                 4,995,606 |
-| Transactions per Block         |                         3 |
-| Total Gas Used                 |                52,647,751 |
-| Gas Used per Block             |                 1,032,308 |
-| Gas Used per Transaction       |                   260,632 |
+| Batch Start                    |                16,240,000 |
+| Batch End                      |                16,240,050 |
+| Execution Duration (seconds)   |                       130 |
+| Total Instruction Count        |             1,776,092,063 |
+| Oracle Verify Cycles           |               237,150,812 |
+| Derivation Cycles              |               493,177,851 |
+| Block Execution Cycles         |               987,885,587 |
+| Blob Verification Cycles       |                84,995,660 |
+| Total SP1 Gas                  |             2,203,604,618 |
+| Number of Blocks               |                        51 |
+| Number of Transactions         |                       160 |
+| Ethereum Gas Used              |                43,859,242 |
+| Cycles per Block               |                74,736,691 |
+| Cycles per Transaction         |                23,422,603 |
+| Transactions per Block         |                        11 |
+| Gas Used per Block             |                 3,509,360 |
+| Gas Used per Transaction       |                 1,105,066 |
+| BN Pair Cycles                 |                         0 |
+| BN Add Cycles                  |                         0 |
+| BN Mul Cycles                  |                         0 |
+| KZG Eval Cycles                |                         0 |
+| EC Recover Cycles              |                 9,407,847 |
 +--------------------------------+---------------------------+
 ```
 
 ## Misc
 - For large enough block ranges, the RISC-V SP1 program will surpass the SP1 memory limit. Recommended limit is 20-30 blocks.
 - Your L2 node must have been synced for the blocks in the range you are proving. 
-   - OP Sepolia Node: Synced from block 15800000 onwards.
-   - OP Mainnet Node: Synced from block 122940000 onwards.
 
diff --git a/book/getting-started/l2-output-oracle.md b/book/getting-started/l2-output-oracle.md
@@ -1,4 +1,4 @@
-# L2 Output Oracle
+# Deploy L2 Output Oracle
 
 The first step in deploying OP Succinct is to deploy a Solidity smart contract that will verify ZKPs of OP derivation (OP's name for their state transition function) and contain the latest state root of your rollup.
 
@@ -25,7 +25,7 @@ Inside the `contracts` folder there is a file called `zkconfig.json` that contai
 
 | Parameter | Description |
 |-----------|-------------|
-| `startingBlockNumber` | The L2 block number at which the rollup starts. Default should be 0. |
+| `startingBlockNumber` | The L2 block number at which to start generating validity proofs. This should be set to the current L2 block number. You can fetch this with `cast bn --rpc-url <L2_RPC>`. |
 | `l2RollupNode` | The URL of the L2 rollup node. (After the tutorial, this is `http://localhost:8545`) |
 | `submissionInterval` | The number of L2 blocks between each L1 output submission. |
 | `l2BlockTime` | The time in seconds between each L2 block. |
@@ -57,29 +57,69 @@ and then run the following command to deploy the contract:
 forge script script/ZKDeployer.s.sol:ZKDeployer \
     --rpc-url $L1_RPC \
     --private-key $PRIVATE_KEY \
+    --ffi \
     --verify \
     --verifier etherscan \
     --etherscan-api-key $ETHERSCAN_API_KEY \
-    --broadcast \
-    --ffi
+    --broadcast
 ```
 
 If successful, you should see the following output:
 
 ```
-Submitting verification for [src/ZKL2OutputOracle.sol:ZKL2OutputOracle] 0xfe6BcbCD9c067d937431b54AfF107D4F8f2aC653.
-Submitted contract for verification:
-        Response: `OK`
-        GUID: `qyc71u8whqpuf3cylumh3bdcf39a4nzv6ffpubfqef2jrzserr`
-        URL: https://sepolia.etherscan.io/address/0xfe6bcbcd9c067d937431b54aff107d4f8f2ac653
-Contract verification status:
-Response: `NOTOK`
-Details: `Pending in queue`
-Contract verification status:
-Response: `NOTOK`
-Details: `Already Verified`
-Contract source code already verified
-All (2) contracts were verified!
+Script ran successfully.
+
+== Return ==
+0: address 0x9b520F7d8031d45Eb8A1D9fE911038576931ab95
+
+## Setting up 1 EVM.
+
+==========================
+
+Chain 11155111
+
+Estimated gas price: 11.826818849 gwei
+
+Estimated total gas used for script: 3012823
+
+Estimated amount required: 0.035632111845100727 ETH
+
+==========================
+
+##### sepolia
+✅  [Success]Hash: 0xc57d97ac588563406183969e8ea15bc06496915547114b1df4e024c142df07b4
+Contract Address: 0x2e4a7Dc6F19BdE1edF1040f855909afF7CcBeDeC
+Block: 6633852
+Paid: 0.00858210364707003 ETH (1503205 gas * 5.709203766 gwei)
+
+
+##### sepolia
+✅  [Success]Hash: 0x1343094b0be4e89594aedb57fb795d920e7cc1a76288485e8cf248fa206321ed
+Block: 6633852
+Paid: 0.001907479233443196 ETH (334106 gas * 5.709203766 gwei)
+
+
+##### sepolia
+✅  [Success]Hash: 0x708ce24c69c2637cadd6cffc654cbe2114e9ea4ec1e69838cd45c1fa27981713
+Contract Address: 0x9b520F7d8031d45Eb8A1D9fE911038576931ab95
+Block: 6633852
+Paid: 0.00250654027540581 ETH (439035 gas * 5.709203766 gwei)
+
+✅ Sequence #1 on sepolia | Total Paid: 0.012996123155919036 ETH (2276346 gas * avg 5.709203766 gwei)
+                                                                                                          
+
+==========================
+
+ONCHAIN EXECUTION COMPLETE & SUCCESSFUL.
+##
+Start verification for (2) contracts
+Start verifying contract `0x9b520F7d8031d45Eb8A1D9fE911038576931ab95` deployed on sepolia
+
+Submitting verification for [lib/optimism/packages/contracts-bedrock/src/universal/Proxy.sol:Proxy] 0x9b520F7d8031d45Eb8A1D9fE911038576931ab95.
+
+...
 ```
 
-Keep note of the address of the `ZKL2OutputOracle` contract that was deployed. You will need it in the next few sections.
+Keep note of the address of the `Proxy` contract that was deployed, which in this case is `0x9b520F7d8031d45Eb8A1D9fE911038576931ab95`. 
+
+It is also returned by the script as `0: address 0x9b520F7d8031d45Eb8A1D9fE911038576931ab95`. 
diff --git a/book/getting-started/prerequisites.md b/book/getting-started/prerequisites.md
@@ -7,6 +7,15 @@ You must have the following installed:
 - [Foundry](https://book.getfoundry.sh/getting-started/installation)
 - [Docker](https://docs.docker.com/get-started/)
 
+You must have the following RPCs available:
+- L1 Archive Node
+- L1 Consensus (Beacon) Node
+- L2 Archive Node
+- L2 Rollup Node
+
+If you do not have an L2 OP Geth node + rollup node running for your rollup, you can follow the [node setup instructions](../node-setup.md) to get started. 
+
+
 ## OP Stack Chain
 
 The rest of this section will assume you have an existing OP Stack Chain running. If you do not have one, there are two ways you can get started:

diff --git a/book/getting-started/proposer.md b/book/getting-started/proposer.md
@@ -6,36 +6,60 @@ The `op-succinct-proposer` service will call to [Succinct's Prover Network](http
 
 The modified proposer  performs the following tasks:
 1. Monitors L1 state to determine when to request a proof.
-2. Requests proofs from the OP Succinct server.
-3. Once proofs have been generated for a sufficiently large range, aggregates batch proofs and submits them on-chain.
+2. Requests proofs from the OP Succinct server. The server sends requests to the Succinct Prover Network.
+3. Once proofs have been generated for a sufficiently large range, aggregates span proofs and submits them on-chain.
 
 We've packaged the `op-succinct-proposer` service in a docker-compose file to make it easier to run.
 
-## 1) Build the Proposer
+## 1) Set Proposer Parameters
+
+In the root directory, create a file called `.env` (mirroring `.env.example`) and set the following environment variables:
+
+| Parameter | Description |
+|-----------|-------------|
+| `L1_RPC` | The RPC URL for the L1 Ethereum node. |
+| `L1_BEACON_RPC` | The RPC URL for the L1 Ethereum consensus node. |
+| `L2_RPC` | The RPC URL for the L2 archive node (OP-Geth). |
+| `L2_NODE_RPC` | The RPC URL for the L2 node. |
+| `POLL_INTERVAL` | The interval at which to poll for new L2 blocks. |
+| `L2OO_ADDRESS` | The address of the L2OutputOracle contract. |
+| `PRIVATE_KEY` | The private key for the `op-proposer` account. |
+| `L2_CHAIN_ID` | The chain ID of the L2 network. |
+| `MAX_CONCURRENT_PROOF_REQUESTS` | The maximum number of concurrent proof requests (default is 20). |
+| `MAX_BLOCK_RANGE_PER_SPAN_PROOF` | The maximum block range per span proof (default is 30). |
+| `OP_SUCCINCT_SERVER_URL` | The URL of the OP Succinct server (default is http://op-succinct-server:3000). |
+| `PROVER_NETWORK_RPC` | The RPC URL for the Succinct Prover Network. |
+| `SP1_PRIVATE_KEY` | The private key for the SP1 account. |
+| `SP1_PROVER` | The type of prover to use (set to "network"). |
+| `SKIP_SIMULATION` | Whether to skip simulation of the proof before sending to the SP1 server (default is true). |
+
+
+## 2) Build the Proposer
 
 Build the docker images for the `op-succinct-proposer` service.
 
 ```bash
-cd proposer
-sudo docker-compose build
+docker-compose build
 ```
 
-## 2) Run the Proposer
+## 3) Run the Proposer
 
 This command launches the `op-succinct-proposer` service in the background. It launches two containers: one container that manages proof generation and another container that is a small fork of the original `op-proposer` service.
 
+After a few minutes, you should see the `op-succinct-proposer` service start to generate span proofs. Once enough span proofs have been generated, they will be verified in an aggregate proof and submitted to the L1.
+
 ```bash
-sudo docker-compose up
+docker-compose up
 ```
 
 To see the logs of the `op-succinct-proposer` service, run:
 
 ```bash
-sudo docker-compose logs -f
+docker-compose logs -f
 ```
 
 and to stop the `op-succinct-proposer` service, run:
 
 ```bash
-sudo docker-compose down
+docker-compose down
 ```
diff --git a/book/node-setup.md b/book/node-setup.md
@@ -0,0 +1,33 @@
+# L2 Node Setup
+
+## Setup Instructions
+
+1. Clone [ops-anton](https://github.com/anton-rs/ops-anton) and follow the instructions in the README to set up your rollup.
+2. Go to [op-node.sh](https://github.com/anton-rs/ops-anton/blob/main/L2/op-mainnet/op-node/op-node.sh#L4-L6) and set the `L2_RPC` to your rollup RPC. Modify the `l1` and `l1.beacon` to your L1 and L1 Beacon RPCs. Note: Your L1 node should be an archive node.
+3. If you are starting a node for a different chain, you will need to modify `op-network` in `op-geth.sh` [here](https://github.com/anton-rs/ops-anton/blob/main/L2/op-mainnet/op-geth/op-geth.sh#L18) and `network` in `op-node.sh` [here](https://github.com/anton-rs/ops-anton/blob/main/L2/op-mainnet/op-node/op-node.sh#L10).
+4. In `/L2/op-mainnet` (or the directory you chose):
+   1. Generate a JWT secret `./generate_jwt.sh`
+   2. `docker network create anton-net` (Creates a Docker network for the nodes to communicate on).
+   3. `just up` (Starts all the services).
+
+Your `op-geth` endpoint will be available at the RPC port chosen [here](https://github.com/anton-rs/ops-anton/blob/main/L2/op-mainnet/op-geth/op-geth.sh#L7), which in this case is `8547` (e.g. `http://localhost:8547`).
+
+Your `op-node` endpoint (rollup node) will be available at the RPC port chosen [here](https://github.com/anton-rs/ops-anton/blob/main/L2/op-mainnet/op-node/op-node.sh#L13), which in this case is `5058` (e.g. `http://localhost:5058`).
+
+## Checking Sync Status
+
+After a few hours, your node should be fully synced and you can use it to begin generating ZKPs.
+
+To check your node's sync status, you can run the following commands:
+
+**op-geth:**
+
+```bash
+curl -H "Content-Type: application/json" -X POST --data '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}' http://localhost:8547
+```
+
+**op-node:**
+
+```bash
+curl -H "Content-Type: application/json" -X POST --data '{"jsonrpc":"2.0","method":"optimism_syncStatus","params":[],"id":1}' http://localhost:5058
+```
diff --git a/contracts/.env.example b/contracts/.env.example
@@ -1,2 +1,5 @@
+L1_RPC=
 # Rollup RPC URL
-L2_NODE_RPC=
+L2_NODE_RPC=
+PRIVATE_KEY=
+ETHERSCAN_API_KEY=
diff --git a/contracts/script/ZKDeployer.s.sol b/contracts/script/ZKDeployer.s.sol
@@ -7,7 +7,7 @@ import {Utils} from "../test/helpers/Utils.sol";
 import {Proxy} from "@optimism/src/universal/Proxy.sol";
 
 contract ZKDeployer is Script, Utils {
-    function run() public {
+    function run() public returns (address) {
         vm.startBroadcast();
 
         Config memory config = readJsonWithRPCFromEnv("zkconfig.json");
@@ -19,5 +19,7 @@ contract ZKDeployer is Script, Utils {
         upgradeAndInitialize(zkL2OutputOracleImpl, config, address(0), bytes32(0), 0);
 
         vm.stopBroadcast();
+
+        return config.l2OutputOracleProxy;
     }
 }
diff --git a/contracts/zkconfig.json b/contracts/zkconfig.json
@@ -1,5 +1,5 @@
 {
-    "startingBlockNumber": 16795981,
+    "startingBlockNumber": 16837928,
     "l2RollupNode": "",
     "submissionInterval": 150,
     "l2BlockTime": 2,

diff --git a/justfile b/justfile
@@ -27,6 +27,11 @@ run-multi start end use-cache="false" prove="false":
 
   cargo run --bin multi --release -- --start {{start}} --end {{end}} $CACHE_FLAG $PROVE_FLAG
 
+# Runs the cost estimator for a given block range.
+cost-estimator start end:
+  #!/usr/bin/env bash
+  cargo run --bin cost_estimator --release -- --start {{start}} --end {{end}}
+
 # Runs the client program in native execution mode. Modified version of Kona Native Client execution:
 # https://github.com/ethereum-optimism/kona/blob/ae71b9df103c941c06b0dc5400223c4f13fe5717/bin/client/justfile#L65-L108
 run-client-native l2_block_num l1_rpc='${L1_RPC}' l1_beacon_rpc='${L1_BEACON_RPC}' l2_rpc='${L2_RPC}' verbosity="-vvvv":

diff --git a/proposer/op/proposer/db/db.go b/proposer/op/proposer/db/db.go
@@ -211,6 +211,26 @@ func (db *ProofDB) GetLatestEndBlock() (uint64, error) {
 	return uint64(maxEnd.EndBlock), nil
 }
 
+// If a proof failed to be sent to the prover network, it's status will be set to FAILED, but the prover request ID will be empty.
+// This function returns all such proofs.
+func (db *ProofDB) GetProofsFailedOnServer() ([]*ent.ProofRequest, error) {
+	proofs, err := db.client.ProofRequest.Query().
+		Where(
+			proofrequest.StatusEQ(proofrequest.StatusFAILED),
+			proofrequest.ProverRequestIDEQ(""),
+		).
+		All(context.Background())
+
+	if err != nil {
+		if ent.IsNotFound(err) {
+			return nil, nil
+		}
+		return nil, fmt.Errorf("failed to query failed proof: %w", err)
+	}
+
+	return proofs, nil
+}
+
 // Get all pending proofs with a status of requested and a prover ID that is not empty.
 func (db *ProofDB) GetAllPendingProofs() ([]*ent.ProofRequest, error) {
 	proofs, err := db.client.ProofRequest.Query().