diff --git a/docs/tutorials/_category_.json b/docs/tutorials/_category_.json new file mode 100644 index 000000000..f27bca92d --- /dev/null +++ b/docs/tutorials/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Advanced Tutorials", + "position": 2, + "link": null +} \ No newline at end of file diff --git a/docs/tutorials/tutorials.md b/docs/tutorials/tutorials.md new file mode 100644 index 000000000..e6828c9fa --- /dev/null +++ b/docs/tutorials/tutorials.md @@ -0,0 +1,12 @@ +--- +sidebar_position: 0 +--- +# Tutorials + +## Advanced Tutorials + +This section provides a concise overview of tutorials focused on implementing vote extensions in the Cosmos SDK. Vote extensions are a powerful feature for enhancing the security and fairness of blockchain applications, particularly in scenarios like implementing oracles and mitigating auction front-running. + +* **Implementing Oracle with Vote Extensions** - This tutorial details how to use vote extensions for the implementation of a secure and reliable oracle within a blockchain application. It demonstrates the use of vote extensions to securely include oracle data submissions in blocks, ensuring the data's integrity and reliability for the blockchain. + +* **Mitigating Auction Front-Running with Vote Extensions** - Explore how to prevent auction front-running using vote extensions. This tutorial outlines the creation of a module aimed at mitigating front-running in nameservice auctions, emphasising the `ExtendVote`, `PrepareProposal`, and `ProcessProposal` functions to facilitate a fair auction process. \ No newline at end of file diff --git a/docs/tutorials/vote-extensions/_category_.json b/docs/tutorials/vote-extensions/_category_.json new file mode 100644 index 000000000..a2aecebd0 --- /dev/null +++ b/docs/tutorials/vote-extensions/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Vote Extensions Tutorials", + "position": 1, + "link": null +} \ No newline at end of file diff --git a/docs/tutorials/vote-extensions/auction-frontrunning/00-getting-started.md b/docs/tutorials/vote-extensions/auction-frontrunning/00-getting-started.md new file mode 100644 index 000000000..212f957da --- /dev/null +++ b/docs/tutorials/vote-extensions/auction-frontrunning/00-getting-started.md @@ -0,0 +1,40 @@ +# Getting Started + +## Table of Contents + +* [Getting Started](#overview-of-the-project) +* [Understanding Front-Running](01-understanding-front-running) +* [Mitigating Front-running with Vote Extensions](02-mitigating-front-running-with-vote-extensions) +* [Demo of Mitigating Front-Running](03-demo-of-mitigating-front-running) + +## Getting Started + +### Overview of the Project + +This tutorial outlines the development of a module designed to mitigate front-running in nameservice auctions. The following functions are central to this module: + +* `ExtendVote`: Gathers bids from the mempool and includes them in the vote extension to ensure a fair and transparent auction process. +* `PrepareProposal`: Processes the vote extensions from the previous block, creating a special transaction that encapsulates bids to be included in the current proposal. +* `ProcessProposal`: Validates that the first transaction in the proposal is the special transaction containing the vote extensions and ensures the integrity of the bids. + +In this advanced tutorial, we will be working with an example application that facilitates the auctioning of nameservices. To see what frontrunning and nameservices are [here](./01-understanding-frontrunning.md) This application provides a practical use case to explore the prevention of auction front-running, also known as "bid sniping", where a validator takes advantage of seeing a bid in the mempool to place their own higher bid before the original bid is processed. + +The tutorial will guide you through using the Cosmos SDK to mitigate front-running using vote extensions. The module will be built on top of the base blockchain provided in the `tutorials/base` directory and will use the `auction` module as a foundation. By the end of this tutorial, you will have a better understanding of how to prevent front-running in blockchain auctions, specifically in the context of nameservice auctioning. + +## What are Vote extensions? + +Vote extensions is arbitrary information which can be inserted into a block. This feature is part of ABCI 2.0, which is available for use in the SDK 0.50 release and part of the 0.38 CometBFT release. + +More information about vote extensions can be seen [here](https://docs.cosmos.network/main/build/abci/vote-extensions). + +## Requirements and Setup + +Before diving into the advanced tutorial on auction front-running simulation, ensure you meet the following requirements: + +* [Golang >1.21.5](https://golang.org/doc/install) installed +* Familiarity with the concepts of front-running and MEV, as detailed in [Understanding Front-Running](./01-understanding-frontrunning.md) +* Understanding of Vote Extensions as described [here](https://docs.cosmos.network/main/build/abci/vote-extensions) + +You will also need a foundational blockchain to build upon coupled with your own module. The `tutorials/base` directory has the necessary blockchain code to start your custom project with the Cosmos SDK. For the module, you can use the `auction` module provided in the `tutorials/auction/x/auction` directory as a reference but please be aware that all of the code needed to implement vote extensions is already implemented in this module. + +This will set up a strong base for your blockchain, enabling the integration of advanced features such as auction front-running simulation. diff --git a/docs/tutorials/vote-extensions/auction-frontrunning/01-understanding-frontrunning.md b/docs/tutorials/vote-extensions/auction-frontrunning/01-understanding-frontrunning.md new file mode 100644 index 000000000..31602b0e6 --- /dev/null +++ b/docs/tutorials/vote-extensions/auction-frontrunning/01-understanding-frontrunning.md @@ -0,0 +1,41 @@ +# Understanding Front-Running and more + +## Introduction + +Blockchain technology is vulnerable to practices that can affect the fairness and security of the network. Two such practices are front-running and Maximal Extractable Value (MEV), which are important for blockchain participants to understand. + +## What is Front-Running? + +Front-running is when someone, such as a validator, uses their ability to see pending transactions to execute their own transactions first, benefiting from the knowledge of upcoming transactions. In nameservice auctions, a front-runner might place a higher bid before the original bid is confirmed, unfairly winning the auction. + +## Nameservices and Nameservice Auctions + +Nameservices are human-readable identifiers on a blockchain, akin to internet domain names, that correspond to specific addresses or resources. They simplify interactions with typically long and complex blockchain addresses, allowing users to have a memorable and unique identifier for their blockchain address or smart contract. + +Nameservice auctions are the process by which these identifiers are bid on and acquired. To combat front-running—where someone might use knowledge of pending bids to place a higher bid first—mechanisms such as commit-reveal schemes, auction extensions, and fair sequencing are implemented. These strategies ensure a transparent and fair bidding process, reducing the potential for Maximal Extractable Value (MEV) exploitation. + +## What is Maximal Extractable Value (MEV)? + +MEV is the highest value that can be extracted by manipulating the order of transactions within a block, beyond the standard block rewards and fees. This has become more prominent with the growth of decentralised finance (DeFi), where transaction order can greatly affect profits. + +## Implications of MEV + +MEV can lead to: + +- **Network Security**: Potential centralisation, as those with more computational power might dominate the process, increasing the risk of attacks. +- **Market Fairness**: An uneven playing field where only a few can gain at the expense of the majority. +- **User Experience**: Higher fees and network congestion due to the competition for MEV. + +## Mitigating MEV and Front-Running + +Some solutions being developed to mitigate MEV and front-running, including: + +- **Time-delayed Transactions**: Random delays to make transaction timing unpredictable. +- **Private Transaction Pools**: Concealing transactions until they are mined. +- **Fair Sequencing Services**: Processing transactions in the order they are received. + +For this tutorial, we will be exploring the last solution, fair sequencing services, in the context of nameservice auctions. + +## Conclusion + +MEV and front-running are challenges to blockchain integrity and fairness. Ongoing innovation and implementation of mitigation strategies are crucial for the ecosystem's health and success. diff --git a/docs/tutorials/vote-extensions/auction-frontrunning/02-mitigating-front-running-with-vote-extensions.md b/docs/tutorials/vote-extensions/auction-frontrunning/02-mitigating-front-running-with-vote-extensions.md new file mode 100644 index 000000000..421b6ed8c --- /dev/null +++ b/docs/tutorials/vote-extensions/auction-frontrunning/02-mitigating-front-running-with-vote-extensions.md @@ -0,0 +1,331 @@ +# Mitigating Front-running with Vote Extensions + +## Table of Contents + +* [Prerequisites](#prerequisites) +* [Implementing Structs for Vote Extensions](#implementing-structs-for-vote-extensions) +* [Implementing Handlers and Configuring Handlers](#implementing-handlers-and-configuring-handlers) + +## Prerequisites + +Before implementing vote extensions to mitigate front-running, ensure you have a module ready to implement the vote extensions with. If you need to create or reference a similar module, see `x/auction` for guidance. + +In this section, we will discuss the steps to mitigate front-running using vote extensions. We will introduce new types within the `abci/types.go` file. These types will be used to handle the process of preparing proposals, processing proposals, and handling vote extensions. + +### Implementing Structs for Vote Extensions + +First, copy the following structs into the `abci/types.go` and each of these structs serves a specific purpose in the process of mitigating front-running using vote extensions: + +```go +package abci + +import ( + //import the necessary files +) + +type PrepareProposalHandler struct { + logger log.Logger + txConfig client.TxConfig + cdc codec.Codec + mempool *mempool.ThresholdMempool + txProvider provider.TxProvider + keyname string + runProvider bool +} +``` + +The `PrepareProposalHandler` struct is used to handle the preparation of a proposal in the consensus process. It contains several fields: logger for logging information and errors, txConfig for transaction configuration, cdc (Codec) for encoding and decoding transactions, mempool for referencing the set of unconfirmed transactions, txProvider for building the proposal with transactions, keyname for the name of the key used for signing transactions, and runProvider, a boolean flag indicating whether the provider should be run to build the proposal. + +```go +type ProcessProposalHandler struct { + TxConfig client.TxConfig + Codec codec.Codec + Logger log.Logger +} +``` + +After the proposal has been prepared and vote extensions have been included, the `ProcessProposalHandler` is used to process the proposal. This includes validating the proposal and the included vote extensions. The `ProcessProposalHandler` allows you to access the transaction configuration and codec, which are necessary for processing the vote extensions. + +```go +type VoteExtHandler struct { + logger log.Logger + currentBlock int64 + mempool *mempool.ThresholdMempool + cdc codec.Codec +} +``` + +This struct is used to handle vote extensions. It contains a logger for logging events, the current block number, a mempool for storing transactions, and a codec for encoding and decoding. Vote extensions are a key part of the process to mitigate front-running, as they allow for additional information to be included with each vote. + +```go +type InjectedVoteExt struct { + VoteExtSigner []byte + Bids [][]byte +} + +type InjectedVotes struct { + Votes []InjectedVoteExt +} +``` + +These structs are used to handle injected vote extensions. They include the signer of the vote extension and the bids associated with the vote extension. Each byte array in Bids is a serialised form of a bid transaction. Injected vote extensions are used to add additional information to a vote after it has been created, which can be useful for adding context or additional data to a vote. The serialised bid transactions provide a way to include complex transaction data in a compact, efficient format. + +```go +type AppVoteExtension struct { + Height int64 + Bids [][]byte +} +``` + +This struct is used for application vote extensions. It includes the height of the block and the bids associated with the vote extension. Application vote extensions are used to add additional information to a vote at the application level, which can be useful for adding context or additional data to a vote that is specific to the application. + +```go +type SpecialTransaction struct { + Height int + Bids [][]byte +} +``` + +This struct is used for special transactions. It includes the height of the block and the bids associated with the transaction. Special transactions are used for transactions that need to be handled differently from regular transactions, such as transactions that are part of the process to mitigate front-running. + +### Implementing Handlers and Configuring Handlers + +To establish the `VoteExtensionHandler`, follow these steps: + +1. Navigate to the `abci/proposal.go` file. This is where we will implement the `VoteExtensionHandler``. + +2. Implement the `NewVoteExtensionHandler` function. This function is a constructor for the `VoteExtHandler` struct. It takes a logger, a mempool, and a codec as parameters and returns a new instance of `VoteExtHandler`. + +```go +func NewVoteExtensionHandler(lg log.Logger, mp *mempool.ThresholdMempool, cdc codec.Codec) *VoteExtHandler { + return &VoteExtHandler{ + logger: lg, + mempool: mp, + cdc: cdc, + } +} +``` + +3. Implement the `ExtendVoteHandler()` method. This method should handle the logic of extending votes, including inspecting the mempool and submitting a list of all pending bids. This will allow you to access the list of unconfirmed transactions in the abci.`RequestPrepareProposal` during the ensuing block. + +```go +func (h *VoteExtHandler) ExtendVoteHandler() sdk.ExtendVoteHandler { + return func(ctx sdk.Context, req *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) { + h.logger.Info(fmt.Sprintf("Extending votes at block height : %v", req.Height)) + + voteExtBids := [][]byte{} + + // Get mempool txs + itr := h.mempool.SelectPending(context.Background(), nil) + for itr != nil { + tmptx := itr.Tx() + sdkMsgs := tmptx.GetMsgs() + + // Iterate through msgs, check for any bids + for _, msg := range sdkMsgs { + switch msg := msg.(type) { + case *nstypes.MsgBid: + // Marshal sdk bids to []byte + bz, err := h.cdc.Marshal(msg) + if err != nil { + h.logger.Error(fmt.Sprintf("Error marshalling VE Bid : %v", err)) + break + } + voteExtBids = append(voteExtBids, bz) + default: + } + } + + // Move tx to ready pool + err := h.mempool.Update(context.Background(), tmptx) + + // Remove tx from app side mempool + if err != nil { + h.logger.Info(fmt.Sprintf("Unable to update mempool tx: %v", err)) + } + + itr = itr.Next() + } + + // Create vote extension + voteExt := AppVoteExtension{ + Height: req.Height, + Bids: voteExtBids, + } + + // Encode Vote Extension + bz, err := json.Marshal(voteExt) + if err != nil { + return nil, fmt.Errorf("Error marshalling VE: %w", err) + } + + return &abci.ResponseExtendVote{VoteExtension: bz}, nil +} +``` + +4. Configure the handler in `app/app.go` as shown below + +```go +bApp := baseapp.NewBaseApp(AppName, logger, db, txConfig.TxDecoder(), baseAppOptions...) +voteExtHandler := abci2.NewVoteExtensionHandler(logger, mempool, appCodec) +bApp.SetExtendVoteHandler(voteExtHandler.ExtendVoteHandler()) +``` + +To give a bit of context on what is happening above, we first create a new instance of `VoteExtensionHandler` with the necessary dependencies (logger, mempool, and codec). Then, we set this handler as the `ExtendVoteHandler` for our application. This means that whenever a vote needs to be extended, our custom `ExtendVoteHandler()` method will be called. + +To test if vote extensions have been propagated, add the following to the `PrepareProposalHandler`: + +```go +if req.Height > 2 { + voteExt := req.GetLocalLastCommit() + h.logger.Info(fmt.Sprintf("🛠️ :: Get vote extensions: %v", voteExt)) +} +``` + +This is how the whole function should look: + +```go +func (h *PrepareProposalHandler) PrepareProposalHandler() sdk.PrepareProposalHandler { + return func(ctx sdk.Context, req *abci.RequestPrepareProposal) (*abci.ResponsePrepareProposal, error) { + h.logger.Info(fmt.Sprintf("🛠️ :: Prepare Proposal")) + var proposalTxs [][]byte + + var txs []sdk.Tx + + // Get Vote Extensions + if req.Height > 2 { + voteExt := req.GetLocalLastCommit() + h.logger.Info(fmt.Sprintf("🛠️ :: Get vote extensions: %v", voteExt)) + } + + itr := h.mempool.Select(context.Background(), nil) + for itr != nil { + tmptx := itr.Tx() + + txs = append(txs, tmptx) + itr = itr.Next() + } + h.logger.Info(fmt.Sprintf("🛠️ :: Number of Transactions available from mempool: %v", len(txs))) + + if h.runProvider { + tmpMsgs, err := h.txProvider.BuildProposal(ctx, txs) + if err != nil { + h.logger.Error(fmt.Sprintf("❌️ :: Error Building Custom Proposal: %v", err)) + } + txs = tmpMsgs + } + + for _, sdkTxs := range txs { + txBytes, err := h.txConfig.TxEncoder()(sdkTxs) + if err != nil { + h.logger.Info(fmt.Sprintf("❌~Error encoding transaction: %v", err.Error())) + } + proposalTxs = append(proposalTxs, txBytes) + } + + h.logger.Info(fmt.Sprintf("🛠️ :: Number of Transactions in proposal: %v", len(proposalTxs))) + + return &abci.ResponsePrepareProposal{Txs: proposalTxs}, nil + } +} +``` + +As mentioned above, we check if vote extensions have been propagated, you can do this by checking the logs for any relevant messages such as `🛠️ :: Get vote extensions:`. If the logs do not provide enough information, you can also reinitialise your local testing environment by running the `./scripts/single_node/setup.sh` script again. + +5. Implement the `ProcessProposalHandler()`. This function is responsible for processing the proposal. It should handle the logic of processing vote extensions, including inspecting the proposal and validating the bids. + +```go +func (h *ProcessProposalHandler) ProcessProposalHandler() sdk.ProcessProposalHandler { + return func(ctx sdk.Context, req *abci.RequestProcessProposal) (resp *abci.ResponseProcessProposal, err error) { + h.Logger.Info(fmt.Sprintf("⚙️ :: Process Proposal")) + + // The first transaction will always be the Special Transaction + numTxs := len(req.Txs) + + h.Logger.Info(fmt.Sprintf("⚙️:: Number of transactions :: %v", numTxs)) + + if numTxs >= 1 { + var st SpecialTransaction + err = json.Unmarshal(req.Txs[0], &st) + if err != nil { + h.Logger.Error(fmt.Sprintf("❌️:: Error unmarshalling special Tx in Process Proposal :: %v", err)) + } + if len(st.Bids) > 0 { + h.Logger.Info(fmt.Sprintf("⚙️:: There are bids in the Special Transaction")) + var bids []nstypes.MsgBid + for i, b := range st.Bids { + var bid nstypes.MsgBid + h.Codec.Unmarshal(b, &bid) + h.Logger.Info(fmt.Sprintf("⚙️:: Special Transaction Bid No %v :: %v", i, bid)) + bids = append(bids, bid) + } + // Validate Bids in Tx + txs := req.Txs[1:] + ok, err := ValidateBids(h.TxConfig, bids, txs, h.Logger) + if err != nil { + h.Logger.Error(fmt.Sprintf("❌️:: Error validating bids in Process Proposal :: %v", err)) + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } + if !ok { + h.Logger.Error(fmt.Sprintf("❌️:: Unable to validate bids in Process Proposal :: %v", err)) + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } + h.Logger.Info("⚙️:: Successfully validated bids in Process Proposal") + } + } + + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_ACCEPT}, nil + } +} +``` + +6. Implement the `ProcessVoteExtensions()` function. This function should handle the logic of processing vote extensions, including validating the bids. + +```go +func processVoteExtensions(req *abci.RequestPrepareProposal, log log.Logger) (SpecialTransaction, error) { + log.Info(fmt.Sprintf("🛠️ :: Process Vote Extensions")) + + // Create empty response + st := SpecialTransaction{ + 0, + [][]byte{}, + } + + // Get Vote Ext for H-1 from Req + voteExt := req.GetLocalLastCommit() + votes := voteExt.Votes + + // Iterate through votes + var ve AppVoteExtension + for _, vote := range votes { + // Unmarshal to AppExt + err := json.Unmarshal(vote.VoteExtension, &ve) + if err != nil { + log.Error(fmt.Sprintf("❌ :: Error unmarshalling Vote Extension")) + } + + st.Height = int(ve.Height) + + // If Bids in VE, append to Special Transaction + if len(ve.Bids) > 0 { + log.Info("🛠️ :: Bids in VE") + for _, b := range ve.Bids { + st.Bids = append(st.Bids, b) + } + } + } + + return st, nil +} +``` + +7. Configure the `ProcessProposalHandler()` in app/app.go: + +```go +processPropHandler := abci2.ProcessProposalHandler{app.txConfig, appCodec, logger} +bApp.SetProcessProposal(processPropHandler.ProcessProposalHandler()) +``` + +This sets the `ProcessProposalHandler()` for our application. This means that whenever a proposal needs to be processed, our custom `ProcessProposalHandler()` method will be called. + +To test if the proposal processing and vote extensions are working correctly, you can check the logs for any relevant messages. If the logs do not provide enough information, you can also reinitialize your local testing environment by running `./scripts/single_node/setup.sh` script. diff --git a/docs/tutorials/vote-extensions/auction-frontrunning/03-demo-of-mitigating-front-running.md b/docs/tutorials/vote-extensions/auction-frontrunning/03-demo-of-mitigating-front-running.md new file mode 100644 index 000000000..48cf22a40 --- /dev/null +++ b/docs/tutorials/vote-extensions/auction-frontrunning/03-demo-of-mitigating-front-running.md @@ -0,0 +1,106 @@ +# Demo of Mitigating Front-Running with Vote Extensions + +The purpose of this demo is to test the implementation of the `VoteExtensionHandler` and `PrepareProposalHandler` that we have just added to the codebase. These handlers are designed to mitigate front-running by ensuring that all validators have a consistent view of the mempool when preparing proposals. + +In this demo, we are using a 3 validator network. The Beacon validator is special because it has a custom transaction provider enabled. This means that it can potentially manipulate the order of transactions in a proposal to its advantage (i.e., front-running). + +1. Bootstrap the validator network: This sets up a network with 3 validators. The script `./scripts/configure.sh is used to configure the network and the validators. + +```shell +cd scripts +configure.sh +``` + +If this doesnt work please ensure you have run `make build` in the `tutorials/nameservice/base` directory. + + +2. Have alice attempt to reserve `bob.cosmos`: This is a normal transaction that alice wants to execute. The script ``./scripts/reserve.sh "bob.cosmos"` is used to send this transaction. + +```shell +bash reserve.sh "bob.cosmos" +``` + +3. Query to verify the name has been reserved: This is to check the result of the transaction. The script `./scripts/whois.sh "bob.cosmos"` is used to query the state of the blockchain. + +```shell +bash whois.sh "bob.cosmos" +``` + +It should return: + +```{ + "name": { + "name": "bob.cosmos", + "owner": "cosmos1nq9wuvuju4jdmpmzvxmg8zhhu2ma2y2l2pnu6w", + "resolve_address": "cosmos1h6zy2kn9efxtw5z22rc5k9qu7twl70z24kr3ht", + "amount": [ + { + "denom": "uatom", + "amount": "1000" + } + ] + } +} +``` + +To detect front-running attempts by the beacon, scrutinise the logs during the `ProcessProposal` stage. Open the logs for each validator, including the beacon, `val1`, and `val2`, to observe the following behavior. Open the log file of the validator node. The location of this file can vary depending on your setup, but it's typically located in a directory like `$HOME/cosmos/nodes/#{validator}/logs`. The directory in this case will be under the validator so, `beacon` `val1` or `val2`. Run the following to tail the logs of the validator or beacon: + +```shell +tail -f $HOME/cosmos/nodes/#{validator}/logs +``` + +```shell +2:47PM ERR ❌️:: Detected invalid proposal bid :: name:"bob.cosmos" resolveAddress:"cosmos1wmuwv38pdur63zw04t0c78r2a8dyt08hf9tpvd" owner:"cosmos1wmuwv38pdur63zw04t0c78r2a8dyt08hf9tpvd" amount: module=server +2:47PM ERR ❌️:: Unable to validate bids in Process Proposal :: module=server +2:47PM ERR prevote step: state machine rejected a proposed block; this should not happen:the proposer may be misbehaving; prevoting nil err=null height=142 module=consensus round=0 +``` + + +4. List the Beacon's keys: This is to verify the addresses of the validators. The script `./scripts/list-beacon-keys.sh` is used to list the keys. + +```shell +bash list-beacon-keys.sh +``` + +We should receive something similar to the following: + +```shell +[ + { + "name": "alice", + "type": "local", + "address": "cosmos1h6zy2kn9efxtw5z22rc5k9qu7twl70z24kr3ht", + "pubkey": "{\"@type\":\"/cosmos.crypto.secp256k1.PubKey\",\"key\":\"A32cvBUkNJz+h2vld4A5BxvU5Rd+HyqpR3aGtvEhlm4C\"}" + }, + { + "name": "barbara", + "type": "local", + "address": "cosmos1nq9wuvuju4jdmpmzvxmg8zhhu2ma2y2l2pnu6w", + "pubkey": "{\"@type\":\"/cosmos.crypto.secp256k1.PubKey\",\"key\":\"Ag9PFsNyTQPoJdbyCWia5rZH9CrvSrjMsk7Oz4L3rXQ5\"}" + }, + { + "name": "beacon-key", + "type": "local", + "address": "cosmos1ez9a6x7lz4gvn27zr368muw8jeyas7sv84lfup", + "pubkey": "{\"@type\":\"/cosmos.crypto.secp256k1.PubKey\",\"key\":\"AlzJZMWyN7lass710TnAhyuFKAFIaANJyw5ad5P2kpcH\"}" + }, + { + "name": "cindy", + "type": "local", + "address": "cosmos1m5j6za9w4qc2c5ljzxmm2v7a63mhjeag34pa3g", + "pubkey": "{\"@type\":\"/cosmos.crypto.secp256k1.PubKey\",\"key\":\"A6F1/3yot5OpyXoSkBbkyl+3rqBkxzRVSJfvSpm/AvW5\"}" + } +] +``` + +This allows us to match up the addresses and see that the bid was not front run by the beacon due as the resolve address is Alice's address and not the beacons address. + +By running this demo, we can verify that the `VoteExtensionHandler` and `PrepareProposalHandler` are working as expected and that they are able to prevent front-running. + +## Conclusion + +In this tutorial, we've tackled front-running and MEV, focusing on nameservice auctions' vulnerability to these issues. We've explored vote extensions, a key feature of ABCI 2.0, and integrated them into a Cosmos SDK application. + +Through practical exercises, you've implemented vote extensions, and tested their effectiveness in creating a fair auction system. You've gained practical insights by configuring a validator network and analysing blockchain logs. + +Keep experimenting with these concepts, engage with the community, and stay updated on new advancements. The knowledge you've acquired here is crucial for developing secure and fair blockchain applications. diff --git a/docs/tutorials/vote-extensions/auction-frontrunning/_category_.json b/docs/tutorials/vote-extensions/auction-frontrunning/_category_.json new file mode 100644 index 000000000..aab0cfdf6 --- /dev/null +++ b/docs/tutorials/vote-extensions/auction-frontrunning/_category_.json @@ -0,0 +1,5 @@ +{ + "label": " Mitigating Auction Front-Running Tutorial", + "position": 0, + "link": null +} \ No newline at end of file diff --git a/docs/tutorials/vote-extensions/oracle/00-getting-started.md b/docs/tutorials/vote-extensions/oracle/00-getting-started.md new file mode 100644 index 000000000..c96886a64 --- /dev/null +++ b/docs/tutorials/vote-extensions/oracle/00-getting-started.md @@ -0,0 +1,35 @@ +# Getting Started + +## Table of Contents + +- [Getting Started](00-getting-started.md) +- [What is an Oracle?](01-what-is-an-oracle.md) +- [Implementing Vote Extensions](02-implementing-vote-extensions.md) +- [Testing the Oracle Module](03-testing-oracle.md) + +## Prerequisites + +Before you start with this tutorial, make sure you have: + +- A working chain project. This tutorial won't cover the steps of creating a new chain/module. +- Familiarity with the Cosmos SDK. If you're not, we suggest you start with [Cosmos SDK Tutorials](https://tutorials.cosmos.network), as ABCI++ is considered an advanced topic. +- Read and understood [What is an Oracle?](01-what-is-an-oracle.md). This provides necessary background information for understanding the Oracle module. +- Basic understanding of Go programming language. + +## What are Vote extensions? + +Vote extensions is arbitrary information which can be inserted into a block. This feature is part of ABCI 2.0, which is available for use in the SDK 0.50 release and part of the 0.38 CometBFT release. + +More information about vote extensions can be seen [here](https://docs.cosmos.network/main/build/abci/vote-extensions). + +## Overview of the project + +We’ll go through the creation of a simple price oracle module focusing on the vote extensions implementation, ignoring the details inside the price oracle itself. + +We’ll go through the implementation of: + +- `ExtendVote` to get information from external price APIs. +- `VerifyVoteExtension` to check that the format of the provided votes is correct. +- `PrepareProposal` to process the vote extensions from the previous block and include them into the proposal as a transaction. +- `ProcessProposal` to check that the first transaction in the proposal is actually a “special tx” that contains the price information. +- `PreBlocker` to make price information available during FinalizeBlock. diff --git a/docs/tutorials/vote-extensions/oracle/01-what-is-an-oracle.md b/docs/tutorials/vote-extensions/oracle/01-what-is-an-oracle.md new file mode 100644 index 000000000..9d50ddb36 --- /dev/null +++ b/docs/tutorials/vote-extensions/oracle/01-what-is-an-oracle.md @@ -0,0 +1,13 @@ +# What is an Oracle? + +An oracle in blockchain technology is a system that provides external data to a blockchain network. It acts as a source of information that is not natively accessible within the blockchain's closed environment. This can range from financial market prices to real-world event, making it crucial for decentralised applications. + +## Oracle in the Cosmos SDK + +In the Cosmos SDK, an oracle module can be implemented to provide external data to the blockchain. This module can use features like vote extensions to submit additional data during the consensus process, which can then be used by the blockchain to update its state with information from the outside world. + +For instance, a price oracle module in the Cosmos SDK could supply timely and accurate asset price information, which is vital for various financial operations within the blockchain ecosystem. + +## Conclusion + +Oracles are essential for blockchains to interact with external data, enabling them to respond to real-world information and events. Their implementation is key to the reliability and robustness of blockchain networks. diff --git a/docs/tutorials/vote-extensions/oracle/02-implementing-vote-extensions.md b/docs/tutorials/vote-extensions/oracle/02-implementing-vote-extensions.md new file mode 100644 index 000000000..0a33e11f6 --- /dev/null +++ b/docs/tutorials/vote-extensions/oracle/02-implementing-vote-extensions.md @@ -0,0 +1,219 @@ +# Implementing Vote Extensions + +## Implement ExtendVote + +First we’ll create the `OracleVoteExtension` struct, this is the object that will be marshaled as bytes and signed by the validator. + +In our example we’ll use JSON to marshal the vote extension for simplicity but we recommend to find an encoding that produces a smaller output, given that large vote extensions could impact CometBFT’s performance. Custom encodings and compressed bytes can be used out of the box. + +```go +// OracleVoteExtension defines the canonical vote extension structure. +type OracleVoteExtension struct { + Height int64 + Prices map[string]math.LegacyDec +} +``` + +Then we’ll create a `VoteExtensionsHandler` struct that contains everything we need to query for prices. + +```go +type VoteExtHandler struct { + logger log.Logger + currentBlock int64 // current block height + lastPriceSyncTS time.Time // last time we synced prices + providerTimeout time.Duration // timeout for fetching prices from providers + providers map[string]Provider // mapping of provider name to provider (e.g. Binance -> BinanceProvider) + providerPairs map[string][]keeper.CurrencyPair // mapping of provider name to supported pairs (e.g. Binance -> [ATOM/USD]) + + Keeper keeper.Keeper // keeper of our oracle module +} +``` + +Finally, a function that returns `sdk.ExtendVoteHandler` is needed too, and this is where our vote extension logic will live. + +```go +func (h *VoteExtHandler) ExtendVoteHandler() sdk.ExtendVoteHandler { + return func(ctx sdk.Context, req *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) { + // here we'd have a helper function that gets all the prices and does a weighted average using the volume of each market + prices := h.getAllVolumeWeightedPrices() + + voteExt := OracleVoteExtension{ + Height: req.Height, + Prices: prices, + } + + bz, err := json.Marshal(voteExt) + if err != nil { + return nil, fmt.Errorf("failed to marshal vote extension: %w", err) + } + + return &abci.ResponseExtendVote{VoteExtension: bz}, nil + } +} +``` + +As you can see above, the creation of a vote extension is pretty simple and we just have to return bytes. CometBFT will handle the signing of these bytes for us. We ignored the process of getting the prices but you can see a more complete example [here:](../base/x/oracle/abci/vote_extensions.go) + +Here we’ll do some simple checks like: + +* Is the vote extension unmarshaled correctly? +* Is the vote extension for the right height? +* Some other validation, for example, are the prices from this extension too deviated from my own prices? Or maybe checks that can detect malicious behavior. + +```go +func (h *VoteExtHandler) VerifyVoteExtensionHandler() sdk.VerifyVoteExtensionHandler { + return func(ctx sdk.Context, req *abci.RequestVerifyVoteExtension) (*abci.ResponseVerifyVoteExtension, error) { + var voteExt OracleVoteExtension + err := json.Unmarshal(req.VoteExtension, &voteExt) + if err != nil { + return nil, fmt.Errorf("failed to unmarshal vote extension: %w", err) + } + + if voteExt.Height != req.Height { + return nil, fmt.Errorf("vote extension height does not match request height; expected: %d, got: %d", req.Height, voteExt.Height) + } + + // Verify incoming prices from a validator are valid. Note, verification during + // VerifyVoteExtensionHandler MUST be deterministic. For brevity and demo + // purposes, we omit implementation. + if err := h.verifyOraclePrices(ctx, voteExt.Prices); err != nil { + return nil, fmt.Errorf("failed to verify oracle prices from validator %X: %w", req.ValidatorAddress, err) + } + + return &abci.ResponseVerifyVoteExtension{Status: abci.ResponseVerifyVoteExtension_ACCEPT}, nil + } +} +``` + +## Implement PrepareProposal + +```go +type ProposalHandler struct { + logger log.Logger + keeper keeper.Keeper // our oracle module keeper + valStore baseapp.ValidatorStore // to get the current validators' pubkeys +} +``` + +And we create the struct for our “special tx”, that will contain the prices and the votes so validators can later re-check in ProcessPRoposal that they get the same result than the block’s proposer. With this we could also check if all the votes have been used by comparing the votes received in ProcessProposal. + +```go +type StakeWeightedPrices struct { + StakeWeightedPrices map[string]math.LegacyDec + ExtendedCommitInfo abci.ExtendedCommitInfo +} +``` + +Now we create the `PrepareProposalHandler`. In this step we’ll first check if the vote extensions’ signatures are correct using a helper function called ValidateVoteExtensions from the baseapp pacakge. + +```go +func (h *ProposalHandler) PrepareProposal() sdk.PrepareProposalHandler { + return func(ctx sdk.Context, req *abci.RequestPrepareProposal) (*abci.ResponsePrepareProposal, error) { + err := baseapp.ValidateVoteExtensions(ctx, h.valStore, req.Height, ctx.ChainID(), req.LocalLastCommit) + if err != nil { + return nil, err + } +... +``` + +Then we proceed to make the calculations only if the current height if higher than the height at which vote extensions have been enabled. Remember that vote extensions are made available to the block proposer on the next block at which they are produced/enabled. + +```go +... + proposalTxs := req.Txs + + if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight { + stakeWeightedPrices, err := h.computeStakeWeightedOraclePrices(ctx, req.LocalLastCommit) + if err != nil { + return nil, errors.New("failed to compute stake-weighted oracle prices") + } + + injectedVoteExtTx := StakeWeightedPrices{ + StakeWeightedPrices: stakeWeightedPrices, + ExtendedCommitInfo: req.LocalLastCommit, + } +... +``` + +Finally we inject the result as a transaction at a specific location, usually at the beginning of the block: + +## Implement ProcessProposal + +Now we can implement the method that all validators will execute to ensure the proposer is doing his work correctly. + +Here, if vote extensions are enabled, we’ll check if the tx at index 0 is an injected vote extension + +```go +func (h *ProposalHandler) ProcessProposal() sdk.ProcessProposalHandler { + return func(ctx sdk.Context, req *abci.RequestProcessProposal) (*abci.ResponseProcessProposal, error) { + if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight { + var injectedVoteExtTx StakeWeightedPrices + if err := json.Unmarshal(req.Txs[0], &injectedVoteExtTx); err != nil { + h.logger.Error("failed to decode injected vote extension tx", "err", err) + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } +... +``` + +Then we re-validate the vote extensions signatures using +baseapp.ValidateVoteExtensions, re-calculate the results (just like in PrepareProposal) and compare them with the results we got from the injected tx. + +```go + err := baseapp.ValidateVoteExtensions(ctx, h.valStore, req.Height, ctx.ChainID(), injectedVoteExtTx.ExtendedCommitInfo) + if err != nil { + return nil, err + } + + // Verify the proposer's stake-weighted oracle prices by computing the same + // calculation and comparing the results. We omit verification for brevity + // and demo purposes. + stakeWeightedPrices, err := h.computeStakeWeightedOraclePrices(ctx, injectedVoteExtTx.ExtendedCommitInfo) + if err != nil { + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } + + if err := compareOraclePrices(injectedVoteExtTx.StakeWeightedPrices, stakeWeightedPrices); err != nil { + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } + } + + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_ACCEPT}, nil + } +} +``` + +Important: In this example we avoided using the mempool and other basics, please refer to the DefaultProposalHandler for a complete implementation: [https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/baseapp/abci_utils.go](https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/baseapp/abci_utils.go) + +## Implement PreBlocker + +Now validators are extending their vote, verifying other votes and including the result in the block. But how do we actually make use of this result? This is done in the PreBlocker which is code that is run before any other code during FinalizeBlock so we make sure we make this information available to the chain and its modules during the entire block execution (from BeginBlock). + +At this step we know that the injected tx is well-formatted and has been verified by the validators participating in consensus, so making use of it is straightforward. Just check if vote extensions are enabled, pick up the first transaction and use a method in your module’s keeper to set the result. + +```go +func (h *ProposalHandler) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) (*sdk.ResponsePreBlock, error) { + res := &sdk.ResponsePreBlock{} + if len(req.Txs) == 0 { + return res, nil + } + + if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight { + var injectedVoteExtTx StakeWeightedPrices + if err := json.Unmarshal(req.Txs[0], &injectedVoteExtTx); err != nil { + h.logger.Error("failed to decode injected vote extension tx", "err", err) + return nil, err + } + + // set oracle prices using the passed in context, which will make these prices available in the current block + if err := h.keeper.SetOraclePrices(ctx, injectedVoteExtTx.StakeWeightedPrices); err != nil { + return nil, err + } + } + return res, nil +} + +``` + +## Conclusion + +In this tutorial, we've created a simple price oracle module that incorporates vote extensions. We've seen how to implement `ExtendVote`, `VerifyVoteExtension`, `PrepareProposal`, `ProcessProposal`, and `PreBlocker` to handle the voting and verification process of vote extensions, as well as how to make use of the results during the block execution. diff --git a/docs/tutorials/vote-extensions/oracle/03-testing-oracle.md b/docs/tutorials/vote-extensions/oracle/03-testing-oracle.md new file mode 100644 index 000000000..99dd467c3 --- /dev/null +++ b/docs/tutorials/vote-extensions/oracle/03-testing-oracle.md @@ -0,0 +1,57 @@ +# Testing the Oracle Module + +We will guide you through the process of testing the Oracle module in your application. The Oracle module uses vote extensions to provide current price data. + +## Step 1: Compile and Install the Application + +First, we need to compile and install the application. Please ensure you are in the `tutorials/oracle/base` directory. Run the following command in your terminal: + +```shell +make install +``` + +This command compiles the application and moves the resulting binary to a location in your system's PATH. + +## Step 2: Initialise the Application + +Next, we need to initialise the application. Run the following command in your terminal: + +```shell +make init +``` + +This command runs the script `tutorials/oracle/base/scripts/init.sh`, which sets up the necessary configuration for your application to run. This includes creating the `app.toml` configuration file and initialising the blockchain with a genesis block. + +## Step 3: Start the Application + +Now, we can start the application. Run the following command in your terminal: + +```shell +tutoriald start +``` + +This command starts your application, begins the blockchain node, and starts processing transactions. + +## Step 4: Query the Oracle Prices + +Finally, we can query the current prices from the Oracle module. Run the following command in your terminal: + +```shell +bash tutoriald q oracle prices +``` + +This command queries the current prices from the Oracle module. The expected output shows that the vote extensions were successfully included in the block and the Oracle module was able to retrieve the price data. + +## Understanding Vote Extensions in Oracle + +In the Oracle module, the `ExtendVoteHandler` function is responsible for creating the vote extensions. This function fetches the current prices from the provider, creates a `OracleVoteExtension` struct with these prices, and then marshals this struct into bytes. These bytes are then set as the vote extension. + +In the context of testing, the Oracle module uses a mock provider to simulate the behavior of a real price provider. This mock provider is defined in the mockprovider package and is used to return predefined prices for specific currency pairs. + +## Conclusion + +In this tutorial, we've delved into the concept of Oracle's in blockchain technology, focusing on their role in providing external data to a blockchain network. We've explored vote extensions, a powerful feature of ABCI++, and integrated them into a Cosmos SDK application to create a price oracle module. + +Through hands-on exercises, you've implemented vote extensions, and tested their effectiveness in providing timely and accurate asset price information. You've gained practical insights by setting up a mock provider for testing and analysing the process of extending votes, verifying vote extensions, and preparing and processing proposals. + +Keep experimenting with these concepts, engage with the community, and stay updated on new advancements. The knowledge you've acquired here is crucial for developing robust and reliable blockchain applications that can interact with real-world data. diff --git a/docs/tutorials/vote-extensions/oracle/_category_.json b/docs/tutorials/vote-extensions/oracle/_category_.json new file mode 100644 index 000000000..b63ffe2ff --- /dev/null +++ b/docs/tutorials/vote-extensions/oracle/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Oracle Tutorial", + "position": 1, + "link": null +} \ No newline at end of file diff --git a/docusaurus.config.js b/docusaurus.config.js index a42ea0fb8..8020e02a4 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -123,7 +123,14 @@ const config = { { type: "doc", label: "User Guides", - docId: "user/user", // I find it weird that it points to the keyring and not a common page + docId: "user/user", + position: "left", + }, + { + type: "doc", + label: "Tutorials", + collapsed: false, + docId: "tutorials/vote-extensions/auction-frontrunning/getting-started", position: "left", }, { diff --git a/sidebars.js b/sidebars.js index 40c127c58..fa154a6c3 100644 --- a/sidebars.js +++ b/sidebars.js @@ -17,6 +17,7 @@ const sidebars = { learnSidebar: [{ type: "autogenerated", dirName: "learn" }], buildSidebar: [{ type: "autogenerated", dirName: "build" }], userSidebar: [{ type: "autogenerated", dirName: "user" }], + tutorialsSidebar: [{ type: "autogenerated", dirName: "tutorials" }], }; module.exports = sidebars; diff --git a/versioned_docs/version-0.50/tutorials/_category_.json b/versioned_docs/version-0.50/tutorials/_category_.json new file mode 100644 index 000000000..f27bca92d --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Advanced Tutorials", + "position": 2, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.50/tutorials/tutorials.md b/versioned_docs/version-0.50/tutorials/tutorials.md new file mode 100644 index 000000000..e6828c9fa --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/tutorials.md @@ -0,0 +1,12 @@ +--- +sidebar_position: 0 +--- +# Tutorials + +## Advanced Tutorials + +This section provides a concise overview of tutorials focused on implementing vote extensions in the Cosmos SDK. Vote extensions are a powerful feature for enhancing the security and fairness of blockchain applications, particularly in scenarios like implementing oracles and mitigating auction front-running. + +* **Implementing Oracle with Vote Extensions** - This tutorial details how to use vote extensions for the implementation of a secure and reliable oracle within a blockchain application. It demonstrates the use of vote extensions to securely include oracle data submissions in blocks, ensuring the data's integrity and reliability for the blockchain. + +* **Mitigating Auction Front-Running with Vote Extensions** - Explore how to prevent auction front-running using vote extensions. This tutorial outlines the creation of a module aimed at mitigating front-running in nameservice auctions, emphasising the `ExtendVote`, `PrepareProposal`, and `ProcessProposal` functions to facilitate a fair auction process. \ No newline at end of file diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/_category_.json b/versioned_docs/version-0.50/tutorials/vote-extensions/_category_.json new file mode 100644 index 000000000..a2aecebd0 --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Vote Extensions Tutorials", + "position": 1, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/00-getting-started.md b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/00-getting-started.md new file mode 100644 index 000000000..212f957da --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/00-getting-started.md @@ -0,0 +1,40 @@ +# Getting Started + +## Table of Contents + +* [Getting Started](#overview-of-the-project) +* [Understanding Front-Running](01-understanding-front-running) +* [Mitigating Front-running with Vote Extensions](02-mitigating-front-running-with-vote-extensions) +* [Demo of Mitigating Front-Running](03-demo-of-mitigating-front-running) + +## Getting Started + +### Overview of the Project + +This tutorial outlines the development of a module designed to mitigate front-running in nameservice auctions. The following functions are central to this module: + +* `ExtendVote`: Gathers bids from the mempool and includes them in the vote extension to ensure a fair and transparent auction process. +* `PrepareProposal`: Processes the vote extensions from the previous block, creating a special transaction that encapsulates bids to be included in the current proposal. +* `ProcessProposal`: Validates that the first transaction in the proposal is the special transaction containing the vote extensions and ensures the integrity of the bids. + +In this advanced tutorial, we will be working with an example application that facilitates the auctioning of nameservices. To see what frontrunning and nameservices are [here](./01-understanding-frontrunning.md) This application provides a practical use case to explore the prevention of auction front-running, also known as "bid sniping", where a validator takes advantage of seeing a bid in the mempool to place their own higher bid before the original bid is processed. + +The tutorial will guide you through using the Cosmos SDK to mitigate front-running using vote extensions. The module will be built on top of the base blockchain provided in the `tutorials/base` directory and will use the `auction` module as a foundation. By the end of this tutorial, you will have a better understanding of how to prevent front-running in blockchain auctions, specifically in the context of nameservice auctioning. + +## What are Vote extensions? + +Vote extensions is arbitrary information which can be inserted into a block. This feature is part of ABCI 2.0, which is available for use in the SDK 0.50 release and part of the 0.38 CometBFT release. + +More information about vote extensions can be seen [here](https://docs.cosmos.network/main/build/abci/vote-extensions). + +## Requirements and Setup + +Before diving into the advanced tutorial on auction front-running simulation, ensure you meet the following requirements: + +* [Golang >1.21.5](https://golang.org/doc/install) installed +* Familiarity with the concepts of front-running and MEV, as detailed in [Understanding Front-Running](./01-understanding-frontrunning.md) +* Understanding of Vote Extensions as described [here](https://docs.cosmos.network/main/build/abci/vote-extensions) + +You will also need a foundational blockchain to build upon coupled with your own module. The `tutorials/base` directory has the necessary blockchain code to start your custom project with the Cosmos SDK. For the module, you can use the `auction` module provided in the `tutorials/auction/x/auction` directory as a reference but please be aware that all of the code needed to implement vote extensions is already implemented in this module. + +This will set up a strong base for your blockchain, enabling the integration of advanced features such as auction front-running simulation. diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/01-understanding-frontrunning.md b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/01-understanding-frontrunning.md new file mode 100644 index 000000000..31602b0e6 --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/01-understanding-frontrunning.md @@ -0,0 +1,41 @@ +# Understanding Front-Running and more + +## Introduction + +Blockchain technology is vulnerable to practices that can affect the fairness and security of the network. Two such practices are front-running and Maximal Extractable Value (MEV), which are important for blockchain participants to understand. + +## What is Front-Running? + +Front-running is when someone, such as a validator, uses their ability to see pending transactions to execute their own transactions first, benefiting from the knowledge of upcoming transactions. In nameservice auctions, a front-runner might place a higher bid before the original bid is confirmed, unfairly winning the auction. + +## Nameservices and Nameservice Auctions + +Nameservices are human-readable identifiers on a blockchain, akin to internet domain names, that correspond to specific addresses or resources. They simplify interactions with typically long and complex blockchain addresses, allowing users to have a memorable and unique identifier for their blockchain address or smart contract. + +Nameservice auctions are the process by which these identifiers are bid on and acquired. To combat front-running—where someone might use knowledge of pending bids to place a higher bid first—mechanisms such as commit-reveal schemes, auction extensions, and fair sequencing are implemented. These strategies ensure a transparent and fair bidding process, reducing the potential for Maximal Extractable Value (MEV) exploitation. + +## What is Maximal Extractable Value (MEV)? + +MEV is the highest value that can be extracted by manipulating the order of transactions within a block, beyond the standard block rewards and fees. This has become more prominent with the growth of decentralised finance (DeFi), where transaction order can greatly affect profits. + +## Implications of MEV + +MEV can lead to: + +- **Network Security**: Potential centralisation, as those with more computational power might dominate the process, increasing the risk of attacks. +- **Market Fairness**: An uneven playing field where only a few can gain at the expense of the majority. +- **User Experience**: Higher fees and network congestion due to the competition for MEV. + +## Mitigating MEV and Front-Running + +Some solutions being developed to mitigate MEV and front-running, including: + +- **Time-delayed Transactions**: Random delays to make transaction timing unpredictable. +- **Private Transaction Pools**: Concealing transactions until they are mined. +- **Fair Sequencing Services**: Processing transactions in the order they are received. + +For this tutorial, we will be exploring the last solution, fair sequencing services, in the context of nameservice auctions. + +## Conclusion + +MEV and front-running are challenges to blockchain integrity and fairness. Ongoing innovation and implementation of mitigation strategies are crucial for the ecosystem's health and success. diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/02-mitigating-front-running-with-vote-extensions.md.md b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/02-mitigating-front-running-with-vote-extensions.md.md new file mode 100644 index 000000000..421b6ed8c --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/02-mitigating-front-running-with-vote-extensions.md.md @@ -0,0 +1,331 @@ +# Mitigating Front-running with Vote Extensions + +## Table of Contents + +* [Prerequisites](#prerequisites) +* [Implementing Structs for Vote Extensions](#implementing-structs-for-vote-extensions) +* [Implementing Handlers and Configuring Handlers](#implementing-handlers-and-configuring-handlers) + +## Prerequisites + +Before implementing vote extensions to mitigate front-running, ensure you have a module ready to implement the vote extensions with. If you need to create or reference a similar module, see `x/auction` for guidance. + +In this section, we will discuss the steps to mitigate front-running using vote extensions. We will introduce new types within the `abci/types.go` file. These types will be used to handle the process of preparing proposals, processing proposals, and handling vote extensions. + +### Implementing Structs for Vote Extensions + +First, copy the following structs into the `abci/types.go` and each of these structs serves a specific purpose in the process of mitigating front-running using vote extensions: + +```go +package abci + +import ( + //import the necessary files +) + +type PrepareProposalHandler struct { + logger log.Logger + txConfig client.TxConfig + cdc codec.Codec + mempool *mempool.ThresholdMempool + txProvider provider.TxProvider + keyname string + runProvider bool +} +``` + +The `PrepareProposalHandler` struct is used to handle the preparation of a proposal in the consensus process. It contains several fields: logger for logging information and errors, txConfig for transaction configuration, cdc (Codec) for encoding and decoding transactions, mempool for referencing the set of unconfirmed transactions, txProvider for building the proposal with transactions, keyname for the name of the key used for signing transactions, and runProvider, a boolean flag indicating whether the provider should be run to build the proposal. + +```go +type ProcessProposalHandler struct { + TxConfig client.TxConfig + Codec codec.Codec + Logger log.Logger +} +``` + +After the proposal has been prepared and vote extensions have been included, the `ProcessProposalHandler` is used to process the proposal. This includes validating the proposal and the included vote extensions. The `ProcessProposalHandler` allows you to access the transaction configuration and codec, which are necessary for processing the vote extensions. + +```go +type VoteExtHandler struct { + logger log.Logger + currentBlock int64 + mempool *mempool.ThresholdMempool + cdc codec.Codec +} +``` + +This struct is used to handle vote extensions. It contains a logger for logging events, the current block number, a mempool for storing transactions, and a codec for encoding and decoding. Vote extensions are a key part of the process to mitigate front-running, as they allow for additional information to be included with each vote. + +```go +type InjectedVoteExt struct { + VoteExtSigner []byte + Bids [][]byte +} + +type InjectedVotes struct { + Votes []InjectedVoteExt +} +``` + +These structs are used to handle injected vote extensions. They include the signer of the vote extension and the bids associated with the vote extension. Each byte array in Bids is a serialised form of a bid transaction. Injected vote extensions are used to add additional information to a vote after it has been created, which can be useful for adding context or additional data to a vote. The serialised bid transactions provide a way to include complex transaction data in a compact, efficient format. + +```go +type AppVoteExtension struct { + Height int64 + Bids [][]byte +} +``` + +This struct is used for application vote extensions. It includes the height of the block and the bids associated with the vote extension. Application vote extensions are used to add additional information to a vote at the application level, which can be useful for adding context or additional data to a vote that is specific to the application. + +```go +type SpecialTransaction struct { + Height int + Bids [][]byte +} +``` + +This struct is used for special transactions. It includes the height of the block and the bids associated with the transaction. Special transactions are used for transactions that need to be handled differently from regular transactions, such as transactions that are part of the process to mitigate front-running. + +### Implementing Handlers and Configuring Handlers + +To establish the `VoteExtensionHandler`, follow these steps: + +1. Navigate to the `abci/proposal.go` file. This is where we will implement the `VoteExtensionHandler``. + +2. Implement the `NewVoteExtensionHandler` function. This function is a constructor for the `VoteExtHandler` struct. It takes a logger, a mempool, and a codec as parameters and returns a new instance of `VoteExtHandler`. + +```go +func NewVoteExtensionHandler(lg log.Logger, mp *mempool.ThresholdMempool, cdc codec.Codec) *VoteExtHandler { + return &VoteExtHandler{ + logger: lg, + mempool: mp, + cdc: cdc, + } +} +``` + +3. Implement the `ExtendVoteHandler()` method. This method should handle the logic of extending votes, including inspecting the mempool and submitting a list of all pending bids. This will allow you to access the list of unconfirmed transactions in the abci.`RequestPrepareProposal` during the ensuing block. + +```go +func (h *VoteExtHandler) ExtendVoteHandler() sdk.ExtendVoteHandler { + return func(ctx sdk.Context, req *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) { + h.logger.Info(fmt.Sprintf("Extending votes at block height : %v", req.Height)) + + voteExtBids := [][]byte{} + + // Get mempool txs + itr := h.mempool.SelectPending(context.Background(), nil) + for itr != nil { + tmptx := itr.Tx() + sdkMsgs := tmptx.GetMsgs() + + // Iterate through msgs, check for any bids + for _, msg := range sdkMsgs { + switch msg := msg.(type) { + case *nstypes.MsgBid: + // Marshal sdk bids to []byte + bz, err := h.cdc.Marshal(msg) + if err != nil { + h.logger.Error(fmt.Sprintf("Error marshalling VE Bid : %v", err)) + break + } + voteExtBids = append(voteExtBids, bz) + default: + } + } + + // Move tx to ready pool + err := h.mempool.Update(context.Background(), tmptx) + + // Remove tx from app side mempool + if err != nil { + h.logger.Info(fmt.Sprintf("Unable to update mempool tx: %v", err)) + } + + itr = itr.Next() + } + + // Create vote extension + voteExt := AppVoteExtension{ + Height: req.Height, + Bids: voteExtBids, + } + + // Encode Vote Extension + bz, err := json.Marshal(voteExt) + if err != nil { + return nil, fmt.Errorf("Error marshalling VE: %w", err) + } + + return &abci.ResponseExtendVote{VoteExtension: bz}, nil +} +``` + +4. Configure the handler in `app/app.go` as shown below + +```go +bApp := baseapp.NewBaseApp(AppName, logger, db, txConfig.TxDecoder(), baseAppOptions...) +voteExtHandler := abci2.NewVoteExtensionHandler(logger, mempool, appCodec) +bApp.SetExtendVoteHandler(voteExtHandler.ExtendVoteHandler()) +``` + +To give a bit of context on what is happening above, we first create a new instance of `VoteExtensionHandler` with the necessary dependencies (logger, mempool, and codec). Then, we set this handler as the `ExtendVoteHandler` for our application. This means that whenever a vote needs to be extended, our custom `ExtendVoteHandler()` method will be called. + +To test if vote extensions have been propagated, add the following to the `PrepareProposalHandler`: + +```go +if req.Height > 2 { + voteExt := req.GetLocalLastCommit() + h.logger.Info(fmt.Sprintf("🛠️ :: Get vote extensions: %v", voteExt)) +} +``` + +This is how the whole function should look: + +```go +func (h *PrepareProposalHandler) PrepareProposalHandler() sdk.PrepareProposalHandler { + return func(ctx sdk.Context, req *abci.RequestPrepareProposal) (*abci.ResponsePrepareProposal, error) { + h.logger.Info(fmt.Sprintf("🛠️ :: Prepare Proposal")) + var proposalTxs [][]byte + + var txs []sdk.Tx + + // Get Vote Extensions + if req.Height > 2 { + voteExt := req.GetLocalLastCommit() + h.logger.Info(fmt.Sprintf("🛠️ :: Get vote extensions: %v", voteExt)) + } + + itr := h.mempool.Select(context.Background(), nil) + for itr != nil { + tmptx := itr.Tx() + + txs = append(txs, tmptx) + itr = itr.Next() + } + h.logger.Info(fmt.Sprintf("🛠️ :: Number of Transactions available from mempool: %v", len(txs))) + + if h.runProvider { + tmpMsgs, err := h.txProvider.BuildProposal(ctx, txs) + if err != nil { + h.logger.Error(fmt.Sprintf("❌️ :: Error Building Custom Proposal: %v", err)) + } + txs = tmpMsgs + } + + for _, sdkTxs := range txs { + txBytes, err := h.txConfig.TxEncoder()(sdkTxs) + if err != nil { + h.logger.Info(fmt.Sprintf("❌~Error encoding transaction: %v", err.Error())) + } + proposalTxs = append(proposalTxs, txBytes) + } + + h.logger.Info(fmt.Sprintf("🛠️ :: Number of Transactions in proposal: %v", len(proposalTxs))) + + return &abci.ResponsePrepareProposal{Txs: proposalTxs}, nil + } +} +``` + +As mentioned above, we check if vote extensions have been propagated, you can do this by checking the logs for any relevant messages such as `🛠️ :: Get vote extensions:`. If the logs do not provide enough information, you can also reinitialise your local testing environment by running the `./scripts/single_node/setup.sh` script again. + +5. Implement the `ProcessProposalHandler()`. This function is responsible for processing the proposal. It should handle the logic of processing vote extensions, including inspecting the proposal and validating the bids. + +```go +func (h *ProcessProposalHandler) ProcessProposalHandler() sdk.ProcessProposalHandler { + return func(ctx sdk.Context, req *abci.RequestProcessProposal) (resp *abci.ResponseProcessProposal, err error) { + h.Logger.Info(fmt.Sprintf("⚙️ :: Process Proposal")) + + // The first transaction will always be the Special Transaction + numTxs := len(req.Txs) + + h.Logger.Info(fmt.Sprintf("⚙️:: Number of transactions :: %v", numTxs)) + + if numTxs >= 1 { + var st SpecialTransaction + err = json.Unmarshal(req.Txs[0], &st) + if err != nil { + h.Logger.Error(fmt.Sprintf("❌️:: Error unmarshalling special Tx in Process Proposal :: %v", err)) + } + if len(st.Bids) > 0 { + h.Logger.Info(fmt.Sprintf("⚙️:: There are bids in the Special Transaction")) + var bids []nstypes.MsgBid + for i, b := range st.Bids { + var bid nstypes.MsgBid + h.Codec.Unmarshal(b, &bid) + h.Logger.Info(fmt.Sprintf("⚙️:: Special Transaction Bid No %v :: %v", i, bid)) + bids = append(bids, bid) + } + // Validate Bids in Tx + txs := req.Txs[1:] + ok, err := ValidateBids(h.TxConfig, bids, txs, h.Logger) + if err != nil { + h.Logger.Error(fmt.Sprintf("❌️:: Error validating bids in Process Proposal :: %v", err)) + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } + if !ok { + h.Logger.Error(fmt.Sprintf("❌️:: Unable to validate bids in Process Proposal :: %v", err)) + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } + h.Logger.Info("⚙️:: Successfully validated bids in Process Proposal") + } + } + + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_ACCEPT}, nil + } +} +``` + +6. Implement the `ProcessVoteExtensions()` function. This function should handle the logic of processing vote extensions, including validating the bids. + +```go +func processVoteExtensions(req *abci.RequestPrepareProposal, log log.Logger) (SpecialTransaction, error) { + log.Info(fmt.Sprintf("🛠️ :: Process Vote Extensions")) + + // Create empty response + st := SpecialTransaction{ + 0, + [][]byte{}, + } + + // Get Vote Ext for H-1 from Req + voteExt := req.GetLocalLastCommit() + votes := voteExt.Votes + + // Iterate through votes + var ve AppVoteExtension + for _, vote := range votes { + // Unmarshal to AppExt + err := json.Unmarshal(vote.VoteExtension, &ve) + if err != nil { + log.Error(fmt.Sprintf("❌ :: Error unmarshalling Vote Extension")) + } + + st.Height = int(ve.Height) + + // If Bids in VE, append to Special Transaction + if len(ve.Bids) > 0 { + log.Info("🛠️ :: Bids in VE") + for _, b := range ve.Bids { + st.Bids = append(st.Bids, b) + } + } + } + + return st, nil +} +``` + +7. Configure the `ProcessProposalHandler()` in app/app.go: + +```go +processPropHandler := abci2.ProcessProposalHandler{app.txConfig, appCodec, logger} +bApp.SetProcessProposal(processPropHandler.ProcessProposalHandler()) +``` + +This sets the `ProcessProposalHandler()` for our application. This means that whenever a proposal needs to be processed, our custom `ProcessProposalHandler()` method will be called. + +To test if the proposal processing and vote extensions are working correctly, you can check the logs for any relevant messages. If the logs do not provide enough information, you can also reinitialize your local testing environment by running `./scripts/single_node/setup.sh` script. diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/03-demo-of-mitigating-front-running.md b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/03-demo-of-mitigating-front-running.md new file mode 100644 index 000000000..48cf22a40 --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/03-demo-of-mitigating-front-running.md @@ -0,0 +1,106 @@ +# Demo of Mitigating Front-Running with Vote Extensions + +The purpose of this demo is to test the implementation of the `VoteExtensionHandler` and `PrepareProposalHandler` that we have just added to the codebase. These handlers are designed to mitigate front-running by ensuring that all validators have a consistent view of the mempool when preparing proposals. + +In this demo, we are using a 3 validator network. The Beacon validator is special because it has a custom transaction provider enabled. This means that it can potentially manipulate the order of transactions in a proposal to its advantage (i.e., front-running). + +1. Bootstrap the validator network: This sets up a network with 3 validators. The script `./scripts/configure.sh is used to configure the network and the validators. + +```shell +cd scripts +configure.sh +``` + +If this doesnt work please ensure you have run `make build` in the `tutorials/nameservice/base` directory. + + +2. Have alice attempt to reserve `bob.cosmos`: This is a normal transaction that alice wants to execute. The script ``./scripts/reserve.sh "bob.cosmos"` is used to send this transaction. + +```shell +bash reserve.sh "bob.cosmos" +``` + +3. Query to verify the name has been reserved: This is to check the result of the transaction. The script `./scripts/whois.sh "bob.cosmos"` is used to query the state of the blockchain. + +```shell +bash whois.sh "bob.cosmos" +``` + +It should return: + +```{ + "name": { + "name": "bob.cosmos", + "owner": "cosmos1nq9wuvuju4jdmpmzvxmg8zhhu2ma2y2l2pnu6w", + "resolve_address": "cosmos1h6zy2kn9efxtw5z22rc5k9qu7twl70z24kr3ht", + "amount": [ + { + "denom": "uatom", + "amount": "1000" + } + ] + } +} +``` + +To detect front-running attempts by the beacon, scrutinise the logs during the `ProcessProposal` stage. Open the logs for each validator, including the beacon, `val1`, and `val2`, to observe the following behavior. Open the log file of the validator node. The location of this file can vary depending on your setup, but it's typically located in a directory like `$HOME/cosmos/nodes/#{validator}/logs`. The directory in this case will be under the validator so, `beacon` `val1` or `val2`. Run the following to tail the logs of the validator or beacon: + +```shell +tail -f $HOME/cosmos/nodes/#{validator}/logs +``` + +```shell +2:47PM ERR ❌️:: Detected invalid proposal bid :: name:"bob.cosmos" resolveAddress:"cosmos1wmuwv38pdur63zw04t0c78r2a8dyt08hf9tpvd" owner:"cosmos1wmuwv38pdur63zw04t0c78r2a8dyt08hf9tpvd" amount: module=server +2:47PM ERR ❌️:: Unable to validate bids in Process Proposal :: module=server +2:47PM ERR prevote step: state machine rejected a proposed block; this should not happen:the proposer may be misbehaving; prevoting nil err=null height=142 module=consensus round=0 +``` + + +4. List the Beacon's keys: This is to verify the addresses of the validators. The script `./scripts/list-beacon-keys.sh` is used to list the keys. + +```shell +bash list-beacon-keys.sh +``` + +We should receive something similar to the following: + +```shell +[ + { + "name": "alice", + "type": "local", + "address": "cosmos1h6zy2kn9efxtw5z22rc5k9qu7twl70z24kr3ht", + "pubkey": "{\"@type\":\"/cosmos.crypto.secp256k1.PubKey\",\"key\":\"A32cvBUkNJz+h2vld4A5BxvU5Rd+HyqpR3aGtvEhlm4C\"}" + }, + { + "name": "barbara", + "type": "local", + "address": "cosmos1nq9wuvuju4jdmpmzvxmg8zhhu2ma2y2l2pnu6w", + "pubkey": "{\"@type\":\"/cosmos.crypto.secp256k1.PubKey\",\"key\":\"Ag9PFsNyTQPoJdbyCWia5rZH9CrvSrjMsk7Oz4L3rXQ5\"}" + }, + { + "name": "beacon-key", + "type": "local", + "address": "cosmos1ez9a6x7lz4gvn27zr368muw8jeyas7sv84lfup", + "pubkey": "{\"@type\":\"/cosmos.crypto.secp256k1.PubKey\",\"key\":\"AlzJZMWyN7lass710TnAhyuFKAFIaANJyw5ad5P2kpcH\"}" + }, + { + "name": "cindy", + "type": "local", + "address": "cosmos1m5j6za9w4qc2c5ljzxmm2v7a63mhjeag34pa3g", + "pubkey": "{\"@type\":\"/cosmos.crypto.secp256k1.PubKey\",\"key\":\"A6F1/3yot5OpyXoSkBbkyl+3rqBkxzRVSJfvSpm/AvW5\"}" + } +] +``` + +This allows us to match up the addresses and see that the bid was not front run by the beacon due as the resolve address is Alice's address and not the beacons address. + +By running this demo, we can verify that the `VoteExtensionHandler` and `PrepareProposalHandler` are working as expected and that they are able to prevent front-running. + +## Conclusion + +In this tutorial, we've tackled front-running and MEV, focusing on nameservice auctions' vulnerability to these issues. We've explored vote extensions, a key feature of ABCI 2.0, and integrated them into a Cosmos SDK application. + +Through practical exercises, you've implemented vote extensions, and tested their effectiveness in creating a fair auction system. You've gained practical insights by configuring a validator network and analysing blockchain logs. + +Keep experimenting with these concepts, engage with the community, and stay updated on new advancements. The knowledge you've acquired here is crucial for developing secure and fair blockchain applications. diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/_category_.json b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/_category_.json new file mode 100644 index 000000000..d949f628c --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/auction-frontrunning/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Mitigating Auction Front-Running Tutorial", + "position": 0, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/00-getting-started.md b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/00-getting-started.md new file mode 100644 index 000000000..c96886a64 --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/00-getting-started.md @@ -0,0 +1,35 @@ +# Getting Started + +## Table of Contents + +- [Getting Started](00-getting-started.md) +- [What is an Oracle?](01-what-is-an-oracle.md) +- [Implementing Vote Extensions](02-implementing-vote-extensions.md) +- [Testing the Oracle Module](03-testing-oracle.md) + +## Prerequisites + +Before you start with this tutorial, make sure you have: + +- A working chain project. This tutorial won't cover the steps of creating a new chain/module. +- Familiarity with the Cosmos SDK. If you're not, we suggest you start with [Cosmos SDK Tutorials](https://tutorials.cosmos.network), as ABCI++ is considered an advanced topic. +- Read and understood [What is an Oracle?](01-what-is-an-oracle.md). This provides necessary background information for understanding the Oracle module. +- Basic understanding of Go programming language. + +## What are Vote extensions? + +Vote extensions is arbitrary information which can be inserted into a block. This feature is part of ABCI 2.0, which is available for use in the SDK 0.50 release and part of the 0.38 CometBFT release. + +More information about vote extensions can be seen [here](https://docs.cosmos.network/main/build/abci/vote-extensions). + +## Overview of the project + +We’ll go through the creation of a simple price oracle module focusing on the vote extensions implementation, ignoring the details inside the price oracle itself. + +We’ll go through the implementation of: + +- `ExtendVote` to get information from external price APIs. +- `VerifyVoteExtension` to check that the format of the provided votes is correct. +- `PrepareProposal` to process the vote extensions from the previous block and include them into the proposal as a transaction. +- `ProcessProposal` to check that the first transaction in the proposal is actually a “special tx” that contains the price information. +- `PreBlocker` to make price information available during FinalizeBlock. diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/01-what-is-an-oracle.md b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/01-what-is-an-oracle.md new file mode 100644 index 000000000..9d50ddb36 --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/01-what-is-an-oracle.md @@ -0,0 +1,13 @@ +# What is an Oracle? + +An oracle in blockchain technology is a system that provides external data to a blockchain network. It acts as a source of information that is not natively accessible within the blockchain's closed environment. This can range from financial market prices to real-world event, making it crucial for decentralised applications. + +## Oracle in the Cosmos SDK + +In the Cosmos SDK, an oracle module can be implemented to provide external data to the blockchain. This module can use features like vote extensions to submit additional data during the consensus process, which can then be used by the blockchain to update its state with information from the outside world. + +For instance, a price oracle module in the Cosmos SDK could supply timely and accurate asset price information, which is vital for various financial operations within the blockchain ecosystem. + +## Conclusion + +Oracles are essential for blockchains to interact with external data, enabling them to respond to real-world information and events. Their implementation is key to the reliability and robustness of blockchain networks. diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/02-implementing-vote-extensions.md b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/02-implementing-vote-extensions.md new file mode 100644 index 000000000..0a33e11f6 --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/02-implementing-vote-extensions.md @@ -0,0 +1,219 @@ +# Implementing Vote Extensions + +## Implement ExtendVote + +First we’ll create the `OracleVoteExtension` struct, this is the object that will be marshaled as bytes and signed by the validator. + +In our example we’ll use JSON to marshal the vote extension for simplicity but we recommend to find an encoding that produces a smaller output, given that large vote extensions could impact CometBFT’s performance. Custom encodings and compressed bytes can be used out of the box. + +```go +// OracleVoteExtension defines the canonical vote extension structure. +type OracleVoteExtension struct { + Height int64 + Prices map[string]math.LegacyDec +} +``` + +Then we’ll create a `VoteExtensionsHandler` struct that contains everything we need to query for prices. + +```go +type VoteExtHandler struct { + logger log.Logger + currentBlock int64 // current block height + lastPriceSyncTS time.Time // last time we synced prices + providerTimeout time.Duration // timeout for fetching prices from providers + providers map[string]Provider // mapping of provider name to provider (e.g. Binance -> BinanceProvider) + providerPairs map[string][]keeper.CurrencyPair // mapping of provider name to supported pairs (e.g. Binance -> [ATOM/USD]) + + Keeper keeper.Keeper // keeper of our oracle module +} +``` + +Finally, a function that returns `sdk.ExtendVoteHandler` is needed too, and this is where our vote extension logic will live. + +```go +func (h *VoteExtHandler) ExtendVoteHandler() sdk.ExtendVoteHandler { + return func(ctx sdk.Context, req *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) { + // here we'd have a helper function that gets all the prices and does a weighted average using the volume of each market + prices := h.getAllVolumeWeightedPrices() + + voteExt := OracleVoteExtension{ + Height: req.Height, + Prices: prices, + } + + bz, err := json.Marshal(voteExt) + if err != nil { + return nil, fmt.Errorf("failed to marshal vote extension: %w", err) + } + + return &abci.ResponseExtendVote{VoteExtension: bz}, nil + } +} +``` + +As you can see above, the creation of a vote extension is pretty simple and we just have to return bytes. CometBFT will handle the signing of these bytes for us. We ignored the process of getting the prices but you can see a more complete example [here:](../base/x/oracle/abci/vote_extensions.go) + +Here we’ll do some simple checks like: + +* Is the vote extension unmarshaled correctly? +* Is the vote extension for the right height? +* Some other validation, for example, are the prices from this extension too deviated from my own prices? Or maybe checks that can detect malicious behavior. + +```go +func (h *VoteExtHandler) VerifyVoteExtensionHandler() sdk.VerifyVoteExtensionHandler { + return func(ctx sdk.Context, req *abci.RequestVerifyVoteExtension) (*abci.ResponseVerifyVoteExtension, error) { + var voteExt OracleVoteExtension + err := json.Unmarshal(req.VoteExtension, &voteExt) + if err != nil { + return nil, fmt.Errorf("failed to unmarshal vote extension: %w", err) + } + + if voteExt.Height != req.Height { + return nil, fmt.Errorf("vote extension height does not match request height; expected: %d, got: %d", req.Height, voteExt.Height) + } + + // Verify incoming prices from a validator are valid. Note, verification during + // VerifyVoteExtensionHandler MUST be deterministic. For brevity and demo + // purposes, we omit implementation. + if err := h.verifyOraclePrices(ctx, voteExt.Prices); err != nil { + return nil, fmt.Errorf("failed to verify oracle prices from validator %X: %w", req.ValidatorAddress, err) + } + + return &abci.ResponseVerifyVoteExtension{Status: abci.ResponseVerifyVoteExtension_ACCEPT}, nil + } +} +``` + +## Implement PrepareProposal + +```go +type ProposalHandler struct { + logger log.Logger + keeper keeper.Keeper // our oracle module keeper + valStore baseapp.ValidatorStore // to get the current validators' pubkeys +} +``` + +And we create the struct for our “special tx”, that will contain the prices and the votes so validators can later re-check in ProcessPRoposal that they get the same result than the block’s proposer. With this we could also check if all the votes have been used by comparing the votes received in ProcessProposal. + +```go +type StakeWeightedPrices struct { + StakeWeightedPrices map[string]math.LegacyDec + ExtendedCommitInfo abci.ExtendedCommitInfo +} +``` + +Now we create the `PrepareProposalHandler`. In this step we’ll first check if the vote extensions’ signatures are correct using a helper function called ValidateVoteExtensions from the baseapp pacakge. + +```go +func (h *ProposalHandler) PrepareProposal() sdk.PrepareProposalHandler { + return func(ctx sdk.Context, req *abci.RequestPrepareProposal) (*abci.ResponsePrepareProposal, error) { + err := baseapp.ValidateVoteExtensions(ctx, h.valStore, req.Height, ctx.ChainID(), req.LocalLastCommit) + if err != nil { + return nil, err + } +... +``` + +Then we proceed to make the calculations only if the current height if higher than the height at which vote extensions have been enabled. Remember that vote extensions are made available to the block proposer on the next block at which they are produced/enabled. + +```go +... + proposalTxs := req.Txs + + if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight { + stakeWeightedPrices, err := h.computeStakeWeightedOraclePrices(ctx, req.LocalLastCommit) + if err != nil { + return nil, errors.New("failed to compute stake-weighted oracle prices") + } + + injectedVoteExtTx := StakeWeightedPrices{ + StakeWeightedPrices: stakeWeightedPrices, + ExtendedCommitInfo: req.LocalLastCommit, + } +... +``` + +Finally we inject the result as a transaction at a specific location, usually at the beginning of the block: + +## Implement ProcessProposal + +Now we can implement the method that all validators will execute to ensure the proposer is doing his work correctly. + +Here, if vote extensions are enabled, we’ll check if the tx at index 0 is an injected vote extension + +```go +func (h *ProposalHandler) ProcessProposal() sdk.ProcessProposalHandler { + return func(ctx sdk.Context, req *abci.RequestProcessProposal) (*abci.ResponseProcessProposal, error) { + if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight { + var injectedVoteExtTx StakeWeightedPrices + if err := json.Unmarshal(req.Txs[0], &injectedVoteExtTx); err != nil { + h.logger.Error("failed to decode injected vote extension tx", "err", err) + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } +... +``` + +Then we re-validate the vote extensions signatures using +baseapp.ValidateVoteExtensions, re-calculate the results (just like in PrepareProposal) and compare them with the results we got from the injected tx. + +```go + err := baseapp.ValidateVoteExtensions(ctx, h.valStore, req.Height, ctx.ChainID(), injectedVoteExtTx.ExtendedCommitInfo) + if err != nil { + return nil, err + } + + // Verify the proposer's stake-weighted oracle prices by computing the same + // calculation and comparing the results. We omit verification for brevity + // and demo purposes. + stakeWeightedPrices, err := h.computeStakeWeightedOraclePrices(ctx, injectedVoteExtTx.ExtendedCommitInfo) + if err != nil { + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } + + if err := compareOraclePrices(injectedVoteExtTx.StakeWeightedPrices, stakeWeightedPrices); err != nil { + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil + } + } + + return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_ACCEPT}, nil + } +} +``` + +Important: In this example we avoided using the mempool and other basics, please refer to the DefaultProposalHandler for a complete implementation: [https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/baseapp/abci_utils.go](https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/baseapp/abci_utils.go) + +## Implement PreBlocker + +Now validators are extending their vote, verifying other votes and including the result in the block. But how do we actually make use of this result? This is done in the PreBlocker which is code that is run before any other code during FinalizeBlock so we make sure we make this information available to the chain and its modules during the entire block execution (from BeginBlock). + +At this step we know that the injected tx is well-formatted and has been verified by the validators participating in consensus, so making use of it is straightforward. Just check if vote extensions are enabled, pick up the first transaction and use a method in your module’s keeper to set the result. + +```go +func (h *ProposalHandler) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) (*sdk.ResponsePreBlock, error) { + res := &sdk.ResponsePreBlock{} + if len(req.Txs) == 0 { + return res, nil + } + + if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight { + var injectedVoteExtTx StakeWeightedPrices + if err := json.Unmarshal(req.Txs[0], &injectedVoteExtTx); err != nil { + h.logger.Error("failed to decode injected vote extension tx", "err", err) + return nil, err + } + + // set oracle prices using the passed in context, which will make these prices available in the current block + if err := h.keeper.SetOraclePrices(ctx, injectedVoteExtTx.StakeWeightedPrices); err != nil { + return nil, err + } + } + return res, nil +} + +``` + +## Conclusion + +In this tutorial, we've created a simple price oracle module that incorporates vote extensions. We've seen how to implement `ExtendVote`, `VerifyVoteExtension`, `PrepareProposal`, `ProcessProposal`, and `PreBlocker` to handle the voting and verification process of vote extensions, as well as how to make use of the results during the block execution. diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/03-testing-oracle.md b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/03-testing-oracle.md new file mode 100644 index 000000000..859b9518e --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/03-testing-oracle.md @@ -0,0 +1,57 @@ +# Testing the Oracle Module + +We will guide you through the process of testing the Oracle module in your application. The Oracle module uses vote extensions to provide current price data. + +## Step 1: Compile and Install the Application + +First, we need to compile and install the application. Please ensure you are in the `tutorials/oracle/base` directory. Run the following command in your terminal: + +```shell +bash make install +``` + +This command compiles the application and moves the resulting binary to a location in your system's PATH. + +## Step 2: Initialise the Application + +Next, we need to initialise the application. Run the following command in your terminal: + +```shell +bash make init +``` + +This command runs the script `tutorials/oracle/base/scripts/init.sh`, which sets up the necessary configuration for your application to run. This includes creating the `app.toml` configuration file and initialising the blockchain with a genesis block. + +## Step 3: Start the Application + +Now, we can start the application. Run the following command in your terminal: + +```shell +tutoriald start +``` + +This command starts your application, begins the blockchain node, and starts processing transactions. + +## Step 4: Query the Oracle Prices + +Finally, we can query the current prices from the Oracle module. Run the following command in your terminal: + +```shell +bash tutoriald q oracle prices +``` + +This command queries the current prices from the Oracle module. The expected output shows that the vote extensions were successfully included in the block and the Oracle module was able to retrieve the price data. + +## Understanding Vote Extensions in Oracle + +In the Oracle module, the `ExtendVoteHandler` function is responsible for creating the vote extensions. This function fetches the current prices from the provider, creates a `OracleVoteExtension` struct with these prices, and then marshals this struct into bytes. These bytes are then set as the vote extension. + +In the context of testing, the Oracle module uses a mock provider to simulate the behavior of a real price provider. This mock provider is defined in the mockprovider package and is used to return predefined prices for specific currency pairs. + +## Conclusion + +In this tutorial, we've delved into the concept of Oracle's in blockchain technology, focusing on their role in providing external data to a blockchain network. We've explored vote extensions, a powerful feature of ABCI++, and integrated them into a Cosmos SDK application to create a price oracle module. + +Through hands-on exercises, you've implemented vote extensions, and tested their effectiveness in providing timely and accurate asset price information. You've gained practical insights by setting up a mock provider for testing and analysing the process of extending votes, verifying vote extensions, and preparing and processing proposals. + +Keep experimenting with these concepts, engage with the community, and stay updated on new advancements. The knowledge you've acquired here is crucial for developing robust and reliable blockchain applications that can interact with real-world data. diff --git a/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/_category_.json b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/_category_.json new file mode 100644 index 000000000..b63ffe2ff --- /dev/null +++ b/versioned_docs/version-0.50/tutorials/vote-extensions/oracle/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Oracle Tutorial", + "position": 1, + "link": null +} \ No newline at end of file diff --git a/versioned_sidebars/version-0.50-sidebars.json b/versioned_sidebars/version-0.50-sidebars.json index c6be3020a..a9a3bcb28 100644 --- a/versioned_sidebars/version-0.50-sidebars.json +++ b/versioned_sidebars/version-0.50-sidebars.json @@ -16,5 +16,11 @@ "type": "autogenerated", "dirName": "user" } + ], + "tutorialsSidebar": [ + { + "type": "autogenerated", + "dirName": "tutorials" + } ] }