diff --git a/versioned_docs/version-0.52/build/abci/00-introduction.md b/versioned_docs/version-0.52/build/abci/00-introduction.md new file mode 100644 index 000000000..7d955d41d --- /dev/null +++ b/versioned_docs/version-0.52/build/abci/00-introduction.md @@ -0,0 +1,51 @@ +# Introduction + +## What is ABCI? + +ABCI, Application Blockchain Interface is the interface between CometBFT and the application. More information about ABCI can be found in the specs [here](https://docs.cometbft.com/v1.0/spec/abci/). Within the release of ABCI 2.0 for the 0.38 CometBFT release there were additional methods introduced. + +The 5 methods introduced during ABCI 2.0 (compared to ABCI v0) are: + +* `PrepareProposal` +* `ProcessProposal` +* `ExtendVote` +* `VerifyVoteExtension` +* `FinalizeBlock` + + +## The Flow + +## PrepareProposal + +Based on validator voting power, CometBFT chooses a block proposer and calls `PrepareProposal` on the block proposer's application (Cosmos SDK). The selected block proposer is responsible for collecting outstanding transactions from the mempool, adhering to the application's specifications. The application can enforce custom transaction ordering and incorporate additional transactions, potentially generated from vote extensions in the previous block. + +To perform this manipulation on the application side, a custom handler must be implemented. By default, the Cosmos SDK provides `PrepareProposalHandler`, used in conjunction with an application specific mempool. A custom handler can be written by application developer, if a noop handler provided, all transactions are considered valid. + +Please note that vote extensions will only be available on the following height in which vote extensions are enabled. More information about vote extensions can be found [here](https://docs.cosmos.network/main/build/abci/vote-extensions). + +After creating the proposal, the proposer returns it to CometBFT. + +PrepareProposal CAN be non-deterministic. + +## ProcessProposal + +This method allows validators to perform application-specific checks on the block proposal and is called on all validators. This is an important step in the consensus process, as it ensures that the block is valid and meets the requirements of the application. For example, validators could check that the block contains all the required transactions or that the block does not create any invalid state transitions. + +The implementation of `ProcessProposal` MUST be deterministic. + +## ExtendVote and VerifyVoteExtensions + +These methods allow applications to extend the voting process by requiring validators to perform additional actions beyond simply validating blocks. + +If vote extensions are enabled, `ExtendVote` will be called on every validator and each one will return its vote extension which is in practice a bunch of bytes. As mentioned above this data (vote extension) can only be retrieved in the next block height during `PrepareProposal`. Additionally, this data can be arbitrary, but in the provided tutorials, it serves as an oracle or proof of transactions in the mempool. Essentially, vote extensions are processed and injected as transactions. Examples of use-cases for vote extensions include prices for a price oracle or encryption shares for an encrypted transaction mempool. `ExtendVote` CAN be non-deterministic. + +`VerifyVoteExtensions` is performed on every validator multiple times in order to verify other validators' vote extensions. This check is submitted to validate the integrity and validity of the vote extensions preventing malicious or invalid vote extensions. + +Additionally, applications must keep the vote extension data concise as it can degrade the performance of their chain, see testing results [here](https://docs.cometbft.com/v1.0/references/qa/cometbft-qa-38#vote-extensions-testbed). + +`VerifyVoteExtensions` MUST be deterministic. + + +## FinalizeBlock + +`FinalizeBlock` is then called and is responsible for updating the state of the blockchain and making the block available to users. diff --git a/versioned_docs/version-0.52/build/abci/01-prepare-proposal.md b/versioned_docs/version-0.52/build/abci/01-prepare-proposal.md new file mode 100644 index 000000000..3d480664e --- /dev/null +++ b/versioned_docs/version-0.52/build/abci/01-prepare-proposal.md @@ -0,0 +1,45 @@ +# Prepare Proposal + +`PrepareProposal` handles construction of the block, meaning that when a proposer +is preparing to propose a block, it requests the application to evaluate a +`RequestPrepareProposal`, which contains a series of transactions from CometBFT's +mempool. At this point, the application has complete control over the proposal. +It can modify, delete, and inject transactions from its own app-side mempool into +the proposal or even ignore all the transactions altogether. What the application +does with the transactions provided to it by `RequestPrepareProposal` has no +effect on CometBFT's mempool. + +Note, that the application defines the semantics of the `PrepareProposal` and it +MAY be non-deterministic and is only executed by the current block proposer. + +Now, reading mempool twice in the previous sentence is confusing, lets break it down. +CometBFT has a mempool that handles gossiping transactions to other nodes +in the network. The order of these transactions is determined by CometBFT's mempool, +using FIFO as the sole ordering mechanism. It's worth noting that the priority mempool +in Comet was removed or deprecated. +However, since the application is able to fully inspect +all transactions, it can provide greater control over transaction ordering. +Allowing the application to handle ordering enables the application to define how +it would like the block constructed. + +The Cosmos SDK defines the `DefaultProposalHandler` type, which provides applications with +`PrepareProposal` and `ProcessProposal` handlers. If you decide to implement your +own `PrepareProposal` handler, you must ensure that the transactions +selected DO NOT exceed the maximum block gas (if set) and the maximum bytes provided +by `req.MaxBytes`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/baseapp/abci_utils.go +``` + +This default implementation can be overridden by the application developer in +favor of a custom implementation in [`app_di.go`](https://docs.cosmos.network/main/build/building-apps/app-go-di): + +```go +prepareOpt := func(app *baseapp.BaseApp) { + abciPropHandler := baseapp.NewDefaultProposalHandler(mempool, app) + app.SetPrepareProposal(abciPropHandler.PrepareProposalHandler()) +} + +baseAppOptions = append(baseAppOptions, prepareOpt) +``` diff --git a/versioned_docs/version-0.52/build/abci/02-process-proposal.md b/versioned_docs/version-0.52/build/abci/02-process-proposal.md new file mode 100644 index 000000000..7768890bd --- /dev/null +++ b/versioned_docs/version-0.52/build/abci/02-process-proposal.md @@ -0,0 +1,32 @@ +# Process Proposal + +`ProcessProposal` handles the validation of a proposal from `PrepareProposal`, +which also includes a block header. Meaning, that after a block has been proposed +the other validators have the right to accept or reject that block. The validator in the +default implementation of `PrepareProposal` runs basic validity checks on each +transaction. + +Note, `ProcessProposal` MAY NOT be non-deterministic, i.e. it must be deterministic. +This means if `ProcessProposal` panics or fails and we reject, all honest validator +processes should reject (i.e., prevote nil). If so, then CometBFT will start a new round with a new block proposal, and the same cycle will happen with `PrepareProposal` +and `ProcessProposal` for the new proposal. + +Here is the implementation of the default implementation: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/baseapp/abci_utils.go#L224-L231 +``` + +Like `PrepareProposal` this implementation is the default and can be modified by +the application developer in [`app_di.go`](https://docs.cosmos.network/main/build/building-apps/app-go-di). If you decide to implement +your own `ProcessProposal` handler, you must ensure that the transactions +provided in the proposal DO NOT exceed the maximum block gas and `maxtxbytes` (if set). + +```go +processOpt := func(app *baseapp.BaseApp) { + abciPropHandler := baseapp.NewDefaultProposalHandler(mempool, app) + app.SetProcessProposal(abciPropHandler.ProcessProposalHandler()) +} + +baseAppOptions = append(baseAppOptions, processOpt) +``` diff --git a/versioned_docs/version-0.52/build/abci/03-vote-extensions.md b/versioned_docs/version-0.52/build/abci/03-vote-extensions.md new file mode 100644 index 000000000..26cbfd1a2 --- /dev/null +++ b/versioned_docs/version-0.52/build/abci/03-vote-extensions.md @@ -0,0 +1,122 @@ +# Vote Extensions + +:::note Synopsis +This section describes how the application can define and use vote extensions +defined in ABCI++. +::: + +## Extend Vote + +ABCI2.0 (colloquially called ABCI++) allows an application to extend a pre-commit vote with arbitrary data. This process does NOT have to be deterministic, and the data returned can be unique to the +validator process. The Cosmos SDK defines [`baseapp.ExtendVoteHandler`](https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/types/abci.go#L26-L27): + +```go +type ExtendVoteHandler func(Context, *abci.ExtendVoteRequest) (*abci.ExtendVoteResponse, error) +``` + +An application can set this handler in `app.go` via the `baseapp.SetExtendVoteHandler` +`BaseApp` option function. The `sdk.ExtendVoteHandler`, if defined, is called during +the `ExtendVote` ABCI method. Note, if an application decides to implement +`baseapp.ExtendVoteHandler`, it MUST return a non-nil `VoteExtension`. However, the vote +extension can be empty. See [here](https://docs.cometbft.com/v1.0/spec/abci/abci++_methods#extendvote) +for more details. + +There are many decentralized censorship-resistant use cases for vote extensions. +For example, a validator may want to submit prices for a price oracle or encryption +shares for an encrypted transaction mempool. Note, an application should be careful +to consider the size of the vote extensions as they could increase latency in block +production. See [here](https://docs.cometbft.com/v1.0/references/qa/cometbft-qa-38#vote-extensions-testbed) +for more details. + +Click [here](https://docs.cosmos.network/main/build/abci/vote-extensions) if you would like a walkthrough of how to implement vote extensions. + + +## Verify Vote Extension + +Similar to extending a vote, an application can also verify vote extensions from +other validators when validating their pre-commits. For a given vote extension, +this process MUST be deterministic. The Cosmos SDK defines [`sdk.VerifyVoteExtensionHandler`](https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/types/abci.go#L29-L31): + +```go +type VerifyVoteExtensionHandler func(Context, *abci.VerifyVoteExtensionRequest) (*abci.VerifyVoteExtensionResponse, error) +``` + +An application can set this handler in `app.go` via the `baseapp.SetVerifyVoteExtensionHandler` +`BaseApp` option function. The `sdk.VerifyVoteExtensionHandler`, if defined, is called +during the `VerifyVoteExtension` ABCI method. If an application defines a vote +extension handler, it should also define a verification handler. Note, not all +validators will share the same view of what vote extensions they verify depending +on how votes are propagated. See [here](https://docs.cometbft.com/v1.0/spec/abci/abci++_methods#verifyvoteextension) +for more details. + +Additionally, please keep in mind that performance can be degraded if vote extensions are too big ([see vote extension testbed](https://docs.cometbft.com/v1.0/references/qa/cometbft-qa-38#vote-extensions-testbed)), so we highly recommend a size validation in `VerifyVoteExtensions`. + + +## Vote Extension Propagation + +The agreed upon vote extensions at height `H` are provided to the proposing validator +at height `H+1` during `PrepareProposal`. As a result, the vote extensions are +not natively provided or exposed to the remaining validators during `ProcessProposal`. +As a result, if an application requires that the agreed upon vote extensions from +height `H` are available to all validators at `H+1`, the application must propagate +these vote extensions manually in the block proposal itself. This can be done by +"injecting" them into the block proposal, since the `Txs` field in `PrepareProposal` +is just a slice of byte slices. + +`FinalizeBlock` will ignore any byte slice that doesn't implement an `sdk.Tx`, so +any injected vote extensions will safely be ignored in `FinalizeBlock`. For more +details on propagation, see the [ABCI++ 2.0 ADR](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-064-abci-2.0.md#vote-extension-propagation--verification). + +### Recovery of injected Vote Extensions + +As stated before, vote extensions can be injected into a block proposal (along with +other transactions in the `Txs` field). The Cosmos SDK provides a pre-FinalizeBlock +hook to allow applications to recover vote extensions, perform any necessary +computation on them, and then store the results in the cached store. These results +will be available to the application during the subsequent `FinalizeBlock` call. + +An example of how a pre-FinalizeBlock hook could look like is shown below: + +```go +app.SetPreBlocker(func(ctx sdk.Context, req *abci.RequestFinalizeBlock) error { + allVEs := []VE{} // store all parsed vote extensions here + for _, tx := range req.Txs { + // define a custom function that tries to parse the tx as a vote extension + ve, ok := parseVoteExtension(tx) + if !ok { + continue + } + + allVEs = append(allVEs, ve) + } + + // perform any necessary computation on the vote extensions and store the result + // in the cached store + result := compute(allVEs) + err := storeVEResult(ctx, result) + if err != nil { + return err + } + + return nil +}) + +``` + +Then, in an app's module, the application can retrieve the result of the computation +of vote extensions from the cached store: + +```go +func (k Keeper) BeginBlocker(ctx context.Context) error { + // retrieve the result of the computation of vote extensions from the cached store + result, err := k.GetVEResult(ctx) + if err != nil { + return err + } + + // use the result of the computation of vote extensions + k.setSomething(result) + + return nil +} +``` diff --git a/versioned_docs/version-0.52/build/abci/04-checktx.md b/versioned_docs/version-0.52/build/abci/04-checktx.md new file mode 100644 index 000000000..a2c5f2d70 --- /dev/null +++ b/versioned_docs/version-0.52/build/abci/04-checktx.md @@ -0,0 +1,50 @@ +# CheckTx + +CheckTx is called by the `BaseApp` when comet receives a transaction from a client, over the p2p network or RPC. The CheckTx method is responsible for validating the transaction and returning an error if the transaction is invalid. + +```mermaid +graph TD + subgraph SDK[Cosmos SDK] + B[Baseapp] + A[AnteHandlers] + B <-->|Validate TX| A + end + C[CometBFT] <-->|CheckTx|SDK + U((User)) -->|Submit TX| C + N[P2P] -->|Receive TX| C +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/31c604762a434c7b676b6a89897ecbd7c4653a23/baseapp/abci.go#L350-L390 +``` + +## CheckTx Handler + +`CheckTxHandler` allows users to extend the logic of `CheckTx`. `CheckTxHandler` is called by pasding context and the transaction bytes received through ABCI. It is required that the handler returns deterministic results given the same transaction bytes. + +:::note +we return the raw decoded transaction here to avoid decoding it twice. +::: + +```go +type CheckTxHandler func(ctx sdk.Context, tx []byte) (Tx, error) +``` + +Setting a custom `CheckTxHandler` is optional. It can be done from your app.go file: + +```go +func NewSimApp( + logger log.Logger, + db corestore.KVStoreWithBatch, + traceStore io.Writer, + loadLatest bool, + appOpts servertypes.AppOptions, + baseAppOptions ...func(*baseapp.BaseApp), +) *SimApp { + ... + // Create ChecktxHandler + checktxHandler := abci.NewCustomCheckTxHandler(...) + app.SetCheckTxHandler(checktxHandler) + ... +} +``` diff --git a/versioned_docs/version-0.52/build/abci/_category_.json b/versioned_docs/version-0.52/build/abci/_category_.json new file mode 100644 index 000000000..d4ebb80c3 --- /dev/null +++ b/versioned_docs/version-0.52/build/abci/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "ABCI", + "position": 2, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/architecture/PROCESS.md b/versioned_docs/version-0.52/build/architecture/PROCESS.md new file mode 100644 index 000000000..15e11639c --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/PROCESS.md @@ -0,0 +1,58 @@ +# ADR Creation Process + +1. Copy the `adr-template.md` file. Use the following filename pattern: `adr-next_number-title.md` +2. Create a draft Pull Request if you want to get early feedback. +3. Make sure the context and solution are clear and well documented. +4. Add an entry to a list in the [README](./README.md) file. +5. Create a Pull Request to propose a new ADR. + +## What is an ADR? + +An ADR is a document to document an implementation and design that may or may not have been discussed in an RFC. While an RFC is meant to replace synchronous communication in a distributed environment, an ADR is meant to document an already made decision. An ADR won't come with much of a communication overhead because the discussion was recorded in an RFC or a synchronous discussion. If the consensus came from a synchronous discussion then a short excerpt should be added to the ADR to explain the goals. + +## ADR life cycle + +ADR creation is an **iterative** process. Instead of having a high amount of communication overhead, an ADR is used when there is already a decision made and implementation details need to be added. The ADR should document what the collective consensus for the specific issue is and how to solve it. + +1. Every ADR should start with either an RFC or a discussion where consensus has been met. + +2. Once consensus is met, a GitHub Pull Request (PR) is created with a new document based on the `adr-template.md`. + +3. If a _proposed_ ADR is merged, then it should clearly document outstanding issues either in ADR document notes or in a GitHub Issue. + +4. The PR SHOULD always be merged. In the case of a faulty ADR, we still prefer to merge it with a _rejected_ status. The only time the ADR SHOULD NOT be merged is if the author abandons it. + +5. Merged ADRs SHOULD NOT be pruned. + +### ADR status + +Status has two components: + +```text +{CONSENSUS STATUS} {IMPLEMENTATION STATUS} +``` + +IMPLEMENTATION STATUS is either `Implemented` or `Not Implemented`. + +#### Consensus Status + +```text +DRAFT -> PROPOSED -> LAST CALL yyyy-mm-dd -> ACCEPTED | REJECTED -> SUPERSEDED by ADR-xxx + \ | + \ | + v v + ABANDONED +``` + +* `DRAFT`: [optional] an ADR which is a work in progress, not being ready for a general review. This is to present an early work and get early feedback in a Draft Pull Request form. +* `PROPOSED`: an ADR covering a full solution architecture and still in the review - project stakeholders haven't reached an agreement yet. +* `LAST CALL `: [optional] Notify that we are close to accepting updates. Changing a status to `LAST CALL` means that social consensus (of Cosmos SDK maintainers) has been reached and we still want to give it a time to let the community react or analyze. +* `ACCEPTED`: ADR which will represent a currently implemented or to be implemented architecture design. +* `REJECTED`: ADR can go from PROPOSED or ACCEPTED to rejected if the consensus among project stakeholders will decide so. +* `SUPERSEDED by ADR-xxx`: ADR which has been superseded by a new ADR. +* `ABANDONED`: the ADR is no longer pursued by the original authors. + +## Language used in ADR + +* The context/background should be written in the present tense. +* Avoid using a first, personal form. diff --git a/versioned_docs/version-0.52/build/architecture/README.md b/versioned_docs/version-0.52/build/architecture/README.md new file mode 100644 index 000000000..dc2839423 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/README.md @@ -0,0 +1,100 @@ +--- +sidebar_position: 1 +--- + +# Architecture Decision Records (ADR) + +This is a location to record all high-level architecture decisions in the Cosmos-SDK. + +An Architectural Decision (**AD**) is a software design choice that addresses a functional or non-functional requirement that is architecturally significant. +An Architecturally Significant Requirement (**ASR**) is a requirement that has a measurable effect on a software system’s architecture and quality. +An Architectural Decision Record (**ADR**) captures a single AD, such as often done when writing personal notes or meeting minutes; the collection of ADRs created and maintained in a project constitute its decision log. All these are within the topic of Architectural Knowledge Management (AKM). + +You can read more about the ADR concept in this [blog post](https://product.reverb.com/documenting-architecture-decisions-the-reverb-way-a3563bb24bd0#.78xhdix6t). + +## Rationale + +ADRs are intended to be the primary mechanism for proposing new feature designs and new processes, for collecting community input on an issue, and for documenting the design decisions. +An ADR should provide: + +* Context on the relevant goals and the current state +* Proposed changes to achieve the goals +* Summary of pros and cons +* References +* Changelog + +Note the distinction between an ADR and a spec. The ADR provides the context, intuition, reasoning, and +justification for a change in architecture, or for the architecture of something +new. The spec is much more compressed and streamlined summary of everything as +it stands today. + +If recorded decisions turned out to be lacking, convene a discussion, record the new decisions here, and then modify the code to match. + +## Creating new ADR + +Read about the [PROCESS](./PROCESS.md). + +### Use RFC 2119 Keywords + +When writing ADRs, follow the same best practices for writing RFCs. When writing RFCs, key words are used to signify the requirements in the specification. These words are often capitalized: "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL. They are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). + +## ADR Table of Contents + +### Accepted + +* [ADR 002: SDK Documentation Structure](./adr-002-docs-structure.md) +* [ADR 004: Split Denomination Keys](./adr-004-split-denomination-keys.md) +* [ADR 006: Secret Store Replacement](./adr-006-secret-store-replacement.md) +* [ADR 009: Evidence Module](./adr-009-evidence-module.md) +* [ADR 010: Modular AnteHandler](./adr-010-modular-antehandler.md) +* [ADR 019: Protocol Buffer State Encoding](./adr-019-protobuf-state-encoding.md) +* [ADR 020: Protocol Buffer Transaction Encoding](./adr-020-protobuf-transaction-encoding.md) +* [ADR 021: Protocol Buffer Query Encoding](./adr-021-protobuf-query-encoding.md) +* [ADR 023: Protocol Buffer Naming and Versioning](./adr-023-protobuf-naming.md) +* [ADR 024: Coin Metadata](./adr-024-coin-metadata.md) +* [ADR 029: Fee Grant Module](./adr-029-fee-grant-module.md) +* [ADR 030: Message Authorization Module](./adr-030-authz-module.md) +* [ADR 031: Protobuf Msg Services](./adr-031-msg-service.md) +* [ADR 046: Module Params](./adr-046-module-params.md) +* [ADR 055: ORM](./adr-055-orm.md) +* [ADR 058: Auto-Generated CLI](./adr-058-auto-generated-cli.md) +* [ADR 060: ABCI 1.0 (Phase I)](adr-060-abci-1.0.md) +* [ADR 061: Liquid Staking](./adr-061-liquid-staking.md) +* [ADR 070: Un-Ordered Transaction Inclusion](./adr-070-unordered-transactions.md) +* [ADR 065: Store v2](./adr-065-store-v2.md) +* [ADR 073: Built-in In-process Indexer](./adr-073-indexer.md) + +### Proposed + +* [ADR 003: Dynamic Capability Store](./adr-003-dynamic-capability-store.md) +* [ADR 011: Generalize Genesis Accounts](./adr-011-generalize-genesis-accounts.md) +* [ADR 012: State Accessors](./adr-012-state-accessors.md) +* [ADR 013: Metrics](./adr-013-metrics.md) +* [ADR 016: Validator Consensus Key Rotation](./adr-016-validator-consensus-key-rotation.md) +* [ADR 017: Historical Header Module](./adr-017-historical-header-module.md) +* [ADR 018: Extendable Voting Periods](./adr-018-extendable-voting-period.md) +* [ADR 022: Custom baseapp panic handling](./adr-022-custom-panic-handling.md) +* [ADR 027: Deterministic Protobuf Serialization](./adr-027-deterministic-protobuf-serialization.md) +* [ADR 028: Public Key Addresses](./adr-028-public-key-addresses.md) +* [ADR 032: Typed Events](./adr-032-typed-events.md) +* [ADR 033: Inter-module RPC](./adr-033-protobuf-inter-module-comm.md) +* [ADR 035: Rosetta API Support](./adr-035-rosetta-api-support.md) +* [ADR 037: Governance Split Votes](./adr-037-gov-split-vote.md) +* [ADR 038: State Listening](./adr-038-state-listening.md) +* [ADR 039: Epoched Staking](./adr-039-epoched-staking.md) +* [ADR 040: Storage and SMT State Commitments](./adr-040-storage-and-smt-state-commitments.md) +* [ADR 054: Semver Compatible SDK Modules](./adr-054-semver-compatible-modules.md) +* [ADR 057: App Wiring](./adr-057-app-wiring.md) +* [ADR 059: Test Scopes](./adr-059-test-scopes.md) +* [ADR 062: Collections State Layer](./adr-062-collections-state-layer.md) +* [ADR 063: Core Module API](./adr-063-core-module-api.md) +* [ADR 067: Simulator v2](./adr-067-simulator-v2.md) +* [ADR 069: `x/gov` modularity, multiple choice and optimisic proposals](./adr-069-gov-improvements.md) +* [ADR 074: Messages with implicit signers](./adr-074-implicit-msg-signers.md) + +### Draft + +* [ADR 044: Guidelines for Updating Protobuf Definitions](./adr-044-protobuf-updates-guidelines.md) +* [ADR 047: Extend Upgrade Plan](./adr-047-extend-upgrade-plan.md) +* [ADR 053: Go Module Refactoring](./adr-053-go-module-refactoring.md) +* [ADR 068: Preblock](./adr-068-preblock.md) diff --git a/versioned_docs/version-0.52/build/architecture/_category_.json b/versioned_docs/version-0.52/build/architecture/_category_.json new file mode 100644 index 000000000..e0b1907a9 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "ADRs", + "position": 6, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/architecture/adr-002-docs-structure.md b/versioned_docs/version-0.52/build/architecture/adr-002-docs-structure.md new file mode 100644 index 000000000..aea7d60e8 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-002-docs-structure.md @@ -0,0 +1,86 @@ +# ADR 002: SDK Documentation Structure + +## Context + +There is a need for a scalable structure of the Cosmos SDK documentation. Current documentation includes a lot of non-related Cosmos SDK material, is difficult to maintain and hard to follow as a user. + +Ideally, we would have: + +* All docs related to dev frameworks or tools live in their respective github repos (sdk repo would contain sdk docs, hub repo would contain hub docs, lotion repo would contain lotion docs, etc.) +* All other docs (faqs, whitepaper, high-level material about Cosmos) would live on the website. + +## Decision + +Re-structure the `/docs` folder of the Cosmos SDK github repo as follows: + +```text +docs/ +├── README +├── intro/ +├── concepts/ +│ ├── baseapp +│ ├── types +│ ├── store +│ ├── server +│ ├── modules/ +│ │ ├── keeper +│ │ ├── handler +│ │ ├── cli +│ ├── gas +│ └── commands +├── clients/ +│ ├── lite/ +│ ├── service-providers +├── modules/ +├── spec/ +├── translations/ +└── architecture/ +``` + +The files in each sub-folders do not matter and will likely change. What matters is the sectioning: + +* `README`: Landing page of the docs. +* `intro`: Introductory material. Goal is to have a short explainer of the Cosmos SDK and then channel people to the resources they need. The [Cosmos SDK tutorial](https://github.com/cosmos/sdk-application-tutorial/) will be highlighted, as well as the `godocs`. +* `concepts`: Contains high-level explanations of the abstractions of the Cosmos SDK. It does not contain specific code implementation and does not need to be updated often. **It is not an API specification of the interfaces**. API spec is the `godoc`. +* `clients`: Contains specs and info about the various Cosmos SDK clients. +* `spec`: Contains specs of modules, and others. +* `modules`: Contains links to `godocs` and the spec of the modules. +* `architecture`: Contains architecture-related docs like the present one. +* `translations`: Contains different translations of the documentation. + +Website docs sidebar will only include the following sections: + +* `README` +* `intro` +* `concepts` +* `clients` + +`architecture` need not be displayed on the website. + +## Status + +Accepted + +## Consequences + +### Positive + +* Much clearer organisation of the Cosmos SDK docs. +* The `/docs` folder now only contains Cosmos SDK and gaia related material. Later, it will only contain Cosmos SDK related material. +* Developers only have to update `/docs` folder when they open a PR (and not `/examples` for example). +* Easier for developers to find what they need to update in the docs thanks to reworked architecture. +* Cleaner `vuepress` build for website docs. +* Will help build an executable doc (cf https://github.com/cosmos/cosmos-sdk/issues/2611) + +### Neutral + +* We need to move a bunch of deprecated stuff to `/_attic` folder. +* We need to integrate content in `docs/sdk/docs/core` in `concepts`. +* We need to move all the content that currently lives in `docs` and does not fit in new structure (like `lotion`, intro material, whitepaper) to the website repository. +* Update `DOCS_README.md` + +## References + +* https://github.com/cosmos/cosmos-sdk/issues/1460 +* https://github.com/cosmos/cosmos-sdk/pull/2695 +* https://github.com/cosmos/cosmos-sdk/issues/2611 diff --git a/versioned_docs/version-0.52/build/architecture/adr-003-dynamic-capability-store.md b/versioned_docs/version-0.52/build/architecture/adr-003-dynamic-capability-store.md new file mode 100644 index 000000000..89f0c9967 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-003-dynamic-capability-store.md @@ -0,0 +1,344 @@ +# ADR 3: Dynamic Capability Store + +## Changelog + +* 12 December 2019: Initial version +* 02 April 2020: Memory Store Revisions + +## Context + +Full implementation of the [IBC specification](https://github.com/cosmos/ibc) requires the ability to create and authenticate object-capability keys at runtime (i.e., during transaction execution), +as described in [ICS 5](https://github.com/cosmos/ibc/tree/master/spec/core/ics-005-port-allocation#technical-specification). In the IBC specification, capability keys are created for each newly initialised +port & channel, and are used to authenticate future usage of the port or channel. Since channels and potentially ports can be initialised during transaction execution, the state machine must be able to create +object-capability keys at this time. + +At present, the Cosmos SDK does not have the ability to do this. Object-capability keys are currently pointers (memory addresses) of `StoreKey` structs created at application initialisation in `app.go` ([example](https://github.com/cosmos/gaia/blob/dcbddd9f04b3086c0ad07ee65de16e7adedc7da4/app/app.go#L132)) +and passed to Keepers as fixed arguments ([example](https://github.com/cosmos/gaia/blob/dcbddd9f04b3086c0ad07ee65de16e7adedc7da4/app/app.go#L160)). Keepers cannot create or store capability keys during transaction execution — although they could call `NewKVStoreKey` and take the memory address +of the returned struct, storing this in the Merklised store would result in a consensus fault, since the memory address will be different on each machine (this is intentional — were this not the case, the keys would be predictable and couldn't serve as object capabilities). + +Keepers need a way to keep a private map of store keys which can be altered during transaction execution, along with a suitable mechanism for regenerating the unique memory addresses (capability keys) in this map whenever the application is started or restarted, along with a mechanism to revert capability creation on tx failure. +This ADR proposes such an interface & mechanism. + +## Decision + +The Cosmos SDK will include a new `CapabilityKeeper` abstraction, which is responsible for provisioning, +tracking, and authenticating capabilities at runtime. During application initialisation in `app.go`, +the `CapabilityKeeper` will be hooked up to modules through unique function references +(by calling `ScopeToModule`, defined below) so that it can identify the calling module when later +invoked. + +When the initial state is loaded from disk, the `CapabilityKeeper`'s `Initialise` function will create +new capability keys for all previously allocated capability identifiers (allocated during execution of +past transactions and assigned to particular modes), and keep them in a memory-only store while the +chain is running. + +The `CapabilityKeeper` will include a persistent `KVStore`, a `MemoryStore`, and an in-memory map. +The persistent `KVStore` tracks which capability is owned by which modules. +The `MemoryStore` stores a forward mapping that map from module name, capability tuples to capability names and +a reverse mapping that map from module name, capability name to the capability index. +Since we cannot marshal the capability into a `KVStore` and unmarshal without changing the memory location of the capability, +the reverse mapping in the KVStore will simply map to an index. This index can then be used as a key in the ephemeral +go-map to retrieve the capability at the original memory location. + +The `CapabilityKeeper` will define the following types & functions: + +The `Capability` is similar to `StoreKey`, but has a globally unique `Index()` instead of +a name. A `String()` method is provided for debugging. + +A `Capability` is simply a struct, the address of which is taken for the actual capability. + +```go +type Capability struct { + index uint64 +} +``` + +A `CapabilityKeeper` contains a persistent store key, memory store key, and mapping of allocated module names. + +```go +type CapabilityKeeper struct { + persistentKey StoreKey + memKey StoreKey + capMap map[uint64]*Capability + moduleNames map[string]interface{} + sealed bool +} +``` + +The `CapabilityKeeper` provides the ability to create *scoped* sub-keepers which are tied to a +particular module name. These `ScopedCapabilityKeeper`s must be created at application initialisation +and passed to modules, which can then use them to claim capabilities they receive and retrieve +capabilities which they own by name, in addition to creating new capabilities & authenticating capabilities +passed by other modules. + +```go +type ScopedCapabilityKeeper struct { + persistentKey StoreKey + memKey StoreKey + capMap map[uint64]*Capability + moduleName string +} +``` + +`ScopeToModule` is used to create a scoped sub-keeper with a particular name, which must be unique. +It MUST be called before `InitialiseAndSeal`. + +```go +func (ck CapabilityKeeper) ScopeToModule(moduleName string) ScopedCapabilityKeeper { + if ck.sealed { + panic("cannot scope to module via a sealed capability keeper") + } + + if _, ok := ck.scopedModules[moduleName]; ok { + panic(fmt.Sprintf("cannot create multiple scoped keepers for the same module name: %s", moduleName)) + } + + ck.scopedModules[moduleName] = struct{}{} + + return ScopedKeeper{ + cdc: ck.cdc, + storeKey: ck.storeKey, + memKey: ck.memKey, + capMap: ck.capMap, + module: moduleName, + } +} +``` + +`InitialiseAndSeal` MUST be called exactly once, after loading the initial state and creating all +necessary `ScopedCapabilityKeeper`s, in order to populate the memory store with newly-created +capability keys in accordance with the keys previously claimed by particular modules and prevent the +creation of any new `ScopedCapabilityKeeper`s. + +```go +func (ck CapabilityKeeper) InitialiseAndSeal(ctx Context) { + if ck.sealed { + panic("capability keeper is sealed") + } + + persistentStore := ctx.KVStore(ck.persistentKey) + map := ctx.KVStore(ck.memKey) + + // initialise memory store for all names in persistent store + for index, value := range persistentStore.Iter() { + capability = &CapabilityKey{index: index} + + for moduleAndCapability := range value { + moduleName, capabilityName := moduleAndCapability.Split("/") + memStore.Set(moduleName + "/fwd/" + capability, capabilityName) + memStore.Set(moduleName + "/rev/" + capabilityName, index) + + ck.capMap[index] = capability + } + } + + ck.sealed = true +} +``` + +`NewCapability` can be called by any module to create a new unique, unforgeable object-capability +reference. The newly created capability is automatically persisted; the calling module need not +call `ClaimCapability`. + +```go +func (sck ScopedCapabilityKeeper) NewCapability(ctx Context, name string) (Capability, error) { + // check name not taken in memory store + if capStore.Get("rev/" + name) != nil { + return nil, errors.New("name already taken") + } + + // fetch the current index + index := persistentStore.Get("index") + + // create a new capability + capability := &CapabilityKey{index: index} + + // set persistent store + persistentStore.Set(index, Set.singleton(sck.moduleName + "/" + name)) + + // update the index + index++ + persistentStore.Set("index", index) + + // set forward mapping in memory store from capability to name + memStore.Set(sck.moduleName + "/fwd/" + capability, name) + + // set reverse mapping in memory store from name to index + memStore.Set(sck.moduleName + "/rev/" + name, index) + + // set the in-memory mapping from index to capability pointer + capMap[index] = capability + + // return the newly created capability + return capability +} +``` + +`AuthenticateCapability` can be called by any module to check that a capability +does in fact correspond to a particular name (the name can be untrusted user input) +with which the calling module previously associated it. + +```go +func (sck ScopedCapabilityKeeper) AuthenticateCapability(name string, capability Capability) bool { + // return whether forward mapping in memory store matches name + return memStore.Get(sck.moduleName + "/fwd/" + capability) === name +} +``` + +`ClaimCapability` allows a module to claim a capability key which it has received from another module +so that future `GetCapability` calls will succeed. + +`ClaimCapability` MUST be called if a module which receives a capability wishes to access it by name +in the future. Capabilities are multi-owner, so if multiple modules have a single `Capability` reference, +they will all own it. + +```go +func (sck ScopedCapabilityKeeper) ClaimCapability(ctx Context, capability Capability, name string) error { + persistentStore := ctx.KVStore(sck.persistentKey) + + // set forward mapping in memory store from capability to name + memStore.Set(sck.moduleName + "/fwd/" + capability, name) + + // set reverse mapping in memory store from name to capability + memStore.Set(sck.moduleName + "/rev/" + name, capability) + + // update owner set in persistent store + owners := persistentStore.Get(capability.Index()) + owners.add(sck.moduleName + "/" + name) + persistentStore.Set(capability.Index(), owners) +} +``` + +`GetCapability` allows a module to fetch a capability which it has previously claimed by name. +The module is not allowed to retrieve capabilities which it does not own. + +```go +func (sck ScopedCapabilityKeeper) GetCapability(ctx Context, name string) (Capability, error) { + // fetch the index of capability using reverse mapping in memstore + index := memStore.Get(sck.moduleName + "/rev/" + name) + + // fetch capability from go-map using index + capability := capMap[index] + + // return the capability + return capability +} +``` + +`ReleaseCapability` allows a module to release a capability which it had previously claimed. If no +more owners exist, the capability will be deleted globally. + +```go +func (sck ScopedCapabilityKeeper) ReleaseCapability(ctx Context, capability Capability) err { + persistentStore := ctx.KVStore(sck.persistentKey) + + name := capStore.Get(sck.moduleName + "/fwd/" + capability) + if name == nil { + return error("capability not owned by module") + } + + // delete forward mapping in memory store + memoryStore.Delete(sck.moduleName + "/fwd/" + capability, name) + + // delete reverse mapping in memory store + memoryStore.Delete(sck.moduleName + "/rev/" + name, capability) + + // update owner set in persistent store + owners := persistentStore.Get(capability.Index()) + owners.remove(sck.moduleName + "/" + name) + if owners.size() > 0 { + // there are still other owners, keep the capability around + persistentStore.Set(capability.Index(), owners) + } else { + // no more owners, delete the capability + persistentStore.Delete(capability.Index()) + delete(capMap[capability.Index()]) + } +} +``` + +### Usage patterns + +#### Initialisation + +Any modules which use dynamic capabilities must be provided a `ScopedCapabilityKeeper` in `app.go`: + +```go +ck := NewCapabilityKeeper(persistentKey, memoryKey) +mod1Keeper := NewMod1Keeper(ck.ScopeToModule("mod1"), ....) +mod2Keeper := NewMod2Keeper(ck.ScopeToModule("mod2"), ....) + +// other initialisation logic ... + +// load initial state... + +ck.InitialiseAndSeal(initialContext) +``` + +#### Creating, passing, claiming and using capabilities + +Consider the case where `mod1` wants to create a capability, associate it with a resource (e.g. an IBC channel) by name, then pass it to `mod2` which will use it later: + +Module 1 would have the following code: + +```go +capability := scopedCapabilityKeeper.NewCapability(ctx, "resourceABC") +mod2Keeper.SomeFunction(ctx, capability, args...) +``` + +`SomeFunction`, running in module 2, could then claim the capability: + +```go +func (k Mod2Keeper) SomeFunction(ctx Context, capability Capability) { + k.sck.ClaimCapability(ctx, capability, "resourceABC") + // other logic... +} +``` + +Later on, module 2 can retrieve that capability by name and pass it to module 1, which will authenticate it against the resource: + +```go +func (k Mod2Keeper) SomeOtherFunction(ctx Context, name string) { + capability := k.sck.GetCapability(ctx, name) + mod1.UseResource(ctx, capability, "resourceABC") +} +``` + +Module 1 will then check that this capability key is authenticated to use the resource before allowing module 2 to use it: + +```go +func (k Mod1Keeper) UseResource(ctx Context, capability Capability, resource string) { + if !k.sck.AuthenticateCapability(name, capability) { + return errors.New("unauthenticated") + } + // do something with the resource +} +``` + +If module 2 passed the capability key to module 3, module 3 could then claim it and call module 1 just like module 2 did +(in which case module 1, module 2, and module 3 would all be able to use this capability). + +## Status + +Proposed. + +## Consequences + +### Positive + +* Dynamic capability support. +* Allows CapabilityKeeper to return same capability pointer from go-map while reverting any writes to the persistent `KVStore` and in-memory `MemoryStore` on tx failure. + +### Negative + +* Requires an additional keeper. +* Some overlap with existing `StoreKey` system (in the future they could be combined, since this is a superset functionality-wise). +* Requires an extra level of indirection in the reverse mapping, since MemoryStore must map to index which must then be used as key in a go map to retrieve the actual capability + +### Neutral + +(none known) + +## References + +* [Original discussion](https://github.com/cosmos/cosmos-sdk/pull/5230#discussion_r343978513) diff --git a/versioned_docs/version-0.52/build/architecture/adr-004-split-denomination-keys.md b/versioned_docs/version-0.52/build/architecture/adr-004-split-denomination-keys.md new file mode 100644 index 000000000..465ac7b96 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-004-split-denomination-keys.md @@ -0,0 +1,120 @@ +# ADR 004: Split Denomination Keys + +## Changelog + +* 2020-01-08: Initial version +* 2020-01-09: Alterations to handle vesting accounts +* 2020-01-14: Updates from review feedback +* 2020-01-30: Updates from implementation + +### Glossary + +* denom / denomination key -- unique token identifier. + +## Context + +With permissionless IBC, anyone will be able to send arbitrary denominations to any other account. Currently, all non-zero balances are stored along with the account in an `sdk.Coins` struct, which creates a potential denial-of-service concern, as too many denominations will become expensive to load & store each time the account is modified. See issues [5467](https://github.com/cosmos/cosmos-sdk/issues/5467) and [4982](https://github.com/cosmos/cosmos-sdk/issues/4982) for additional context. + +Simply rejecting incoming deposits after a denomination count limit doesn't work, since it opens up a griefing vector: someone could send a user lots of nonsensical coins over IBC, and then prevent the user from receiving real denominations (such as staking rewards). + +## Decision + +Balances shall be stored per-account & per-denomination under a denomination- and account-unique key, thus enabling O(1) read & write access to the balance of a particular account in a particular denomination. + +### Account interface (x/auth) + +`GetCoins()` and `SetCoins()` will be removed from the account interface, since coin balances will +now be stored in & managed by the bank module. + +The vesting account interface will replace `SpendableCoins` in favor of `LockedCoins` which does +not require the account balance anymore. In addition, `TrackDelegation()` will now accept the +account balance of all tokens denominated in the vesting balance instead of loading the entire +account balance. + +Vesting accounts will continue to store original vesting, delegated free, and delegated +vesting coins (which is safe since these cannot contain arbitrary denominations). + +### Bank keeper (x/bank) + +The following APIs will be added to the `x/bank` keeper: + +* `GetAllBalances(ctx Context, addr AccAddress) Coins` +* `GetBalance(ctx Context, addr AccAddress, denom string) Coin` +* `SetBalance(ctx Context, addr AccAddress, coin Coin)` +* `LockedCoins(ctx Context, addr AccAddress) Coins` +* `SpendableCoins(ctx Context, addr AccAddress) Coins` + +Additional APIs may be added to facilitate iteration and auxiliary functionality not essential to +core functionality or persistence. + +Balances will be stored first by the address, then by the denomination (the reverse is also possible, +but retrieval of all balances for a single account is presumed to be more frequent): + +```go +var BalancesPrefix = []byte("balances") + +func (k Keeper) SetBalance(ctx Context, addr AccAddress, balance Coin) error { + if !balance.IsValid() { + return err + } + + store := ctx.KVStore(k.storeKey) + balancesStore := prefix.NewStore(store, BalancesPrefix) + accountStore := prefix.NewStore(balancesStore, addr.Bytes()) + + bz := Marshal(balance) + accountStore.Set([]byte(balance.Denom), bz) + + return nil +} +``` + +This will result in the balances being indexed by the byte representation of +`balances/{address}/{denom}`. + +`DelegateCoins()` and `UndelegateCoins()` will be altered to only load each individual +account balance by denomination found in the (un)delegation amount. As a result, +any mutations to the account balance will made by denomination. + +`SubtractCoins()` and `AddCoins()` will be altered to read & write the balances +directly instead of calling `GetCoins()` / `SetCoins()` (which no longer exist). + +`trackDelegation()` and `trackUndelegation()` will be altered to no longer update +account balances. + +External APIs will need to scan all balances under an account to retain backwards-compatibility. It +is advised that these APIs use `GetBalance` and `SetBalance` instead of `GetAllBalances` when +possible as to not load the entire account balance. + +### Supply module + +The supply module, in order to implement the total supply invariant, will now need +to scan all accounts & call `GetAllBalances` using the `x/bank` Keeper, then sum +the balances and check that they match the expected total supply. + +## Status + +Accepted. + +## Consequences + +### Positive + +* O(1) reads & writes of balances (with respect to the number of denominations for +which an account has non-zero balances). Note, this does not relate to the actual +I/O cost, rather the total number of direct reads needed. + +### Negative + +* Slightly less efficient reads/writes when reading & writing all balances of a +single account in a transaction. + +### Neutral + +None in particular. + +## References + +* Ref: https://github.com/cosmos/cosmos-sdk/issues/4982 +* Ref: https://github.com/cosmos/cosmos-sdk/issues/5467 +* Ref: https://github.com/cosmos/cosmos-sdk/issues/5492 diff --git a/versioned_docs/version-0.52/build/architecture/adr-006-secret-store-replacement.md b/versioned_docs/version-0.52/build/architecture/adr-006-secret-store-replacement.md new file mode 100644 index 000000000..500ba40ca --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-006-secret-store-replacement.md @@ -0,0 +1,54 @@ +# ADR 006: Secret Store Replacement + +## Changelog + +* July 29th, 2019: Initial draft +* September 11th, 2019: Work has started +* November 4th: Cosmos SDK changes merged in +* November 18th: Gaia changes merged in + +## Context + +Currently, a Cosmos SDK application's CLI directory stores key material and metadata in a plain text database in the user’s home directory. Key material is encrypted by a passphrase, protected by bcrypt hashing algorithm. Metadata (e.g. addresses, public keys, key storage details) is available in plain text. + +This is not desirable for a number of reasons. Perhaps the biggest reason is insufficient security protection of key material and metadata. Leaking the plain text allows an attacker to surveil what keys a given computer controls via a number of techniques, like compromised dependencies without any privilege execution. This could be followed by a more targeted attack on a particular user/computer. + +All modern desktop computers OS (Ubuntu, Debian, MacOS, Windows) provide a built-in secret store that is designed to allow applications to store information that is isolated from all other applications and requires passphrase entry to access the data. + +We are seeking solution that provides a common abstraction layer to the many different backends and reasonable fallback for minimal platforms that don’t provide a native secret store. + +## Decision + +We recommend replacing the current Keybase backend based on LevelDB with [Keyring](https://github.com/99designs/keyring) by 99 designs. This application is designed to provide a common abstraction and uniform interface between many secret stores and is used by AWS Vault application by 99-designs application. + +This appears to fulfill the requirement of protecting both key material and metadata from rogue software on a user’s machine. + +## Status + +Accepted + +## Consequences + +### Positive + +Increased safety for users. + +### Negative + +Users must manually migrate. + +Testing against all supported backends is difficult. + +Running tests locally on a Mac require numerous repetitive password entries. + +### Neutral + +{neutral consequences} + +## References + +* #4754 Switch secret store to the keyring secret store (original PR by @poldsam) [__CLOSED__] +* #5029 Add support for github.com/99designs/keyring-backed keybases [__MERGED__] +* #5097 Add keys migrate command [__MERGED__] +* #5180 Drop on-disk keybase in favor of keyring [_PENDING_REVIEW_] +* cosmos/gaia#164 Drop on-disk keybase in favor of keyring (gaia's changes) [_PENDING_REVIEW_] diff --git a/versioned_docs/version-0.52/build/architecture/adr-007-specialization-groups.md b/versioned_docs/version-0.52/build/architecture/adr-007-specialization-groups.md new file mode 100644 index 000000000..dd5617a49 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-007-specialization-groups.md @@ -0,0 +1,177 @@ +# ADR 007: Specialization Groups + +## Changelog + +* 2019 Jul 31: Initial Draft + +## Context + +This idea was first conceived of in order to fulfill the use case of the +creation of a decentralized Computer Emergency Response Team (dCERT), whose +members would be elected by a governing community and would fulfill the role of +coordinating the community under emergency situations. This thinking +can be further abstracted into the concept of "blockchain specialization +groups". + +The creation of these groups are the beginning of specialization capabilities +within a wider blockchain community which could be used to enable a certain +level of delegated responsibilities. Examples of specialization which could be +beneficial to a blockchain community include: code auditing, emergency response, +code development etc. This type of community organization paves the way for +individual stakeholders to delegate votes by issue type, if in the future +governance proposals include a field for issue type. + +## Decision + +A specialization group can be broadly broken down into the following functions +(herein containing examples): + +* Membership Admittance +* Membership Acceptance +* Membership Revocation + * (probably) Without Penalty + * member steps down (self-Revocation) + * replaced by new member from governance + * (probably) With Penalty + * due to breach of soft-agreement (determined through governance) + * due to breach of hard-agreement (determined by code) +* Execution of Duties + * Special transactions which only execute for members of a specialization + group (for example, dCERT members voting to turn off transaction routes in + an emergency scenario) +* Compensation + * Group compensation (further distribution decided by the specialization group) + * Individual compensation for all constituents of a group from the + greater community + +Membership admission to a specialization group could take place over a wide +variety of mechanisms. The most obvious example is through a general vote among +the entire community, however in certain systems a community may want to allow +the members already in a specialization group to internally elect new members, +or maybe the community may assign a permission to a particular specialization +group to appoint members to other 3rd party groups. The sky is really the limit +as to how membership admittance can be structured. We attempt to capture +some of these possibilities in a common interface dubbed the `Electionator`. For +its initial implementation as a part of this ADR we recommend that the general +election abstraction (`Electionator`) is provided as well as a basic +implementation of that abstraction which allows for a continuous election of +members of a specialization group. + +``` golang +// The Electionator abstraction covers the concept space for +// a wide variety of election kinds. +type Electionator interface { + + // is the election object accepting votes. + Active() bool + + // functionality to execute for when a vote is cast in this election, here + // the vote field is anticipated to be marshalled into a vote type used + // by an election. + // + // NOTE There are no explicit ids here. Just votes which pertain specifically + // to one electionator. Anyone can create and send a vote to the electionator item + // which will presumably attempt to marshal those bytes into a particular struct + // and apply the vote information in some arbitrary way. There can be multiple + // Electionators within the Cosmos-Hub for multiple specialization groups, votes + // would need to be routed to the Electionator upstream of here. + Vote(addr sdk.AccAddress, vote []byte) + + // here lies all functionality to authenticate and execute changes for + // when a member accepts being elected + AcceptElection(sdk.AccAddress) + + // Register a revoker object + RegisterRevoker(Revoker) + + // No more revokers may be registered after this function is called + SealRevokers() + + // register hooks to call when an election actions occur + RegisterHooks(ElectionatorHooks) + + // query for the current winner(s) of this election based on arbitrary + // election ruleset + QueryElected() []sdk.AccAddress + + // query metadata for an address in the election this + // could include for example position that an address + // is being elected for within a group + // + // this metadata may be directly related to + // voting information and/or privileges enabled + // to members within a group. + QueryMetadata(sdk.AccAddress) []byte +} + +// ElectionatorHooks, once registered with an Electionator, +// trigger execution of relevant interface functions when +// Electionator events occur. +type ElectionatorHooks interface { + AfterVoteCast(addr sdk.AccAddress, vote []byte) + AfterMemberAccepted(addr sdk.AccAddress) + AfterMemberRevoked(addr sdk.AccAddress, cause []byte) +} + +// Revoker defines the function required for a membership revocation rule-set +// used by a specialization group. This could be used to create self revoking, +// and evidence based revoking, etc. Revokers types may be created and +// reused for different election types. +// +// When revoking the "cause" bytes may be arbitrarily marshalled into evidence, +// memos, etc. +type Revoker interface { + RevokeName() string // identifier for this revoker type + RevokeMember(addr sdk.AccAddress, cause []byte) error +} +``` + +Certain level of commonality likely exists between the existing code within +`x/governance` and required functionality of elections. This common +functionality should be abstracted during implementation. Similarly for each +vote implementation client CLI/REST functionality should be abstracted +to be reused for multiple elections. + +The specialization group abstraction firstly extends the `Electionator` +but also further defines traits of the group. + +``` golang +type SpecializationGroup interface { + Electionator + GetName() string + GetDescription() string + + // general soft contract the group is expected + // to fulfill with the greater community + GetContract() string + + // messages which can be executed by the members of the group + Handler(ctx sdk.Context, msg sdk.Msg) sdk.Result + + // logic to be executed at endblock, this may for instance + // include payment of a stipend to the group members + // for participation in the security group. + EndBlocker(ctx sdk.Context) +} +``` + +## Status + +> Proposed + +## Consequences + +### Positive + +* increases specialization capabilities of a blockchain +* improve abstractions in `x/gov/` such that they can be used with specialization groups + +### Negative + +* could be used to increase centralization within a community + +### Neutral + +## References + +* [dCERT ADR](./adr-008-dCERT-group.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-008-dCERT-group.md b/versioned_docs/version-0.52/build/architecture/adr-008-dCERT-group.md new file mode 100644 index 000000000..13d2b340a --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-008-dCERT-group.md @@ -0,0 +1,171 @@ +# ADR 008: Decentralized Computer Emergency Response Team (dCERT) Group + +## Changelog + +* 2019 Jul 31: Initial Draft + +## Context + +In order to reduce the number of parties involved with handling sensitive +information in an emergency scenario, we propose the creation of a +specialization group named The Decentralized Computer Emergency Response Team +(dCERT). Initially this group's role is intended to serve as coordinators +between various actors within a blockchain community such as validators, +bug-hunters, and developers. During a time of crisis, the dCERT group would +aggregate and relay input from a variety of stakeholders to the developers who +are actively devising a patch to the software, this way sensitive information +does not need to be publicly disclosed while some input from the community can +still be gained. + +Additionally, a special privilege is proposed for the dCERT group: the capacity +to "circuit-break" (aka. temporarily disable) a particular message path. Note +that this privilege should be enabled/disabled globally with a governance +parameter such that this privilege could start disabled and later be enabled +through a parameter change proposal, once a dCERT group has been established. + +In the future it is foreseeable that the community may wish to expand the roles +of dCERT with further responsibilities such as the capacity to "pre-approve" a +security update on behalf of the community prior to a full community +wide vote whereby the sensitive information would be revealed prior to a +vulnerability being patched on the live network. + +## Decision + +The dCERT group is proposed to include an implementation of a `SpecializationGroup` +as defined in [ADR 007](./adr-007-specialization-groups.md). This will include the +implementation of: + +* continuous voting +* slashing due to breach of soft contract +* revoking a member due to breach of soft contract +* emergency disband of the entire dCERT group (ex. for colluding maliciously) +* compensation stipend from the community pool or other means decided by + governance + +This system necessitates the following new parameters: + +* blockly stipend allowance per dCERT member +* maximum number of dCERT members +* required staked slashable tokens for each dCERT member +* quorum for suspending a particular member +* proposal wager for disbanding the dCERT group +* stabilization period for dCERT member transition +* circuit break dCERT privileges enabled + +These parameters are expected to be implemented through the param keeper such +that governance may change them at any given point. + +### Continuous Voting Electionator + +An `Electionator` object is to be implemented as continuous voting and with the +following specifications: + +* All delegation addresses may submit votes at any point which updates their + preferred representation on the dCERT group. +* Preferred representation may be arbitrarily split between addresses (ex. 50% + to John, 25% to Sally, 25% to Carol) +* In order for a new member to be added to the dCERT group they must + send a transaction accepting their admission at which point the validity of + their admission is to be confirmed. + * A sequence number is assigned when a member is added to dCERT group. + If a member leaves the dCERT group and then enters back, a new sequence number + is assigned. +* Addresses which control the greatest amount of preferred-representation are + eligible to join the dCERT group (up the _maximum number of dCERT members_). + If the dCERT group is already full and new member is admitted, the existing + dCERT member with the lowest amount of votes is kicked from the dCERT group. + * In the split situation where the dCERT group is full but a vying candidate + has the same amount of vote as an existing dCERT member, the existing + member should maintain its position. + * In the split situation where somebody must be kicked out but the two + addresses with the smallest number of votes have the same number of votes, + the address with the smallest sequence number maintains its position. +* A stabilization period can be optionally included to reduce the + "flip-flopping" of the dCERT membership tail members. If a stabilization + period is provided which is greater than 0, when members are kicked due to + insufficient support, a queue entry is created which documents which member is + to replace which other member. While this entry is in the queue, no new entries + to kick that same dCERT member can be made. When the entry matures at the + duration of the stabilization period, the new member is instantiated, and old + member kicked. + +### Staking/Slashing + +All members of the dCERT group must stake tokens _specifically_ to maintain +eligibility as a dCERT member. These tokens can be staked directly by the vying +dCERT member or out of the good will of a 3rd party (who shall gain no on-chain +benefits for doing so). This staking mechanism should use the existing global +unbonding time of tokens staked for network validator security. A dCERT member +can _only be_ a member if it has the required tokens staked under this +mechanism. If those tokens are unbonded then the dCERT member must be +automatically kicked from the group. + +Slashing of a particular dCERT member due to soft-contract breach should be +performed by governance on a per member basis based on the magnitude of the +breach. The process flow is anticipated to be that a dCERT member is suspended +by the dCERT group prior to being slashed by governance. + +Membership suspension by the dCERT group takes place through a voting procedure +by the dCERT group members. After this suspension has taken place, a governance +proposal to slash the dCERT member must be submitted, if the proposal is not +approved by the time the rescinding member has completed unbonding their +tokens, then the tokens are no longer staked and unable to be slashed. + +Additionally in the case of an emergency situation of a colluding and malicious +dCERT group, the community needs the capability to disband the entire dCERT +group and likely fully slash them. This could be achieved through a special new +proposal type (implemented as a general governance proposal) which would halt +the functionality of the dCERT group until the proposal was concluded. This +special proposal type would likely need to also have a fairly large wager which +could be slashed if the proposal creator was malicious. The reason a large +wager should be required is because as soon as the proposal is made, the +capability of the dCERT group to halt message routes is put on temporarily +suspended, meaning that a malicious actor who created such a proposal could +then potentially exploit a bug during this period of time, with no dCERT group +capable of shutting down the exploitable message routes. + +### dCERT membership transactions + +Active dCERT members + +* change of the description of the dCERT group +* circuit break a message route +* vote to suspend a dCERT member. + +Here circuit-breaking refers to the capability to disable a groups of messages, +This could for instance mean: "disable all staking-delegation messages", or +"disable all distribution messages". This could be accomplished by verifying +that the message route has not been "circuit-broken" at CheckTx time (in +`baseapp/baseapp.go`). + +"unbreaking" a circuit is anticipated only to occur during a hard fork upgrade +meaning that no capability to unbreak a message route on a live chain is +required. + +Note also, that if there was a problem with governance voting (for instance a +capability to vote many times) then governance would be broken and should be +halted with this mechanism, it would be then up to the validator set to +coordinate and hard-fork upgrade to a patched version of the software where +governance is re-enabled (and fixed). If the dCERT group abuses this privilege +they should all be severely slashed. + +## Status + +> Proposed + +## Consequences + +### Positive + +* Potential to reduces the number of parties to coordinate with during an emergency +* Reduction in possibility of disclosing sensitive information to malicious parties + +### Negative + +* Centralization risks + +### Neutral + +## References + + [Specialization Groups ADR](./adr-007-specialization-groups.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-009-evidence-module.md b/versioned_docs/version-0.52/build/architecture/adr-009-evidence-module.md new file mode 100644 index 000000000..46ee5ebcb --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-009-evidence-module.md @@ -0,0 +1,182 @@ +# ADR 009: Evidence Module + +## Changelog + +* 2019 July 31: Initial draft +* 2019 October 24: Initial implementation + +## Status + +Accepted + +## Context + +In order to support building highly secure, robust and interoperable blockchain +applications, it is vital for the Cosmos SDK to expose a mechanism in which arbitrary +evidence can be submitted, evaluated and verified resulting in some agreed upon +penalty for any misbehavior committed by a validator, such as equivocation (double-voting), +signing when unbonded, signing an incorrect state transition (in the future), etc. +Furthermore, such a mechanism is paramount for any +[IBC](https://github.com/cosmos/ibc) or +cross-chain validation protocol implementation in order to support the ability +for any misbehavior to be relayed back from a collateralized chain to a primary +chain so that the equivocating validator(s) can be slashed. + +## Decision + +We will implement an evidence module in the Cosmos SDK supporting the following +functionality: + +* Provide developers with the abstractions and interfaces necessary to define + custom evidence messages, message handlers, and methods to slash and penalize + accordingly for misbehavior. +* Support the ability to route evidence messages to handlers in any module to + determine the validity of submitted misbehavior. +* Support the ability, through governance, to modify slashing penalties of any + evidence type. +* Querier implementation to support querying params, evidence types, params, and + all submitted valid misbehavior. + +### Types + +First, we define the `Evidence` interface type. The `x/evidence` module may implement +its own types that can be used by many chains (e.g. `CounterFactualEvidence`). +In addition, other modules may implement their own `Evidence` types in a similar +manner in which governance is extensible. It is important to note any concrete +type implementing the `Evidence` interface may include arbitrary fields such as +an infraction time. We want the `Evidence` type to remain as flexible as possible. + +When submitting evidence to the `x/evidence` module, the concrete type must provide +the validator's consensus address, which should be known by the `x/slashing` +module (assuming the infraction is valid), the height at which the infraction +occurred and the validator's power at same height in which the infraction occurred. + +```go +type Evidence interface { + Route() string + Type() string + String() string + Hash() HexBytes + ValidateBasic() error + + // The consensus address of the malicious validator at time of infraction + GetConsensusAddress() ConsAddress + + // Height at which the infraction occurred + GetHeight() int64 + + // The total power of the malicious validator at time of infraction + GetValidatorPower() int64 + + // The total validator set power at time of infraction + GetTotalPower() int64 +} +``` + +### Routing & Handling + +Each `Evidence` type must map to a specific unique route and be registered with +the `x/evidence` module. It accomplishes this through the `Router` implementation. + +```go +type Router interface { + AddRoute(r string, h Handler) Router + HasRoute(r string) bool + GetRoute(path string) Handler + Seal() +} +``` + +Upon successful routing through the `x/evidence` module, the `Evidence` type +is passed through a `Handler`. This `Handler` is responsible for executing all +corresponding business logic necessary for verifying the evidence as valid. In +addition, the `Handler` may execute any necessary slashing and potential jailing. +Since slashing fractions will typically result from some form of static functions, +allow the `Handler` to do this provides the greatest flexibility. An example could +be `k * evidence.GetValidatorPower()` where `k` is an on-chain parameter controlled +by governance. The `Evidence` type should provide all the external information +necessary in order for the `Handler` to make the necessary state transitions. +If no error is returned, the `Evidence` is considered valid. + +```go +type Handler func(Context, Evidence) error +``` + +### Submission + +`Evidence` is submitted through a `MsgSubmitEvidence` message type which is internally +handled by the `x/evidence` module's `SubmitEvidence`. + +```go +type MsgSubmitEvidence struct { + Evidence +} + +func handleMsgSubmitEvidence(ctx Context, keeper Keeper, msg MsgSubmitEvidence) Result { + if err := keeper.SubmitEvidence(ctx, msg.Evidence); err != nil { + return err.Result() + } + + // emit events... + + return Result{ + // ... + } +} +``` + +The `x/evidence` module's keeper is responsible for matching the `Evidence` against +the module's router and invoking the corresponding `Handler` which may include +slashing and jailing the validator. Upon success, the submitted evidence is persisted. + +```go +func (k Keeper) SubmitEvidence(ctx Context, evidence Evidence) error { + handler := keeper.router.GetRoute(evidence.Route()) + if err := handler(ctx, evidence); err != nil { + return ErrInvalidEvidence(keeper.codespace, err) + } + + keeper.setEvidence(ctx, evidence) + return nil +} +``` + +### Genesis + +Finally, we need to represent the genesis state of the `x/evidence` module. The +module only needs a list of all submitted valid infractions and any necessary params +for which the module needs in order to handle submitted evidence. The `x/evidence` +module will naturally define and route native evidence types for which it'll most +likely need slashing penalty constants for. + +```go +type GenesisState struct { + Params Params + Infractions []Evidence +} +``` + +## Consequences + +### Positive + +* Allows the state machine to process misbehavior submitted on-chain and penalize + validators based on agreed upon slashing parameters. +* Allows evidence types to be defined and handled by any module. This further allows + slashing and jailing to be defined by more complex mechanisms. +* Does not solely rely on Tendermint to submit evidence. + +### Negative + +* No easy way to introduce new evidence types through governance on a live chain + due to the inability to introduce the new evidence type's corresponding handler + +### Neutral + +* Should we persist infractions indefinitely? Or should we rather rely on events? + +## References + +* [ICS](https://github.com/cosmos/ics) +* [IBC Architecture](https://github.com/cosmos/ibc/blob/main/spec/ics-001-ics-standard/README.md) +* [Tendermint Fork Accountability](https://github.com/tendermint/spec/blob/7b3138e69490f410768d9b1ffc7a17abc23ea397/spec/consensus/fork-accountability.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-010-modular-antehandler.md b/versioned_docs/version-0.52/build/architecture/adr-010-modular-antehandler.md new file mode 100644 index 000000000..4eb5b8855 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-010-modular-antehandler.md @@ -0,0 +1,290 @@ +# ADR 010: Modular AnteHandler + +## Changelog + +* 2019 Aug 31: Initial draft +* 2021 Sep 14: Superseded by ADR-045 + +## Status + +SUPERSEDED by ADR-045 + +## Context + +The current AnteHandler design allows users to either use the default AnteHandler provided in `x/auth` or to build their own AnteHandler from scratch. Ideally AnteHandler functionality is split into multiple, modular functions that can be chained together along with custom ante-functions so that users do not have to rewrite common antehandler logic when they want to implement custom behavior. + +For example, let's say a user wants to implement some custom signature verification logic. In the current codebase, the user would have to write their own Antehandler from scratch largely reimplementing much of the same code and then set their own custom, monolithic antehandler in the baseapp. Instead, we would like to allow users to specify custom behavior when necessary and combine them with default ante-handler functionality in a way that is as modular and flexible as possible. + +## Proposals + +### Per-Module AnteHandler + +One approach is to use the [ModuleManager](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/types/module) and have each module implement its own antehandler if it requires custom antehandler logic. The ModuleManager can then be passed in an AnteHandler order in the same way it has an order for BeginBlockers and EndBlockers. The ModuleManager returns a single AnteHandler function that will take in a tx and run each module's `AnteHandle` in the specified order. The module manager's AnteHandler is set as the baseapp's AnteHandler. + +Pros: + +1. Simple to implement +2. Utilizes the existing ModuleManager architecture + +Cons: + +1. Improves granularity but still cannot get more granular than a per-module basis. e.g. If auth's `AnteHandle` function is in charge of validating memo and signatures, users cannot swap the signature-checking functionality while keeping the rest of auth's `AnteHandle` functionality. +2. Module AnteHandler are run one after the other. There is no way for one AnteHandler to wrap or "decorate" another. + +### Decorator Pattern + +The [weave project](https://github.com/iov-one/weave) achieves AnteHandler modularity through the use of a decorator pattern. The interface is designed as follows: + +```go +// Decorator wraps a Handler to provide common functionality +// like authentication, or fee-handling, to many Handlers +type Decorator interface { + Check(ctx Context, store KVStore, tx Tx, next Checker) (*CheckResult, error) + Deliver(ctx Context, store KVStore, tx Tx, next Deliverer) (*DeliverResult, error) +} +``` + +Each decorator works like a modularized Cosmos SDK antehandler function, but it can take in a `next` argument that may be another decorator or a Handler (which does not take in a next argument). These decorators can be chained together, one decorator being passed in as the `next` argument of the previous decorator in the chain. The chain ends in a Router which can take a tx and route to the appropriate msg handler. + +A key benefit of this approach is that one Decorator can wrap its internal logic around the next Checker/Deliverer. A weave Decorator may do the following: + +```go +// Example Decorator's Deliver function +func (example Decorator) Deliver(ctx Context, store KVStore, tx Tx, next Deliverer) { + // Do some pre-processing logic + + res, err := next.Deliver(ctx, store, tx) + + // Do some post-processing logic given the result and error +} +``` + +Pros: + +1. Weave Decorators can wrap over the next decorator/handler in the chain. The ability to both pre-process and post-process may be useful in certain settings. +2. Provides a nested modular structure that isn't possible in the solution above, while also allowing for a linear one-after-the-other structure like the solution above. + +Cons: + +1. It is hard to understand at first glance the state updates that would occur after a Decorator runs given the `ctx`, `store`, and `tx`. A Decorator can have an arbitrary number of nested Decorators being called within its function body, each possibly doing some pre- and post-processing before calling the next decorator on the chain. Thus to understand what a Decorator is doing, one must also understand what every other decorator further along the chain is also doing. This can get quite complicated to understand. A linear, one-after-the-other approach while less powerful, may be much easier to reason about. + +### Chained Micro-Functions + +The benefit of Weave's approach is that the Decorators can be very concise, which when chained together allows for maximum customizability. However, the nested structure can get quite complex and thus hard to reason about. + +Another approach is to split the AnteHandler functionality into tightly scoped "micro-functions", while preserving the one-after-the-other ordering that would come from the ModuleManager approach. + +We can then have a way to chain these micro-functions so that they run one after the other. Modules may define multiple ante micro-functions and then also provide a default per-module AnteHandler that implements a default, suggested order for these micro-functions. + +Users can order the AnteHandlers easily by simply using the ModuleManager. The ModuleManager will take in a list of AnteHandlers and return a single AnteHandler that runs each AnteHandler in the order of the list provided. If the user is comfortable with the default ordering of each module, this is as simple as providing a list with each module's antehandler (exactly the same as BeginBlocker and EndBlocker). + +If however, users wish to change the order or add, modify, or delete ante micro-functions in anyway; they can always define their own ante micro-functions and add them explicitly to the list that gets passed into module manager. + +#### Default Workflow + +This is an example of a user's AnteHandler if they choose not to make any custom micro-functions. + +##### Cosmos SDK code + +```go +// Chains together a list of AnteHandler micro-functions that get run one after the other. +// Returned AnteHandler will abort on first error. +func Chainer(order []AnteHandler) AnteHandler { + return func(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + for _, ante := range order { + ctx, err := ante(ctx, tx, simulate) + if err != nil { + return ctx, err + } + } + return ctx, err + } +} +``` + +```go +// AnteHandler micro-function to verify signatures +func VerifySignatures(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // verify signatures + // Returns InvalidSignature Result and abort=true if sigs invalid + // Return OK result and abort=false if sigs are valid +} + +// AnteHandler micro-function to validate memo +func ValidateMemo(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // validate memo +} + +// Auth defines its own default ante-handler by chaining its micro-functions in a recommended order +AuthModuleAnteHandler := Chainer([]AnteHandler{VerifySignatures, ValidateMemo}) +``` + +```go +// Distribution micro-function to deduct fees from tx +func DeductFees(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // Deduct fees from tx + // Abort if insufficient funds in account to pay for fees +} + +// Distribution micro-function to check if fees > mempool parameter +func CheckMempoolFees(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // If CheckTx: Abort if the fees are less than the mempool's minFee parameter +} + +// Distribution defines its own default ante-handler by chaining its micro-functions in a recommended order +DistrModuleAnteHandler := Chainer([]AnteHandler{CheckMempoolFees, DeductFees}) +``` + +```go +type ModuleManager struct { + // other fields + AnteHandlerOrder []AnteHandler +} + +func (mm ModuleManager) GetAnteHandler() AnteHandler { + return Chainer(mm.AnteHandlerOrder) +} +``` + +##### User Code + +```go +// Note: Since user is not making any custom modifications, we can just SetAnteHandlerOrder with the default AnteHandlers provided by each module in our preferred order +moduleManager.SetAnteHandlerOrder([]AnteHandler(AuthModuleAnteHandler, DistrModuleAnteHandler)) + +app.SetAnteHandler(mm.GetAnteHandler()) +``` + +#### Custom Workflow + +This is an example workflow for a user that wants to implement custom antehandler logic. In this example, the user wants to implement custom signature verification and change the order of antehandler so that validate memo runs before signature verification. + +##### User Code + +```go +// User can implement their own custom signature verification antehandler micro-function +func CustomSigVerify(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // do some custom signature verification logic +} +``` + +```go +// Micro-functions allow users to change order of when they get executed, and swap out default ante-functionality with their own custom logic. +// Note that users can still chain the default distribution module handler, and auth micro-function along with their custom ante function +moduleManager.SetAnteHandlerOrder([]AnteHandler(ValidateMemo, CustomSigVerify, DistrModuleAnteHandler)) +``` + +Pros: + +1. Allows for ante functionality to be as modular as possible. +2. For users that do not need custom ante-functionality, there is little difference between how antehandlers work and how BeginBlock and EndBlock work in ModuleManager. +3. Still easy to understand + +Cons: + +1. Cannot wrap antehandlers with decorators like you can with Weave. + +### Simple Decorators + +This approach takes inspiration from Weave's decorator design while trying to minimize the number of breaking changes to the Cosmos SDK and maximizing simplicity. Like Weave decorators, this approach allows one `AnteDecorator` to wrap the next AnteHandler to do pre- and post-processing on the result. This is useful since decorators can do defer/cleanups after an AnteHandler returns as well as perform some setup beforehand. Unlike Weave decorators, these `AnteDecorator` functions can only wrap over the AnteHandler rather than the entire handler execution path. This is deliberate as we want decorators from different modules to perform authentication/validation on a `tx`. However, we do not want decorators being capable of wrapping and modifying the results of a `MsgHandler`. + +In addition, this approach will not break any core Cosmos SDK API's. Since we preserve the notion of an AnteHandler and still set a single AnteHandler in baseapp, the decorator is simply an additional approach available for users that desire more customization. The API of modules (namely `x/auth`) may break with this approach, but the core API remains untouched. + +Allow Decorator interface that can be chained together to create a Cosmos SDK AnteHandler. + +This allows users to choose between implementing an AnteHandler by themselves and setting it in the baseapp, or use the decorator pattern to chain their custom decorators with the Cosmos SDK provided decorators in the order they wish. + +```go +// An AnteDecorator wraps an AnteHandler, and can do pre- and post-processing on the next AnteHandler +type AnteDecorator interface { + AnteHandle(ctx Context, tx Tx, simulate bool, next AnteHandler) (newCtx Context, err error) +} +``` + +```go +// ChainAnteDecorators will recursively link all of the AnteDecorators in the chain and return a final AnteHandler function +// This is done to preserve the ability to set a single AnteHandler function in the baseapp. +func ChainAnteDecorators(chain ...AnteDecorator) AnteHandler { + if len(chain) == 1 { + return func(ctx Context, tx Tx, simulate bool) { + chain[0].AnteHandle(ctx, tx, simulate, nil) + } + } + return func(ctx Context, tx Tx, simulate bool) { + chain[0].AnteHandle(ctx, tx, simulate, ChainAnteDecorators(chain[1:])) + } +} +``` + +#### Example Code + +Define AnteDecorator functions + +```go +// Setup GasMeter, catch OutOfGasPanic and handle appropriately +type SetUpContextDecorator struct{} + +func (sud SetUpContextDecorator) AnteHandle(ctx Context, tx Tx, simulate bool, next AnteHandler) (newCtx Context, err error) { + ctx.GasMeter = NewGasMeter(tx.Gas) + + defer func() { + // recover from OutOfGas panic and handle appropriately + } + + return next(ctx, tx, simulate) +} + +// Signature Verification decorator. Verify Signatures and move on +type SigVerifyDecorator struct{} + +func (svd SigVerifyDecorator) AnteHandle(ctx Context, tx Tx, simulate bool, next AnteHandler) (newCtx Context, err error) { + // verify sigs. Return error if invalid + + // call next antehandler if sigs ok + return next(ctx, tx, simulate) +} + +// User-defined Decorator. Can choose to pre- and post-process on AnteHandler +type UserDefinedDecorator struct{ + // custom fields +} + +func (udd UserDefinedDecorator) AnteHandle(ctx Context, tx Tx, simulate bool, next AnteHandler) (newCtx Context, err error) { + // pre-processing logic + + ctx, err = next(ctx, tx, simulate) + + // post-processing logic +} +``` + +Link AnteDecorators to create a final AnteHandler. Set this AnteHandler in baseapp. + +```go +// Create final antehandler by chaining the decorators together +antehandler := ChainAnteDecorators(NewSetUpContextDecorator(), NewSigVerifyDecorator(), NewUserDefinedDecorator()) + +// Set chained Antehandler in the baseapp +bapp.SetAnteHandler(antehandler) +``` + +Pros: + +1. Allows one decorator to pre- and post-process the next AnteHandler, similar to the Weave design. +2. Do not need to break baseapp API. Users can still set a single AnteHandler if they choose. + +Cons: + +1. Decorator pattern may have a deeply nested structure that is hard to understand, this is mitigated by having the decorator order explicitly listed in the `ChainAnteDecorators` function. +2. Does not make use of the ModuleManager design. Since this is already being used for BeginBlocker/EndBlocker, this proposal seems unaligned with that design pattern. + +## Consequences + +Since pros and cons are written for each approach, it is omitted from this section + +## References + +* [#4572](https://github.com/cosmos/cosmos-sdk/issues/4572): Modular AnteHandler Issue +* [#4582](https://github.com/cosmos/cosmos-sdk/pull/4583): Initial Implementation of Per-Module AnteHandler Approach +* [Weave Decorator Code](https://github.com/iov-one/weave/blob/master/handler.go#L35) +* [Weave Design Videos](https://vimeo.com/showcase/6189877) diff --git a/versioned_docs/version-0.52/build/architecture/adr-011-generalize-genesis-accounts.md b/versioned_docs/version-0.52/build/architecture/adr-011-generalize-genesis-accounts.md new file mode 100644 index 000000000..92a704ba6 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-011-generalize-genesis-accounts.md @@ -0,0 +1,170 @@ +# ADR 011: Generalize Genesis Accounts + +## Changelog + +* 2019-08-30: initial draft + +## Context + +Currently, the Cosmos SDK allows for custom account types; the `auth` keeper stores any type fulfilling its `Account` interface. However `auth` does not handle exporting or loading accounts to/from a genesis file, this is done by `genaccounts`, which only handles one of 4 concrete account types (`BaseAccount`, `ContinuousVestingAccount`, `DelayedVestingAccount` and `ModuleAccount`). + +Projects desiring to use custom accounts (say custom vesting accounts) need to fork and modify `genaccounts`. + +## Decision + +In summary, we will (un)marshal all accounts (interface types) directly using amino, rather than converting to `genaccounts`’s `GenesisAccount` type. Since doing this removes the majority of `genaccounts`'s code, we will merge `genaccounts` into `auth`. Marshalled accounts will be stored in `auth`'s genesis state. + +Detailed changes: + +### 1) (Un)Marshal accounts directly using amino + +The `auth` module's `GenesisState` gains a new field `Accounts`. Note these aren't of type `exported.Account` for reasons outlined in section 3. + +```go +// GenesisState - all auth state that must be provided at genesis +type GenesisState struct { + Params Params `json:"params" yaml:"params"` + Accounts []GenesisAccount `json:"accounts" yaml:"accounts"` +} +``` + +Now `auth`'s `InitGenesis` and `ExportGenesis` (un)marshal accounts as well as the defined params. + +```go +// InitGenesis - Init store state from genesis data +func InitGenesis(ctx sdk.Context, ak AccountKeeper, data GenesisState) { + ak.SetParams(ctx, data.Params) + // load the accounts + for _, a := range data.Accounts { + acc := ak.NewAccount(ctx, a) // set account number + ak.SetAccount(ctx, acc) + } +} + +// ExportGenesis returns a GenesisState for a given context and keeper +func ExportGenesis(ctx sdk.Context, ak AccountKeeper) GenesisState { + params := ak.GetParams(ctx) + + var genAccounts []exported.GenesisAccount + ak.IterateAccounts(ctx, func(account exported.Account) bool { + genAccount := account.(exported.GenesisAccount) + genAccounts = append(genAccounts, genAccount) + return false + }) + + return NewGenesisState(params, genAccounts) +} +``` + +### 2) Register custom account types on the `auth` codec + +The `auth` codec must have all custom account types registered to marshal them. We will follow the pattern established in `gov` for proposals. + +An example custom account definition: + +```go +import authtypes "github.com/cosmos/cosmos-sdk/x/auth/types" + +// Register the module account type with the auth module codec so it can decode module accounts stored in a genesis file +func init() { + authtypes.RegisterAccountTypeCodec(ModuleAccount{}, "cosmos-sdk/ModuleAccount") +} + +type ModuleAccount struct { + ... +``` + +The `auth` codec definition: + +```go +var ModuleCdc *codec.LegacyAmino + +func init() { + ModuleCdc = codec.NewLegacyAmino() + // register module msg's and Account interface + ... + // leave the codec unsealed +} + +// RegisterAccountTypeCodec registers an external account type defined in another module for the internal ModuleCdc. +func RegisterAccountTypeCodec(o interface{}, name string) { + ModuleCdc.RegisterConcrete(o, name, nil) +} +``` + +### 3) Genesis validation for custom account types + +Modules implement a `ValidateGenesis` method. As `auth` does not know of account implementations, accounts will need to validate themselves. + +We will unmarshal accounts into a `GenesisAccount` interface that includes a `Validate` method. + +```go +type GenesisAccount interface { + exported.Account + Validate() error +} +``` + +Then the `auth` `ValidateGenesis` function becomes: + +```go +// ValidateGenesis performs basic validation of auth genesis data returning an +// error for any failed validation criteria. +func ValidateGenesis(data GenesisState) error { + // Validate params + ... + + // Validate accounts + addrMap := make(map[string]bool, len(data.Accounts)) + for _, acc := range data.Accounts { + + // check for duplicated accounts + addrStr := acc.GetAddress().String() + if _, ok := addrMap[addrStr]; ok { + return fmt.Errorf("duplicate account found in genesis state; address: %s", addrStr) + } + addrMap[addrStr] = true + + // check account specific validation + if err := acc.Validate(); err != nil { + return fmt.Errorf("invalid account found in genesis state; address: %s, error: %s", addrStr, err.Error()) + } + + } + return nil +} +``` + +### 4) Move add-genesis-account cli to `auth` + +The `genaccounts` module contains a cli command to add base or vesting accounts to a genesis file. + +This will be moved to `auth`. We will leave it to projects to write their own commands to add custom accounts. An extensible cli handler, similar to `gov`, could be created but it is not worth the complexity for this minor use case. + +### 5) Update module and vesting accounts + +Under the new scheme, module and vesting account types need some minor updates: + +* Type registration on `auth`'s codec (shown above) +* A `Validate` method for each `Account` concrete type + +## Status + +Proposed + +## Consequences + +### Positive + +* custom accounts can be used without needing to fork `genaccounts` +* reduction in lines of code + +### Negative + +### Neutral + +* `genaccounts` module no longer exists +* accounts in genesis files are stored under `accounts` in `auth` rather than in the `genaccounts` module. +-`add-genesis-account` cli command now in `auth` + +## References diff --git a/versioned_docs/version-0.52/build/architecture/adr-012-state-accessors.md b/versioned_docs/version-0.52/build/architecture/adr-012-state-accessors.md new file mode 100644 index 000000000..93600000f --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-012-state-accessors.md @@ -0,0 +1,155 @@ +# ADR 012: State Accessors + +## Changelog + +* 2019 Sep 04: Initial draft + +## Context + +Cosmos SDK modules currently use the `KVStore` interface and `Codec` to access their respective state. While +this provides a large degree of freedom to module developers, it is hard to modularize and the UX is +mediocre. + +First, each time a module tries to access the state, it has to marshal the value and set or get the +value and finally unmarshal. Usually this is done by declaring `Keeper.GetXXX` and `Keeper.SetXXX` functions, +which are repetitive and hard to maintain. + +Second, this makes it harder to align with the object capability theorem: the right to access the +state is defined as a `StoreKey`, which gives full access on the entire Merkle tree, so a module cannot +send the access right to a specific key-value pair (or a set of key-value pairs) to another module safely. + +Finally, because the getter/setter functions are defined as methods of a module's `Keeper`, the reviewers +have to consider the whole Merkle tree space when they reviewing a function accessing any part of the state. +There is no static way to know which part of the state that the function is accessing (and which is not). + +## Decision + +We will define a type named `Value`: + +```go +type Value struct { + m Mapping + key []byte +} +``` + +The `Value` works as a reference for a key-value pair in the state, where `Value.m` defines the key-value +space it will access and `Value.key` defines the exact key for the reference. + +We will define a type named `Mapping`: + +```go +type Mapping struct { + storeKey sdk.StoreKey + cdc *codec.LegacyAmino + prefix []byte +} +``` + +The `Mapping` works as a reference for a key-value space in the state, where `Mapping.storeKey` defines +the IAVL (sub-)tree and `Mapping.prefix` defines the optional subspace prefix. + +We will define the following core methods for the `Value` type: + +```go +// Get and unmarshal stored data, noop if not exists, panic if cannot unmarshal +func (Value) Get(ctx Context, ptr interface{}) {} + +// Get and unmarshal stored data, return error if not exists or cannot unmarshal +func (Value) GetSafe(ctx Context, ptr interface{}) {} + +// Get stored data as raw byte slice +func (Value) GetRaw(ctx Context) []byte {} + +// Marshal and set a raw value +func (Value) Set(ctx Context, o interface{}) {} + +// Check if a raw value exists +func (Value) Exists(ctx Context) bool {} + +// Delete a raw value value +func (Value) Delete(ctx Context) {} +``` + +We will define the following core methods for the `Mapping` type: + +```go +// Constructs key-value pair reference corresponding to the key argument in the Mapping space +func (Mapping) Value(key []byte) Value {} + +// Get and unmarshal stored data, noop if not exists, panic if cannot unmarshal +func (Mapping) Get(ctx Context, key []byte, ptr interface{}) {} + +// Get and unmarshal stored data, return error if not exists or cannot unmarshal +func (Mapping) GetSafe(ctx Context, key []byte, ptr interface{}) + +// Get stored data as raw byte slice +func (Mapping) GetRaw(ctx Context, key []byte) []byte {} + +// Marshal and set a raw value +func (Mapping) Set(ctx Context, key []byte, o interface{}) {} + +// Check if a raw value exists +func (Mapping) Has(ctx Context, key []byte) bool {} + +// Delete a raw value value +func (Mapping) Delete(ctx Context, key []byte) {} +``` + +Each method of the `Mapping` type that is passed the arguments `ctx`, `key`, and `args...` will proxy +the call to `Mapping.Value(key)` with arguments `ctx` and `args...`. + +In addition, we will define and provide a common set of types derived from the `Value` type: + +```go +type Boolean struct { Value } +type Enum struct { Value } +type Integer struct { Value; enc IntEncoding } +type String struct { Value } +// ... +``` + +Where the encoding schemes can be different, `o` arguments in core methods are typed, and `ptr` arguments +in core methods are replaced by explicit return types. + +Finally, we will define a family of types derived from the `Mapping` type: + +```go +type Indexer struct { + m Mapping + enc IntEncoding +} +``` + +Where the `key` argument in core method is typed. + +Some of the properties of the accessor types are: + +* State access happens only when a function which takes a `Context` as an argument is invoked +* Accessor type structs give rights to access the state only that the struct is referring, no other +* Marshalling/Unmarshalling happens implicitly within the core methods + +## Status + +Proposed + +## Consequences + +### Positive + +* Serialization will be done automatically +* Shorter code size, less boilerplate, better UX +* References to the state can be transferred safely +* Explicit scope of accessing + +### Negative + +* Serialization format will be hidden +* Different architecture from the current, but the use of accessor types can be opt-in +* Type-specific types (e.g. `Boolean` and `Integer`) have to be defined manually + +### Neutral + +## References + +* [#4554](https://github.com/cosmos/cosmos-sdk/issues/4554) diff --git a/versioned_docs/version-0.52/build/architecture/adr-013-metrics.md b/versioned_docs/version-0.52/build/architecture/adr-013-metrics.md new file mode 100644 index 000000000..b0808d462 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-013-metrics.md @@ -0,0 +1,157 @@ +# ADR 013: Observability + +## Changelog + +* 20-01-2020: Initial Draft + +## Status + +Proposed + +## Context + +Telemetry is paramount into debugging and understanding what the application is doing and how it is +performing. We aim to expose metrics from modules and other core parts of the Cosmos SDK. + +In addition, we should aim to support multiple configurable sinks that an operator may choose from. +By default, when telemetry is enabled, the application should track and expose metrics that are +stored in-memory. The operator may choose to enable additional sinks, where we support only +[Prometheus](https://prometheus.io/) for now, as it's battle-tested, simple to setup, open source, +and is rich with ecosystem tooling. + +We must also aim to integrate metrics into the Cosmos SDK in the most seamless way possible such that +metrics may be added or removed at will and without much friction. To do this, we will use the +[go-metrics](https://github.com/hashicorp/go-metrics) library. + +Finally, operators may enable telemetry along with specific configuration options. If enabled, metrics +will be exposed via `/metrics?format={text|prometheus}` via the API server. + +## Decision + +We will add an additional configuration block to `app.toml` that defines telemetry settings: + +```toml +############################################################################### +### Telemetry Configuration ### +############################################################################### + +[telemetry] + +# Prefixed with keys to separate services +service-name = {{ .Telemetry.ServiceName }} + +# Enabled enables the application telemetry functionality. When enabled, +# an in-memory sink is also enabled by default. Operators may also enabled +# other sinks such as Prometheus. +enabled = {{ .Telemetry.Enabled }} + +# Enable prefixing gauge values with hostname +enable-hostname = {{ .Telemetry.EnableHostname }} + +# Enable adding hostname to labels +enable-hostname-label = {{ .Telemetry.EnableHostnameLabel }} + +# Enable adding service to labels +enable-service-label = {{ .Telemetry.EnableServiceLabel }} + +# PrometheusRetentionTime, when positive, enables a Prometheus metrics sink. +prometheus-retention-time = {{ .Telemetry.PrometheusRetentionTime }} +``` + +The given configuration allows for two sinks -- in-memory and Prometheus. We create a `Metrics` +type that performs all the bootstrapping for the operator, so capturing metrics becomes seamless. + +```go +// Metrics defines a wrapper around application telemetry functionality. It allows +// metrics to be gathered at any point in time. When creating a Metrics object, +// internally, a global metrics is registered with a set of sinks as configured +// by the operator. In addition to the sinks, when a process gets a SIGUSR1, a +// dump of formatted recent metrics will be sent to STDERR. +type Metrics struct { + memSink *metrics.InmemSink + prometheusEnabled bool +} + +// Gather collects all registered metrics and returns a GatherResponse where the +// metrics are encoded depending on the type. Metrics are either encoded via +// Prometheus or JSON if in-memory. +func (m *Metrics) Gather(format string) (GatherResponse, error) { + switch format { + case FormatPrometheus: + return m.gatherPrometheus() + + case FormatText: + return m.gatherGeneric() + + case FormatDefault: + return m.gatherGeneric() + + default: + return GatherResponse{}, fmt.Errorf("unsupported metrics format: %s", format) + } +} +``` + +In addition, `Metrics` allows us to gather the current set of metrics at any given point in time. An +operator may also choose to send a signal, SIGUSR1, to dump and print formatted metrics to STDERR. + +During an application's bootstrapping and construction phase, if `Telemetry.Enabled` is `true`, the +API server will create an instance of a reference to `Metrics` object and will register a metrics +handler accordingly. + +```go +func (s *Server) Start(cfg config.Config) error { + // ... + + if cfg.Telemetry.Enabled { + m, err := telemetry.New(cfg.Telemetry) + if err != nil { + return err + } + + s.metrics = m + s.registerMetrics() + } + + // ... +} + +func (s *Server) registerMetrics() { + metricsHandler := func(w http.ResponseWriter, r *http.Request) { + format := strings.TrimSpace(r.FormValue("format")) + + gr, err := s.metrics.Gather(format) + if err != nil { + rest.WriteErrorResponse(w, http.StatusBadRequest, fmt.Sprintf("failed to gather metrics: %s", err)) + return + } + + w.Header().Set("Content-Type", gr.ContentType) + _, _ = w.Write(gr.Metrics) + } + + s.Router.HandleFunc("/metrics", metricsHandler).Methods("GET") +} +``` + +Application developers may track counters, gauges, summaries, and key/value metrics. There is no +additional lifting required by modules to leverage profiling metrics. To do so, it's as simple as: + +```go +func (k BaseKeeper) MintCoins(ctx sdk.Context, moduleName string, amt sdk.Coins) error { + defer metrics.MeasureSince(time.Now(), "MintCoins") + // ... +} +``` + +## Consequences + +### Positive + +* Exposure into the performance and behavior of an application + +### Negative + +### Neutral + +## References diff --git a/versioned_docs/version-0.52/build/architecture/adr-014-proportional-slashing.md b/versioned_docs/version-0.52/build/architecture/adr-014-proportional-slashing.md new file mode 100644 index 000000000..d9aeffb80 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-014-proportional-slashing.md @@ -0,0 +1,85 @@ +# ADR 14: Proportional Slashing + +## Changelog + +* 2019-10-15: Initial draft +* 2020-05-25: Removed correlation root slashing +* 2020-07-01: Updated to include S-curve function instead of linear + +## Context + +In Proof of Stake-based chains, centralization of consensus power amongst a small set of validators can cause harm to the network due to increased risk of censorship, liveness failure, fork attacks, etc. However, while this centralization causes a negative externality to the network, it is not directly felt by the delegators contributing towards delegating towards already large validators. We would like a way to pass on the negative externality cost of centralization onto those large validators and their delegators. + +## Decision + +### Design + +To solve this problem, we will implement a procedure called Proportional Slashing. The desire is that the larger a validator is, the more they should be slashed. The first naive attempt is to make a validator's slash percent proportional to their share of consensus voting power. + +```text +slash_amount = k * power // power is the faulting validator's voting power and k is some on-chain constant +``` + +However, this will incentivize validators with large amounts of stake to split up their voting power amongst accounts (sybil attack), so that if they fault, they all get slashed at a lower percent. The solution to this is to take into account not just a validator's own voting percentage, but also the voting percentage of all the other validators who get slashed in a specified time frame. + +```text +slash_amount = k * (power_1 + power_2 + ... + power_n) // where power_i is the voting power of the ith validator faulting in the specified time frame and k is some on-chain constant +``` + +Now, if someone splits a validator of 10% into two validators of 5% each which both fault, then they both fault in the same time frame, they both will get slashed at the sum 10% amount. + +However in practice, we likely don't want a linear relation between amount of stake at fault, and the percentage of stake to slash. In particular, solely 5% of stake double signing effectively did nothing to majorly threaten security, whereas 30% of stake being at fault clearly merits a large slashing factor, due to being very close to the point at which Tendermint security is threatened. A linear relation would require a factor of 6 gap between these two, whereas the difference in risk posed to the network is much larger. We propose using S-curves (formally [logistic functions](https://en.wikipedia.org/wiki/Logistic_function) to solve this). S-Curves capture the desired criterion quite well. They allow the slashing factor to be minimal for small values, and then grow very rapidly near some threshold point where the risk posed becomes notable. + +#### Parameterization + +This requires parameterizing a logistic function. It is very well understood how to parameterize this. It has four parameters: + +1) A minimum slashing factor +2) A maximum slashing factor +3) The inflection point of the S-curve (essentially where do you want to center the S) +4) The rate of growth of the S-curve (How elongated is the S) + +#### Correlation across non-sybil validators + +One will note, that this model doesn't differentiate between multiple validators run by the same operators vs validators run by different operators. This can be seen as an additional benefit in fact. It incentivizes validators to differentiate their setups from other validators, to avoid having correlated faults with them or else they risk a higher slash. So for example, operators should avoid using the same popular cloud hosting platforms or using the same Staking as a Service providers. This will lead to a more resilient and decentralized network. + +#### Griefing + +Griefing, the act of intentionally getting oneself slashed in order to make another's slash worse, could be a concern here. However, using the protocol described here, the attacker also gets equally impacted by the grief as the victim, so it would not provide much benefit to the griefer. + +### Implementation + +In the slashing module, we will add two queues that will track all of the recent slash events. For double sign faults, we will define "recent slashes" as ones that have occurred within the last `unbonding period`. For liveness faults, we will define "recent slashes" as ones that have occurred within the last `jail period`. + +```go +type SlashEvent struct { + Address sdk.ValAddress + ValidatorVotingPercent sdk.Dec + SlashedSoFar sdk.Dec +} +``` + +These slash events will be pruned from the queue once they are older than their respective "recent slash period". + +Whenever a new slash occurs, a `SlashEvent` struct is created with the faulting validator's voting percent and a `SlashedSoFar` of 0. Because recent slash events are pruned before the unbonding period and unjail period expires, it should not be possible for the same validator to have multiple SlashEvents in the same Queue at the same time. + +We then will iterate over all the SlashEvents in the queue, adding their `ValidatorVotingPercent` to calculate the new percent to slash all the validators in the queue at, using the "Square of Sum of Roots" formula introduced above. + +Once we have the `NewSlashPercent`, we then iterate over all the `SlashEvent`s in the queue once again, and if `NewSlashPercent > SlashedSoFar` for that SlashEvent, we call the `staking.Slash(slashEvent.Address, slashEvent.Power, Math.Min(Math.Max(minSlashPercent, NewSlashPercent - SlashedSoFar), maxSlashPercent))` (we pass in the power of the validator before any slashes occurred, so that we slash the right amount of tokens). We then set `SlashEvent.SlashedSoFar` amount to `NewSlashPercent`. + +## Status + +Proposed + +## Consequences + +### Positive + +* Increases decentralization by disincentivizing delegating to large validators +* Incentivizes Decorrelation of Validators +* More severely punishes attacks than accidental faults +* More flexibility in slashing rates parameterization + +### Negative + +* More computationally expensive than current implementation. Will require more data about "recent slashing events" to be stored on chain. diff --git a/versioned_docs/version-0.52/build/architecture/adr-016-validator-consensus-key-rotation.md b/versioned_docs/version-0.52/build/architecture/adr-016-validator-consensus-key-rotation.md new file mode 100644 index 000000000..7085f033c --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-016-validator-consensus-key-rotation.md @@ -0,0 +1,125 @@ +# ADR 016: Validator Consensus Key Rotation + +## Changelog + +* 2019 Oct 23: Initial draft +* 2019 Nov 28: Add key rotation fee + +## Context + +Validator consensus key rotation feature has been discussed and requested for a long time, for the sake of safer validator key management policy (e.g. https://github.com/tendermint/tendermint/issues/1136). So, we suggest one of the simplest form of validator consensus key rotation implementation mostly onto Cosmos SDK. + +We don't need to make any update on consensus logic in Tendermint because Tendermint does not have any mapping information of consensus key and validator operator key, meaning that from Tendermint point of view, a consensus key rotation of a validator is simply a replacement of a consensus key to another. + +Also, it should be noted that this ADR includes only the simplest form of consensus key rotation without considering multiple consensus keys concept. Such multiple consensus keys concept shall remain a long term goal of Tendermint and Cosmos SDK. + +## Decision + +### Pseudo procedure for consensus key rotation + +* create new random consensus key. +* create and broadcast a transaction with a `MsgRotateConsPubKey` that states the new consensus key is now coupled with the validator operator with signature from the validator's operator key. +* old consensus key becomes unable to participate on consensus immediately after the update of key mapping state on-chain. +* start validating with new consensus key. +* validators using HSM and KMS should update the consensus key in HSM to use the new rotated key after the height `h` when `MsgRotateConsPubKey` committed to the blockchain. + +### Considerations + +* consensus key mapping information management strategy + * store history of each key mapping changes in the kvstore. + * the state machine can search corresponding consensus key paired with given validator operator for any arbitrary height in a recent unbonding period. + * the state machine does not need any historical mapping information which is past more than unbonding period. +* key rotation costs related to LCD and IBC + * LCD and IBC will have traffic/computation burden when there exists frequent power changes + * In current Tendermint design, consensus key rotations are seen as power changes from LCD or IBC perspective + * Therefore, to minimize unnecessary frequent key rotation behavior, we limited maximum number of rotation in recent unbonding period and also applied exponentially increasing rotation fee +* limits + * rotations are limited to 1 time in an unbonding window. In future rewrites of the staking module it could be made to happen more times than 1 + * parameters can be decided by governance and stored in genesis file. +* key rotation fee + * a validator should pay `KeyRotationFee` to rotate the consensus key which is calculated as below + * `KeyRotationFee` = (max(`VotingPowerPercentage`, 1)* `InitialKeyRotationFee`) * 2^(number of rotations in `ConsPubKeyRotationHistory` in recent unbonding period) +* evidence module + * evidence module can search corresponding consensus key for any height from slashing keeper so that it can decide which consensus key is supposed to be used for given height. +* abci.ValidatorUpdate + * tendermint already has ability to change a consensus key by ABCI communication(`ValidatorUpdate`). + * validator consensus key update can be done via creating new + delete old by change the power to zero. + * therefore, we expect we even do not need to change tendermint codebase at all to implement this feature. +* new genesis parameters in `staking` module + * `MaxConsPubKeyRotations` : maximum number of rotation can be executed by a validator in recent unbonding period. default value 10 is suggested(11th key rotation will be rejected) + * `InitialKeyRotationFee` : the initial key rotation fee when no key rotation has happened in recent unbonding period. default value 1atom is suggested(1atom fee for the first key rotation in recent unbonding period) + +### Workflow + +1. The validator generates a new consensus keypair. +2. The validator generates and signs a `MsgRotateConsPubKey` tx with their operator key and new ConsPubKey + + ```go + type MsgRotateConsPubKey struct { + ValidatorAddress sdk.ValAddress + NewPubKey crypto.PubKey + } + ``` + +3. `handleMsgRotateConsPubKey` gets `MsgRotateConsPubKey`, calls `RotateConsPubKey` with emits event +4. `RotateConsPubKey` + * checks if `NewPubKey` is not duplicated on `ValidatorsByConsAddr` + * checks if the validator is does not exceed parameter `MaxConsPubKeyRotations` by iterating `ConsPubKeyRotationHistory` + * checks if the signing account has enough balance to pay `KeyRotationFee` + * pays `KeyRotationFee` to community fund + * overwrites `NewPubKey` in `validator.ConsPubKey` + * deletes old `ValidatorByConsAddr` + * `SetValidatorByConsAddr` for `NewPubKey` + * Add `ConsPubKeyRotationHistory` for tracking rotation + + ```go + type ConsPubKeyRotationHistory struct { + OperatorAddress sdk.ValAddress + OldConsPubKey crypto.PubKey + NewConsPubKey crypto.PubKey + RotatedHeight int64 + } + ``` + +5. `ApplyAndReturnValidatorSetUpdates` checks if there is `ConsPubKeyRotationHistory` with `ConsPubKeyRotationHistory.RotatedHeight == ctx.BlockHeight()` and if so, generates 2 `ValidatorUpdate` , one for a remove validator and one for create new validator + + ```go + abci.ValidatorUpdate{ + PubKey: cmttypes.TM2PB.PubKey(OldConsPubKey), + Power: 0, + } + + abci.ValidatorUpdate{ + PubKey: cmttypes.TM2PB.PubKey(NewConsPubKey), + Power: v.ConsensusPower(), + } + ``` + +6. at `previousVotes` Iteration logic of `AllocateTokens`, `previousVote` using `OldConsPubKey` match up with `ConsPubKeyRotationHistory`, and replace validator for token allocation +7. Migrate `ValidatorSigningInfo` and `ValidatorMissedBlockBitArray` from `OldConsPubKey` to `NewConsPubKey` + +* Note : All above features shall be implemented in `staking` module. + +## Status + +Proposed + +## Consequences + +### Positive + +* Validators can immediately or periodically rotate their consensus key to have better security policy +* improved security against Long-Range attacks (https://near.org/blog/long-range-attacks-and-a-new-fork-choice-rule) given a validator throws away the old consensus key(s) + +### Negative + +* Slash module needs more computation because it needs to lookup corresponding consensus key of validators for each height +* frequent key rotations will make light client bisection less efficient + +### Neutral + +## References + +* on tendermint repo : https://github.com/tendermint/tendermint/issues/1136 +* on cosmos-sdk repo : https://github.com/cosmos/cosmos-sdk/issues/5231 +* about multiple consensus keys : https://github.com/tendermint/tendermint/issues/1758#issuecomment-545291698 diff --git a/versioned_docs/version-0.52/build/architecture/adr-017-historical-header-module.md b/versioned_docs/version-0.52/build/architecture/adr-017-historical-header-module.md new file mode 100644 index 000000000..54cd99845 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-017-historical-header-module.md @@ -0,0 +1,63 @@ +# ADR 17: Historical Header Module + +## Changelog + +* 26 November 2019: Start of first version +* 2 December 2019: Final draft of first version +* 7 September 2023: Reduce HistoricalInfo type + +## Context + +In order for the Cosmos SDK to implement the [IBC specification](https://github.com/cosmos/ics), modules within the Cosmos SDK must have the ability to introspect recent consensus states (validator sets & commitment roots) as proofs of these values on other chains must be checked during the handshakes. + +## Decision + +The application MUST store the most recent `n` headers in a persistent store. At first, this store MAY be the current Merklised store. A non-Merklised store MAY be used later as no proofs are necessary. + +The application MUST store this information by storing new headers immediately when handling `abci.RequestBeginBlock`: + +```go +func BeginBlock(ctx sdk.Context, keeper HistoricalHeaderKeeper) error { + info := HistoricalInfo{ + apphash: ctx.HeaderInfo().AppHash, + Time: ctx.HeaderInfo().Time, + NextValidatorsHash: ctx.CometInfo().NextValidatorsHash, + } + keeper.SetHistoricalInfo(ctx, ctx.BlockHeight(), info) + n := keeper.GetParamRecentHeadersToStore() + keeper.PruneHistoricalInfo(ctx, ctx.BlockHeight() - n) + // continue handling request +} +``` + +Alternatively, the application MAY store only the hash of the validator set. + +The application MUST make these past `n` committed headers available for querying by Cosmos SDK modules through the `Keeper`'s `GetHistoricalInfo` function. This MAY be implemented in a new module, or it MAY also be integrated into an existing one (likely `x/staking` or `x/ibc`). + +`n` MAY be configured as a parameter store parameter, in which case it could be changed by `ParameterChangeProposal`s, although it will take some blocks for the stored information to catch up if `n` is increased. + +## Status + +Proposed. + +## Consequences + +Implementation of this ADR will require changes to the Cosmos SDK. It will not require changes to Tendermint. + +### Positive + +* Easy retrieval of headers & state roots for recent past heights by modules anywhere in the Cosmos SDK. +* No RPC calls to Tendermint required. +* No ABCI alterations required. + +### Negative + +* Duplicates `n` headers data in Tendermint & the application (additional disk usage) - in the long term, an approach such as [this](https://github.com/tendermint/tendermint/issues/4210) might be preferable. + +### Neutral + +(none known) + +## References + +* [ICS 2: "Consensus state introspection"](https://github.com/cosmos/ibc/tree/master/spec/core/ics-002-client-semantics#consensus-state-introspection) diff --git a/versioned_docs/version-0.52/build/architecture/adr-018-extendable-voting-period.md b/versioned_docs/version-0.52/build/architecture/adr-018-extendable-voting-period.md new file mode 100644 index 000000000..bf7733908 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-018-extendable-voting-period.md @@ -0,0 +1,66 @@ +# ADR 18: Extendable Voting Periods + +## Changelog + +* 1 January 2020: Start of first version + +## Context + +Currently the voting period for all governance proposals is the same. However, this is suboptimal as all governance proposals do not require the same time period. For more non-contentious proposals, they can be dealt with more efficiently with a faster period, while more contentious or complex proposals may need a longer period for extended discussion/consideration. + +## Decision + +We would like to design a mechanism for making the voting period of a governance proposal variable based on the demand of voters. We would like it to be based on the view of the governance participants, rather than just the proposer of a governance proposal (thus, allowing the proposer to select the voting period length is not sufficient). + +However, we would like to avoid the creation of an entire second voting process to determine the length of the voting period, as it just pushed the problem to determining the length of that first voting period. + +Thus, we propose the following mechanism: + +### Params + +* The current gov param `VotingPeriod` is to be replaced by a `MinVotingPeriod` param. This is the default voting period that all governance proposal voting periods start with. +* There is a new gov param called `MaxVotingPeriodExtension`. + +### Mechanism + +There is a new `Msg` type called `MsgExtendVotingPeriod`, which can be sent by any staked account during a proposal's voting period. It allows the sender to unilaterally extend the length of the voting period by `MaxVotingPeriodExtension * sender's share of voting power`. Every address can only call `MsgExtendVotingPeriod` once per proposal. + +So for example, if the `MaxVotingPeriodExtension` is set to 100 Days, then anyone with 1% of voting power can extend the voting power by 1 day. If 33% of voting power has sent the message, the voting period will be extended by 33 days. Thus, if absolutely everyone chooses to extend the voting period, the absolute maximum voting period will be `MinVotingPeriod + MaxVotingPeriodExtension`. + +This system acts as a sort of distributed coordination, where individual stakers choosing to extend or not, allows the system the gauge the conentiousness/complexity of the proposal. It is extremely unlikely that many stakers will choose to extend at the exact same time, it allows stakers to view how long others have already extended thus far, to decide whether or not to extend further. + +### Dealing with Unbonding/Redelegation + +There is one thing that needs to be addressed. How to deal with redelegation/unbonding during the voting period. If a staker of 5% calls `MsgExtendVotingPeriod` and then unbonds, does the voting period then decrease by 5 days again? This is not good as it can give people a false sense of how long they have to make their decision. For this reason, we want to design it such that the voting period length can only be extended, not shortened. To do this, the current extension amount is based on the highest percent that voted extension at any time. This is best explained by example: + +1. Let's say 2 stakers of voting power 4% and 3% respectively vote to extend. The voting period will be extended by 7 days. +2. Now the staker of 3% decides to unbond before the end of the voting period. The voting period extension remains 7 days. +3. Now, let's say another staker of 2% voting power decides to extend voting period. There is now 6% of active voting power choosing the extend. The voting power remains 7 days. +4. If a fourth staker of 10% chooses to extend now, there is a total of 16% of active voting power wishing to extend. The voting period will be extended to 16 days. + +### Delegators + +Just like votes in the actual voting period, delegators automatically inherit the extension of their validators. If their validator chooses to extend, their voting power will be used in the validator's extension. However, the delegator is unable to override their validator and "unextend" as that would contradict the "voting power length can only be ratcheted up" principle described in the previous section. However, a delegator may choose the extend using their personal voting power, if their validator has not done so. + +## Status + +Proposed + +## Consequences + +### Positive + +* More complex/contentious governance proposals will have more time to properly digest and deliberate + +### Negative + +* Governance process becomes more complex and requires more understanding to interact with effectively +* Can no longer predict when a governance proposal will end. Can't assume order in which governance proposals will end. + +### Neutral + +* The minimum voting period can be made shorter + +## References + +* [Cosmos Forum post where idea first originated](https://forum.cosmos.network/t/proposal-draft-reduce-governance-voting-period-to-7-days/3032/9) diff --git a/versioned_docs/version-0.52/build/architecture/adr-019-protobuf-state-encoding.md b/versioned_docs/version-0.52/build/architecture/adr-019-protobuf-state-encoding.md new file mode 100644 index 000000000..267896d96 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-019-protobuf-state-encoding.md @@ -0,0 +1,379 @@ +# ADR 019: Protocol Buffer State Encoding + +## Changelog + +* 2020 Feb 15: Initial Draft +* 2020 Feb 24: Updates to handle messages with interface fields +* 2020 Apr 27: Convert usages of `oneof` for interfaces to `Any` +* 2020 May 15: Describe `cosmos_proto` extensions and amino compatibility +* 2020 Dec 4: Move and rename `MarshalAny` and `UnmarshalAny` into the `codec.Codec` interface. +* 2021 Feb 24: Remove mentions of `HybridCodec`, which has been abandoned in [#6843](https://github.com/cosmos/cosmos-sdk/pull/6843). + +## Status + +Accepted + +## Context + +Currently, the Cosmos SDK utilizes [go-amino](https://github.com/tendermint/go-amino/) for binary +and JSON object encoding over the wire bringing parity between logical objects and persistence objects. + +From the Amino docs: + +> Amino is an object encoding specification. It is a subset of Proto3 with an extension for interface +> support. See the [Proto3 spec](https://protobuf.dev/programming-guides/proto3/) for more +> information on Proto3, which Amino is largely compatible with (but not with Proto2). +> +> The goal of the Amino encoding protocol is to bring parity into logic objects and persistence objects. + +Amino also aims to have the following goals (not a complete list): + +* Binary bytes must be decode-able with a schema. +* Schema must be upgradeable. +* The encoder and decoder logic must be reasonably simple. + +However, we believe that Amino does not fulfill these goals completely and does not fully meet the +needs of a truly flexible cross-language and multi-client compatible encoding protocol in the Cosmos SDK. +Namely, Amino has proven to be a big pain-point in regards to supporting object serialization across +clients written in various languages while providing virtually little in the way of true backwards +compatibility and upgradeability. Furthermore, through profiling and various benchmarks, Amino has +been shown to be an extremely large performance bottleneck in the Cosmos SDK 1. This is +largely reflected in the performance of simulations and application transaction throughput. + +Thus, we need to adopt an encoding protocol that meets the following criteria for state serialization: + +* Language agnostic +* Platform agnostic +* Rich client support and thriving ecosystem +* High performance +* Minimal encoded message size +* Codegen-based over reflection-based +* Supports backward and forward compatibility + +Note, migrating away from Amino should be viewed as a two-pronged approach, state and client encoding. +This ADR focuses on state serialization in the Cosmos SDK state machine. A corresponding ADR will be +made to address client-side encoding. + +## Decision + +We will adopt [Protocol Buffers](https://protobuf.dev) for serializing +persisted structured data in the Cosmos SDK while providing a clean mechanism and developer UX for +applications wishing to continue to use Amino. We will provide this mechanism by updating modules to +accept a codec interface, `Marshaler`, instead of a concrete Amino codec. Furthermore, the Cosmos SDK +will provide two concrete implementations of the `Marshaler` interface: `AminoCodec` and `ProtoCodec`. + +* `AminoCodec`: Uses Amino for both binary and JSON encoding. +* `ProtoCodec`: Uses Protobuf for both binary and JSON encoding. + +Modules will use whichever codec that is instantiated in the app. By default, the Cosmos SDK's `simapp` +instantiates a `ProtoCodec` as the concrete implementation of `Marshaler`, inside the `MakeTestEncodingConfig` +function. This can be easily overwritten by app developers if they so desire. + +The ultimate goal will be to replace Amino JSON encoding with Protobuf encoding and thus have +modules accept and/or extend `ProtoCodec`. Until then, Amino JSON is still provided for legacy use-cases. +A handful of places in the Cosmos SDK still have Amino JSON hardcoded, such as the Legacy API REST endpoints +and the `x/params` store. They are planned to be converted to Protobuf in a gradual manner. + +### Module Codecs + +Modules that do not require the ability to work with and serialize interfaces, the path to Protobuf +migration is pretty straightforward. These modules are to simply migrate any existing types that +are encoded and persisted via their concrete Amino codec to Protobuf and have their keeper accept a +`Marshaler` that will be a `ProtoCodec`. This migration is simple as things will just work as-is. + +Note, any business logic that needs to encode primitive types like `bool` or `int64` should use +[gogoprotobuf](https://github.com/cosmos/gogoproto) Value types. + +Example: + +```go + ts, err := gogotypes.TimestampProto(completionTime) + if err != nil { + // ... + } + + bz := cdc.MustMarshal(ts) +``` + +However, modules can vary greatly in purpose and design and so we must support the ability for modules +to be able to encode and work with interfaces (e.g. `Account` or `Content`). For these modules, they +must define their own codec interface that extends `Marshaler`. These specific interfaces are unique +to the module and will contain method contracts that know how to serialize the needed interfaces. + +Example: + +```go +// x/auth/types/codec.go + +type Codec interface { + codec.Codec + + MarshalAccount(acc exported.Account) ([]byte, error) + UnmarshalAccount(bz []byte) (exported.Account, error) + + MarshalAccountJSON(acc exported.Account) ([]byte, error) + UnmarshalAccountJSON(bz []byte) (exported.Account, error) +} +``` + +### Usage of `Any` to encode interfaces + +In general, module-level .proto files should define messages which encode interfaces +using [`google.protobuf.Any`](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto). +After [extension discussion](https://github.com/cosmos/cosmos-sdk/issues/6030), +this was chosen as the preferred alternative to application-level `oneof`s +as in our original protobuf design. The arguments in favor of `Any` can be +summarized as follows: + +* `Any` provides a simpler, more consistent client UX for dealing with +interfaces than app-level `oneof`s that will need to be coordinated more +carefully across applications. Creating a generic transaction +signing library using `oneof`s may be cumbersome and critical logic may need +to be reimplemented for each chain +* `Any` provides more resistance against human error than `oneof` +* `Any` is generally simpler to implement for both modules and apps + +The main counter-argument to using `Any` centers around its additional space +and possibly performance overhead. The space overhead could be dealt with using +compression at the persistence layer in the future and the performance impact +is likely to be small. Thus, not using `Any` is seem as a pre-mature optimization, +with user experience as the higher order concern. + +Note, that given the Cosmos SDK's decision to adopt the `Codec` interfaces described +above, apps can still choose to use `oneof` to encode state and transactions +but it is not the recommended approach. If apps do choose to use `oneof`s +instead of `Any` they will likely lose compatibility with client apps that +support multiple chains. Thus developers should think carefully about whether +they care more about what is possibly a pre-mature optimization or end-user +and client developer UX. + +### Safe usage of `Any` + +By default, the [gogo protobuf implementation of `Any`](https://pkg.go.dev/github.com/cosmos/gogoproto/types) +uses [global type registration]( https://github.com/cosmos/gogoproto/blob/master/proto/properties.go#L540) +to decode values packed in `Any` into concrete +go types. This introduces a vulnerability where any malicious module +in the dependency tree could register a type with the global protobuf registry +and cause it to be loaded and unmarshaled by a transaction that referenced +it in the `type_url` field. + +To prevent this, we introduce a type registration mechanism for decoding `Any` +values into concrete types through the `InterfaceRegistry` interface which +bears some similarity to type registration with Amino: + +```go +type InterfaceRegistry interface { + // RegisterInterface associates protoName as the public name for the + // interface passed in as iface + // Ex: + // registry.RegisterInterface("cosmos_sdk.Msg", (*sdk.Msg)(nil)) + RegisterInterface(protoName string, iface interface{}) + + // RegisterImplementations registers impls as a concrete implementations of + // the interface iface + // Ex: + // registry.RegisterImplementations((*sdk.Msg)(nil), &MsgSend{}, &MsgMultiSend{}) + RegisterImplementations(iface interface{}, impls ...proto.Message) + +} +``` + +In addition to serving as a whitelist, `InterfaceRegistry` can also serve +to communicate the list of concrete types that satisfy an interface to clients. + +In .proto files: + +* fields which accept interfaces should be annotated with `cosmos_proto.accepts_interface` +using the same full-qualified name passed as `protoName` to `InterfaceRegistry.RegisterInterface` +* interface implementations should be annotated with `cosmos_proto.implements_interface` +using the same full-qualified name passed as `protoName` to `InterfaceRegistry.RegisterInterface` + +In the future, `protoName`, `cosmos_proto.accepts_interface`, `cosmos_proto.implements_interface` +may be used via code generation, reflection &/or static linting. + +The same struct that implements `InterfaceRegistry` will also implement an +interface `InterfaceUnpacker` to be used for unpacking `Any`s: + +```go +type InterfaceUnpacker interface { + // UnpackAny unpacks the value in any to the interface pointer passed in as + // iface. Note that the type in any must have been registered with + // RegisterImplementations as a concrete type for that interface + // Ex: + // var msg sdk.Msg + // err := ctx.UnpackAny(any, &msg) + // ... + UnpackAny(any *Any, iface interface{}) error +} +``` + +Note that `InterfaceRegistry` usage does not deviate from standard protobuf +usage of `Any`, it just introduces a security and introspection layer for +golang usage. + +`InterfaceRegistry` will be a member of `ProtoCodec` +described above. In order for modules to register interface types, app modules +can optionally implement the following interface: + +```go +type InterfaceModule interface { + RegisterInterfaceTypes(InterfaceRegistry) +} +``` + +The module manager will include a method to call `RegisterInterfaceTypes` on +every module that implements it in order to populate the `InterfaceRegistry`. + +### Using `Any` to encode state + +The Cosmos SDK will provide support methods `MarshalInterface` and `UnmarshalInterface` to hide a complexity of wrapping interface types into `Any` and allow easy serialization. + +```go +import "github.com/cosmos/cosmos-sdk/codec" + +// note: eviexported.Evidence is an interface type +func MarshalEvidence(cdc codec.BinaryCodec, e eviexported.Evidence) ([]byte, error) { + return cdc.MarshalInterface(e) +} + +func UnmarshalEvidence(cdc codec.BinaryCodec, bz []byte) (eviexported.Evidence, error) { + var evi eviexported.Evidence + err := cdc.UnmarshalInterface(&evi, bz) + return err, nil +} +``` + +### Using `Any` in `sdk.Msg`s + +A similar concept is to be applied for messages that contain interfaces fields. +For example, we can define `MsgSubmitEvidence` as follows where `Evidence` is +an interface: + +```protobuf +// x/evidence/types/types.proto + +message MsgSubmitEvidence { + bytes submitter = 1 + [ + (gogoproto.casttype) = "github.com/cosmos/cosmos-sdk/types.AccAddress" + ]; + google.protobuf.Any evidence = 2; +} +``` + +Note that in order to unpack the evidence from `Any` we do need a reference to +`InterfaceRegistry`. In order to reference evidence in methods like +`ValidateBasic` which shouldn't have to know about the `InterfaceRegistry`, we +introduce an `UnpackInterfaces` phase to deserialization which unpacks +interfaces before they're needed. + +### Unpacking Interfaces + +To implement the `UnpackInterfaces` phase of deserialization which unpacks +interfaces wrapped in `Any` before they're needed, we create an interface +that `sdk.Msg`s and other types can implement: + +```go +type UnpackInterfacesMessage interface { + UnpackInterfaces(InterfaceUnpacker) error +} +``` + +We also introduce a private `cachedValue interface{}` field onto the `Any` +struct itself with a public getter `GetCachedValue() interface{}`. + +The `UnpackInterfaces` method is to be invoked during message deserialization right +after `Unmarshal` and any interface values packed in `Any`s will be decoded +and stored in `cachedValue` for reference later. + +Then unpacked interface values can safely be used in any code afterwards +without knowledge of the `InterfaceRegistry` +and messages can introduce a simple getter to cast the cached value to the +correct interface type. + +This has the added benefit that unmarshaling of `Any` values only happens once +during initial deserialization rather than every time the value is read. Also, +when `Any` values are first packed (for instance in a call to +`NewMsgSubmitEvidence`), the original interface value is cached so that +unmarshaling isn't needed to read it again. + +`MsgSubmitEvidence` could implement `UnpackInterfaces`, plus a convenience getter +`GetEvidence` as follows: + +```go +func (msg MsgSubmitEvidence) UnpackInterfaces(ctx sdk.InterfaceRegistry) error { + var evi eviexported.Evidence + return ctx.UnpackAny(msg.Evidence, *evi) +} + +func (msg MsgSubmitEvidence) GetEvidence() eviexported.Evidence { + return msg.Evidence.GetCachedValue().(eviexported.Evidence) +} +``` + +### Amino Compatibility + +Our custom implementation of `Any` can be used transparently with Amino if used +with the proper codec instance. What this means is that interfaces packed within +`Any`s will be amino marshaled like regular Amino interfaces (assuming they +have been registered properly with Amino). + +In order for this functionality to work: + +* **all legacy code must use `*codec.LegacyAmino` instead of `*amino.Codec` which is + now a wrapper which properly handles `Any`** +* **all new code should use `Marshaler` which is compatible with both amino and + protobuf** +* Also, before v0.39, `codec.LegacyAmino` will be renamed to `codec.LegacyAmino`. + +### Why Wasn't X Chosen Instead + +For a more complete comparison to alternative protocols, see [here](https://codeburst.io/json-vs-protocol-buffers-vs-flatbuffers-a4247f8bda6f). + +### Cap'n Proto + +While [Cap’n Proto](https://capnproto.org/) does seem like an advantageous alternative to Protobuf +due to it's native support for interfaces/generics and built in canonicalization, it does lack the +rich client ecosystem compared to Protobuf and is a bit less mature. + +### FlatBuffers + +[FlatBuffers](https://google.github.io/flatbuffers/) is also a potentially viable alternative, with the +primary difference being that FlatBuffers does not need a parsing/unpacking step to a secondary +representation before you can access data, often coupled with per-object memory allocation. + +However, it would require great efforts into research and full understanding the scope of the migration +and path forward -- which isn't immediately clear. In addition, FlatBuffers aren't designed for +untrusted inputs. + +## Future Improvements & Roadmap + +In the future we may consider a compression layer right above the persistence +layer which doesn't change tx or merkle tree hashes, but reduces the storage +overhead of `Any`. In addition, we may adopt protobuf naming conventions which +make type URLs a bit more concise while remaining descriptive. + +Additional code generation support around the usage of `Any` is something that +could also be explored in the future to make the UX for go developers more +seamless. + +## Consequences + +### Positive + +* Significant performance gains. +* Supports backward and forward type compatibility. +* Better support for cross-language clients. + +### Negative + +* Learning curve required to understand and implement Protobuf messages. +* Slightly larger message size due to use of `Any`, although this could be offset + by a compression layer in the future + +### Neutral + +## References + +1. https://github.com/cosmos/cosmos-sdk/issues/4977 +2. https://github.com/cosmos/cosmos-sdk/issues/5444 diff --git a/versioned_docs/version-0.52/build/architecture/adr-020-protobuf-transaction-encoding.md b/versioned_docs/version-0.52/build/architecture/adr-020-protobuf-transaction-encoding.md new file mode 100644 index 000000000..db9a52b6f --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-020-protobuf-transaction-encoding.md @@ -0,0 +1,467 @@ +# ADR 020: Protocol Buffer Transaction Encoding + +## Changelog + +* 2020 March 06: Initial Draft +* 2020 March 12: API Updates +* 2020 April 13: Added details on interface `oneof` handling +* 2020 April 30: Switch to `Any` +* 2020 May 14: Describe public key encoding +* 2020 June 08: Store `TxBody` and `AuthInfo` as bytes in `SignDoc`; Document `TxRaw` as broadcast and storage type. +* 2020 August 07: Use ADR 027 for serializing `SignDoc`. +* 2020 August 19: Move sequence field from `SignDoc` to `SignerInfo`, as discussed in [#6966](https://github.com/cosmos/cosmos-sdk/issues/6966). +* 2020 September 25: Remove `PublicKey` type in favor of `secp256k1.PubKey`, `ed25519.PubKey` and `multisig.LegacyAminoPubKey`. +* 2020 October 15: Add `GetAccount` and `GetAccountWithHeight` methods to the `AccountRetriever` interface. +* 2021 Feb 24: The Cosmos SDK does not use Tendermint's `PubKey` interface anymore, but its own `cryptotypes.PubKey`. Updates to reflect this. +* 2021 May 3: Rename `clientCtx.JSONMarshaler` to `clientCtx.JSONCodec`. +* 2021 June 10: Add `clientCtx.Codec: codec.Codec`. +* 2024 February 5: Account creation step + +## Status + +Accepted + +## Context + +This ADR is a continuation of the motivation, design, and context established in +[ADR 019](./adr-019-protobuf-state-encoding.md), namely, we aim to design the +Protocol Buffer migration path for the client-side of the Cosmos SDK. + +Specifically, the client-side migration path primarily includes tx generation and +signing, message construction and routing, in addition to CLI & REST handlers and +business logic (i.e. queriers). + +With this in mind, we will tackle the migration path via two main areas, txs and +querying. However, this ADR solely focuses on transactions. Querying should be +addressed in a future ADR, but it should build off of these proposals. + +Based on detailed discussions ([\#6030](https://github.com/cosmos/cosmos-sdk/issues/6030) +and [\#6078](https://github.com/cosmos/cosmos-sdk/issues/6078)), the original +design for transactions was changed substantially from an `oneof` /JSON-signing +approach to the approach described below. + +## Decision + +### Transactions + +Since interface values are encoded with `google.protobuf.Any` in state (see [ADR 019](adr-019-protobuf-state-encoding.md)), +`sdk.Msg`s are encoding with `Any` in transactions. + +One of the main goals of using `Any` to encode interface values is to have a +core set of types which is reused by apps so that +clients can safely be compatible with as many chains as possible. + +It is one of the goals of this specification to provide a flexible cross-chain transaction +format that can serve a wide variety of use cases without breaking client +compatibility. + +In order to facilitate signing, transactions are separated into `TxBody`, +which will be re-used by `SignDoc` below, and `signatures`: + +```protobuf +// types/types.proto +package cosmos_sdk.v1; + +message Tx { + TxBody body = 1; + AuthInfo auth_info = 2; + // A list of signatures that matches the length and order of AuthInfo's signer_infos to + // allow connecting signature meta information like public key and signing mode by position. + repeated bytes signatures = 3; +} + +// A variant of Tx that pins the signer's exact binary representation of body and +// auth_info. This is used for signing, broadcasting and verification. The binary +// `serialize(tx: TxRaw)` is stored in Tendermint and the hash `sha256(serialize(tx: TxRaw))` +// becomes the "txhash", commonly used as the transaction ID. +message TxRaw { + // A protobuf serialization of a TxBody that matches the representation in SignDoc. + bytes body = 1; + // A protobuf serialization of an AuthInfo that matches the representation in SignDoc. + bytes auth_info = 2; + // A list of signatures that matches the length and order of AuthInfo's signer_infos to + // allow connecting signature meta information like public key and signing mode by position. + repeated bytes signatures = 3; +} + +message TxBody { + // A list of messages to be executed. The required signers of those messages define + // the number and order of elements in AuthInfo's signer_infos and Tx's signatures. + // Each required signer address is added to the list only the first time it occurs. + // + // By convention, the first required signer (usually from the first message) is referred + // to as the primary signer and pays the fee for the whole transaction. + repeated google.protobuf.Any messages = 1; + string memo = 2; + int64 timeout_height = 3; + repeated google.protobuf.Any extension_options = 1023; +} + +message AuthInfo { + // This list defines the signing modes for the required signers. The number + // and order of elements must match the required signers from TxBody's messages. + // The first element is the primary signer and the one which pays the fee. + repeated SignerInfo signer_infos = 1; + // The fee can be calculated based on the cost of evaluating the body and doing signature verification of the signers. This can be estimated via simulation. + Fee fee = 2; +} + +message SignerInfo { + // The public key is optional for accounts that already exist in state. If unset, the + // verifier can use the required signer address for this position and lookup the public key. + google.protobuf.Any public_key = 1; + // ModeInfo describes the signing mode of the signer and is a nested + // structure to support nested multisig pubkey's + ModeInfo mode_info = 2; + // sequence is the sequence of the account, which describes the + // number of committed transactions signed by a given address. It is used to prevent + // replay attacks. + uint64 sequence = 3; +} + +message ModeInfo { + oneof sum { + Single single = 1; + Multi multi = 2; + } + + // Single is the mode info for a single signer. It is structured as a message + // to allow for additional fields such as locale for SIGN_MODE_TEXTUAL in the future + message Single { + SignMode mode = 1; + } + + // Multi is the mode info for a multisig public key + message Multi { + // bitarray specifies which keys within the multisig are signing + CompactBitArray bitarray = 1; + // mode_infos is the corresponding modes of the signers of the multisig + // which could include nested multisig public keys + repeated ModeInfo mode_infos = 2; + } +} + +enum SignMode { + SIGN_MODE_UNSPECIFIED = 0; + + SIGN_MODE_DIRECT = 1; + + SIGN_MODE_TEXTUAL = 2; + + SIGN_MODE_LEGACY_AMINO_JSON = 127; +} +``` + +As will be discussed below, in order to include as much of the `Tx` as possible +in the `SignDoc`, `SignerInfo` is separated from signatures so that only the +raw signatures themselves live outside of what is signed over. + +Because we are aiming for a flexible, extensible cross-chain transaction +format, new transaction processing options should be added to `TxBody` as soon +those use cases are discovered, even if they can't be implemented yet. + +Because there is coordination overhead in this, `TxBody` includes an +`extension_options` field which can be used for any transaction processing +options that are not already covered. App developers should, nevertheless, +attempt to upstream important improvements to `Tx`. + +### Signing + +All of the signing modes below aim to provide the following guarantees: + +* **No Malleability**: `TxBody` and `AuthInfo` cannot change once the transaction + is signed +* **Predictable Gas**: if I am signing a transaction where I am paying a fee, + the final gas is fully dependent on what I am signing + +These guarantees give the maximum amount confidence to message signers that +manipulation of `Tx`s by intermediaries can't result in any meaningful changes. + +#### `SIGN_MODE_DIRECT` + +The "direct" signing behavior is to sign the raw `TxBody` bytes as broadcast over +the wire. This has the advantages of: + +* requiring the minimum additional client capabilities beyond a standard protocol + buffers implementation +* leaving effectively zero holes for transaction malleability (i.e. there are no + subtle differences between the signing and encoding formats which could + potentially be exploited by an attacker) + +Signatures are structured using the `SignDoc` below which reuses the serialization of +`TxBody` and `AuthInfo` and only adds the fields which are needed for signatures: + +```protobuf +// types/types.proto +message SignDoc { + // A protobuf serialization of a TxBody that matches the representation in TxRaw. + bytes body = 1; + // A protobuf serialization of an AuthInfo that matches the representation in TxRaw. + bytes auth_info = 2; + string chain_id = 3; + uint64 account_number = 4; +} +``` + +In order to sign in the default mode, clients take the following steps: + +1. Serialize `TxBody` and `AuthInfo` using any valid protobuf implementation. +2. Create a `SignDoc` and serialize it using [ADR 027](./adr-027-deterministic-protobuf-serialization.md). +3. Sign the encoded `SignDoc` bytes. +4. Build a `TxRaw` and serialize it for broadcasting. + +Signature verification is based on comparing the raw `TxBody` and `AuthInfo` +bytes encoded in `TxRaw` not based on any ["canonicalization"](https://github.com/regen-network/canonical-proto3) +algorithm which creates added complexity for clients in addition to preventing +some forms of upgradeability (to be addressed later in this document). + +Signature verifiers do: + +1. Deserialize a `TxRaw` and pull out `body` and `auth_info`. +2. Create a list of required signer addresses from the messages. +3. For each required signer: + * Pull account number and sequence from the state. + * Obtain the public key either from state or `AuthInfo`'s `signer_infos`. + * Create a `SignDoc` and serialize it using [ADR 027](./adr-027-deterministic-protobuf-serialization.md). + * Verify the signature at the same list position against the serialized `SignDoc`. + +#### `SIGN_MODE_LEGACY_AMINO` + +In order to support legacy wallets and exchanges, Amino JSON will be temporarily +supported transaction signing. Once wallets and exchanges have had a +chance to upgrade to protobuf based signing, this option will be disabled. In +the meantime, it is foreseen that disabling the current Amino signing would cause +too much breakage to be feasible. Note that this is mainly a requirement of the +Cosmos Hub and other chains may choose to disable Amino signing immediately. + +Legacy clients will be able to sign a transaction using the current Amino +JSON format and have it encoded to protobuf using the REST `/tx/encode` +endpoint before broadcasting. + +#### `SIGN_MODE_TEXTUAL` + +As was discussed extensively in [\#6078](https://github.com/cosmos/cosmos-sdk/issues/6078), +there is a desire for a human-readable signing encoding, especially for hardware +wallets like the [Ledger](https://www.ledger.com) which display +transaction contents to users before signing. JSON was an attempt at this but +falls short of the ideal. + +`SIGN_MODE_TEXTUAL` is intended as a placeholder for a human-readable +encoding which will replace Amino JSON. This new encoding should be even more +focused on readability than JSON, possibly based on formatting strings like +[MessageFormat](http://userguide.icu-project.org/formatparse/messages). + +In order to ensure that the new human-readable format does not suffer from +transaction malleability issues, `SIGN_MODE_TEXTUAL` +requires that the _human-readable bytes are concatenated with the raw `SignDoc`_ +to generate sign bytes. + +Multiple human-readable formats (maybe even localized messages) may be supported +by `SIGN_MODE_TEXTUAL` when it is implemented. + +### Unknown Field Filtering + +Unknown fields in protobuf messages should generally be rejected by transaction +processors because: + +* important data may be present in the unknown fields, that if ignored, will + cause unexpected behavior for clients +* they present a malleability vulnerability where attackers can bloat tx size + by adding random uninterpreted data to unsigned content (i.e. the master `Tx`, + not `TxBody`) + +There are also scenarios where we may choose to safely ignore unknown fields +(https://github.com/cosmos/cosmos-sdk/issues/6078#issuecomment-624400188) to +provide graceful forwards compatibility with newer clients. + +We propose that field numbers with bit 11 set (for most use cases this is +the range of 1024-2047) be considered non-critical fields that can safely be +ignored if unknown. + +To handle this we will need an unknown field filter that: + +* always rejects unknown fields in unsigned content (i.e. top-level `Tx` and + unsigned parts of `AuthInfo` if present based on the signing mode) +* rejects unknown fields in all messages (including nested `Any`s) other than + fields with bit 11 set + +This will likely need to be a custom protobuf parser pass that takes message bytes +and `FileDescriptor`s and returns a boolean result. + +### Public Key Encoding + +Public keys in the Cosmos SDK implement the `cryptotypes.PubKey` interface. +We propose to use `Any` for protobuf encoding as we are doing with other interfaces (for example, in `BaseAccount.PubKey` and `SignerInfo.PublicKey`). +The following public keys are implemented: secp256k1, secp256r1, ed25519 and legacy-multisignature. + +Ex: + +```protobuf +message PubKey { + bytes key = 1; +} +``` + +`multisig.LegacyAminoPubKey` has an array of `Any`'s member to support any +protobuf public key type. + +Apps should only attempt to handle a registered set of public keys that they +have tested. The provided signature verification ante handler decorators will +enforce this. + +### CLI & REST + +Currently, the REST and CLI handlers encode and decode types and txs via Amino +JSON encoding using a concrete Amino codec. Being that some of the types dealt with +in the client can be interfaces, similar to how we described in [ADR 019](./adr-019-protobuf-state-encoding.md), +the client logic will now need to take a codec interface that knows not only how +to handle all the types, but also knows how to generate transactions, signatures, +and messages. + +If the account is sending its first transaction, the account number must be set to 0. This is due to the account not being created yet. + +```go +type AccountRetriever interface { + GetAccount(clientCtx Context, addr sdk.AccAddress) (client.Account, error) + GetAccountWithHeight(clientCtx Context, addr sdk.AccAddress) (client.Account, int64, error) + EnsureExists(clientCtx client.Context, addr sdk.AccAddress) error + GetAccountNumberSequence(clientCtx client.Context, addr sdk.AccAddress) (uint64, uint64, error) +} + +type Generator interface { + NewTx() TxBuilder + NewFee() ClientFee + NewSignature() ClientSignature + MarshalTx(tx types.Tx) ([]byte, error) +} + +type TxBuilder interface { + GetTx() sdk.Tx + + SetMsgs(...sdk.Msg) error + GetSignatures() []sdk.Signature + SetSignatures(...sdk.Signature) + GetFee() sdk.Fee + SetFee(sdk.Fee) + GetMemo() string + SetMemo(string) +} +``` + +We then update `Context` to have new fields: `Codec`, `TxGenerator`, +and `AccountRetriever`, and we update `AppModule.GetTxCmd` to take +a `Context` which should have all of these fields pre-populated. + +Each client method should then use one of the `Init` methods to re-initialize +the pre-populated `Context`. `tx.GenerateOrBroadcastTx` can be used to +generate or broadcast a transaction. For example: + +```go +import "github.com/spf13/cobra" +import "github.com/cosmos/cosmos-sdk/client" +import "github.com/cosmos/cosmos-sdk/client/tx" + +func NewCmdDoSomething(clientCtx client.Context) *cobra.Command { + return &cobra.Command{ + RunE: func(cmd *cobra.Command, args []string) error { + clientCtx := ctx.InitWithInput(cmd.InOrStdin()) + msg := NewSomeMsg{...} + tx.GenerateOrBroadcastTx(clientCtx, msg) + }, + } +} +``` + +## Future Improvements + +### `SIGN_MODE_TEXTUAL` specification + +A concrete specification and implementation of `SIGN_MODE_TEXTUAL` is intended +as a near-term future improvement so that the ledger app and other wallets +can gracefully transition away from Amino JSON. + +### `SIGN_MODE_DIRECT_AUX` + +(\*Documented as option (3) in https://github.com/cosmos/cosmos-sdk/issues/6078#issuecomment-628026933) + +We could add a mode `SIGN_MODE_DIRECT_AUX` +to support scenarios where multiple signatures +are being gathered into a single transaction but the message composer does not +yet know which signatures will be included in the final transaction. For instance, +I may have a 3/5 multisig wallet and want to send a `TxBody` to all 5 +signers to see who signs first. As soon as I have 3 signatures then I will go +ahead and build the full transaction. + +With `SIGN_MODE_DIRECT`, each signer needs +to sign the full `AuthInfo` which includes the full list of all signers and +their signing modes, making the above scenario very hard. + +`SIGN_MODE_DIRECT_AUX` would allow "auxiliary" signers to create their signature +using only `TxBody` and their own `PublicKey`. This allows the full list of +signers in `AuthInfo` to be delayed until signatures have been collected. + +An "auxiliary" signer is any signer besides the primary signer who is paying +the fee. For the primary signer, the full `AuthInfo` is actually needed to calculate gas and fees +because that is dependent on how many signers and which key types and signing +modes they are using. Auxiliary signers, however, do not need to worry about +fees or gas and thus can just sign `TxBody`. + +To generate a signature in `SIGN_MODE_DIRECT_AUX` these steps would be followed: + +1. Encode `SignDocAux` (with the same requirement that fields must be serialized + in order): + + ```protobuf + // types/types.proto + message SignDocAux { + bytes body_bytes = 1; + // PublicKey is included in SignDocAux : + // 1. as a special case for multisig public keys. For multisig public keys, + // the signer should use the top-level multisig public key they are signing + // against, not their own public key. This is to prevent against a form + // of malleability where a signature could be taken out of context of the + // multisig key that was intended to be signed for + // 2. to guard against scenario where configuration information is encoded + // in public keys (it has been proposed) such that two keys can generate + // the same signature but have different security properties + // + // By including it here, the composer of AuthInfo cannot reference the + // a public key variant the signer did not intend to use + PublicKey public_key = 2; + string chain_id = 3; + uint64 account_number = 4; + } + ``` + +2. Sign the encoded `SignDocAux` bytes +3. Send their signature and `SignerInfo` to primary signer who will then + sign and broadcast the final transaction (with `SIGN_MODE_DIRECT` and `AuthInfo` + added) once enough signatures have been collected + +### `SIGN_MODE_DIRECT_RELAXED` + +(_Documented as option (1)(a) in https://github.com/cosmos/cosmos-sdk/issues/6078#issuecomment-628026933_) + +This is a variation of `SIGN_MODE_DIRECT` where multiple signers wouldn't need to +coordinate public keys and signing modes in advance. It would involve an alternate +`SignDoc` similar to `SignDocAux` above with fee. This could be added in the future +if client developers found the burden of collecting public keys and modes in advance +too burdensome. + +## Consequences + +### Positive + +* Significant performance gains. +* Supports backward and forward type compatibility. +* Better support for cross-language clients. +* Multiple signing modes allow for greater protocol evolution + +### Negative + +* `google.protobuf.Any` type URLs increase transaction size although the effect + may be negligible or compression may be able to mitigate it. + +### Neutral + +## References diff --git a/versioned_docs/version-0.52/build/architecture/adr-021-protobuf-query-encoding.md b/versioned_docs/version-0.52/build/architecture/adr-021-protobuf-query-encoding.md new file mode 100644 index 000000000..a87c6c7ea --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-021-protobuf-query-encoding.md @@ -0,0 +1,256 @@ +# ADR 021: Protocol Buffer Query Encoding + +## Changelog + +* 2020 March 27: Initial Draft + +## Status + +Accepted + +## Context + +This ADR is a continuation of the motivation, design, and context established in +[ADR 019](./adr-019-protobuf-state-encoding.md) and +[ADR 020](./adr-020-protobuf-transaction-encoding.md), namely, we aim to design the +Protocol Buffer migration path for the client-side of the Cosmos SDK. + +This ADR continues from [ADD 020](./adr-020-protobuf-transaction-encoding.md) +to specify the encoding of queries. + +## Decision + +### Custom Query Definition + +Modules define custom queries through a protocol buffers `service` definition. +These `service` definitions are generally associated with and used by the +GRPC protocol. However, the protocol buffers specification indicates that +they can be used more generically by any request/response protocol that uses +protocol buffer encoding. Thus, we can use `service` definitions for specifying +custom ABCI queries and even reuse a substantial amount of the GRPC infrastructure. + +Each module with custom queries should define a service canonically named `Query`: + +```protobuf +// x/bank/types/types.proto + +service Query { + rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) { } + rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) { } +} +``` + +#### Handling of Interface Types + +Modules that use interface types and need true polymorphism generally force a +`oneof` up to the app-level that provides the set of concrete implementations of +that interface that the app supports. While app's are welcome to do the same for +queries and implement an app-level query service, it is recommended that modules +provide query methods that expose these interfaces via `google.protobuf.Any`. +There is a concern on the transaction level that the overhead of `Any` is too +high to justify its usage. However for queries this is not a concern, and +providing generic module-level queries that use `Any` does not preclude apps +from also providing app-level queries that return use the app-level `oneof`s. + +A hypothetical example for the `gov` module would look something like: + +```protobuf +// x/gov/types/types.proto + +import "google/protobuf/any.proto"; + +service Query { + rpc GetProposal(GetProposalParams) returns (AnyProposal) { } +} + +message AnyProposal { + ProposalBase base = 1; + google.protobuf.Any content = 2; +} +``` + +### Custom Query Implementation + +In order to implement the query service, we can reuse the existing [gogo protobuf](https://github.com/cosmos/gogoproto) +grpc plugin, which for a service named `Query` generates an interface named +`QueryServer` as below: + +```go +type QueryServer interface { + QueryBalance(context.Context, *QueryBalanceParams) (*types.Coin, error) + QueryAllBalances(context.Context, *QueryAllBalancesParams) (*QueryAllBalancesResponse, error) +} +``` + +The custom queries for our module are implemented by implementing this interface. + +The first parameter in this generated interface is a generic `context.Context`, +whereas querier methods generally need an instance of `sdk.Context` to read +from the store. Since arbitrary values can be attached to `context.Context` +using the `WithValue` and `Value` methods, the Cosmos SDK should provide a function +`sdk.UnwrapSDKContext` to retrieve the `sdk.Context` from the provided +`context.Context`. + +An example implementation of `QueryBalance` for the bank module as above would +look something like: + +```go +type Querier struct { + Keeper +} + +func (q Querier) QueryBalance(ctx context.Context, params *types.QueryBalanceParams) (*sdk.Coin, error) { + balance := q.GetBalance(sdk.UnwrapSDKContext(ctx), params.Address, params.Denom) + return &balance, nil +} +``` + +### Custom Query Registration and Routing + +Query server implementations as above would be registered with `AppModule`s using +a new method `RegisterQueryService(grpc.Server)` which could be implemented simply +as below: + +```go +// x/bank/module.go +func (am AppModule) RegisterQueryService(server grpc.Server) { + types.RegisterQueryServer(server, keeper.Querier{am.keeper}) +} +``` + +Underneath the hood, a new method `RegisterService(sd *grpc.ServiceDesc, handler interface{})` +will be added to the existing `baseapp.QueryRouter` to add the queries to the custom +query routing table (with the routing method being described below). +The signature for this method matches the existing +`RegisterServer` method on the GRPC `Server` type where `handler` is the custom +query server implementation described above. + +GRPC-like requests are routed by the service name (ex. `cosmos_sdk.x.bank.v1.Query`) +and method name (ex. `QueryBalance`) combined with `/`s to form a full +method name (ex. `/cosmos_sdk.x.bank.v1.Query/QueryBalance`). This gets translated +into an ABCI query as `custom/cosmos_sdk.x.bank.v1.Query/QueryBalance`. Service handlers +registered with `QueryRouter.RegisterService` will be routed this way. + +Beyond the method name, GRPC requests carry a protobuf encoded payload, which maps naturally +to `RequestQuery.Data`, and receive a protobuf encoded response or error. Thus +there is a quite natural mapping of GRPC-like rpc methods to the existing +`sdk.Query` and `QueryRouter` infrastructure. + +This basic specification allows us to reuse protocol buffer `service` definitions +for ABCI custom queries substantially reducing the need for manual decoding and +encoding in query methods. + +### GRPC Protocol Support + +In addition to providing an ABCI query pathway, we can easily provide a GRPC +proxy server that routes requests in the GRPC protocol to ABCI query requests +under the hood. In this way, clients could use their host languages' existing +GRPC implementations to make direct queries against Cosmos SDK app's using +these `service` definitions. In order for this server to work, the `QueryRouter` +on `BaseApp` will need to expose the service handlers registered with +`QueryRouter.RegisterService` to the proxy server implementation. Nodes could +launch the proxy server on a separate port in the same process as the ABCI app +with a command-line flag. + +### REST Queries and Swagger Generation + +[grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) is a project that +translates REST calls into GRPC calls using special annotations on service +methods. Modules that want to expose REST queries should add `google.api.http` +annotations to their `rpc` methods as in this example below. + +```protobuf +// x/bank/types/types.proto + +service Query { + rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) { + option (google.api.http) = { + get: "/x/bank/v1/balance/{address}/{denom}" + }; + } + rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) { + option (google.api.http) = { + get: "/x/bank/v1/balances/{address}" + }; + } +} +``` + +grpc-gateway will work directly against the GRPC proxy described above which will +translate requests to ABCI queries under the hood. grpc-gateway can also +generate Swagger definitions automatically. + +In the current implementation of REST queries, each module needs to implement +REST queries manually in addition to ABCI querier methods. Using the grpc-gateway +approach, there will be no need to generate separate REST query handlers, just +query servers as described above as grpc-gateway handles the translation of protobuf +to REST as well as Swagger definitions. + +The Cosmos SDK should provide CLI commands for apps to start GRPC gateway either in +a separate process or the same process as the ABCI app, as well as provide a +command for generating grpc-gateway proxy `.proto` files and the `swagger.json` +file. + +### Client Usage + +The gogo protobuf grpc plugin generates client interfaces in addition to server +interfaces. For the `Query` service defined above we would get a `QueryClient` +interface like: + +```go +type QueryClient interface { + QueryBalance(ctx context.Context, in *QueryBalanceParams, opts ...grpc.CallOption) (*types.Coin, error) + QueryAllBalances(ctx context.Context, in *QueryAllBalancesParams, opts ...grpc.CallOption) (*QueryAllBalancesResponse, error) +} +``` + +Via a small patch to gogo protobuf ([gogo/protobuf#675](https://github.com/gogo/protobuf/pull/675)) +we have tweaked the grpc codegen to use an interface rather than concrete type +for the generated client struct. This allows us to also reuse the GRPC infrastructure +for ABCI client queries. + +1Context`will receive a new method`QueryConn`that returns a`ClientConn` +that routes calls to ABCI queries + +Clients (such as CLI methods) will then be able to call query methods like this: + +```go +clientCtx := client.NewContext() +queryClient := types.NewQueryClient(clientCtx.QueryConn()) +params := &types.QueryBalanceParams{addr, denom} +result, err := queryClient.QueryBalance(gocontext.Background(), params) +``` + +### Testing + +Tests would be able to create a query client directly from keeper and `sdk.Context` +references using a `QueryServerTestHelper` as below: + +```go +queryHelper := baseapp.NewQueryServerTestHelper(ctx) +types.RegisterQueryServer(queryHelper, keeper.Querier{app.BankKeeper}) +queryClient := types.NewQueryClient(queryHelper) +``` + +## Future Improvements + +## Consequences + +### Positive + +* greatly simplified querier implementation (no manual encoding/decoding) +* easy query client generation (can use existing grpc and swagger tools) +* no need for REST query implementations +* type safe query methods (generated via grpc plugin) +* going forward, there will be less breakage of query methods because of the +backwards compatibility guarantees provided by buf + +### Negative + +* all clients using the existing ABCI/REST queries will need to be refactored +for both the new GRPC/REST query paths as well as protobuf/proto-json encoded +data, but this is more or less unavoidable in the protobuf refactoring + +### Neutral + +## References diff --git a/versioned_docs/version-0.52/build/architecture/adr-022-custom-panic-handling.md b/versioned_docs/version-0.52/build/architecture/adr-022-custom-panic-handling.md new file mode 100644 index 000000000..2cdce59f4 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-022-custom-panic-handling.md @@ -0,0 +1,218 @@ +# ADR 022: Custom BaseApp panic handling + +## Changelog + +* 2020 Apr 24: Initial Draft +* 2021 Sep 14: Superseded by ADR-045 + +## Status + +SUPERSEDED by ADR-045 + +## Context + +The current implementation of BaseApp does not allow developers to write custom error handlers during panic recovery +[runTx()](https://github.com/cosmos/cosmos-sdk/blob/bad4ca75f58b182f600396ca350ad844c18fc80b/baseapp/baseapp.go#L539) +method. We think that this method can be more flexible and can give Cosmos SDK users more options for customizations without +the need to rewrite whole BaseApp. Also there's one special case for `sdk.ErrorOutOfGas` error handling, that case +might be handled in a "standard" way (middleware) alongside the others. + +We propose middleware-solution, which could help developers implement the following cases: + +* add external logging (let's say sending reports to external services like [Sentry](https://sentry.io)); +* call panic for specific error cases; + +It will also make `OutOfGas` case and `default` case one of the middlewares. +`Default` case wraps recovery object to an error and logs it ([example middleware implementation](#recovery-middleware)). + +Our project has a sidecar service running alongside the blockchain node (smart contracts virtual machine). It is +essential that node <-> sidecar connectivity stays stable for TXs processing. So when the communication breaks we need +to crash the node and reboot it once the problem is solved. That behaviour makes node's state machine execution +deterministic. As all keeper panics are caught by runTx's `defer()` handler, we have to adjust the BaseApp code +in order to customize it. + +## Decision + +### Design + +#### Overview + +Instead of hardcoding custom error handling into BaseApp we suggest using set of middlewares which can be customized +externally and will allow developers use as many custom error handlers as they want. Implementation with tests +can be found [here](https://github.com/cosmos/cosmos-sdk/pull/6053). + +#### Implementation details + +##### Recovery handler + +New `RecoveryHandler` type added. `recoveryObj` input argument is an object returned by the standard Go function +`recover()` from the `builtin` package. + +```go +type RecoveryHandler func(recoveryObj interface{}) error +``` + +Handler should type assert (or other methods) an object to define if object should be handled. +`nil` should be returned if input object can't be handled by that `RecoveryHandler` (not a handler's target type). +Not `nil` error should be returned if input object was handled and middleware chain execution should be stopped. + +An example: + +```go +func exampleErrHandler(recoveryObj interface{}) error { + err, ok := recoveryObj.(error) + if !ok { return nil } + + if someSpecificError.Is(err) { + panic(customPanicMsg) + } else { + return nil + } +} +``` + +This example breaks the application execution, but it also might enrich the error's context like the `OutOfGas` handler. + +##### Recovery middleware + +We also add a middleware type (decorator). That function type wraps `RecoveryHandler` and returns the next middleware in +execution chain and handler's `error`. Type is used to separate actual `recovery()` object handling from middleware +chain processing. + +```go +type recoveryMiddleware func(recoveryObj interface{}) (recoveryMiddleware, error) + +func newRecoveryMiddleware(handler RecoveryHandler, next recoveryMiddleware) recoveryMiddleware { + return func(recoveryObj interface{}) (recoveryMiddleware, error) { + if err := handler(recoveryObj); err != nil { + return nil, err + } + return next, nil + } +} +``` + +Function receives a `recoveryObj` object and returns: + +* (next `recoveryMiddleware`, `nil`) if object wasn't handled (not a target type) by `RecoveryHandler`; +* (`nil`, not nil `error`) if input object was handled and other middlewares in the chain should not be executed; +* (`nil`, `nil`) in case of invalid behavior. Panic recovery might not have been properly handled; +this can be avoided by always using a `default` as a rightmost middleware in the chain (always returns an `error`'); + +`OutOfGas` middleware example: + +```go +func newOutOfGasRecoveryMiddleware(gasWanted uint64, ctx sdk.Context, next recoveryMiddleware) recoveryMiddleware { + handler := func(recoveryObj interface{}) error { + err, ok := recoveryObj.(sdk.ErrorOutOfGas) + if !ok { return nil } + + return errorsmod.Wrap( + sdkerrors.ErrOutOfGas, fmt.Sprintf( + "out of gas in location: %v; gasWanted: %d, gasUsed: %d", err.Descriptor, gasWanted, ctx.GasMeter().GasConsumed(), + ), + ) + } + + return newRecoveryMiddleware(handler, next) +} +``` + +`Default` middleware example: + +```go +func newDefaultRecoveryMiddleware() recoveryMiddleware { + handler := func(recoveryObj interface{}) error { + return errorsmod.Wrap( + sdkerrors.ErrPanic, fmt.Sprintf("recovered: %v\nstack:\n%v", recoveryObj, string(debug.Stack())), + ) + } + + return newRecoveryMiddleware(handler, nil) +} +``` + +##### Recovery processing + +Basic chain of middlewares processing would look like: + +```go +func processRecovery(recoveryObj interface{}, middleware recoveryMiddleware) error { + if middleware == nil { return nil } + + next, err := middleware(recoveryObj) + if err != nil { return err } + if next == nil { return nil } + + return processRecovery(recoveryObj, next) +} +``` + +That way we can create a middleware chain which is executed from left to right, the rightmost middleware is a +`default` handler which must return an `error`. + +##### BaseApp changes + +The `default` middleware chain must exist in a `BaseApp` object. `Baseapp` modifications: + +```go +type BaseApp struct { + // ... + runTxRecoveryMiddleware recoveryMiddleware +} + +func NewBaseApp(...) { + // ... + app.runTxRecoveryMiddleware = newDefaultRecoveryMiddleware() +} + +func (app *BaseApp) runTx(...) { + // ... + defer func() { + if r := recover(); r != nil { + recoveryMW := newOutOfGasRecoveryMiddleware(gasWanted, ctx, app.runTxRecoveryMiddleware) + err, result = processRecovery(r, recoveryMW), nil + } + + gInfo = sdk.GasInfo{GasWanted: gasWanted, GasUsed: ctx.GasMeter().GasConsumed()} + }() + // ... +} +``` + +Developers can add their custom `RecoveryHandler`s by providing `AddRunTxRecoveryHandler` as a BaseApp option parameter to the `NewBaseapp` constructor: + +```go +func (app *BaseApp) AddRunTxRecoveryHandler(handlers ...RecoveryHandler) { + for _, h := range handlers { + app.runTxRecoveryMiddleware = newRecoveryMiddleware(h, app.runTxRecoveryMiddleware) + } +} +``` + +This method would prepend handlers to an existing chain. + +## Consequences + +### Positive + +* Developers of Cosmos SDK based projects can add custom panic handlers to: + * add error context for custom panic sources (panic inside of custom keepers); + * emit `panic()`: passthrough recovery object to the Tendermint core; + * other necessary handling; +* Developers can use standard Cosmos SDK `BaseApp` implementation, rather that rewriting it in their projects; +* Proposed solution doesn't break the current "standard" `runTx()` flow; + +### Negative + +* Introduces changes to the execution model design. + +### Neutral + +* `OutOfGas` error handler becomes one of the middlewares; +* Default panic handler becomes one of the middlewares; + +## References + +* [PR-6053 with proposed solution](https://github.com/cosmos/cosmos-sdk/pull/6053) +* [Similar solution. ADR-010 Modular AnteHandler](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-010-modular-antehandler.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-023-protobuf-naming.md b/versioned_docs/version-0.52/build/architecture/adr-023-protobuf-naming.md new file mode 100644 index 000000000..01bfd8bd9 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-023-protobuf-naming.md @@ -0,0 +1,263 @@ +# ADR 023: Protocol Buffer Naming and Versioning Conventions + +## Changelog + +* 2020 April 27: Initial Draft +* 2020 August 5: Update guidelines + +## Status + +Accepted + +## Context + +Protocol Buffers provide a basic [style guide](https://protobuf.dev/programming-guides/style/) +and [Buf](https://buf.build/docs/style-guide) builds upon that. To the +extent possible, we want to follow industry accepted guidelines and wisdom for +the effective usage of protobuf, deviating from those only when there is clear +rationale for our use case. + +### Adoption of `Any` + +The adoption of `google.protobuf.Any` as the recommended approach for encoding +interface types (as opposed to `oneof`) makes package naming a central part +of the encoding as fully-qualified message names now appear in encoded +messages. + +### Current Directory Organization + +Thus far we have mostly followed [Buf's](https://buf.build) [DEFAULT](https://buf.build/docs/lint-checkers#default) +recommendations, with the minor deviation of disabling [`PACKAGE_DIRECTORY_MATCH`](https://buf.build/docs/lint-checkers#file_layout) +which although being convenient for developing code comes with the warning +from Buf that: + +> you will have a very bad time with many Protobuf plugins across various languages if you do not do this + +### Adoption of gRPC Queries + +In [ADR 021](adr-021-protobuf-query-encoding.md), gRPC was adopted for Protobuf +native queries. The full gRPC service path thus becomes a key part of ABCI query +path. In the future, gRPC queries may be allowed from within persistent scripts +by technologies such as CosmWasm and these query routes would be stored within +script binaries. + +## Decision + +The goal of this ADR is to provide thoughtful naming conventions that: + +* encourage a good user experience for when users interact directly with +.proto files and fully-qualified protobuf names +* balance conciseness against the possibility of either over-optimizing (making +names too short and cryptic) or under-optimizing (just accepting bloated names +with lots of redundant information) + +These guidelines are meant to act as a style guide for both the Cosmos SDK and +third-party modules. + +As a starting point, we should adopt all of the [DEFAULT](https://buf.build/docs/lint-checkers#default) +checkers in [Buf's](https://buf.build) including [`PACKAGE_DIRECTORY_MATCH`](https://buf.build/docs/lint-checkers#file_layout), +except: + +* [PACKAGE_VERSION_SUFFIX](https://buf.build/docs/lint-checkers#package_version_suffix) +* [SERVICE_SUFFIX](https://buf.build/docs/lint-checkers#service_suffix) + +Further guidelines to be described below. + +### Principles + +#### Concise and Descriptive Names + +Names should be descriptive enough to convey their meaning and distinguish +them from other names. + +Given that we are using fully-qualifed names within +`google.protobuf.Any` as well as within gRPC query routes, we should aim to +keep names concise, without going overboard. The general rule of thumb should +be if a shorter name would convey more or else the same thing, pick the shorter +name. + +For instance, `cosmos.bank.MsgSend` (19 bytes) conveys roughly the same information +as `cosmos_sdk.x.bank.v1.MsgSend` (28 bytes) but is more concise. + +Such conciseness makes names both more pleasant to work with and take up less +space within transactions and on the wire. + +We should also resist the temptation to over-optimize, by making names +cryptically short with abbreviations. For instance, we shouldn't try to +reduce `cosmos.bank.MsgSend` to `csm.bk.MSnd` just to save a few bytes. + +The goal is to make names **_concise but not cryptic_**. + +#### Names are for Clients First + +Package and type names should be chosen for the benefit of users, not +necessarily because of legacy concerns related to the go code-base. + +#### Plan for Longevity + +In the interests of long-term support, we should plan on the names we do +choose to be in usage for a long time, so now is the opportunity to make +the best choices for the future. + +### Versioning + +#### Guidelines on Stable Package Versions + +In general, schema evolution is the way to update protobuf schemas. That means that new fields, +messages, and RPC methods are _added_ to existing schemas and old fields, messages and RPC methods +are maintained as long as possible. + +Breaking things is often unacceptable in a blockchain scenario. For instance, immutable smart contracts +may depend on certain data schemas on the host chain. If the host chain breaks those schemas, the smart +contract may be irreparably broken. Even when things can be fixed (for instance in client software), +this often comes at a high cost. + +Instead of breaking things, we should make every effort to evolve schemas rather than just breaking them. +[Buf](https://buf.build) breaking change detection should be used on all stable (non-alpha or beta) packages +to prevent such breakage. + +With that in mind, different stable versions (i.e. `v1` or `v2`) of a package should more or less be considered +different packages and this should be last resort approach for upgrading protobuf schemas. Scenarios where creating +a `v2` may make sense are: + +* we want to create a new module with similar functionality to an existing module and adding `v2` is the most natural +way to do this. In that case, there are really just two different, but similar modules with different APIs. +* we want to add a new revamped API for an existing module and it's just too cumbersome to add it to the existing package, +so putting it in `v2` is cleaner for users. In this case, care should be made to not deprecate support for +`v1` if it is actively used in immutable smart contracts. + +#### Guidelines on unstable (alpha and beta) package versions + +The following guidelines are recommended for marking packages as alpha or beta: + +* marking something as `alpha` or `beta` should be a last resort and just putting something in the +stable package (i.e. `v1` or `v2`) should be preferred +* a package _should_ be marked as `alpha` _if and only if_ there are active discussions to remove +or significantly alter the package in the near future +* a package _should_ be marked as `beta` _if and only if_ there is an active discussion to +significantly refactor/rework the functionality in the near future but not remove it +* modules _can and should_ have types in both stable (i.e. `v1` or `v2`) and unstable (`alpha` or `beta`) packages. + +_`alpha` and `beta` should not be used to avoid responsibility for maintaining compatibility._ +Whenever code is released into the wild, especially on a blockchain, there is a high cost to changing things. In some +cases, for instance with immutable smart contracts, a breaking change may be impossible to fix. + +When marking something as `alpha` or `beta`, maintainers should ask the questions: + +* what is the cost of asking others to change their code vs the benefit of us maintaining the optionality to change it? +* what is the plan for moving this to `v1` and how will that affect users? + +`alpha` or `beta` should really be used to communicate "changes are planned". + +As a case study, gRPC reflection is in the package `grpc.reflection.v1alpha`. It hasn't been changed since +2017 and it is now used in other widely used software like gRPCurl. Some folks probably use it in production services +and so if they actually went and changed the package to `grpc.reflection.v1`, some software would break and +they probably don't want to do that... So now the `v1alpha` package is more or less the de-facto `v1`. Let's not do that. + +The following are guidelines for working with non-stable packages: + +* [Buf's recommended version suffix](https://buf.build/docs/lint-checkers#package_version_suffix) +(ex. `v1alpha1`) _should_ be used for non-stable packages +* non-stable packages should generally be excluded from breaking change detection +* immutable smart contract modules (i.e. CosmWasm) _should_ block smart contracts/persistent +scripts from interacting with `alpha`/`beta` packages + +#### Omit v1 suffix + +Instead of using [Buf's recommended version suffix](https://buf.build/docs/lint-checkers#package_version_suffix), +we can omit `v1` for packages that don't actually have a second version. This +allows for more concise names for common use cases like `cosmos.bank.Send`. +Packages that do have a second or third version can indicate that with `.v2` +or `.v3`. + +### Package Naming + +#### Adopt a short, unique top-level package name + +Top-level packages should adopt a short name that is known to not collide with +other names in common usage within the Cosmos ecosystem. In the near future, a +registry should be created to reserve and index top-level package names used +within the Cosmos ecosystem. Because the Cosmos SDK is intended to provide +the top-level types for the Cosmos project, the top-level package name `cosmos` +is recommended for usage within the Cosmos SDK instead of the longer `cosmos_sdk`. +[ICS](https://github.com/cosmos/ics) specifications could consider a +short top-level package like `ics23` based upon the standard number. + +#### Limit sub-package depth + +Sub-package depth should be increased with caution. Generally a single +sub-package is needed for a module or a library. Even though `x` or `modules` +is used in source code to denote modules, this is often unnecessary for .proto +files as modules are the primary thing sub-packages are used for. Only items which +are known to be used infrequently should have deep sub-package depths. + +For the Cosmos SDK, it is recommended that we simply write `cosmos.bank`, +`cosmos.gov`, etc. rather than `cosmos.x.bank`. In practice, most non-module +types can go straight in the `cosmos` package or we can introduce a +`cosmos.base` package if needed. Note that this naming _will not_ change +go package names, i.e. the `cosmos.bank` protobuf package will still live in +`x/bank`. + +### Message Naming + +Message type names should be as concise possible without losing clarity. `sdk.Msg` +types which are used in transactions will retain the `Msg` prefix as that provides +helpful context. + +### Service and RPC Naming + +[ADR 021](adr-021-protobuf-query-encoding.md) specifies that modules should +implement a gRPC query service. We should consider the principle of conciseness +for query service and RPC names as these may be called from persistent script +modules such as CosmWasm. Also, users may use these query paths from tools like +[gRPCurl](https://github.com/fullstorydev/grpcurl). As an example, we can shorten +`/cosmos_sdk.x.bank.v1.QueryService/QueryBalance` to +`/cosmos.bank.Query/Balance` without losing much useful information. + +RPC request and response types _should_ follow the `ServiceNameMethodNameRequest`/ +`ServiceNameMethodNameResponse` naming convention. i.e. for an RPC method named `Balance` +on the `Query` service, the request and response types would be `QueryBalanceRequest` +and `QueryBalanceResponse`. This will be more self-explanatory than `BalanceRequest` +and `BalanceResponse`. + +#### Use just `Query` for the query service + +Instead of [Buf's default service suffix recommendation](https://github.com/cosmos/cosmos-sdk/pull/6033), +we should simply use the shorter `Query` for query services. + +For other types of gRPC services, we should consider sticking with Buf's +default recommendation. + +#### Omit `Get` and `Query` from query service RPC names + +`Get` and `Query` should be omitted from `Query` service names because they are +redundant in the fully-qualified name. For instance, `/cosmos.bank.Query/QueryBalance` +just says `Query` twice without any new information. + +## Future Improvements + +A registry of top-level package names should be created to coordinate naming +across the ecosystem, prevent collisions, and also help developers discover +useful schemas. A simple starting point would be a git repository with +community-based governance. + +## Consequences + +### Positive + +* names will be more concise and easier to read and type +* all transactions using `Any` will be at shorter (`_sdk.x` and `.v1` will be removed) +* `.proto` file imports will be more standard (without `"third_party/proto"` in +the path) +* code generation will be easier for clients because .proto files will be +in a single `proto/` directory which can be copied rather than scattered +throughout the Cosmos SDK + +### Negative + +### Neutral + +* `.proto` files will need to be reorganized and refactored +* some modules may need to be marked as alpha or beta + +## References diff --git a/versioned_docs/version-0.52/build/architecture/adr-024-coin-metadata.md b/versioned_docs/version-0.52/build/architecture/adr-024-coin-metadata.md new file mode 100644 index 000000000..91c7c4d9f --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-024-coin-metadata.md @@ -0,0 +1,140 @@ +# ADR 024: Coin Metadata + +## Changelog + +* 05/19/2020: Initial draft + +## Status + +ACCEPTED + +## Context + +Assets in the Cosmos SDK are represented via a `Coins` type that consists of an `amount` and a `denom`, +where the `amount` can be any arbitrarily large or small value. In addition, the Cosmos SDK uses an +account-based model where there are two types of primary accounts -- basic accounts and module accounts. +All account types have a set of balances that are composed of `Coins`. The `x/bank` module keeps +track of all balances for all accounts and also keeps track of the total supply of balances in an +application. + +With regards to a balance `amount`, the Cosmos SDK assumes a static and fixed unit of denomination, +regardless of the denomination itself. In other words, clients and apps built atop a Cosmos-SDK-based +chain may choose to define and use arbitrary units of denomination to provide a richer UX, however, by +the time a tx or operation reaches the Cosmos SDK state machine, the `amount` is treated as a single +unit. For example, for the Cosmos Hub (Gaia), clients assume 1 ATOM = 10^6 uatom, and so all txs and +operations in the Cosmos SDK work off of units of 10^6. + +This clearly provides a poor and limited UX especially as interoperability of networks increases and +as a result the total amount of asset types increases. We propose to have `x/bank` additionally keep +track of metadata per `denom` in order to help clients, wallet providers, and explorers improve their +UX and remove the requirement for making any assumptions on the unit of denomination. + +## Decision + +The `x/bank` module will be updated to store and index metadata by `denom`, specifically the "base" or +smallest unit -- the unit the Cosmos SDK state-machine works with. + +Metadata may also include a non-zero length list of denominations. Each entry contains the name of +the denomination `denom`, the exponent to the base and a list of aliases. An entry is to be +interpreted as `1 denom = 10^exponent base_denom` (e.g. `1 ETH = 10^18 wei` and `1 uatom = 10^0 uatom`). + +There are two denominations that are of high importance for clients: the `base`, which is the smallest +possible unit and the `display`, which is the unit that is commonly referred to in human communication +and on exchanges. The values in those fields link to an entry in the list of denominations. + +The list in `denom_units` and the `display` entry may be changed via governance. + +As a result, we can define the type as follows: + +```protobuf +message DenomUnit { + string denom = 1; + uint32 exponent = 2; + repeated string aliases = 3; +} + +message Metadata { + string description = 1; + repeated DenomUnit denom_units = 2; + string base = 3; + string display = 4; +} +``` + +As an example, the ATOM's metadata can be defined as follows: + +```json +{ + "name": "atom", + "description": "The native staking token of the Cosmos Hub.", + "denom_units": [ + { + "denom": "uatom", + "exponent": 0, + "aliases": [ + "microatom" + ], + }, + { + "denom": "matom", + "exponent": 3, + "aliases": [ + "milliatom" + ] + }, + { + "denom": "atom", + "exponent": 6, + } + ], + "base": "uatom", + "display": "atom", +} +``` + +Given the above metadata, a client may infer the following things: + +* 4.3atom = 4.3 * (10^6) = 4,300,000uatom +* The string "atom" can be used as a display name in a list of tokens. +* The balance 4300000 can be displayed as 4,300,000uatom or 4,300matom or 4.3atom. + The `display` denomination 4.3atom is a good default if the authors of the client don't make + an explicit decision to choose a different representation. + +A client should be able to query for metadata by denom both via the CLI and REST interfaces. In +addition, we will add handlers to these interfaces to convert from any unit to another given unit, +as the base framework for this already exists in the Cosmos SDK. + +Finally, we need to ensure metadata exists in the `GenesisState` of the `x/bank` module which is also +indexed by the base `denom`. + +```go +type GenesisState struct { + SendEnabled bool `json:"send_enabled" yaml:"send_enabled"` + Balances []Balance `json:"balances" yaml:"balances"` + Supply sdk.Coins `json:"supply" yaml:"supply"` + DenomMetadata []Metadata `json:"denom_metadata" yaml:"denom_metadata"` +} +``` + +## Future Work + +In order for clients to avoid having to convert assets to the base denomination -- either manually or +via an endpoint, we may consider supporting automatic conversion of a given unit input. + +## Consequences + +### Positive + +* Provides clients, wallet providers and block explorers with additional data on + asset denomination to improve UX and remove any need to make assumptions on + denomination units. + +### Negative + +* A small amount of required additional storage in the `x/bank` module. The amount + of additional storage should be minimal as the amount of total assets should not + be large. + +### Neutral + +## References diff --git a/versioned_docs/version-0.52/build/architecture/adr-027-deterministic-protobuf-serialization.md b/versioned_docs/version-0.52/build/architecture/adr-027-deterministic-protobuf-serialization.md new file mode 100644 index 000000000..41e0d28e3 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-027-deterministic-protobuf-serialization.md @@ -0,0 +1,314 @@ +# ADR 027: Deterministic Protobuf Serialization + +## Changelog + +* 2020-08-07: Initial Draft +* 2020-09-01: Further clarify rules + +## Status + +Proposed + +## Abstract + +Fully deterministic structure serialization, which works across many languages and clients, +is needed when signing messages. We need to be sure that whenever we serialize +a data structure, no matter in which supported language, the raw bytes +will stay the same. +[Protobuf](https://protobuf.dev/programming-guides/proto3/) +serialization is not bijective (i.e. there exist a practically unlimited number of +valid binary representations for a given protobuf document)1. + +This document describes a deterministic serialization scheme for +a subset of protobuf documents, that covers this use case but can be reused in +other cases as well. + +### Context + +For signature verification in Cosmos SDK, the signer and verifier need to agree on +the same serialization of a `SignDoc` as defined in +[ADR-020](./adr-020-protobuf-transaction-encoding.md) without transmitting the +serialization. + +Currently, for block signatures we are using a workaround: we create a new [TxRaw](https://github.com/cosmos/cosmos-sdk/blob/9e85e81e0e8140067dd893421290c191529c148c/proto/cosmos/tx/v1beta1/tx.proto#L30) +instance (as defined in [adr-020-protobuf-transaction-encoding](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-020-protobuf-transaction-encoding.md#transactions)) +by converting all [Tx](https://github.com/cosmos/cosmos-sdk/blob/9e85e81e0e8140067dd893421290c191529c148c/proto/cosmos/tx/v1beta1/tx.proto#L13) +fields to bytes on the client side. This adds an additional manual +step when sending and signing transactions. + +### Decision + +The following encoding scheme is to be used by other ADRs, +and in particular for `SignDoc` serialization. + +## Specification + +### Scope + +This ADR defines a protobuf3 serializer. The output is a valid protobuf +serialization, such that every protobuf parser can parse it. + +No maps are supported in version 1 due to the complexity of defining a +deterministic serialization. This might change in future. Implementations must +reject documents containing maps as invalid input. + +### Background - Protobuf3 Encoding + +Most numeric types in protobuf3 are encoded as +[varints](https://protobuf.dev/programming-guides/encoding/#varints). +Varints are at most 10 bytes, and since each varint byte has 7 bits of data, +varints are a representation of `uint70` (70-bit unsigned integer). When +encoding, numeric values are casted from their base type to `uint70`, and when +decoding, the parsed `uint70` is casted to the appropriate numeric type. + +The maximum valid value for a varint that complies with protobuf3 is +`FF FF FF FF FF FF FF FF FF 7F` (i.e. `2**70 -1`). If the field type is +`{,u,s}int64`, the highest 6 bits of the 70 are dropped during decoding, +introducing 6 bits of malleability. If the field type is `{,u,s}int32`, the +highest 38 bits of the 70 are dropped during decoding, introducing 38 bits of +malleability. + +Among other sources of non-determinism, this ADR eliminates the possibility of +encoding malleability. + +### Serialization rules + +The serialization is based on the +[protobuf3 encoding](https://protobuf.dev/programming-guides/encoding/) +with the following additions: + +1. Fields must be serialized only once in ascending order +2. Extra fields or any extra data must not be added +3. [Default values](https://protobuf.dev/programming-guides/proto3/#default) + must be omitted +4. `repeated` fields of scalar numeric types must use + [packed encoding](https://protobuf.dev/programming-guides/encoding/#packed) +5. Varint encoding must not be longer than needed: + * No trailing zero bytes (in little endian, i.e. no leading zeroes in big + endian). Per rule 3 above, the default value of `0` must be omitted, so + this rule does not apply in such cases. + * The maximum value for a varint must be `FF FF FF FF FF FF FF FF FF 01`. + In other words, when decoded, the highest 6 bits of the 70-bit unsigned + integer must be `0`. (10-byte varints are 10 groups of 7 bits, i.e. + 70 bits, of which only the lowest 70-6=64 are useful.) + * The maximum value for 32-bit values in varint encoding must be `FF FF FF FF 0F` + with one exception (below). In other words, when decoded, the highest 38 + bits of the 70-bit unsigned integer must be `0`. + * The one exception to the above is _negative_ `int32`, which must be + encoded using the full 10 bytes for sign extension2. + * The maximum value for Boolean values in varint encoding must be `01` (i.e. + it must be `0` or `1`). Per rule 3 above, the default value of `0` must + be omitted, so if a Boolean is included it must have a value of `1`. + +While rule number 1. and 2. should be pretty straight forward and describe the +default behavior of all protobuf encoders the author is aware of, the 3rd rule +is more interesting. After a protobuf3 deserialization you cannot differentiate +between unset fields and fields set to the default value3. At +serialization level however, it is possible to set the fields with an empty +value or omitting them entirely. This is a significant difference to e.g. JSON +where a property can be empty (`""`, `0`), `null` or undefined, leading to 3 +different documents. + +Omitting fields set to default values is valid because the parser must assign +the default value to fields missing in the serialization4. For scalar +types, omitting defaults is required by the spec5. For `repeated` +fields, not serializing them is the only way to express empty lists. Enums must +have a first element of numeric value 0, which is the default6. And +message fields default to unset7. + +Omitting defaults allows for some amount of forward compatibility: users of +newer versions of a protobuf schema produce the same serialization as users of +older versions as long as newly added fields are not used (i.e. set to their +default value). + +### Implementation + +There are three main implementation strategies, ordered from the least to the +most custom development: + +* **Use a protobuf serializer that follows the above rules by default.** E.g. + [gogoproto](https://pkg.go.dev/github.com/cosmos/gogoproto/gogoproto) is known to + be compliant by in most cases, but not when certain annotations such as + `nullable = false` are used. It might also be an option to configure an + existing serializer accordingly. +* **Normalize default values before encoding them.** If your serializer follows + rule 1. and 2. and allows you to explicitly unset fields for serialization, + you can normalize default values to unset. This can be done when working with + [protobuf.js](https://www.npmjs.com/package/protobufjs): + + ```js + const bytes = SignDoc.encode({ + bodyBytes: body.length > 0 ? body : null, // normalize empty bytes to unset + authInfoBytes: authInfo.length > 0 ? authInfo : null, // normalize empty bytes to unset + chainId: chainId || null, // normalize "" to unset + accountNumber: accountNumber || null, // normalize 0 to unset + accountSequence: accountSequence || null, // normalize 0 to unset + }).finish(); + ``` + +* **Use a hand-written serializer for the types you need.** If none of the above + ways works for you, you can write a serializer yourself. For SignDoc this + would look something like this in Go, building on existing protobuf utilities: + + ```go + if !signDoc.body_bytes.empty() { + buf.WriteUVarInt64(0xA) // wire type and field number for body_bytes + buf.WriteUVarInt64(signDoc.body_bytes.length()) + buf.WriteBytes(signDoc.body_bytes) + } + + if !signDoc.auth_info.empty() { + buf.WriteUVarInt64(0x12) // wire type and field number for auth_info + buf.WriteUVarInt64(signDoc.auth_info.length()) + buf.WriteBytes(signDoc.auth_info) + } + + if !signDoc.chain_id.empty() { + buf.WriteUVarInt64(0x1a) // wire type and field number for chain_id + buf.WriteUVarInt64(signDoc.chain_id.length()) + buf.WriteBytes(signDoc.chain_id) + } + + if signDoc.account_number != 0 { + buf.WriteUVarInt64(0x20) // wire type and field number for account_number + buf.WriteUVarInt(signDoc.account_number) + } + + if signDoc.account_sequence != 0 { + buf.WriteUVarInt64(0x28) // wire type and field number for account_sequence + buf.WriteUVarInt(signDoc.account_sequence) + } + ``` + +### Test vectors + +Given the protobuf definition `Article.proto` + +```protobuf +package blog; +syntax = "proto3"; + +enum Type { + UNSPECIFIED = 0; + IMAGES = 1; + NEWS = 2; +}; + +enum Review { + UNSPECIFIED = 0; + ACCEPTED = 1; + REJECTED = 2; +}; + +message Article { + string title = 1; + string description = 2; + uint64 created = 3; + uint64 updated = 4; + bool public = 5; + bool promoted = 6; + Type type = 7; + Review review = 8; + repeated string comments = 9; + repeated string backlinks = 10; +}; +``` + +serializing the values + +```yaml +title: "The world needs change 🌳" +description: "" +created: 1596806111080 +updated: 0 +public: true +promoted: false +type: Type.NEWS +review: Review.UNSPECIFIED +comments: ["Nice one", "Thank you"] +backlinks: [] +``` + +must result in the serialization + +```text +0a1b54686520776f726c64206e65656473206368616e676520f09f8cb318e8bebec8bc2e280138024a084e696365206f6e654a095468616e6b20796f75 +``` + +When inspecting the serialized document, you see that every second field is +omitted: + +```shell +$ echo 0a1b54686520776f726c64206e65656473206368616e676520f09f8cb318e8bebec8bc2e280138024a084e696365206f6e654a095468616e6b20796f75 | xxd -r -p | protoc --decode_raw +1: "The world needs change \360\237\214\263" +3: 1596806111080 +5: 1 +7: 2 +9: "Nice one" +9: "Thank you" +``` + +## Consequences + +Having such an encoding available allows us to get deterministic serialization +for all protobuf documents we need in the context of Cosmos SDK signing. + +### Positive + +* Well defined rules that can be verified independent of a reference + implementation +* Simple enough to keep the barrier to implement transaction signing low +* It allows us to continue to use 0 and other empty values in SignDoc, avoiding + the need to work around 0 sequences. This does not imply the change from + https://github.com/cosmos/cosmos-sdk/pull/6949 should not be merged, but not + too important anymore. + +### Negative + +* When implementing transaction signing, the encoding rules above must be + understood and implemented. +* The need for rule number 3. adds some complexity to implementations. +* Some data structures may require custom code for serialization. Thus + the code is not very portable - it will require additional work for each + client implementing serialization to properly handle custom data structures. + +### Neutral + +### Usage in Cosmos SDK + +For the reasons mentioned above ("Negative" section) we prefer to keep workarounds +for shared data structure. Example: the aforementioned `TxRaw` is using raw bytes +as a workaround. This allows them to use any valid Protobuf library without +the need of implementing a custom serializer that adheres to this standard (and related risks of bugs). + +## References + +* 1 _When a message is serialized, there is no guaranteed order for + how its known or unknown fields should be written. Serialization order is an + implementation detail and the details of any particular implementation may + change in the future. Therefore, protocol buffer parsers must be able to parse + fields in any order._ from + https://protobuf.dev/programming-guides/encoding/#order +* 2 https://protobuf.dev/programming-guides/encoding/#signed_integers +* 3 _Note that for scalar message fields, once a message is parsed + there's no way of telling whether a field was explicitly set to the default + value (for example whether a boolean was set to false) or just not set at all: + you should bear this in mind when defining your message types. For example, + don't have a boolean that switches on some behavior when set to false if you + don't want that behavior to also happen by default._ from + https://protobuf.dev/programming-guides/proto3/#default +* 4 _When a message is parsed, if the encoded message does not + contain a particular singular element, the corresponding field in the parsed + object is set to the default value for that field._ from + https://protobuf.dev/programming-guides/proto3/#default +* 5 _Also note that if a scalar message field is set to its default, + the value will not be serialized on the wire._ from + https://protobuf.dev/programming-guides/proto3/#default +* 6 _For enums, the default value is the first defined enum value, + which must be 0._ from + https://protobuf.dev/programming-guides/proto3/#default +* 7 _For message fields, the field is not set. Its exact value is + language-dependent._ from + https://protobuf.dev/programming-guides/proto3/#default +* Encoding rules and parts of the reasoning taken from + [canonical-proto3 Aaron Craelius](https://github.com/regen-network/canonical-proto3) diff --git a/versioned_docs/version-0.52/build/architecture/adr-028-public-key-addresses.md b/versioned_docs/version-0.52/build/architecture/adr-028-public-key-addresses.md new file mode 100644 index 000000000..100049b55 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-028-public-key-addresses.md @@ -0,0 +1,342 @@ +# ADR 028: Public Key Addresses + +## Changelog + +* 2020/08/18: Initial version +* 2021/01/15: Analysis and algorithm update + +## Status + +Proposed + +## Abstract + +This ADR defines an address format for all addressable Cosmos SDK accounts. That includes: new public key algorithms, multisig public keys, and module accounts. + +## Context + +Issue [\#3685](https://github.com/cosmos/cosmos-sdk/issues/3685) identified that public key +address spaces are currently overlapping. We confirmed that it significantly decreases security of Cosmos SDK. + +### Problem + +An attacker can control an input for an address generation function. This leads to a birthday attack, which significantly decreases the security space. +To overcome this, we need to separate the inputs for different kind of account types: +a security break of one account type shouldn't impact the security of other account types. + +### Initial proposals + +One initial proposal was extending the address length and +adding prefixes for different types of addresses. + +@ethanfrey explained an alternate approach originally used in https://github.com/iov-one/weave: + +> I spent quite a bit of time thinking about this issue while building weave... The other cosmos Sdk. +> Basically I define a condition to be a type and format as human readable string with some binary data appended. This condition is hashed into an Address (again at 20 bytes). The use of this prefix makes it impossible to find a preimage for a given address with a different condition (eg ed25519 vs secp256k1). +> This is explained in depth here https://weave.readthedocs.io/en/latest/design/permissions.html +> And the code is here, look mainly at the top where we process conditions. https://github.com/iov-one/weave/blob/master/conditions.go + +And explained how this approach should be sufficiently collision resistant: + +> Yeah, AFAIK, 20 bytes should be collision resistance when the preimages are unique and not malleable. A space of 2^160 would expect some collision to be likely around 2^80 elements (birthday paradox). And if you want to find a collision for some existing element in the database, it is still 2^160. 2^80 only is if all these elements are written to state. +> The good example you brought up was eg. a public key bytes being a valid public key on two algorithms supported by the codec. Meaning if either was broken, you would break accounts even if they were secured with the safer variant. This is only as the issue when no differentiating type info is present in the preimage (before hashing into an address). +> I would like to hear an argument if the 20 bytes space is an actual issue for security, as I would be happy to increase my address sizes in weave. I just figured cosmos and ethereum and bitcoin all use 20 bytes, it should be good enough. And the arguments above which made me feel it was secure. But I have not done a deeper analysis. + +This led to the first proposal (which we proved to be not good enough): +we concatenate a key type with a public key, hash it and take the first 20 bytes of that hash, summarized as `sha256(keyTypePrefix || keybytes)[:20]`. + +### Review and Discussions + +In [\#5694](https://github.com/cosmos/cosmos-sdk/issues/5694) we discussed various solutions. +We agreed that 20 bytes it's not future proof, and extending the address length is the only way to allow addresses of different types, various signature types, etc. +This disqualifies the initial proposal. + +In the issue we discussed various modifications: + +* Choice of the hash function. +* Move the prefix out of the hash function: `keyTypePrefix + sha256(keybytes)[:20]` [post-hash-prefix-proposal]. +* Use double hashing: `sha256(keyTypePrefix + sha256(keybytes)[:20])`. +* Increase to keybytes hash slice from 20 byte to 32 or 40 bytes. We concluded that 32 bytes, produced by a good hash functions is future secure. + +### Requirements + +* Support currently used tools - we don't want to break an ecosystem, or add a long adaptation period. Ref: https://github.com/cosmos/cosmos-sdk/issues/8041 +* Try to keep the address length small - addresses are widely used in state, both as part of a key and object value. + +### Scope + +This ADR only defines a process for the generation of address bytes. For end-user interactions with addresses (through the API, or CLI, etc.), we still use bech32 to format these addresses as strings. This ADR doesn't change that. +Using Bech32 for string encoding gives us support for checksum error codes and handling of user typos. + +## Decision + +We define the following account types, for which we define the address function: + +1. simple accounts: represented by a regular public key (ie: secp256k1, sr25519) +2. naive multisig: accounts composed by other addressable objects (ie: naive multisig) +3. composed accounts with a native address key (ie: bls, group module accounts) +4. module accounts: basically any accounts which cannot sign transactions and which are managed internally by modules + +### Legacy Public Key Addresses Don't Change + +Currently (Jan 2021), the only officially supported Cosmos SDK user accounts are `secp256k1` basic accounts and legacy amino multisig. +They are used in existing Cosmos SDK zones. They use the following address formats: + +* secp256k1: `ripemd160(sha256(pk_bytes))[:20]` +* legacy amino multisig: `sha256(aminoCdc.Marshal(pk))[:20]` + +We don't want to change existing addresses. So the addresses for these two key types will remain the same. + +The current multisig public keys use amino serialization to generate the address. We will retain +those public keys and their address formatting, and call them "legacy amino" multisig public keys +in protobuf. We will also create multisig public keys without amino addresses to be described below. + +### Hash Function Choice + +As in other parts of the Cosmos SDK, we will use `sha256`. + +### Basic Address + +We start with defining a base algorithm for generating addresses which we will call `Hash`. Notably, it's used for accounts represented by a single key pair. For each public key schema we have to have an associated `typ` string, explained in the next section. `hash` is the cryptographic hash function defined in the previous section. + +```go +const A_LEN = 32 + +func Hash(typ string, key []byte) []byte { + return hash(hash(typ) + key)[:A_LEN] +} +``` + +The `+` is bytes concatenation, which doesn't use any separator. + +This algorithm is the outcome of a consultation session with a professional cryptographer. +Motivation: this algorithm keeps the address relatively small (length of the `typ` doesn't impact the length of the final address) +and it's more secure than [post-hash-prefix-proposal] (which uses the first 20 bytes of a pubkey hash, significantly reducing the address space). +Moreover the cryptographer motivated the choice of adding `typ` in the hash to protect against a switch table attack. + +`address.Hash` is a low level function to generate _base_ addresses for new key types. Example: + +* BLS: `address.Hash("bls", pubkey)` + +### Composed Addresses + +For simple composed accounts (like a new naive multisig) we generalize the `address.Hash`. The address is constructed by recursively creating addresses for the sub accounts, sorting the addresses and composing them into a single address. It ensures that the ordering of keys doesn't impact the resulting address. + +```go +// We don't need a PubKey interface - we need anything which is addressable. +type Addressable interface { + Address() []byte +} + +func Composed(typ string, subaccounts []Addressable) []byte { + addresses = map(subaccounts, \a -> LengthPrefix(a.Address())) + addresses = sort(addresses) + return address.Hash(typ, addresses[0] + ... + addresses[n]) +} +``` + +The `typ` parameter should be a schema descriptor, containing all significant attributes with deterministic serialization (eg: utf8 string). +`LengthPrefix` is a function which prepends 1 byte to the address. The value of that byte is the length of the address bits before prepending. The address must be at most 255 bits long. +We are using `LengthPrefix` to eliminate conflicts - it assures, that for 2 lists of addresses: `as = {a1, a2, ..., an}` and `bs = {b1, b2, ..., bm}` such that every `bi` and `ai` is at most 255 long, `concatenate(map(as, (a) => LengthPrefix(a))) = map(bs, (b) => LengthPrefix(b))` if `as = bs`. + +Implementation Tip: account implementations should cache addresses. + +#### Multisig Addresses + +For a new multisig public keys, we define the `typ` parameter not based on any encoding scheme (amino or protobuf). This avoids issues with non-determinism in the encoding scheme. + +Example: + +```protobuf +package cosmos.crypto.multisig; + +message PubKey { + uint32 threshold = 1; + repeated google.protobuf.Any pubkeys = 2; +} +``` + +```go +func (multisig PubKey) Address() { + // first gather all nested pub keys + var keys []address.Addressable // cryptotypes.PubKey implements Addressable + for _, _key := range multisig.Pubkeys { + keys = append(keys, key.GetCachedValue().(cryptotypes.PubKey)) + } + + // form the type from the message name (cosmos.crypto.multisig.PubKey) and the threshold joined together + prefix := fmt.Sprintf("%s/%d", proto.MessageName(multisig), multisig.Threshold) + + // use the Composed function defined above + return address.Composed(prefix, keys) +} +``` + + +### Derived Addresses + +We must be able to cryptographically derive one address from another one. The derivation process must guarantee hash properties, hence we use the already defined `Hash` function: + +```go +func Derive(address, derivationKey []byte) []byte { + return Hash(address, derivationKey) +} +``` + +### Module Account Addresses + +A module account will have `"module"` type. Module accounts can have sub accounts. The submodule account will be created based on module name, and sequence of derivation keys. Typically, the first derivation key should be a class of the derived accounts. The derivation process has a defined order: module name, submodule key, subsubmodule key... An example module account is created using: + +```go +address.Module(moduleName, key) +``` + +An example sub-module account is created using: + +```go +groupPolicyAddresses := []byte{1} +address.Module(moduleName, groupPolicyAddresses, policyID) +``` + +The `address.Module` function is using `address.Hash` with `"module"` as the type argument, and byte representation of the module name concatenated with submodule key. The two last component must be uniquely separated to avoid potential clashes (example: modulename="ab" & submodulekey="bc" will have the same derivation key as modulename="a" & submodulekey="bbc"). +We use a null byte (`'\x00'`) to separate module name from the submodule key. This works, because null byte is not a part of a valid module name. Finally, the sub-submodule accounts are created by applying the `Derive` function recursively. +We could use `Derive` function also in the first step (rather than concatenating module name with zero byte and the submodule key). We decided to do concatenation to avoid one level of derivation and speed up computation. + +For backward compatibility with the existing `authtypes.NewModuleAddress`, we add a special case in `Module` function: when no derivation key is provided, we fallback to the "legacy" implementation. + +```go +func Module(moduleName string, derivationKeys ...[]byte) []byte{ + if len(derivationKeys) == 0 { + return authtypes.NewModuleAddress(moduleName) // legacy case + } + submoduleAddress := Hash("module", []byte(moduleName) + 0 + key) + return fold((a, k) => Derive(a, k), subsubKeys, submoduleAddress) +} +``` + +**Example 1** A lending BTC pool address would be: + +```go +btcPool := address.Module("lending", btc.Address()}) +``` + +If we want to create an address for a module account depending on more than one key, we can concatenate them: + +```go +btcAtomAMM := address.Module("amm", btc.Address() + atom.Address()}) +``` + +**Example 2** a smart-contract address could be constructed by: + +```go +smartContractAddr = Module("mySmartContractVM", smartContractsNamespace, smartContractKey}) + +// which equals to: +smartContractAddr = Derived( + Module("mySmartContractVM", smartContractsNamespace), + []{smartContractKey}) +``` + +### Schema Types + +A `typ` parameter used in `Hash` function SHOULD be unique for each account type. +Since all Cosmos SDK account types are serialized in the state, we propose to use the protobuf message name string. + +Example: all public key types have a unique protobuf message type similar to: + +```protobuf +package cosmos.crypto.sr25519; + +message PubKey { + bytes key = 1; +} +``` + +All protobuf messages have unique fully qualified names, in this example `cosmos.crypto.sr25519.PubKey`. +These names are derived directly from .proto files in a standardized way and used +in other places such as the type URL in `Any`s. We can easily obtain the name using +`proto.MessageName(msg)`. + +## Consequences + +### Backwards Compatibility + +This ADR is compatible with what was committed and directly supported in the Cosmos SDK repository. + +### Positive + +* a simple algorithm for generating addresses for new public keys, complex accounts and modules +* the algorithm generalizes _native composed keys_ +* increased security and collision resistance of addresses +* the approach is extensible for future use-cases - one can use other address types, as long as they don't conflict with the address length specified here (20 or 32 bytes). +* support new account types. + +### Negative + +* addresses do not communicate key type, a prefixed approach would have done this +* addresses are 60% longer and will consume more storage space +* requires a refactor of KVStore store keys to handle variable length addresses + +### Neutral + +* protobuf message names are used as key type prefixes + +## Further Discussions + +Some accounts can have a fixed name or may be constructed in other way (eg: modules). We were discussing an idea of an account with a predefined name (eg: `me.regen`), which could be used by institutions. +Without going into details, these kinds of addresses are compatible with the hash based addresses described here as long as they don't have the same length. +More specifically, any special account address must not have a length equal to 20 or 32 bytes. + +## Appendix: Consulting session + +End of Dec 2020 we had a session with [Alan Szepieniec](https://scholar.google.be/citations?user=4LyZn8oAAAAJ&hl=en) to consult the approach presented above. + +Alan general observations: + +* we don’t need 2-preimage resistance +* we need 32bytes address space for collision resistance +* when an attacker can control an input for object with an address then we have a problem with birthday attack +* there is an issue with smart-contracts for hashing +* sha2 mining can be use to breaking address pre-image + +Hashing algorithm + +* any attack breaking blake3 will break blake2 +* Alan is pretty confident about the current security analysis of the blake hash algorithm. It was a finalist, and the author is well known in security analysis. + +Algorithm: + +* Alan recommends to hash the prefix: `address(pub_key) = hash(hash(key_type) + pub_key)[:32]`, main benefits: + * we are free to user arbitrary long prefix names + * we still don’t risk collisions + * switch tables +* discussion about penalization -> about adding prefix post hash +* Aaron asked about post hash prefixes (`address(pub_key) = key_type + hash(pub_key)`) and differences. Alan noted that this approach has longer address space and it’s stronger. + +Algorithm for complex / composed keys: + +* merging tree like addresses with same algorithm are fine + +Module addresses: Should module addresses have different size to differentiate it? + +* we will need to set a pre-image prefix for module address to keep them in 32-byte space: `hash(hash('module') + module_key)` +* Aaron observation: we already need to deal with variable length (to not break secp256k1 keys). + +Discussion about arithmetic hash function for ZKP + +* Posseidon / Rescue +* Problem: much bigger risk because we don’t know much techniques and history of crypto-analysis of arithmetic constructions. It’s still a new ground and area of active research. + +Post quantum signature size + +* Alan suggestion: Falcon: speed / size ration - very good. +* Aaron - should we think about it? + Alan: based on early extrapolation this thing will get able to break EC cryptography in 2050 . But that’s a lot of uncertainty. But there is magic happening with recursions / linking / simulation and that can speedup the progress. + +Other ideas + +* Let’s say we use same key and two different address algorithms for 2 different use cases. Is it still safe to use it? Alan: if we want to hide the public key (which is not our use case), then it’s less secure but there are fixes. + +### References + +* [Notes](https://hackmd.io/_NGWI4xZSbKzj1BkCqyZMw) diff --git a/versioned_docs/version-0.52/build/architecture/adr-029-fee-grant-module.md b/versioned_docs/version-0.52/build/architecture/adr-029-fee-grant-module.md new file mode 100644 index 000000000..6b52556ff --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-029-fee-grant-module.md @@ -0,0 +1,153 @@ +# ADR 029: Fee Grant Module + +## Changelog + +* 2020/08/18: Initial Draft +* 2021/05/05: Removed height based expiration support and simplified naming. + +## Status + +Accepted + +## Context + +In order to make blockchain transactions, the signing account must possess a sufficient balance of the right denomination +in order to pay fees. There are classes of transactions where needing to maintain a wallet with sufficient fees is a +barrier to adoption. + +For instance, when proper permissions are setup, someone may temporarily delegate the ability to vote on proposals to +a "burner" account that is stored on a mobile phone with only minimal security. + +Other use cases include workers tracking items in a supply chain or farmers submitting field data for analytics +or compliance purposes. + +For all of these use cases, UX would be significantly enhanced by obviating the need for these accounts to always +maintain the appropriate fee balance. This is especially true if we wanted to achieve enterprise adoption for something +like supply chain tracking. + +While one solution would be to have a service that fills up these accounts automatically with the appropriate fees, a better UX +would be provided by allowing these accounts to pull from a common fee pool account with proper spending limits. +A single pool would reduce the churn of making lots of small "fill up" transactions and also more effectively leverages +the resources of the organization setting up the pool. + +## Decision + +As a solution we propose a module, `x/feegrant` which allows one account, the "granter" to grant another account, the "grantee" +an allowance to spend the granter's account balance for fees within certain well-defined limits. + +Fee allowances are defined by the extensible `FeeAllowanceI` interface: + +```go +type FeeAllowanceI { + // Accept can use fee payment requested as well as timestamp of the current block + // to determine whether or not to process this. This is checked in + // Keeper.UseGrantedFees and the return values should match how it is handled there. + // + // If it returns an error, the fee payment is rejected, otherwise it is accepted. + // The FeeAllowance implementation is expected to update it's internal state + // and will be saved again after an acceptance. + // + // If remove is true (regardless of the error), the FeeAllowance will be deleted from storage + // (eg. when it is used up). (See call to RevokeFeeAllowance in Keeper.UseGrantedFees) + Accept(ctx sdk.Context, fee sdk.Coins, msgs []sdk.Msg) (remove bool, err error) + + // ValidateBasic should evaluate this FeeAllowance for internal consistency. + // Don't allow negative amounts, or negative periods for example. + ValidateBasic() error +} +``` + +Two basic fee allowance types, `BasicAllowance` and `PeriodicAllowance` are defined to support known use cases: + +```protobuf +// BasicAllowance implements FeeAllowanceI with a one-time grant of tokens +// that optionally expires. The delegatee can use up to SpendLimit to cover fees. +message BasicAllowance { + // spend_limit specifies the maximum amount of tokens that can be spent + // by this allowance and will be updated as tokens are spent. If it is + // empty, there is no spend limit and any amount of coins can be spent. + repeated cosmos_sdk.v1.Coin spend_limit = 1; + + // expiration specifies an optional time when this allowance expires + google.protobuf.Timestamp expiration = 2; +} + +// PeriodicAllowance extends FeeAllowanceI to allow for both a maximum cap, +// as well as a limit per time period. +message PeriodicAllowance { + BasicAllowance basic = 1; + + // period specifies the time duration in which period_spend_limit coins can + // be spent before that allowance is reset + google.protobuf.Duration period = 2; + + // period_spend_limit specifies the maximum number of coins that can be spent + // in the period + repeated cosmos_sdk.v1.Coin period_spend_limit = 3; + + // period_can_spend is the number of coins left to be spent before the period_reset time + repeated cosmos_sdk.v1.Coin period_can_spend = 4; + + // period_reset is the time at which this period resets and a new one begins, + // it is calculated from the start time of the first transaction after the + // last period ended + google.protobuf.Timestamp period_reset = 5; +} + +``` + +Allowances can be granted and revoked using `MsgGrantAllowance` and `MsgRevokeAllowance`: + +```protobuf +// MsgGrantAllowance adds permission for Grantee to spend up to Allowance +// of fees from the account of Granter. +message MsgGrantAllowance { + string granter = 1; + string grantee = 2; + google.protobuf.Any allowance = 3; + } + + // MsgRevokeAllowance removes any existing FeeAllowance from Granter to Grantee. + message MsgRevokeAllowance { + string granter = 1; + string grantee = 2; + } +``` + +In order to use allowances in transactions, we add a new field `granter` to the transaction `Fee` type: + +```protobuf +package cosmos.tx.v1beta1; + +message Fee { + repeated cosmos.base.v1beta1.Coin amount = 1; + uint64 gas_limit = 2; + string payer = 3; + string granter = 4; +} +``` + +`granter` must either be left empty or must correspond to an account which has granted +a fee allowance to fee payer (either the first signer or the value of the `payer` field). + +A new `AnteDecorator` named `DeductGrantedFeeDecorator` will be created in order to process transactions with `fee_payer` +set and correctly deduct fees based on fee allowances. + +## Consequences + +### Positive + +* improved UX for use cases where it is cumbersome to maintain an account balance just for fees + +### Negative + +### Neutral + +* a new field must be added to the transaction `Fee` message and a new `AnteDecorator` must be +created to use it + +## References + +* Blog article describing initial work: https://medium.com/regen-network/hacking-the-cosmos-cosmwasm-and-key-management-a08b9f561d1b +* Initial public specification: https://gist.github.com/aaronc/b60628017352df5983791cad30babe56 +* Original subkeys proposal from B-harvest which influenced this design: https://github.com/cosmos/cosmos-sdk/issues/4480 diff --git a/versioned_docs/version-0.52/build/architecture/adr-030-authz-module.md b/versioned_docs/version-0.52/build/architecture/adr-030-authz-module.md new file mode 100644 index 000000000..5aab72c5c --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-030-authz-module.md @@ -0,0 +1,258 @@ +# ADR 030: Authorization Module + +## Changelog + +* 2019-11-06: Initial Draft +* 2020-10-12: Updated Draft +* 2020-11-13: Accepted +* 2020-05-06: proto API updates, use `sdk.Msg` instead of `sdk.ServiceMsg` (the latter concept was removed from Cosmos SDK) +* 2022-04-20: Updated the `SendAuthorization` proto docs to clarify the `SpendLimit` is a required field. (Generic authorization can be used with bank msg type url to create limit less bank authorization) + +## Status + +Accepted + +## Abstract + +This ADR defines the `x/authz` module which allows accounts to grant authorizations to perform actions +on behalf of that account to other accounts. + +## Context + +The concrete use cases which motivated this module include: + +* the desire to delegate the ability to vote on proposals to other accounts besides the account which one has +delegated stake +* "sub-keys" functionality, as originally proposed in [\#4480](https://github.com/cosmos/cosmos-sdk/issues/4480) which +is a term used to describe the functionality provided by this module together with +the `fee_grant` module from [ADR 029](./adr-029-fee-grant-module.md) and the [group module](https://github.com/cosmos/cosmos-sdk/tree/main/x/group). + +The "sub-keys" functionality roughly refers to the ability for one account to grant some subset of its capabilities to +other accounts with possibly less robust, but easier to use security measures. For instance, a master account representing +an organization could grant the ability to spend small amounts of the organization's funds to individual employee accounts. +Or an individual (or group) with a multisig wallet could grant the ability to vote on proposals to any one of the member +keys. + +The current implementation is based on work done by the [Gaian's team at Hackatom Berlin 2019](https://github.com/cosmos-gaians/cosmos-sdk/tree/hackatom/x/delegation). + +## Decision + +We will create a module named `authz` which provides functionality for +granting arbitrary privileges from one account (the _granter_) to another account (the _grantee_). Authorizations +must be granted for a particular `Msg` service methods one by one using an implementation +of `Authorization` interface. + +### Types + +Authorizations determine exactly what privileges are granted. They are extensible +and can be defined for any `Msg` service method even outside of the module where +the `Msg` method is defined. `Authorization`s reference `Msg`s using their TypeURL. + +#### Authorization + +```go +type Authorization interface { + proto.Message + + // MsgTypeURL returns the fully-qualified Msg TypeURL (as described in ADR 020), + // which will process and accept or reject a request. + MsgTypeURL() string + + // Accept determines whether this grant permits the provided sdk.Msg to be performed, and if + // so provides an upgraded authorization instance. + Accept(ctx sdk.Context, msg sdk.Msg) (AcceptResponse, error) + + // ValidateBasic does a simple validation check that + // doesn't require access to any other information. + ValidateBasic() error +} + +// AcceptResponse instruments the controller of an authz message if the request is accepted +// and if it should be updated or deleted. +type AcceptResponse struct { + // If Accept=true, the controller can accept and authorization and handle the update. + Accept bool + // If Delete=true, the controller must delete the authorization object and release + // storage resources. + Delete bool + // Controller, who is calling Authorization.Accept must check if `Updated != nil`. If yes, + // it must use the updated version and handle the update on the storage level. + Updated Authorization +} +``` + +For example a `SendAuthorization` like this is defined for `MsgSend` that takes +a `SpendLimit` and updates it down to zero: + +```go +type SendAuthorization struct { + // SpendLimit specifies the maximum amount of tokens that can be spent + // by this authorization and will be updated as tokens are spent. This field is required. (Generic authorization + // can be used with bank msg type url to create limit less bank authorization). + SpendLimit sdk.Coins +} + +func (a SendAuthorization) MsgTypeURL() string { + return sdk.MsgTypeURL(&MsgSend{}) +} + +func (a SendAuthorization) Accept(ctx sdk.Context, msg sdk.Msg) (authz.AcceptResponse, error) { + mSend, ok := msg.(*MsgSend) + if !ok { + return authz.AcceptResponse{}, sdkerrors.ErrInvalidType.Wrap("type mismatch") + } + limitLeft, isNegative := a.SpendLimit.SafeSub(mSend.Amount) + if isNegative { + return authz.AcceptResponse{}, sdkerrors.ErrInsufficientFunds.Wrapf("requested amount is more than spend limit") + } + if limitLeft.IsZero() { + return authz.AcceptResponse{Accept: true, Delete: true}, nil + } + + return authz.AcceptResponse{Accept: true, Delete: false, Updated: &SendAuthorization{SpendLimit: limitLeft}}, nil +} +``` + +A different type of capability for `MsgSend` could be implemented +using the `Authorization` interface with no need to change the underlying +`bank` module. + +##### Small notes on `AcceptResponse` + +* The `AcceptResponse.Accept` field will be set to `true` if the authorization is accepted. +However, if it is rejected, the function `Accept` will raise an error (without setting `AcceptResponse.Accept` to `false`). + +* The `AcceptResponse.Updated` field will be set to a non-nil value only if there is a real change to the authorization. +If authorization remains the same (as is, for instance, always the case for a [`GenericAuthorization`](#genericauthorization)), +the field will be `nil`. + +### `Msg` Service + +```protobuf +service Msg { + // Grant grants the provided authorization to the grantee on the granter's + // account with the provided expiration time. + rpc Grant(MsgGrant) returns (MsgGrantResponse); + + // Exec attempts to execute the provided messages using + // authorizations granted to the grantee. Each message should have only + // one signer corresponding to the granter of the authorization. + rpc Exec(MsgExec) returns (MsgExecResponse); + + // Revoke revokes any authorization corresponding to the provided method name on the + // granter's account that has been granted to the grantee. + rpc Revoke(MsgRevoke) returns (MsgRevokeResponse); +} + +// Grant gives permissions to execute +// the provided method with expiration time. +message Grant { + google.protobuf.Any authorization = 1 [(cosmos_proto.accepts_interface) = "cosmos.authz.v1beta1.Authorization"]; + google.protobuf.Timestamp expiration = 2 [(gogoproto.stdtime) = true, (gogoproto.nullable) = false]; +} + +message MsgGrant { + string granter = 1; + string grantee = 2; + + Grant grant = 3 [(gogoproto.nullable) = false]; +} + +message MsgExecResponse { + cosmos.base.abci.v1beta1.Result result = 1; +} + +message MsgExec { + string grantee = 1; + // Authorization Msg requests to execute. Each msg must implement Authorization interface + repeated google.protobuf.Any msgs = 2 [(cosmos_proto.accepts_interface) = "cosmos.base.v1beta1.Msg"];; +} +``` + +### Router Middleware + +The `authz` `Keeper` will expose a `DispatchActions` method which allows other modules to send `Msg`s +to the router based on `Authorization` grants: + +```go +type Keeper interface { + // DispatchActions routes the provided msgs to their respective handlers if the grantee was granted an authorization + // to send those messages by the first (and only) signer of each msg. + DispatchActions(ctx sdk.Context, grantee sdk.AccAddress, msgs []sdk.Msg) sdk.Result` +} +``` + +### CLI + +#### `tx exec` Method + +When a CLI user wants to run a transaction on behalf of another account using `MsgExec`, they +can use the `exec` method. For instance `gaiacli tx gov vote 1 yes --from --generate-only | gaiacli tx authz exec --send-as --from ` +would send a transaction like this: + +```go +MsgExec { + Grantee: mykey, + Msgs: []sdk.Msg{ + MsgVote { + ProposalID: 1, + Voter: cosmos3thsdgh983egh823 + Option: Yes + } + } +} +``` + +#### `tx grant --from ` + +This CLI command will send a `MsgGrant` transaction. `authorization` should be encoded as +JSON on the CLI. + +#### `tx revoke --from ` + +This CLI command will send a `MsgRevoke` transaction. + +### Built-in Authorizations + +#### `SendAuthorization` + +```protobuf +// SendAuthorization allows the grantee to spend up to spend_limit coins from +// the granter's account. +message SendAuthorization { + repeated cosmos.base.v1beta1.Coin spend_limit = 1; +} +``` + +#### `GenericAuthorization` + +```protobuf +// GenericAuthorization gives the grantee unrestricted permissions to execute +// the provided method on behalf of the granter's account. +message GenericAuthorization { + option (cosmos_proto.implements_interface) = "Authorization"; + + // Msg, identified by it's type URL, to grant unrestricted permissions to execute + string msg = 1; +} +``` + +## Consequences + +### Positive + +* Users will be able to authorize arbitrary actions on behalf of their accounts to other +users, improving key management for many use cases +* The solution is more generic than previously considered approaches and the +`Authorization` interface approach can be extended to cover other use cases by +SDK users + +### Negative + +### Neutral + +## References + +* Initial Hackatom implementation: https://github.com/cosmos-gaians/cosmos-sdk/tree/hackatom/x/delegation +* Post-Hackatom spec: https://gist.github.com/aaronc/b60628017352df5983791cad30babe56#delegation-module +* B-Harvest subkeys spec: https://github.com/cosmos/cosmos-sdk/issues/4480 diff --git a/versioned_docs/version-0.52/build/architecture/adr-031-msg-service.md b/versioned_docs/version-0.52/build/architecture/adr-031-msg-service.md new file mode 100644 index 000000000..8185509bd --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-031-msg-service.md @@ -0,0 +1,201 @@ +# ADR 031: Protobuf Msg Services + +## Changelog + +* 2020-10-05: Initial Draft +* 2021-04-21: Remove `ServiceMsg`s to follow Protobuf `Any`'s spec, see [#9063](https://github.com/cosmos/cosmos-sdk/issues/9063). + +## Status + +Accepted + +## Abstract + +We want to leverage protobuf `service` definitions for defining `Msg`s which will give us significant developer UX +improvements in terms of the code that is generated and the fact that return types will now be well defined. + +## Context + +Currently `Msg` handlers in the Cosmos SDK do have return values that are placed in the `data` field of the response. +These return values, however, are not specified anywhere except in the golang handler code. + +In early conversations [it was proposed](https://docs.google.com/document/d/1eEgYgvgZqLE45vETjhwIw4VOqK-5hwQtZtjVbiXnIGc/edit) +that `Msg` return types be captured using a protobuf extension field, ex: + +```protobuf +package cosmos.gov; + +message MsgSubmitProposal + option (cosmos_proto.msg_return) = “uint64”; + string delegator_address = 1; + string validator_address = 2; + repeated sdk.Coin amount = 3; +} +``` + +This was never adopted, however. + +Having a well-specified return value for `Msg`s would improve client UX. For instance, +in `x/gov`, `MsgSubmitProposal` returns the proposal ID as a big-endian `uint64`. +This isn’t really documented anywhere and clients would need to know the internals +of the Cosmos SDK to parse that value and return it to users. + +Also, there may be cases where we want to use these return values programmatically. +For instance, https://github.com/cosmos/cosmos-sdk/issues/7093 proposes a method for +doing inter-module Ocaps using the `Msg` router. A well-defined return type would +improve the developer UX for this approach. + +In addition, handler registration of `Msg` types tends to add a bit of +boilerplate on top of keepers and is usually done through manual type switches. +This isn't necessarily bad, but it does add overhead to creating modules. + +## Decision + +We decide to use protobuf `service` definitions for defining `Msg`s as well as +the code generated by them as a replacement for `Msg` handlers. + +Below we define how this will look for the `SubmitProposal` message from `x/gov` module. +We start with a `Msg` `service` definition: + +```protobuf +package cosmos.gov; + +service Msg { + rpc SubmitProposal(MsgSubmitProposal) returns (MsgSubmitProposalResponse); +} + +// Note that for backwards compatibility this uses MsgSubmitProposal as the request +// type instead of the more canonical MsgSubmitProposalRequest +message MsgSubmitProposal { + google.protobuf.Any content = 1; + string proposer = 2; +} + +message MsgSubmitProposalResponse { + uint64 proposal_id; +} +``` + +While this is most commonly used for gRPC, overloading protobuf `service` definitions like this does not violate +the intent of the [protobuf spec](https://protobuf.dev/programming-guides/proto3/#services) which says: +> If you don’t want to use gRPC, it’s also possible to use protocol buffers with your own RPC implementation. +With this approach, we would get an auto-generated `MsgServer` interface: + +In addition to clearly specifying return types, this has the benefit of generating client and server code. On the server +side, this is almost like an automatically generated keeper method and could maybe be used instead of keepers eventually +(see [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093)): + +```go +package gov + +type MsgServer interface { + SubmitProposal(context.Context, *MsgSubmitProposal) (*MsgSubmitProposalResponse, error) +} +``` + +On the client side, developers could take advantage of this by creating RPC implementations that encapsulate transaction +logic. Protobuf libraries that use asynchronous callbacks, like [protobuf.js](https://github.com/protobufjs/protobuf.js#using-services) +could use this to register callbacks for specific messages even for transactions that include multiple `Msg`s. + +Each `Msg` service method should have exactly one request parameter: its corresponding `Msg` type. For example, the `Msg` service method `/cosmos.gov.v1beta1.Msg/SubmitProposal` above has exactly one request parameter, namely the `Msg` type `/cosmos.gov.v1beta1.MsgSubmitProposal`. It is important the reader understands clearly the nomenclature difference between a `Msg` service (a Protobuf service) and a `Msg` type (a Protobuf message), and the differences in their fully-qualified name. + +This convention has been decided over the more canonical `Msg...Request` names mainly for backwards compatibility, but also for better readability in `TxBody.messages` (see [Encoding section](#encoding) below): transactions containing `/cosmos.gov.MsgSubmitProposal` read better than those containing `/cosmos.gov.v1beta1.MsgSubmitProposalRequest`. + +One consequence of this convention is that each `Msg` type can be the request parameter of only one `Msg` service method. However, we consider this limitation a good practice in explicitness. + +### Encoding + +Encoding of transactions generated with `Msg` services do not differ from current Protobuf transaction encoding as defined in [ADR-020](./adr-020-protobuf-transaction-encoding.md). We are encoding `Msg` types (which are exactly `Msg` service methods' request parameters) as `Any` in `Tx`s which involves packing the +binary-encoded `Msg` with its type URL. + +### Decoding + +Since `Msg` types are packed into `Any`, decoding transactions messages are done by unpacking `Any`s into `Msg` types. For more information, please refer to [ADR-020](./adr-020-protobuf-transaction-encoding.md#transactions). + +### Routing + +We propose to add a `msg_service_router` in BaseApp. This router is a key/value map which maps `Msg` types' `type_url`s to their corresponding `Msg` service method handler. Since there is a 1-to-1 mapping between `Msg` types and `Msg` service method, the `msg_service_router` has exactly one entry per `Msg` service method. + +When a transaction is processed by BaseApp (in CheckTx or in DeliverTx), its `TxBody.messages` are decoded as `Msg`s. Each `Msg`'s `type_url` is matched against an entry in the `msg_service_router`, and the respective `Msg` service method handler is called. + +For backward compatibility, the old handlers are not removed yet. If BaseApp receives a legacy `Msg` with no corresponding entry in the `msg_service_router`, it will be routed via its legacy `Route()` method into the legacy handler. + +### Module Configuration + +In [ADR 021](./adr-021-protobuf-query-encoding.md), we introduced a method `RegisterQueryService` +to `AppModule` which allows for modules to register gRPC queriers. + +To register `Msg` services, we attempt a more extensible approach by converting `RegisterQueryService` +to a more generic `RegisterServices` method: + +```go +type AppModule interface { + RegisterServices(Configurator) + ... +} + +type Configurator interface { + QueryServer() grpc.Server + MsgServer() grpc.Server +} + +// example module: +func (am AppModule) RegisterServices(cfg Configurator) { + types.RegisterQueryServer(cfg.QueryServer(), keeper) + types.RegisterMsgServer(cfg.MsgServer(), keeper) +} +``` + +The `RegisterServices` method and the `Configurator` interface are intended to +evolve to satisfy the use cases discussed in [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093) +and [\#7122](https://github.com/cosmos/cosmos-sdk/issues/7421). + +When `Msg` services are registered, the framework _should_ verify that all `Msg` types +implement the `sdk.Msg` interface and throw an error during initialization rather +than later when transactions are processed. + +### `Msg` Service Implementation + +Just like query services, `Msg` service methods can retrieve the `sdk.Context` +from the `context.Context` parameter method using the `sdk.UnwrapSDKContext` +method: + +```go +package gov + +func (k Keeper) SubmitProposal(goCtx context.Context, params *types.MsgSubmitProposal) (*MsgSubmitProposalResponse, error) { + ctx := sdk.UnwrapSDKContext(goCtx) + ... +} +``` + +The `sdk.Context` should have an `EventManager` already attached by BaseApp's `msg_service_router`. + +Separate handler definition is no longer needed with this approach. + +## Consequences + +This design changes how a module functionality is exposed and accessed. It deprecates the existing `Handler` interface and `AppModule.Route` in favor of [Protocol Buffer Services](https://protobuf.dev/programming-guides/proto3/#services) and Service Routing described above. This dramatically simplifies the code. We don't need to create handlers and keepers any more. Use of Protocol Buffer auto-generated clients clearly separates the communication interfaces between the module and a modules user. The control logic (aka handlers and keepers) is not exposed any more. A module interface can be seen as a black box accessible through a client API. It's worth to note that the client interfaces are also generated by Protocol Buffers. + +This also allows us to change how we perform functional tests. Instead of mocking AppModules and Router, we will mock a client (server will stay hidden). More specifically: we will never mock `moduleA.MsgServer` in `moduleB`, but rather `moduleA.MsgClient`. One can think about it as working with external services (eg DBs, or online servers...). We assume that the transmission between clients and servers is correctly handled by generated Protocol Buffers. + +Finally, closing a module to client API opens desirable OCAP patterns discussed in ADR-033. Since server implementation and interface is hidden, nobody can hold "keepers"/servers and will be forced to relay on the client interface, which will drive developers for correct encapsulation and software engineering patterns. + +### Pros + +* communicates return type clearly +* manual handler registration and return type marshaling is no longer needed, just implement the interface and register it +* communication interface is automatically generated, the developer can now focus only on the state transition methods - this would improve the UX of [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093) approach (1) if we chose to adopt that +* generated client code could be useful for clients and tests +* dramatically reduces and simplifies the code + +### Cons + +* using `service` definitions outside the context of gRPC could be confusing (but doesn’t violate the proto3 spec) + +## References + +* [Initial Github Issue \#7122](https://github.com/cosmos/cosmos-sdk/issues/7122) +* [proto 3 Language Guide: Defining Services](https://protobuf.dev/programming-guides/proto3/#services) +* [ADR 020](./adr-020-protobuf-transaction-encoding.md) +* [ADR 021](./adr-021-protobuf-query-encoding.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-032-typed-events.md b/versioned_docs/version-0.52/build/architecture/adr-032-typed-events.md new file mode 100644 index 000000000..1c043038d --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-032-typed-events.md @@ -0,0 +1,319 @@ +# ADR 032: Typed Events + +## Changelog + +* 28-Sept-2020: Initial Draft + +## Authors + +* Anil Kumar (@anilcse) +* Jack Zampolin (@jackzampolin) +* Adam Bozanich (@boz) + +## Status + +Proposed + +## Abstract + +Currently in the Cosmos SDK, events are defined in the handlers for each message as well as `BeginBlock` and `EndBlock`. Each module doesn't have types defined for each event, they are implemented as `map[string]string`. Above all else this makes these events difficult to consume as it requires a great deal of raw string matching and parsing. This proposal focuses on updating the events to use **typed events** defined in each module such that emitting and subscribing to events will be much easier. This workflow comes from the experience of the Akash Network team. + +## Context + +Currently in the Cosmos SDK, events are defined in the handlers for each message, meaning each module doesn't have a canonical set of types for each event. Above all else this makes these events difficult to consume as it requires a great deal of raw string matching and parsing. This proposal focuses on updating the events to use **typed events** defined in each module such that emitting and subscribing to events will be much easier. This workflow comes from the experience of the Akash Network team. + +[Our platform](http://github.com/ovrclk/akash) requires a number of programmatic on chain interactions both on the provider (datacenter - to bid on new orders and listen for leases created) and user (application developer - to send the app manifest to the provider) side. In addition the Akash team is now maintaining the IBC [`relayer`](https://github.com/ovrclk/relayer), another very event driven process. In working on these core pieces of infrastructure, and integrating lessons learned from Kubernetes development, our team has developed a standard method for defining and consuming typed events in Cosmos SDK modules. We have found that it is extremely useful in building this type of event driven application. + +As the Cosmos SDK gets used more extensively for apps like `peggy`, other peg zones, IBC, DeFi, etc... there will be an exploding demand for event driven applications to support new features desired by users. We propose upstreaming our findings into the Cosmos SDK to enable all Cosmos SDK applications to quickly and easily build event driven apps to aid their core application. Wallets, exchanges, explorers, and defi protocols all stand to benefit from this work. + +If this proposal is accepted, users will be able to build event driven Cosmos SDK apps in go by just writing `EventHandler`s for their specific event types and passing them to `EventEmitters` that are defined in the Cosmos SDK. + +The end of this proposal contains a detailed example of how to consume events after this refactor. + +This proposal is specifically about how to consume these events as a client of the blockchain, not for intermodule communication. + +## Decision + +**Step-1**: Implement additional functionality in the `types` package: `EmitTypedEvent` and `ParseTypedEvent` functions + +```go +// types/events.go + +// EmitTypedEvent takes typed event and emits converting it into sdk.Event +func (em *EventManager) EmitTypedEvent(event proto.Message) error { + evtType := proto.MessageName(event) + evtJSON, err := codec.ProtoMarshalJSON(event) + if err != nil { + return err + } + + var attrMap map[string]json.RawMessage + err = json.Unmarshal(evtJSON, &attrMap) + if err != nil { + return err + } + + var attrs []abci.EventAttribute + for k, v := range attrMap { + attrs = append(attrs, abci.EventAttribute{ + Key: []byte(k), + Value: v, + }) + } + + em.EmitEvent(Event{ + Type: evtType, + Attributes: attrs, + }) + + return nil +} + +// ParseTypedEvent converts abci.Event back to typed event +func ParseTypedEvent(event abci.Event) (proto.Message, error) { + concreteGoType := proto.MessageType(event.Type) + if concreteGoType == nil { + return nil, fmt.Errorf("failed to retrieve the message of type %q", event.Type) + } + + var value reflect.Value + if concreteGoType.Kind() == reflect.Ptr { + value = reflect.New(concreteGoType.Elem()) + } else { + value = reflect.Zero(concreteGoType) + } + + protoMsg, ok := value.Interface().(proto.Message) + if !ok { + return nil, fmt.Errorf("%q does not implement proto.Message", event.Type) + } + + attrMap := make(map[string]json.RawMessage) + for _, attr := range event.Attributes { + attrMap[string(attr.Key)] = attr.Value + } + + attrBytes, err := json.Marshal(attrMap) + if err != nil { + return nil, err + } + + err = jsonpb.Unmarshal(strings.NewReader(string(attrBytes)), protoMsg) + if err != nil { + return nil, err + } + + return protoMsg, nil +} +``` + +Here, the `EmitTypedEvent` is a method on `EventManager` which takes typed event as input and apply json serialization on it. Then it maps the JSON key/value pairs to `event.Attributes` and emits it in form of `sdk.Event`. `Event.Type` will be the type URL of the proto message. + +When we subscribe to emitted events on the CometBFT websocket, they are emitted in the form of an `abci.Event`. `ParseTypedEvent` parses the event back to it's original proto message. + +**Step-2**: Add proto definitions for typed events for msgs in each module: + +For example, let's take `MsgSubmitProposal` of `gov` module and implement this event's type. + +```protobuf +// proto/cosmos/gov/v1beta1/gov.proto +// Add typed event definition + +package cosmos.gov.v1beta1; + +message EventSubmitProposal { + string from_address = 1; + uint64 proposal_id = 2; + TextProposal proposal = 3; +} +``` + +**Step-3**: Refactor event emission to use the typed event created and emit using `sdk.EmitTypedEvent`: + +```go +// x/gov/handler.go +func handleMsgSubmitProposal(ctx sdk.Context, keeper keeper.Keeper, msg types.MsgSubmitProposalI) (*sdk.Result, error) { + ... + types.Context.EventManager().EmitTypedEvent( + &EventSubmitProposal{ + FromAddress: fromAddress, + ProposalId: id, + Proposal: proposal, + }, + ) + ... +} +``` + +### How to subscribe to these typed events in `Client` + +> NOTE: Full code example below + +Users will be able to subscribe using `client.Context.Client.Subscribe` and consume events which are emitted using `EventHandler`s. + +Akash Network has built a simple [`pubsub`](https://github.com/ovrclk/akash/blob/90d258caeb933b611d575355b8df281208a214f8/pubsub/bus.go#L20). This can be used to subscribe to `abci.Events` and [publish](https://github.com/ovrclk/akash/blob/90d258caeb933b611d575355b8df281208a214f8/events/publish.go#L21) them as typed events. + +Please see the below code sample for more detail on this flow looks for clients. + +## Consequences + +### Positive + +* Improves consistency of implementation for the events currently in the Cosmos SDK +* Provides a much more ergonomic way to handle events and facilitates writing event driven applications +* This implementation will support a middleware ecosystem of `EventHandler`s + +### Negative + +## Detailed code example of publishing events + +This ADR also proposes adding affordances to emit and consume these events. This way developers will only need to write +`EventHandler`s which define the actions they desire to take. + +```go +// EventEmitter is a type that describes event emitter functions +// This should be defined in `types/events.go` +type EventEmitter func(context.Context, client.Context, ...EventHandler) error + +// EventHandler is a type of function that handles events coming out of the event bus +// This should be defined in `types/events.go` +type EventHandler func(proto.Message) error + +// Sample use of the functions below +func main() { + ctx, cancel := context.WithCancel(context.Background()) + + if err := TxEmitter(ctx, client.Context{}.WithNodeURI("tcp://localhost:26657"), SubmitProposalEventHandler); err != nil { + cancel() + panic(err) + } + + return +} + +// SubmitProposalEventHandler is an example of an event handler that prints proposal details +// when any EventSubmitProposal is emitted. +func SubmitProposalEventHandler(ev proto.Message) (err error) { + switch event := ev.(type) { + // Handle governance proposal events creation events + case govtypes.EventSubmitProposal: + // Users define business logic here e.g. + fmt.Println(ev.FromAddress, ev.ProposalId, ev.Proposal) + return nil + default: + return nil + } +} + +// TxEmitter is an example of an event emitter that emits just transaction events. This can and +// should be implemented somewhere in the Cosmos SDK. The Cosmos SDK can include an EventEmitters for tm.event='Tx' +// and/or tm.event='NewBlock' (the new block events may contain typed events) +func TxEmitter(ctx context.Context, cliCtx client.Context, ehs ...EventHandler) (err error) { + // Instantiate and start CometBFT RPC client + client, err := cliCtx.GetNode() + if err != nil { + return err + } + + if err = client.Start(); err != nil { + return err + } + + // Start the pubsub bus + bus := pubsub.NewBus() + defer bus.Close() + + // Initialize a new error group + eg, ctx := errgroup.WithContext(ctx) + + // Publish chain events to the pubsub bus + eg.Go(func() error { + return PublishChainTxEvents(ctx, client, bus, simapp.ModuleBasics) + }) + + // Subscribe to the bus events + subscriber, err := bus.Subscribe() + if err != nil { + return err + } + + // Handle all the events coming out of the bus + eg.Go(func() error { + var err error + for { + select { + case <-ctx.Done(): + return nil + case <-subscriber.Done(): + return nil + case ev := <-subscriber.Events(): + for _, eh := range ehs { + if err = eh(ev); err != nil { + break + } + } + } + } + return nil + }) + + return group.Wait() +} + +// PublishChainTxEvents events using cmtclient. Waits on context shutdown signals to exit. +func PublishChainTxEvents(ctx context.Context, client cmtclient.EventsClient, bus pubsub.Bus) (err error) { + // Subscribe to transaction events + txch, err := client.Subscribe(ctx, "txevents", "tm.event='Tx'", 100) + if err != nil { + return err + } + + // Unsubscribe from transaction events on function exit + defer func() { + err = client.UnsubscribeAll(ctx, "txevents") + }() + + // Use errgroup to manage concurrency + g, ctx := errgroup.WithContext(ctx) + + // Publish transaction events in a goroutine + g.Go(func() error { + var err error + for { + select { + case <-ctx.Done(): + break + case ed := <-ch: + switch evt := ed.Data.(type) { + case cmttypes.EventDataTx: + if !evt.Result.IsOK() { + continue + } + // range over events and parse them + // send them to the pubsub bus + for _, abciEv := range events { + typedEvent, err := sdk.ParseTypedEvent(abciEv) + if err != nil { + return err + } + if err := bus.Publish(typedEvent); err != nil { + bus.Close() + return + } + continue + } + } + } + } + return err + }) + + // Exit on error or context cancellation + return g.Wait() +} +``` + +## References + +* [Publish Custom Events via a bus](https://github.com/ovrclk/akash/blob/90d258caeb933b611d575355b8df281208a214f8/events/publish.go#L19-L58) +* [Consuming the events in `Client`](https://github.com/ovrclk/deploy/blob/bf6c633ab6c68f3026df59efd9982d6ca1bf0561/cmd/event-handlers.go#L57) diff --git a/versioned_docs/version-0.52/build/architecture/adr-033-protobuf-inter-module-comm.md b/versioned_docs/version-0.52/build/architecture/adr-033-protobuf-inter-module-comm.md new file mode 100644 index 000000000..4f2769e67 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-033-protobuf-inter-module-comm.md @@ -0,0 +1,400 @@ +# ADR 033: Protobuf-based Inter-Module Communication + +## Changelog + +* 2020-10-05: Initial Draft + +## Status + +Proposed + +## Abstract + +This ADR introduces a system for permissioned inter-module communication leveraging the protobuf `Query` and `Msg` +service definitions defined in [ADR 021](./adr-021-protobuf-query-encoding.md) and +[ADR 031](./adr-031-msg-service.md) which provides: + +* stable protobuf based module interfaces to potentially later replace the keeper paradigm +* stronger inter-module object capabilities (OCAPs) guarantees +* module accounts and sub-account authorization + +## Context + +In the current Cosmos SDK documentation on the [Object-Capability Model](https://docs.cosmos.network/main/learn/advanced/ocap#ocaps-in-practice), it is stated that: + +> We assume that a thriving ecosystem of Cosmos SDK modules that are easy to compose into a blockchain application will contain faulty or malicious modules. + +There is currently not a thriving ecosystem of Cosmos SDK modules. We hypothesize that this is in part due to: + +1. lack of a stable v1.0 Cosmos SDK to build modules off of. Module interfaces are changing, sometimes dramatically, from +point release to point release, often for good reasons, but this does not create a stable foundation to build on. +2. lack of a properly implemented object capability or even object-oriented encapsulation system which makes refactors +of module keeper interfaces inevitable because the current interfaces are poorly constrained. + +### `x/bank` Case Study + +Currently the `x/bank` keeper gives pretty much unrestricted access to any module which references it. For instance, the +`SetBalance` method allows the caller to set the balance of any account to anything, bypassing even proper tracking of supply. + +There appears to have been some later attempts to implement some semblance of OCAPs using module-level minting, staking +and burning permissions. These permissions allow a module to mint, burn or delegate tokens with reference to the module’s +own account. These permissions are actually stored as a `[]string` array on the `ModuleAccount` type in state. + +However, these permissions don’t really do much. They control what modules can be referenced in the `MintCoins`, +`BurnCoins` and `DelegateCoins***` methods, but for one there is no unique object capability token that controls access — +just a simple string. So the `x/upgrade` module could mint tokens for the `x/staking` module simple by calling +`MintCoins(“staking”)`. Furthermore, all modules which have access to these keeper methods, also have access to +`SetBalance` negating any other attempt at OCAPs and breaking even basic object-oriented encapsulation. + +## Decision + +Based on [ADR-021](./adr-021-protobuf-query-encoding.md) and [ADR-031](./adr-031-msg-service.md), we introduce the +Inter-Module Communication framework for secure module authorization and OCAPs. +When implemented, this could also serve as an alternative to the existing paradigm of passing keepers between +modules. The approach outlined here-in is intended to form the basis of a Cosmos SDK v1.0 that provides the necessary +stability and encapsulation guarantees that allow a thriving module ecosystem to emerge. + +Of particular note — the decision is to _enable_ this functionality for modules to adopt at their own discretion. +Proposals to migrate existing modules to this new paradigm will have to be a separate conversation, potentially +addressed as amendments to this ADR. + +### New "Keeper" Paradigm + +In [ADR 021](./adr-021-protobuf-query-encoding.md), a mechanism for using protobuf service definitions to define queriers +was introduced and in [ADR 31](./adr-031-msg-service.md), a mechanism for using protobuf service to define `Msg`s was added. +Protobuf service definitions generate two golang interfaces representing the client and server sides of a service plus +some helper code. Here is a minimal example for the bank `cosmos.bank.Msg/Send` message type: + +```go +package bank + +type MsgClient interface { + Send(context.Context, *MsgSend, opts ...grpc.CallOption) (*MsgSendResponse, error) +} + +type MsgServer interface { + Send(context.Context, *MsgSend) (*MsgSendResponse, error) +} +``` + +[ADR 021](./adr-021-protobuf-query-encoding.md) and [ADR 31](./adr-031-msg-service.md) specifies how modules can implement the generated `QueryServer` +and `MsgServer` interfaces as replacements for the legacy queriers and `Msg` handlers respectively. + +In this ADR we explain how modules can make queries and send `Msg`s to other modules using the generated `QueryClient` +and `MsgClient` interfaces and propose this mechanism as a replacement for the existing `Keeper` paradigm. To be clear, +this ADR does not necessitate the creation of new protobuf definitions or services. Rather, it leverages the same proto +based service interfaces already used by clients for inter-module communication. + +Using this `QueryClient`/`MsgClient` approach has the following key benefits over exposing keepers to external modules: + +1. Protobuf types are checked for breaking changes using [buf](https://buf.build/docs/breaking-overview) and because of +the way protobuf is designed this will give us strong backwards compatibility guarantees while allowing for forward +evolution. +2. The separation between the client and server interfaces will allow us to insert permission checking code in between +the two which checks if one module is authorized to send the specified `Msg` to the other module providing a proper +object capability system (see below). +3. The router for inter-module communication gives us a convenient place to handle rollback of transactions, +enabling atomicy of operations ([currently a problem](https://github.com/cosmos/cosmos-sdk/issues/8030)). Any failure within a module-to-module call would result in a failure of the entire +transaction + +This mechanism has the added benefits of: + +* reducing boilerplate through code generation, and +* allowing for modules in other languages either via a VM like CosmWasm or sub-processes using gRPC + +### Inter-module Communication + +To use the `Client` generated by the protobuf compiler we need a `grpc.ClientConn` [interface](https://github.com/grpc/grpc-go/blob/v1.49.x/clientconn.go#L441-L450) +implementation. For this we introduce +a new type, `ModuleKey`, which implements the `grpc.ClientConn` interface. `ModuleKey` can be thought of as the "private +key" corresponding to a module account, where authentication is provided through use of a special `Invoker()` function, +described in more detail below. + +Blockchain users (external clients) use their account's private key to sign transactions containing `Msg`s where they are listed as signers (each +message specifies required signers with `Msg.GetSigner`). The authentication checks is performed by `AnteHandler`. + +Here, we extend this process, by allowing modules to be identified in `Msg.GetSigners`. When a module wants to trigger the execution a `Msg` in another module, +its `ModuleKey` acts as the sender (through the `ClientConn` interface we describe below) and is set as a sole "signer". It's worth to note +that we don't use any cryptographic signature in this case. +For example, module `A` could use its `A.ModuleKey` to create `MsgSend` object for `/cosmos.bank.Msg/Send` transaction. `MsgSend` validation +will assure that the `from` account (`A.ModuleKey` in this case) is the signer. + +Here's an example of a hypothetical module `foo` interacting with `x/bank`: + +```go +package foo + + +type FooMsgServer { + // ... + + bankQuery bank.QueryClient + bankMsg bank.MsgClient +} + +func NewFooMsgServer(moduleKey RootModuleKey, ...) FooMsgServer { + // ... + + return FooMsgServer { + // ... + modouleKey: moduleKey, + bankQuery: bank.NewQueryClient(moduleKey), + bankMsg: bank.NewMsgClient(moduleKey), + } +} + +func (foo *FooMsgServer) Bar(ctx context.Context, req *MsgBarRequest) (*MsgBarResponse, error) { + balance, err := foo.bankQuery.Balance(&bank.QueryBalanceRequest{Address: fooMsgServer.moduleKey.Address(), Denom: "foo"}) + + ... + + res, err := foo.bankMsg.Send(ctx, &bank.MsgSendRequest{FromAddress: fooMsgServer.moduleKey.Address(), ...}) + + ... +} +``` + +This design is also intended to be extensible to cover use cases of more fine grained permissioning like minting by +denom prefix being restricted to certain modules (as discussed in +[#7459](https://github.com/cosmos/cosmos-sdk/pull/7459#discussion_r529545528)). + +### `ModuleKey`s and `ModuleID`s + +A `ModuleKey` can be thought of as a "private key" for a module account and a `ModuleID` can be thought of as the +corresponding "public key". From the [ADR 028](./adr-028-public-key-addresses.md), modules can have both a root module account and any number of sub-accounts +or derived accounts that can be used for different pools (ex. staking pools) or managed accounts (ex. group +accounts). We can also think of module sub-accounts as similar to derived keys - there is a root key and then some +derivation path. `ModuleID` is a simple struct which contains the module name and optional "derivation" path, +and forms its address based on the `AddressHash` method from [the ADR-028](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-028-public-key-addresses.md): + +```go +type ModuleID struct { + ModuleName string + Path []byte +} + +func (key ModuleID) Address() []byte { + return AddressHash(key.ModuleName, key.Path) +} +``` + +In addition to being able to generate a `ModuleID` and address, a `ModuleKey` contains a special function called +`Invoker` which is the key to safe inter-module access. The `Invoker` creates an `InvokeFn` closure which is used as an `Invoke` method in +the `grpc.ClientConn` interface and under the hood is able to route messages to the appropriate `Msg` and `Query` handlers +performing appropriate security checks on `Msg`s. This allows for even safer inter-module access than keeper's whose +private member variables could be manipulated through reflection. Golang does not support reflection on a function +closure's captured variables and direct manipulation of memory would be needed for a truly malicious module to bypass +the `ModuleKey` security. + +The two `ModuleKey` types are `RootModuleKey` and `DerivedModuleKey`: + +```go +type Invoker func(callInfo CallInfo) func(ctx context.Context, request, response interface{}, opts ...interface{}) error + +type CallInfo { + Method string + Caller ModuleID +} + +type RootModuleKey struct { + moduleName string + invoker Invoker +} + +func (rm RootModuleKey) Derive(path []byte) DerivedModuleKey { /* ... */} + +type DerivedModuleKey struct { + moduleName string + path []byte + invoker Invoker +} +``` + +A module can get access to a `DerivedModuleKey`, using the `Derive(path []byte)` method on `RootModuleKey` and then +would use this key to authenticate `Msg`s from a sub-account. Ex: + +```go +package foo + +func (fooMsgServer *MsgServer) Bar(ctx context.Context, req *MsgBar) (*MsgBarResponse, error) { + derivedKey := fooMsgServer.moduleKey.Derive(req.SomePath) + bankMsgClient := bank.NewMsgClient(derivedKey) + res, err := bankMsgClient.Balance(ctx, &bank.MsgSend{FromAddress: derivedKey.Address(), ...}) + ... +} +``` + +In this way, a module can gain permissioned access to a root account and any number of sub-accounts and send +authenticated `Msg`s from these accounts. The `Invoker` `callInfo.Caller` parameter is used under the hood to +distinguish between different module accounts, but either way the function returned by `Invoker` only allows `Msg`s +from either the root or a derived module account to pass through. + +Note that `Invoker` itself returns a function closure based on the `CallInfo` passed in. This will allow client implementations +in the future that cache the invoke function for each method type avoiding the overhead of hash table lookup. +This would reduce the performance overhead of this inter-module communication method to the bare minimum required for +checking permissions. + +To re-iterate, the closure only allows access to authorized calls. There is no access to anything else regardless of any +name impersonation. + +Below is a rough sketch of the implementation of `grpc.ClientConn.Invoke` for `RootModuleKey`: + +```go +func (key RootModuleKey) Invoke(ctx context.Context, method string, args, reply interface{}, opts ...grpc.CallOption) error { + f := key.invoker(CallInfo {Method: method, Caller: ModuleID {ModuleName: key.moduleName}}) + return f(ctx, args, reply) +} +``` + +### `AppModule` Wiring and Requirements + +In [ADR 031](./adr-031-msg-service.md), the `AppModule.RegisterService(Configurator)` method was introduced. To support +inter-module communication, we extend the `Configurator` interface to pass in the `ModuleKey` and to allow modules to +specify their dependencies on other modules using `RequireServer()`: + +```go +type Configurator interface { + MsgServer() grpc.Server + QueryServer() grpc.Server + + ModuleKey() ModuleKey + RequireServer(msgServer interface{}) +} +``` + +The `ModuleKey` is passed to modules in the `RegisterService` method itself so that `RegisterServices` serves as a single +entry point for configuring module services. This is intended to also have the side-effect of greatly reducing boilerplate in +`app.go`. For now, `ModuleKey`s will be created based on `AppModule.Name()`, but a more flexible system may be +introduced in the future. The `ModuleManager` will handle creation of module accounts behind the scenes. + +Because modules do not get direct access to each other anymore, modules may have unfulfilled dependencies. To make sure +that module dependencies are resolved at startup, the `Configurator.RequireServer` method should be added. The `ModuleManager` +will make sure that all dependencies declared with `RequireServer` can be resolved before the app starts. An example +module `foo` could declare it's dependency on `x/bank` like this: + +```go +package foo + +func (am AppModule) RegisterServices(cfg Configurator) { + cfg.RequireServer((*bank.QueryServer)(nil)) + cfg.RequireServer((*bank.MsgServer)(nil)) +} +``` + +### Security Considerations + +In addition to checking for `ModuleKey` permissions, a few additional security precautions will need to be taken by +the underlying router infrastructure. + +#### Recursion and Re-entry + +Recursive or re-entrant method invocations pose a potential security threat. This can be a problem if Module A +calls Module B and Module B calls module A again in the same call. + +One basic way for the router system to deal with this is to maintain a call stack which prevents a module from +being referenced more than once in the call stack so that there is no re-entry. A `map[string]interface{}` table +in the router could be used to perform this security check. + +#### Queries + +Queries in Cosmos SDK are generally un-permissioned so allowing one module to query another module should not pose +any major security threats assuming basic precautions are taken. The basic precaution that the router system will +need to take is making sure that the `sdk.Context` passed to query methods does not allow writing to the store. This +can be done for now with a `CacheMultiStore` as is currently done for `BaseApp` queries. + +### Internal Methods + +In many cases, we may wish for modules to call methods on other modules which are not exposed to clients at all. For this +purpose, we add the `InternalServer` method to `Configurator`: + +```go +type Configurator interface { + MsgServer() grpc.Server + QueryServer() grpc.Server + InternalServer() grpc.Server +} +``` + +As an example, x/slashing's Slash must call x/staking's Slash, but we don't want to expose x/staking's Slash to end users +and clients. + +Internal protobuf services will be defined in a corresponding `internal.proto` file in the given module's +proto package. + +Services registered against `InternalServer` will be callable from other modules but not by external clients. + +An alternative solution to internal-only methods could involve hooks / plugins as discussed [here](https://github.com/cosmos/cosmos-sdk/pull/7459#issuecomment-733807753). +A more detailed evaluation of a hooks / plugin system will be addressed later in follow-ups to this ADR or as a separate +ADR. + +### Authorization + +By default, the inter-module router requires that messages are sent by the first signer returned by `GetSigners`. The +inter-module router should also accept authorization middleware such as that provided by [ADR 030](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md). +This middleware will allow accounts to otherwise specific module accounts to perform actions on their behalf. +Authorization middleware should take into account the need to grant certain modules effectively "admin" privileges to +other modules. This will be addressed in separate ADRs or updates to this ADR. + +### Future Work + +Other future improvements may include: + +* custom code generation that: + * simplifies interfaces (ex. generates code with `sdk.Context` instead of `context.Context`) + * optimizes inter-module calls - for instance caching resolved methods after first invocation +* combining `StoreKey`s and `ModuleKey`s into a single interface so that modules have a single OCAPs handle +* code generation which makes inter-module communication more performant +* decoupling `ModuleKey` creation from `AppModule.Name()` so that app's can override root module account names +* inter-module hooks and plugins + +## Alternatives + +### MsgServices vs `x/capability` + +The `x/capability` module does provide a proper object-capability implementation that can be used by any module in the +Cosmos SDK and could even be used for inter-module OCAPs as described in [\#5931](https://github.com/cosmos/cosmos-sdk/issues/5931). + +The advantages of the approach described in this ADR are mostly around how it integrates with other parts of the Cosmos SDK, +specifically: + +* protobuf so that: + * code generation of interfaces can be leveraged for a better dev UX + * module interfaces are versioned and checked for breakage using [buf](https://docs.buf.build/breaking-overview) +* sub-module accounts as per ADR 028 +* the general `Msg` passing paradigm and the way signers are specified by `GetSigners` + +Also, this is a complete replacement for keepers and could be applied to _all_ inter-module communication whereas the +`x/capability` approach in #5931 would need to be applied method by method. + +## Consequences + +### Backwards Compatibility + +This ADR is intended to provide a pathway to a scenario where there is greater long term compatibility between modules. +In the short-term, this will likely result in breaking certain `Keeper` interfaces which are too permissive and/or +replacing `Keeper` interfaces altogether. + +### Positive + +* an alternative to keepers which can more easily lead to stable inter-module interfaces +* proper inter-module OCAPs +* improved module developer DevX, as commented on by several particpants on + [Architecture Review Call, Dec 3](https://hackmd.io/E0wxxOvRQ5qVmTf6N_k84Q) +* lays the groundwork for what can be a greatly simplified `app.go` +* router can be setup to enforce atomic transactions for module-to-module calls + +### Negative + +* modules which adopt this will need significant refactoring + +### Neutral + +## Test Cases [optional] + +## References + +* [ADR 021](./adr-021-protobuf-query-encoding.md) +* [ADR 031](./adr-031-msg-service.md) +* [ADR 028](./adr-028-public-key-addresses.md) +* [ADR 030 draft](https://github.com/cosmos/cosmos-sdk/pull/7105) +* [Object-Capability Model](https://docs.cosmos.network/main/learn/advanced/ocap#ocaps-in-practice) diff --git a/versioned_docs/version-0.52/build/architecture/adr-034-account-rekeying.md b/versioned_docs/version-0.52/build/architecture/adr-034-account-rekeying.md new file mode 100644 index 000000000..111a162ad --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-034-account-rekeying.md @@ -0,0 +1,76 @@ +# ADR 034: Account Rekeying + +## Changelog + +* 30-09-2020: Initial Draft + +## Status + +PROPOSED + +## Abstract + +Account rekeying is a process hat allows an account to replace its authentication pubkey with a new one. + +## Context + +Currently, in the Cosmos SDK, the address of an auth `BaseAccount` is based on the hash of the public key. Once an account is created, the public key for the account is set in stone, and cannot be changed. This can be a problem for users, as key rotation is a useful security practice, but is not possible currently. Furthermore, as multisigs are a type of pubkey, once a multisig for an account is set, it can not be updated. This is problematic, as multisigs are often used by organizations or companies, who may need to change their set of multisig signers for internal reasons. + +Transferring all the assets of an account to a new account with the updated pubkey is not sufficient, because some "engagements" of an account are not easily transferable. For example, in staking, to transfer bonded Atoms, an account would have to unbond all delegations and wait the three week unbonding period. Even more significantly, for validator operators, ownership over a validator is not transferrable at all, meaning that the operator key for a validator can never be updated, leading to poor operational security for validators. + +## Decision + +We propose the addition of a new feature to `x/auth` that allows accounts to update the public key associated with their account, while keeping the address the same. + +This is possible because the Cosmos SDK `BaseAccount` stores the public key for an account in state, instead of making the assumption that the public key is included in the transaction (whether explicitly or implicitly through the signature) as in other blockchains such as Bitcoin and Ethereum. Because the public key is stored on chain, it is okay for the public key to not hash to the address of an account, as the address is not pertinent to the signature checking process. + +To build this system, we design a new Msg type as follows: + +```protobuf +service Msg { + rpc ChangePubKey(MsgChangePubKey) returns (MsgChangePubKeyResponse); +} + +message MsgChangePubKey { + string address = 1; + google.protobuf.Any pub_key = 2; +} + +message MsgChangePubKeyResponse {} +``` + +The MsgChangePubKey transaction needs to be signed by the existing pubkey in state. + +Once, approved, the handler for this message type, which takes in the AccountKeeper, will update the in-state pubkey for the account and replace it with the pubkey from the Msg. + +An account that has had its pubkey changed cannot be automatically pruned from state. This is because if pruned, the original pubkey of the account would be needed to recreate the same address, but the owner of the address may not have the original pubkey anymore. Currently, we do not automatically prune any accounts anyways, but we would like to keep this option open the road (this is the purpose of account numbers). To resolve this, we charge an additional gas fee for this operation to compensate for this externality (this bound gas amount is configured as parameter `PubKeyChangeCost`). The bonus gas is charged inside the handler, using the `ConsumeGas` function. Furthermore, in the future, we can allow accounts that have rekeyed manually prune themselves using a new Msg type such as `MsgDeleteAccount`. Manually pruning accounts can give a gas refund as an incentive for performing the action. + +```go + amount := ak.GetParams(ctx).PubKeyChangeCost + ctx.GasMeter().ConsumeGas(amount, "pubkey change fee") +``` + +Every time a key for an address is changed, we will store a log of this change in the state of the chain, thus creating a stack of all previous keys for an address and the time intervals for which they were active. This allows dapps and clients to easily query past keys for an account which may be useful for features such as verifying timestamped off-chain signed messages. + +## Consequences + +### Positive + +* Will allow users and validator operators to employ better operational security practices with key rotation. +* Will allow organizations or groups to easily change and add/remove multisig signers. + +### Negative + +Breaks the current assumed relationship between address and pubkeys as H(pubkey) = address. This has a couple of consequences. + +* This makes wallets that support this feature more complicated. For example, if an address on chain was updated, the corresponding key in the CLI wallet also needs to be updated. +* Cannot automatically prune accounts with 0 balance that have had their pubkey changed. + +### Neutral + +* While the purpose of this is intended to allow the owner of an account to update to a new pubkey they own, this could technically also be used to transfer ownership of an account to a new owner. For example, this could be use used to sell a staked position without unbonding or an account that has vesting tokens. However, the friction of this is very high as this would essentially have to be done as a very specific OTC trade. Furthermore, additional constraints could be added to prevent accounts with Vesting tokens to use this feature. +* Will require that PubKeys for an account are included in the genesis exports. + +## References + +* https://algorandtechnologies.com/news/announcing-rekeying diff --git a/versioned_docs/version-0.52/build/architecture/adr-035-rosetta-api-support.md b/versioned_docs/version-0.52/build/architecture/adr-035-rosetta-api-support.md new file mode 100644 index 000000000..01a81048b --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-035-rosetta-api-support.md @@ -0,0 +1,211 @@ +# ADR 035: Rosetta API Support + +## Authors + +* Jonathan Gimeno (@jgimeno) +* David Grierson (@senormonito) +* Alessio Treglia (@alessio) +* Frojdy Dymylja (@fdymylja) + +## Changelog + +* 2021-05-12: the external library [cosmos-rosetta-gateway](https://github.com/tendermint/cosmos-rosetta-gateway) has been moved within the Cosmos SDK. + +## Context + +[Rosetta API](https://www.rosetta-api.org/) is an open-source specification and set of tools developed by Coinbase to +standardise blockchain interactions. + +Through the use of a standard API for integrating blockchain applications it will + +* Be easier for a user to interact with a given blockchain +* Allow exchanges to integrate new blockchains quickly and easily +* Enable application developers to build cross-blockchain applications such as block explorers, wallets and dApps at + considerably lower cost and effort. + +## Decision + +It is clear that adding Rosetta API support to the Cosmos SDK will bring value to all the developers and +Cosmos SDK based chains in the ecosystem. How it is implemented is key. + +The driving principles of the proposed design are: + +1. **Extensibility:** it must be as riskless and painless as possible for application developers to set-up network + configurations to expose Rosetta API-compliant services. +2. **Long term support:** This proposal aims to provide support for all the supported Cosmos SDK release series. +3. **Cost-efficiency:** Backporting changes to Rosetta API specifications from `master` to the various stable + branches of Cosmos SDK is a cost that needs to be reduced. + +We will achieve these delivering on these principles by the following: + +1. There will be a package `rosetta/lib` + for the implementation of the core Rosetta API features, particularly: + a. The types and interfaces (`Client`, `OfflineClient`...), this separates design from implementation detail. + b. The `Server` functionality as this is independent of the Cosmos SDK version. + c. The `Online/OfflineNetwork`, which is not exported, and implements the rosetta API using the `Client` interface to query the node, build tx and so on. + d. The `errors` package to extend rosetta errors. +2. Due to differences between the Cosmos release series, each series will have its own specific implementation of `Client` interface. +3. There will be two options for starting an API service in applications: + a. API shares the application process + b. API-specific process. + +## Architecture + +### The External Repo + +As section will describe the proposed external library, including the service implementation, plus the defined types and interfaces. + +#### Server + +`Server` is a simple `struct` that is started and listens to the port specified in the settings. This is meant to be used across all the Cosmos SDK versions that are actively supported. + +The constructor follows: + +`func NewServer(settings Settings) (Server, error)` + +`Settings`, which are used to construct a new server, are the following: + +```go +// Settings define the rosetta server settings +type Settings struct { + // Network contains the information regarding the network + Network *types.NetworkIdentifier + // Client is the online API handler + Client crgtypes.Client + // Listen is the address the handler will listen at + Listen string + // Offline defines if the rosetta service should be exposed in offline mode + Offline bool + // Retries is the number of readiness checks that will be attempted when instantiating the handler + // valid only for online API + Retries int + // RetryWait is the time that will be waited between retries + RetryWait time.Duration +} +``` + +#### Types + +Package types uses a mixture of rosetta types and custom defined type wrappers, that the client must parse and return while executing operations. + +##### Interfaces + +Every SDK version uses a different format to connect (rpc, gRPC, etc), query and build transactions, we have abstracted this in what is the `Client` interface. +The client uses rosetta types, whilst the `Online/OfflineNetwork` takes care of returning correctly parsed rosetta responses and errors. + +Each Cosmos SDK release series will have their own `Client` implementations. +Developers can implement their own custom `Client`s as required. + +```go +// Client defines the API the client implementation should provide. +type Client interface { + // Needed if the client needs to perform some action before connecting. + Bootstrap() error + // Ready checks if the servicer constraints for queries are satisfied + // for example the node might still not be ready, it's useful in process + // when the rosetta instance might come up before the node itself + // the servicer must return nil if the node is ready + Ready() error + + // Data API + + // Balances fetches the balance of the given address + // if height is not nil, then the balance will be displayed + // at the provided height, otherwise last block balance will be returned + Balances(ctx context.Context, addr string, height *int64) ([]*types.Amount, error) + // BlockByHashAlt gets a block and its transaction at the provided height + BlockByHash(ctx context.Context, hash string) (BlockResponse, error) + // BlockByHeightAlt gets a block given its height, if height is nil then last block is returned + BlockByHeight(ctx context.Context, height *int64) (BlockResponse, error) + // BlockTransactionsByHash gets the block, parent block and transactions + // given the block hash. + BlockTransactionsByHash(ctx context.Context, hash string) (BlockTransactionsResponse, error) + // BlockTransactionsByHash gets the block, parent block and transactions + // given the block hash. + BlockTransactionsByHeight(ctx context.Context, height *int64) (BlockTransactionsResponse, error) + // GetTx gets a transaction given its hash + GetTx(ctx context.Context, hash string) (*types.Transaction, error) + // GetUnconfirmedTx gets an unconfirmed Tx given its hash + // NOTE(fdymylja): NOT IMPLEMENTED YET! + GetUnconfirmedTx(ctx context.Context, hash string) (*types.Transaction, error) + // Mempool returns the list of the current non confirmed transactions + Mempool(ctx context.Context) ([]*types.TransactionIdentifier, error) + // Peers gets the peers currently connected to the node + Peers(ctx context.Context) ([]*types.Peer, error) + // Status returns the node status, such as sync data, version etc + Status(ctx context.Context) (*types.SyncStatus, error) + + // Construction API + + // PostTx posts txBytes to the node and returns the transaction identifier plus metadata related + // to the transaction itself. + PostTx(txBytes []byte) (res *types.TransactionIdentifier, meta map[string]interface{}, err error) + // ConstructionMetadataFromOptions + ConstructionMetadataFromOptions(ctx context.Context, options map[string]interface{}) (meta map[string]interface{}, err error) + OfflineClient +} + +// OfflineClient defines the functionalities supported without having access to the node +type OfflineClient interface { + NetworkInformationProvider + // SignedTx returns the signed transaction given the tx bytes (msgs) plus the signatures + SignedTx(ctx context.Context, txBytes []byte, sigs []*types.Signature) (signedTxBytes []byte, err error) + // TxOperationsAndSignersAccountIdentifiers returns the operations related to a transaction and the account + // identifiers if the transaction is signed + TxOperationsAndSignersAccountIdentifiers(signed bool, hexBytes []byte) (ops []*types.Operation, signers []*types.AccountIdentifier, err error) + // ConstructionPayload returns the construction payload given the request + ConstructionPayload(ctx context.Context, req *types.ConstructionPayloadsRequest) (resp *types.ConstructionPayloadsResponse, err error) + // PreprocessOperationsToOptions returns the options given the preprocess operations + PreprocessOperationsToOptions(ctx context.Context, req *types.ConstructionPreprocessRequest) (options map[string]interface{}, err error) + // AccountIdentifierFromPublicKey returns the account identifier given the public key + AccountIdentifierFromPublicKey(pubKey *types.PublicKey) (*types.AccountIdentifier, error) +} +``` + +### 2. Cosmos SDK Implementation + +The Cosmos SDK implementation, based on version, takes care of satisfying the `Client` interface. +In Stargate, Launchpad and 0.37, we have introduced the concept of rosetta.Msg, this message is not in the shared repository as the sdk.Msg type differs between Cosmos SDK versions. + +The rosetta.Msg interface follows: + +```go +// Msg represents a cosmos-sdk message that can be converted from and to a rosetta operation. +type Msg interface { + sdk.Msg + ToOperations(withStatus, hasError bool) []*types.Operation + FromOperations(ops []*types.Operation) (sdk.Msg, error) +} +``` + +Hence developers who want to extend the rosetta set of supported operations just need to extend their module's sdk.Msgs with the `ToOperations` and `FromOperations` methods. + +### 3. API service invocation + +As stated at the start, application developers will have two methods for invocation of the Rosetta API service: + +1. Shared process for both application and API +2. Standalone API service + +#### Shared Process (Only Stargate) + +Rosetta API service could run within the same execution process as the application. This would be enabled via app.toml settings, and if gRPC is not enabled the rosetta instance would be spinned in offline mode (tx building capabilities only). + +#### Separate API service + +Client application developers can write a new command to launch a Rosetta API server as a separate process too, using the rosetta command contained in the `/server/rosetta` package. Construction of the command depends on Cosmos SDK version. Examples can be found inside `simd` for stargate, and `contrib/rosetta/simapp` for other release series. + +## Status + +Proposed + +## Consequences + +### Positive + +* Out-of-the-box Rosetta API support within Cosmos SDK. +* Blockchain interface standardisation + +## References + +* https://www.rosetta-api.org/ diff --git a/versioned_docs/version-0.52/build/architecture/adr-036-arbitrary-signature.md b/versioned_docs/version-0.52/build/architecture/adr-036-arbitrary-signature.md new file mode 100644 index 000000000..fe9dada54 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-036-arbitrary-signature.md @@ -0,0 +1,132 @@ +# ADR 036: Arbitrary Message Signature Specification + +## Changelog + +* 28/10/2020 - Initial draft + +## Authors + +* Antoine Herzog (@antoineherzog) +* Zaki Manian (@zmanian) +* Aleksandr Bezobchuk (alexanderbez) [1] +* Frojdi Dymylja (@fdymylja) + +## Status + +Draft + +## Abstract + +Currently, in the Cosmos SDK, there is no convention to sign arbitrary message like on Ethereum. We propose with this specification, for Cosmos SDK ecosystem, a way to sign and validate off-chain arbitrary messages. + +This specification serves the purpose of covering every use case, this means that cosmos-sdk applications developers decide how to serialize and represent `Data` to users. + +## Context + +Having the ability to sign messages off-chain has proven to be a fundamental aspect of nearly any blockchain. The notion of signing messages off-chain has many added benefits such as saving on computational costs and reducing transaction throughput and overhead. Within the context of the Cosmos, some of the major applications of signing such data includes, but is not limited to, providing a cryptographic secure and verifiable means of proving validator identity and possibly associating it with some other framework or organization. In addition, having the ability to sign Cosmos messages with a Ledger or similar HSM device. + +Further context and use cases can be found in the references links. + +## Decision + +The aim is being able to sign arbitrary messages, even using Ledger or similar HSM devices. + +As a result signed messages should look roughly like Cosmos SDK messages but **must not** be a valid on-chain transaction. `chain-id`, `account_number` and `sequence` can all be assigned invalid values. + +Cosmos SDK 0.40 also introduces a concept of “auth_info” this can specify SIGN_MODES. + +A spec should include an `auth_info` that supports SIGN_MODE_DIRECT and SIGN_MODE_LEGACY_AMINO. + +Create the `offchain` proto definitions, we extend the auth module with `offchain` package to offer functionalities to verify and sign offline messages. + +An offchain transaction follows these rules: + +* the memo must be empty +* nonce, sequence number must be equal to 0 +* chain-id must be equal to “” +* fee gas must be equal to 0 +* fee amount must be an empty array + +Verification of an offchain transaction follows the same rules as an onchain one, except for the spec differences highlighted above. + +The first message added to the `offchain` package is `MsgSignData`. + +`MsgSignData` allows developers to sign arbitrary bytes valid offchain only. Where `Signer` is the account address of the signer. `Data` is arbitrary bytes which can represent `text`, `files`, `object`s. It's applications developers decision how `Data` should be deserialized, serialized and the object it can represent in their context. + +It's applications developers decision how `Data` should be treated, by treated we mean the serialization and deserialization process and the Object `Data` should represent. + +Proto definition: + +```protobuf +// MsgSignData defines an arbitrary, general-purpose, off-chain message +message MsgSignData { + // Signer is the sdk.AccAddress of the message signer + bytes Signer = 1 [(gogoproto.jsontag) = "signer", (gogoproto.casttype) = "github.com/cosmos/cosmos-sdk/types.AccAddress"]; + // Data represents the raw bytes of the content that is signed (text, json, etc) + bytes Data = 2 [(gogoproto.jsontag) = "data"]; +} +``` + +Signed MsgSignData json example: + +```json +{ + "type": "cosmos-sdk/StdTx", + "value": { + "msg": [ + { + "type": "sign/MsgSignData", + "value": { + "signer": "cosmos1hftz5ugqmpg9243xeegsqqav62f8hnywsjr4xr", + "data": "cmFuZG9t" + } + } + ], + "fee": { + "amount": [], + "gas": "0" + }, + "signatures": [ + { + "pub_key": { + "type": "tendermint/PubKeySecp256k1", + "value": "AqnDSiRoFmTPfq97xxEb2VkQ/Hm28cPsqsZm9jEVsYK9" + }, + "signature": "8y8i34qJakkjse9pOD2De+dnlc4KvFgh0wQpes4eydN66D9kv7cmCEouRrkka9tlW9cAkIL52ErB+6ye7X5aEg==" + } + ], + "memo": "" + } +} +``` + +## Consequences + +There is a specification on how messages, that are not meant to be broadcast to a live chain, should be formed. + +### Backwards Compatibility + +Backwards compatibility is maintained as this is a new message spec definition. + +### Positive + +* A common format that can be used by multiple applications to sign and verify off-chain messages. +* The specification is primitive which means it can cover every use case without limiting what is possible to fit inside it. +* It gives room for other off-chain messages specifications that aim to target more specific and common use cases such as off-chain-based authN/authZ layers [2]. + +### Negative + +* Current proposal requires a fixed relationship between an account address and a public key. +* Doesn't work with multisig accounts. + +## Further discussion + +* Regarding security in `MsgSignData`, the developer using `MsgSignData` is in charge of making the content laying in `Data` non-replayable when, and if, needed. +* the offchain package will be further extended with extra messages that target specific use cases such as, but not limited to, authentication in applications, payment channels, L2 solutions in general. + +## References + +1. https://github.com/cosmos/ics/pull/33 +2. https://github.com/cosmos/cosmos-sdk/pull/7727#discussion_r515668204 +3. https://github.com/cosmos/cosmos-sdk/pull/7727#issuecomment-722478477 +4. https://github.com/cosmos/cosmos-sdk/pull/7727#issuecomment-721062923 diff --git a/versioned_docs/version-0.52/build/architecture/adr-037-gov-split-vote.md b/versioned_docs/version-0.52/build/architecture/adr-037-gov-split-vote.md new file mode 100644 index 000000000..c369ad8b1 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-037-gov-split-vote.md @@ -0,0 +1,111 @@ +# ADR 037: Governance split votes + +## Changelog + +* 2020/10/28: Initial draft + +## Status + +Accepted + +## Abstract + +This ADR defines a modification to the governance module that would allow a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No. + +## Context + +Currently, an address can cast a vote with only one options (Yes/No/Abstain/NoWithVeto) and use their full voting power behind that choice. + +However, often times the entity owning that address might not be a single individual. For example, a company might have different stakeholders who want to vote differently, and so it makes sense to allow them to split their voting power. Another example use case is exchanges. Many centralized exchanges often stake a portion of their users' tokens in their custody. Currently, it is not possible for them to do "passthrough voting" and giving their users voting rights over their tokens. However, with this system, exchanges can poll their users for voting preferences, and then vote on-chain proportionally to the results of the poll. + +## Decision + +We modify the vote structs to be + +```go +type WeightedVoteOption struct { + Option string + Weight sdk.Dec +} + +type Vote struct { + ProposalID int64 + Voter sdk.Address + Options []WeightedVoteOption +} +``` + +And for backwards compatibility, we introduce `MsgVoteWeighted` while keeping `MsgVote`. + +```go +type MsgVote struct { + ProposalID int64 + Voter sdk.Address + Option Option +} + +type MsgVoteWeighted struct { + ProposalID int64 + Voter sdk.Address + Options []WeightedVoteOption +} +``` + +The `ValidateBasic` of a `MsgVoteWeighted` struct would require that + +1. The sum of all the Rates is equal to 1.0 +2. No Option is repeated + +The governance tally function will iterate over all the options in a vote and add to the tally the result of the voter's voting power * the rate for that option. + +```go +tally() { + results := map[types.VoteOption]sdk.Dec + + for _, vote := range votes { + for i, weightedOption := range vote.Options { + results[weightedOption.Option] += getVotingPower(vote.voter) * weightedOption.Weight + } + } +} +``` + +The CLI command for creating a multi-option vote would be as such: + +```shell +simd tx gov vote 1 "yes=0.6,no=0.3,abstain=0.05,no_with_veto=0.05" --from mykey +``` + +To create a single-option vote a user can do either + +```shell +simd tx gov vote 1 "yes=1" --from mykey +``` + +or + +```shell +simd tx gov vote 1 yes --from mykey +``` + +to maintain backwards compatibility. + +## Consequences + +### Backwards Compatibility + +* Previous VoteMsg types will remain the same and so clients will not have to update their procedure unless they want to support the WeightedVoteMsg feature. +* When querying a Vote struct from state, its structure will be different, and so clients wanting to display all voters and their respective votes will have to handle the new format and the fact that a single voter can have split votes. +* The result of querying the tally function should have the same API for clients. + +### Positive + +* Can make the voting process more accurate for addresses representing multiple stakeholders, often some of the largest addresses. + +### Negative + +* Is more complex than simple voting, and so may be harder to explain to users. However, this is mostly mitigated because the feature is opt-in. + +### Neutral + +* Relatively minor change to governance tally function. diff --git a/versioned_docs/version-0.52/build/architecture/adr-038-state-listening.md b/versioned_docs/version-0.52/build/architecture/adr-038-state-listening.md new file mode 100644 index 000000000..db0620c55 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-038-state-listening.md @@ -0,0 +1,724 @@ +# ADR 038: KVStore state listening + +## Changelog + +* 11/23/2020: Initial draft +* 10/06/2022: Introduce plugin system based on hashicorp/go-plugin +* 10/14/2022: + * Add `ListenCommit`, flatten the state writes in a block to a single batch. + * Remove listeners from cache stores, should only listen to `rootmulti.Store`. + * Remove `HaltAppOnDeliveryError()`, the errors are propagated by default, the implementations should return nil if don't want to propagate errors. +* 26/05/2023: Update with ABCI 2.0 + +## Status + +Proposed + +## Abstract + +This ADR defines a set of changes to enable listening to state changes of individual KVStores and exposing these data to consumers. + +## Context + +Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/main/docs/build/building-modules/02-messages-and-queries.md#queries) +which proceed either through Tendermint and the ABCI, or through the gRPC server. +In addition to these request/response queries, it would be beneficial to have a means of listening to state changes as they occur in real time. + +## Decision + +We will modify the `CommitMultiStore` interface and its concrete (`rootmulti`) implementations and introduce a new `listenkv.Store` to allow listening to state changes in underlying KVStores. We don't need to listen to cache stores, because we can't be sure that the writes will be committed eventually, and the writes are duplicated in `rootmulti.Store` eventually, so we should only listen to `rootmulti.Store`. +We will introduce a plugin system for configuring and running streaming services that write these state changes and their surrounding ABCI message context to different destinations. + +### Listening + +In a new file, `store/types/listening.go`, we will create a `MemoryListener` struct for streaming out protobuf encoded KV pairs state changes from a KVStore. +The `MemoryListener` will be used internally by the concrete `rootmulti` implementation to collect state changes from KVStores. + +```go +// MemoryListener listens to the state writes and accumulate the records in memory. +type MemoryListener struct { + stateCache []StoreKVPair +} + +// NewMemoryListener creates a listener that accumulate the state writes in memory. +func NewMemoryListener() *MemoryListener { + return &MemoryListener{} +} + +// OnWrite writes state change events to the internal cache +func (fl *MemoryListener) OnWrite(storeKey StoreKey, key []byte, value []byte, delete bool) { + fl.stateCache = append(fl.stateCache, StoreKVPair{ + StoreKey: storeKey.Name(), + Delete: delete, + Key: key, + Value: value, + }) +} + +// PopStateCache returns the current state caches and set to nil +func (fl *MemoryListener) PopStateCache() []StoreKVPair { + res := fl.stateCache + fl.stateCache = nil + return res +} +``` + +We will also define a protobuf type for the KV pairs. In addition to the key and value fields this message +will include the StoreKey for the originating KVStore so that we can collect information from separate KVStores and determine the source of each KV pair. + +```protobuf +message StoreKVPair { + optional string store_key = 1; // the store key for the KVStore this pair originates from + required bool set = 2; // true indicates a set operation, false indicates a delete operation + required bytes key = 3; + required bytes value = 4; +} +``` + +### ListenKVStore + +We will create a new `Store` type `listenkv.Store` that the `rootmulti` store will use to wrap a `KVStore` to enable state listening. +We will configure the `Store` with a `MemoryListener` which will collect state changes for output to specific destinations. + +```go +// Store implements the KVStore interface with listening enabled. +// Operations are traced on each core KVStore call and written to any of the +// underlying listeners with the proper key and operation permissions +type Store struct { + parent types.KVStore + listener *types.MemoryListener + parentStoreKey types.StoreKey +} + +// NewStore returns a reference to a new traceKVStore given a parent +// KVStore implementation and a buffered writer. +func NewStore(parent types.KVStore, psk types.StoreKey, listener *types.MemoryListener) *Store { + return &Store{parent: parent, listener: listener, parentStoreKey: psk} +} + +// Set implements the KVStore interface. It traces a write operation and +// delegates the Set call to the parent KVStore. +func (s *Store) Set(key []byte, value []byte) { + types.AssertValidKey(key) + s.parent.Set(key, value) + s.listener.OnWrite(s.parentStoreKey, key, value, false) +} + +// Delete implements the KVStore interface. It traces a write operation and +// delegates the Delete call to the parent KVStore. +func (s *Store) Delete(key []byte) { + s.parent.Delete(key) + s.listener.OnWrite(s.parentStoreKey, key, nil, true) +} +``` + +### MultiStore interface updates + +We will update the `CommitMultiStore` interface to allow us to wrap a `Memorylistener` to a specific `KVStore`. +Note that the `MemoryListener` will be attached internally by the concrete `rootmulti` implementation. + +```go +type CommitMultiStore interface { + ... + + // AddListeners adds a listener for the KVStore belonging to the provided StoreKey + AddListeners(keys []StoreKey) + + // PopStateCache returns the accumulated state change messages from MemoryListener + PopStateCache() []StoreKVPair +} +``` + + +### MultiStore implementation updates + +We will adjust the `rootmulti` `GetKVStore` method to wrap the returned `KVStore` with a `listenkv.Store` if listening is turned on for that `Store`. + +```go +func (rs *Store) GetKVStore(key types.StoreKey) types.KVStore { + store := rs.stores[key].(types.KVStore) + + if rs.TracingEnabled() { + store = tracekv.NewStore(store, rs.traceWriter, rs.traceContext) + } + if rs.ListeningEnabled(key) { + store = listenkv.NewStore(store, key, rs.listeners[key]) + } + + return store +} +``` + +We will implement `AddListeners` to manage KVStore listeners internally and implement `PopStateCache` +for a means of retrieving the current state. + +```go +// AddListeners adds state change listener for a specific KVStore +func (rs *Store) AddListeners(keys []types.StoreKey) { + listener := types.NewMemoryListener() + for i := range keys { + rs.listeners[keys[i]] = listener + } +} +``` + +```go +func (rs *Store) PopStateCache() []types.StoreKVPair { + var cache []types.StoreKVPair + for _, ls := range rs.listeners { + cache = append(cache, ls.PopStateCache()...) + } + sort.SliceStable(cache, func(i, j int) bool { + return cache[i].StoreKey < cache[j].StoreKey + }) + return cache +} +``` + +We will also adjust the `rootmulti` `CacheMultiStore` and `CacheMultiStoreWithVersion` methods to enable listening in +the cache layer. + +```go +func (rs *Store) CacheMultiStore() types.CacheMultiStore { + stores := make(map[types.StoreKey]types.CacheWrapper) + for k, v := range rs.stores { + store := v.(types.KVStore) + // Wire the listenkv.Store to allow listeners to observe the writes from the cache store, + // set same listeners on cache store will observe duplicated writes. + if rs.ListeningEnabled(k) { + store = listenkv.NewStore(store, k, rs.listeners[k]) + } + stores[k] = store + } + return cachemulti.NewStore(rs.db, stores, rs.keysByName, rs.traceWriter, rs.getTracingContext()) +} +``` + +```go +func (rs *Store) CacheMultiStoreWithVersion(version int64) (types.CacheMultiStore, error) { + // ... + + // Wire the listenkv.Store to allow listeners to observe the writes from the cache store, + // set same listeners on cache store will observe duplicated writes. + if rs.ListeningEnabled(key) { + cacheStore = listenkv.NewStore(cacheStore, key, rs.listeners[key]) + } + + cachedStores[key] = cacheStore + } + + return cachemulti.NewStore(rs.db, cachedStores, rs.keysByName, rs.traceWriter, rs.getTracingContext()), nil +} +``` + +### Exposing the data + +#### Streaming Service + +We will introduce a new `ABCIListener` interface that plugs into the BaseApp and relays ABCI requests and responses +so that the service can group the state changes with the ABCI requests. + +```go +// baseapp/streaming.go + +// ABCIListener is the interface that we're exposing as a streaming service. +type ABCIListener interface { + // ListenFinalizeBlock updates the streaming service with the latest FinalizeBlock messages + ListenFinalizeBlock(ctx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) error + // ListenCommit updates the steaming service with the latest Commit messages and state changes + ListenCommit(ctx context.Context, res abci.ResponseCommit, changeSet []*StoreKVPair) error +} +``` + +#### BaseApp Registration + +We will add a new method to the `BaseApp` to enable the registration of `StreamingService`s: + + ```go + // SetStreamingService is used to set a streaming service into the BaseApp hooks and load the listeners into the multistore +func (app *BaseApp) SetStreamingService(s ABCIListener) { + // register the StreamingService within the BaseApp + // BaseApp will pass BeginBlock, DeliverTx, and EndBlock requests and responses to the streaming services to update their ABCI context + app.abciListeners = append(app.abciListeners, s) +} +``` + +We will add two new fields to the `BaseApp` struct: + +```go +type BaseApp struct { + + ... + + // abciListenersAsync for determining if abciListeners will run asynchronously. + // When abciListenersAsync=false and stopNodeOnABCIListenerErr=false listeners will run synchronized but will not stop the node. + // When abciListenersAsync=true stopNodeOnABCIListenerErr will be ignored. + abciListenersAsync bool + + // stopNodeOnABCIListenerErr halts the node when ABCI streaming service listening results in an error. + // stopNodeOnABCIListenerErr=true must be paired with abciListenersAsync=false. + stopNodeOnABCIListenerErr bool +} +``` + +#### ABCI Event Hooks + +We will modify the `FinalizeBlock` and `Commit` methods to pass ABCI requests and responses +to any streaming service hooks registered with the `BaseApp`. + +```go +func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) abci.ResponseFinalizeBlock { + + var abciRes abci.ResponseFinalizeBlock + defer func() { + // call the streaming service hook with the FinalizeBlock messages + for _, abciListener := range app.abciListeners { + ctx := app.finalizeState.ctx + blockHeight := ctx.BlockHeight() + if app.abciListenersAsync { + go func(req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) { + if err := app.abciListener.FinalizeBlock(blockHeight, req, res); err != nil { + app.logger.Error("FinalizeBlock listening hook failed", "height", blockHeight, "err", err) + } + }(req, abciRes) + } else { + if err := app.abciListener.ListenFinalizeBlock(blockHeight, req, res); err != nil { + app.logger.Error("FinalizeBlock listening hook failed", "height", blockHeight, "err", err) + if app.stopNodeOnABCIListenerErr { + os.Exit(1) + } + } + } + } + }() + + ... + + return abciRes +} +``` + +```go +func (app *BaseApp) Commit() abci.ResponseCommit { + + ... + + res := abci.ResponseCommit{ + Data: commitID.Hash, + RetainHeight: retainHeight, + } + + // call the streaming service hook with the Commit messages + for _, abciListener := range app.abciListeners { + ctx := app.deliverState.ctx + blockHeight := ctx.BlockHeight() + changeSet := app.cms.PopStateCache() + if app.abciListenersAsync { + go func(res abci.ResponseCommit, changeSet []store.StoreKVPair) { + if err := app.abciListener.ListenCommit(ctx, res, changeSet); err != nil { + app.logger.Error("ListenCommit listening hook failed", "height", blockHeight, "err", err) + } + }(res, changeSet) + } else { + if err := app.abciListener.ListenCommit(ctx, res, changeSet); err != nil { + app.logger.Error("ListenCommit listening hook failed", "height", blockHeight, "err", err) + if app.stopNodeOnABCIListenerErr { + os.Exit(1) + } + } + } + } + + ... + + return res +} +``` + +#### Go Plugin System + +We propose a plugin architecture to load and run `Streaming` plugins and other types of implementations. We will introduce a plugin +system over gRPC that is used to load and run Cosmos-SDK plugins. The plugin system uses [hashicorp/go-plugin](https://github.com/hashicorp/go-plugin). +Each plugin must have a struct that implements the `plugin.Plugin` interface and an `Impl` interface for processing messages over gRPC. +Each plugin must also have a message protocol defined for the gRPC service: + +```go +// streaming/plugins/abci/{plugin_version}/interface.go + +// Handshake is a common handshake that is shared by streaming and host. +// This prevents users from executing bad plugins or executing a plugin +// directory. It is a UX feature, not a security feature. +var Handshake = plugin.HandshakeConfig{ + ProtocolVersion: 1, + MagicCookieKey: "ABCI_LISTENER_PLUGIN", + MagicCookieValue: "ef78114d-7bdf-411c-868f-347c99a78345", +} + +// ListenerPlugin is the base struct for all kinds of go-plugin implementations +// It will be included in interfaces of different Plugins +type ABCIListenerPlugin struct { + // GRPCPlugin must still implement the Plugin interface + plugin.Plugin + // Concrete implementation, written in Go. This is only used for plugins + // that are written in Go. + Impl baseapp.ABCIListener +} + +func (p *ListenerGRPCPlugin) GRPCServer(_ *plugin.GRPCBroker, s *grpc.Server) error { + RegisterABCIListenerServiceServer(s, &GRPCServer{Impl: p.Impl}) + return nil +} + +func (p *ListenerGRPCPlugin) GRPCClient( + _ context.Context, + _ *plugin.GRPCBroker, + c *grpc.ClientConn, +) (interface{}, error) { + return &GRPCClient{client: NewABCIListenerServiceClient(c)}, nil +} +``` + +The `plugin.Plugin` interface has two methods `Client` and `Server`. For our GRPC service these are `GRPCClient` and `GRPCServer` +The `Impl` field holds the concrete implementation of our `baseapp.ABCIListener` interface written in Go. +Note: this is only used for plugin implementations written in Go. + +The advantage of having such a plugin system is that within each plugin authors can define the message protocol in a way that fits their use case. +For example, when state change listening is desired, the `ABCIListener` message protocol can be defined as below (*for illustrative purposes only*). +When state change listening is not desired than `ListenCommit` can be omitted from the protocol. + +```protobuf +syntax = "proto3"; + +... + +message Empty {} + +message ListenFinalizeBlockRequest { + RequestFinalizeBlock req = 1; + ResponseFinalizeBlock res = 2; +} +message ListenCommitRequest { + int64 block_height = 1; + ResponseCommit res = 2; + repeated StoreKVPair changeSet = 3; +} + +// plugin that listens to state changes +service ABCIListenerService { + rpc ListenFinalizeBlock(ListenFinalizeBlockRequest) returns (Empty); + rpc ListenCommit(ListenCommitRequest) returns (Empty); +} +``` + +```protobuf +... +// plugin that doesn't listen to state changes +service ABCIListenerService { + rpc ListenFinalizeBlock(ListenFinalizeBlockRequest) returns (Empty); + rpc ListenCommit(ListenCommitRequest) returns (Empty); +} +``` + +Implementing the service above: + +```go +// streaming/plugins/abci/{plugin_version}/grpc.go + +var ( + _ baseapp.ABCIListener = (*GRPCClient)(nil) +) + +// GRPCClient is an implementation of the ABCIListener and ABCIListenerPlugin interfaces that talks over RPC. +type GRPCClient struct { + client ABCIListenerServiceClient +} + +func (m *GRPCClient) ListenFinalizeBlock(goCtx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) error { + ctx := sdk.UnwrapSDKContext(goCtx) + _, err := m.client.ListenDeliverTx(ctx, &ListenDeliverTxRequest{BlockHeight: ctx.BlockHeight(), Req: req, Res: res}) + return err +} + +func (m *GRPCClient) ListenCommit(goCtx context.Context, res abci.ResponseCommit, changeSet []store.StoreKVPair) error { + ctx := sdk.UnwrapSDKContext(goCtx) + _, err := m.client.ListenCommit(ctx, &ListenCommitRequest{BlockHeight: ctx.BlockHeight(), Res: res, ChangeSet: changeSet}) + return err +} + +// GRPCServer is the gRPC server that GRPCClient talks to. +type GRPCServer struct { + // This is the real implementation + Impl baseapp.ABCIListener +} + +func (m *GRPCServer) ListenFinalizeBlock(ctx context.Context, req *ListenFinalizeBlockRequest) (*Empty, error) { + return &Empty{}, m.Impl.ListenFinalizeBlock(ctx, req.Req, req.Res) +} + +func (m *GRPCServer) ListenCommit(ctx context.Context, req *ListenCommitRequest) (*Empty, error) { + return &Empty{}, m.Impl.ListenCommit(ctx, req.Res, req.ChangeSet) +} + +``` + +And the pre-compiled Go plugin `Impl`(*this is only used for plugins that are written in Go*): + +```go +// streaming/plugins/abci/{plugin_version}/impl/plugin.go + +// Plugins are pre-compiled and loaded by the plugin system + +// ABCIListener is the implementation of the baseapp.ABCIListener interface +type ABCIListener struct{} + +func (m *ABCIListenerPlugin) ListenFinalizeBlock(ctx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) error { + // send data to external system +} + +func (m *ABCIListenerPlugin) ListenCommit(ctx context.Context, res abci.ResponseCommit, changeSet []store.StoreKVPair) error { + // send data to external system +} + +func main() { + plugin.Serve(&plugin.ServeConfig{ + HandshakeConfig: grpc_abci_v1.Handshake, + Plugins: map[string]plugin.Plugin{ + "grpc_plugin_v1": &grpc_abci_v1.ABCIListenerGRPCPlugin{Impl: &ABCIListenerPlugin{}}, + }, + + // A non-nil value here enables gRPC serving for this streaming... + GRPCServer: plugin.DefaultGRPCServer, + }) +} +``` + +We will introduce a plugin loading system that will return `(interface{}, error)`. +This provides the advantage of using versioned plugins where the plugin interface and gRPC protocol change over time. +In addition, it allows for building independent plugin that can expose different parts of the system over gRPC. + +```go +func NewStreamingPlugin(name string, logLevel string) (interface{}, error) { + logger := hclog.New(&hclog.LoggerOptions{ + Output: hclog.DefaultOutput, + Level: toHclogLevel(logLevel), + Name: fmt.Sprintf("plugin.%s", name), + }) + + // We're a host. Start by launching the streaming process. + env := os.Getenv(GetPluginEnvKey(name)) + client := plugin.NewClient(&plugin.ClientConfig{ + HandshakeConfig: HandshakeMap[name], + Plugins: PluginMap, + Cmd: exec.Command("sh", "-c", env), + Logger: logger, + AllowedProtocols: []plugin.Protocol{ + plugin.ProtocolNetRPC, plugin.ProtocolGRPC}, + }) + + // Connect via RPC + rpcClient, err := client.Client() + if err != nil { + return nil, err + } + + // Request streaming plugin + return rpcClient.Dispense(name) +} + +``` + +We propose a `RegisterStreamingPlugin` function for the App to register `NewStreamingPlugin`s with the App's BaseApp. +Streaming plugins can be of `Any` type; therefore, the function takes in an interface vs a concrete type. +For example, we could have plugins of `ABCIListener`, `WasmListener` or `IBCListener`. Note that `RegisterStreamingPluing` function +is helper function and not a requirement. Plugin registration can easily be moved from the App to the BaseApp directly. + +```go +// baseapp/streaming.go + +// RegisterStreamingPlugin registers streaming plugins with the App. +// This method returns an error if a plugin is not supported. +func RegisterStreamingPlugin( + bApp *BaseApp, + appOpts servertypes.AppOptions, + keys map[string]*types.KVStoreKey, + streamingPlugin interface{}, +) error { + switch t := streamingPlugin.(type) { + case ABCIListener: + registerABCIListenerPlugin(bApp, appOpts, keys, t) + default: + return fmt.Errorf("unexpected plugin type %T", t) + } + return nil +} +``` + +```go +func registerABCIListenerPlugin( + bApp *BaseApp, + appOpts servertypes.AppOptions, + keys map[string]*store.KVStoreKey, + abciListener ABCIListener, +) { + asyncKey := fmt.Sprintf("%s.%s.%s", StreamingTomlKey, StreamingABCITomlKey, StreamingABCIAsync) + async := cast.ToBool(appOpts.Get(asyncKey)) + stopNodeOnErrKey := fmt.Sprintf("%s.%s.%s", StreamingTomlKey, StreamingABCITomlKey, StreamingABCIStopNodeOnErrTomlKey) + stopNodeOnErr := cast.ToBool(appOpts.Get(stopNodeOnErrKey)) + keysKey := fmt.Sprintf("%s.%s.%s", StreamingTomlKey, StreamingABCITomlKey, StreamingABCIKeysTomlKey) + exposeKeysStr := cast.ToStringSlice(appOpts.Get(keysKey)) + exposedKeys := exposeStoreKeysSorted(exposeKeysStr, keys) + bApp.cms.AddListeners(exposedKeys) + app.SetStreamingManager( + storetypes.StreamingManager{ + ABCIListeners: []storetypes.ABCIListener{abciListener}, + StopNodeOnErr: stopNodeOnErr, + }, + ) +} +``` + +```go +func exposeAll(list []string) bool { + for _, ele := range list { + if ele == "*" { + return true + } + } + return false +} + +func exposeStoreKeys(keysStr []string, keys map[string]*types.KVStoreKey) []types.StoreKey { + var exposeStoreKeys []types.StoreKey + if exposeAll(keysStr) { + exposeStoreKeys = make([]types.StoreKey, 0, len(keys)) + for _, storeKey := range keys { + exposeStoreKeys = append(exposeStoreKeys, storeKey) + } + } else { + exposeStoreKeys = make([]types.StoreKey, 0, len(keysStr)) + for _, keyStr := range keysStr { + if storeKey, ok := keys[keyStr]; ok { + exposeStoreKeys = append(exposeStoreKeys, storeKey) + } + } + } + // sort storeKeys for deterministic output + sort.SliceStable(exposeStoreKeys, func(i, j int) bool { + return exposeStoreKeys[i].Name() < exposeStoreKeys[j].Name() + }) + + return exposeStoreKeys +} +``` + +The `NewStreamingPlugin` and `RegisterStreamingPlugin` functions are used to register a plugin with the App's BaseApp. + +e.g. in `NewSimApp`: + +```go +func NewSimApp( + logger log.Logger, + db corestore.KVStoreWithBatch, + traceStore io.Writer, + loadLatest bool, + appOpts servertypes.AppOptions, + baseAppOptions ...func(*baseapp.BaseApp), +) *SimApp { + + ... + + keys := sdk.NewKVStoreKeys( + authtypes.StoreKey, banktypes.StoreKey, stakingtypes.StoreKey, + minttypes.StoreKey, distrtypes.StoreKey, slashingtypes.StoreKey, + govtypes.StoreKey, paramstypes.StoreKey, ibchost.StoreKey, upgradetypes.StoreKey, + evidencetypes.StoreKey, ibctransfertypes.StoreKey, capabilitytypes.StoreKey, + ) + + ... + + // register streaming services + streamingCfg := cast.ToStringMap(appOpts.Get(baseapp.StreamingTomlKey)) + for service := range streamingCfg { + pluginKey := fmt.Sprintf("%s.%s.%s", baseapp.StreamingTomlKey, service, baseapp.StreamingPluginTomlKey) + pluginName := strings.TrimSpace(cast.ToString(appOpts.Get(pluginKey))) + if len(pluginName) > 0 { + logLevel := cast.ToString(appOpts.Get(flags.FlagLogLevel)) + plugin, err := streaming.NewStreamingPlugin(pluginName, logLevel) + if err != nil { + tmos.Exit(err.Error()) + } + if err := baseapp.RegisterStreamingPlugin(bApp, appOpts, keys, plugin); err != nil { + tmos.Exit(err.Error()) + } + } + } + + return app +``` + +#### Configuration + +The plugin system will be configured within an App's TOML configuration files. + +```toml +# gRPC streaming +[streaming] + +# ABCI streaming service +[streaming.abci] + +# The plugin version to use for ABCI listening +plugin = "abci_v1" + +# List of kv store keys to listen to for state changes. +# Set to ["*"] to expose all keys. +keys = ["*"] + +# Enable abciListeners to run asynchronously. +# When abciListenersAsync=false and stopNodeOnABCIListenerErr=false listeners will run synchronized but will not stop the node. +# When abciListenersAsync=true stopNodeOnABCIListenerErr will be ignored. +async = false + +# Whether to stop the node on message deliver error. +stop-node-on-err = true +``` + +There will be four parameters for configuring `ABCIListener` plugin: `streaming.abci.plugin`, `streaming.abci.keys`, `streaming.abci.async` and `streaming.abci.stop-node-on-err`. +`streaming.abci.plugin` is the name of the plugin we want to use for streaming, `streaming.abci.keys` is a set of store keys for stores it listens to, +`streaming.abci.async` is bool enabling asynchronous listening and `streaming.abci.stop-node-on-err` is a bool that stops the node when true and when operating +on synchronized mode `streaming.abci.async=false`. Note that `streaming.abci.stop-node-on-err=true` will be ignored if `streaming.abci.async=true`. + +The configuration above support additional streaming plugins by adding the plugin to the `[streaming]` configuration section +and registering the plugin with `RegisterStreamingPlugin` helper function. + +Note the that each plugin must include `streaming.{service}.plugin` property as it is a requirement for doing the lookup and registration of the plugin +with the App. All other properties are unique to the individual services. + +#### Encoding and decoding streams + +ADR-038 introduces the interfaces and types for streaming state changes out from KVStores, associating this +data with their related ABCI requests and responses, and registering a service for consuming this data and streaming it to some destination in a final format. +Instead of prescribing a final data format in this ADR, it is left to a specific plugin implementation to define and document this format. +We take this approach because flexibility in the final format is necessary to support a wide range of streaming service plugins. For example, +the data format for a streaming service that writes the data out to a set of files will differ from the data format that is written to a Kafka topic. + +## Consequences + +These changes will provide a means of subscribing to KVStore state changes in real time. + +### Backwards Compatibility + +* This ADR changes the `CommitMultiStore` interface, implementations supporting the previous version of this interface will not support the new one + +### Positive + +* Ability to listen to KVStore state changes in real time and expose these events to external consumers + +### Negative + +* Changes `CommitMultiStore` interface and its implementations + +### Neutral + +* Introduces additional- but optional- complexity to configuring and running a cosmos application +* If an application developer opts to use these features to expose data, they need to be aware of the ramifications/risks of that data exposure as it pertains to the specifics of their application diff --git a/versioned_docs/version-0.52/build/architecture/adr-039-epoched-staking.md b/versioned_docs/version-0.52/build/architecture/adr-039-epoched-staking.md new file mode 100644 index 000000000..bfdddbe20 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-039-epoched-staking.md @@ -0,0 +1,122 @@ +# ADR 039: Epoched Staking + +## Changelog + +* 10-Feb-2021: Initial Draft + +## Authors + +* Dev Ojha (@valardragon) +* Sunny Aggarwal (@sunnya97) + +## Status + +Proposed + +## Abstract + +This ADR updates the proof of stake module to buffer the staking weight updates for a number of blocks before updating the consensus' staking weights. The length of the buffer is dubbed an epoch. The prior functionality of the staking module is then a special case of the abstracted module, with the epoch being set to 1 block. + +## Context + +The current proof of stake module takes the design decision to apply staking weight changes to the consensus engine immediately. This means that delegations and unbonds get applied immediately to the validator set. This decision was primarily done as it was implementationally simplest, and because we at the time believed that this would lead to better UX for clients. + +An alternative design choice is to allow buffering staking updates (delegations, unbonds, validators joining) for a number of blocks. This 'epoch'ed proof of stake consensus provides the guarantee that the consensus weights for validators will not change mid-epoch, except in the event of a slash condition. + +Additionally, the UX hurdle may not be as significant as was previously thought. This is because it is possible to provide users immediate acknowledgement that their bond was recorded and will be executed. + +Furthermore, it has become clearer over time that immediate execution of staking events comes with limitations, such as: + +* Threshold based cryptography. One of the main limitations is that because the validator set can change so regularly, it makes the running of multiparty computation by a fixed validator set difficult. Many threshold-based cryptographic features for blockchains such as randomness beacons and threshold decryption require a computationally-expensive DKG process (will take much longer than 1 block to create). To productively use these, we need to guarantee that the result of the DKG will be used for a reasonably long time. It wouldn't be feasible to rerun the DKG every block. By epoching staking, it guarantees we'll only need to run a new DKG once every epoch. + +* Light client efficiency. This would lessen the overhead for IBC when there is high churn in the validator set. In the Tendermint light client bisection algorithm, the number of headers you need to verify is related to bounding the difference in validator sets between a trusted header and the latest header. If the difference is too great, you verify more header in between the two. By limiting the frequency of validator set changes, we can reduce the worst case size of IBC lite client proofs, which occurs when a validator set has high churn. + +* Fairness of deterministic leader election. Currently we have no ways of reasoning of fairness of deterministic leader election in the presence of staking changes without epochs (tendermint/spec#217). Breaking fairness of leader election is profitable for validators, as they earn additional rewards from being the proposer. Adding epochs at least makes it easier for our deterministic leader election to match something we can prove secure. (Albeit, we still haven’t proven if our current algorithm is fair with > 2 validators in the presence of stake changes) + +* Staking derivative design. Currently, reward distribution is done lazily using the F1 fee distribution. While saving computational complexity, lazy accounting requires a more stateful staking implementation. Right now, each delegation entry has to track the time of last withdrawal. Handling this can be a challenge for some staking derivatives designs that seek to provide fungibility for all tokens staked to a single validator. Force-withdrawing rewards to users can help solve this, however it is infeasible to force-withdraw rewards to users on a per block basis. With epochs, a chain could more easily alter the design to have rewards be forcefully withdrawn (iterating over delegator accounts only once per-epoch), and can thus remove delegation timing from state. This may be useful for certain staking derivative designs. + +## Design considerations + +### Slashing + +There is a design consideration for whether to apply a slash immediately or at the end of an epoch. A slash event should apply to only members who are actually staked during the time of the infraction, namely during the epoch the slash event occurred. + +Applying it immediately can be viewed as offering greater consensus layer security, at potential costs to the aforementioned usecases. The benefits of immediate slashing for consensus layer security can be all be obtained by executing the validator jailing immediately (thus removing it from the validator set), and delaying the actual slash change to the validator's weight until the epoch boundary. For the use cases mentioned above, workarounds can be integrated to avoid problems, as follows: + +* For threshold based cryptography, this setting will have the threshold cryptography use the original epoch weights, while consensus has an update that lets it more rapidly benefit from additional security. If the threshold based cryptography blocks liveness of the chain, then we have effectively raised the liveness threshold of the remaining validators for the rest of the epoch. (Alternatively, jailed nodes could still contribute shares) This plan will fail in the extreme case that more than 1/3rd of the validators have been jailed within a single epoch. For such an extreme scenario, the chain already have its own custom incident response plan, and defining how to handle the threshold cryptography should be a part of that. +* For light client efficiency, there can be a bit included in the header indicating an intra-epoch slash (ala https://github.com/tendermint/spec/issues/199). +* For fairness of deterministic leader election, applying a slash or jailing within an epoch would break the guarantee we were seeking to provide. This then re-introduces a new (but significantly simpler) problem for trying to provide fairness guarantees. Namely, that validators can adversarially elect to remove themself from the set of proposers. From a security perspective, this could potentially be handled by two different mechanisms (or prove to still be too difficult to achieve). One is making a security statement acknowledging the ability for an adversary to force an ahead-of-time fixed threshold of users to drop out of the proposer set within an epoch. The second method would be to parameterize such that the cost of a slash within the epoch far outweighs benefits due to being a proposer. However, this latter criterion is quite dubious, since being a proposer can have many advantageous side-effects in chains with complex state machines. (Namely, DeFi games such as Fomo3D) +* For staking derivative design, there is no issue introduced. This does not increase the state size of staking records, since whether a slash has occurred is fully queryable given the validator address. + +### Token lockup + +When someone makes a transaction to delegate, even though they are not immediately staked, their tokens should be moved into a pool managed by the staking module which will then be used at the end of an epoch. This prevents concerns where they stake, and then spend those tokens not realizing they were already allocated for staking, and thus having their staking tx fail. + +### Pipelining the epochs + +For threshold based cryptography in particular, we need a pipeline for epoch changes. This is because when we are in epoch N, we want the epoch N+1 weights to be fixed so that the validator set can do the DKG accordingly. So if we are currently in epoch N, the stake weights for epoch N+1 should already be fixed, and new stake changes should be getting applied to epoch N + 2. + +This can be handled by making a parameter for the epoch pipeline length. This parameter should not be alterable except during hard forks, to mitigate implementation complexity of switching the pipeline length. + +With pipeline length 1, if I redelegate during epoch N, then my redelegation is applied prior to the beginning of epoch N+1. +With pipeline length 2, if I redelegate during epoch N, then my redelegation is applied prior to the beginning of epoch N+2. + +### Rewards + +Even though all staking updates are applied at epoch boundaries, rewards can still be distributed immediately when they are claimed. This is because they do not affect the current stake weights, as we do not implement auto-bonding of rewards. If such a feature were to be implemented, it would have to be setup so that rewards are auto-bonded at the epoch boundary. + +### Parameterizing the epoch length + +When choosing the epoch length, there is a trade-off queued state/computation buildup, and countering the previously discussed limitations of immediate execution if they apply to a given chain. + +Until an ABCI mechanism for variable block times is introduced, it is ill-advised to be using high epoch lengths due to the computation buildup. This is because when a block's execution time is greater than the expected block time from Tendermint, rounds may increment. + +## Decision + +**Step-1**: Implement buffering of all staking and slashing messages. + +First we create a pool for storing tokens that are being bonded, but should be applied at the epoch boundary called the `EpochDelegationPool`. Then, we have two separate queues, one for staking, one for slashing. We describe what happens on each message being delivered below: + +### Staking messages + +* **MsgCreateValidator**: Move user's self-bond to `EpochDelegationPool` immediately. Queue a message for the epoch boundary to handle the self-bond, taking the funds from the `EpochDelegationPool`. If Epoch execution fail, return back funds from `EpochDelegationPool` to user's account. +* **MsgEditValidator**: Validate message and if valid queue the message for execution at the end of the Epoch. +* **MsgDelegate**: Move user's funds to `EpochDelegationPool` immediately. Queue a message for the epoch boundary to handle the delegation, taking the funds from the `EpochDelegationPool`. If Epoch execution fail, return back funds from `EpochDelegationPool` to user's account. +* **MsgBeginRedelegate**: Validate message and if valid queue the message for execution at the end of the Epoch. +* **MsgUndelegate**: Validate message and if valid queue the message for execution at the end of the Epoch. + +### Slashing messages + +* **MsgUnjail**: Validate message and if valid queue the message for execution at the end of the Epoch. +* **Slash Event**: Whenever a slash event is created, it gets queued in the slashing module to apply at the end of the epoch. The queues should be setup such that this slash applies immediately. + +### Evidence Messages + +* **MsgSubmitEvidence**: This gets executed immediately, and the validator gets jailed immediately. However in slashing, the actual slash event gets queued. + +Then we add methods to the end blockers, to ensure that at the epoch boundary the queues are cleared and delegation updates are applied. + +**Step-2**: Implement querying of queued staking txs. + +When querying the staking activity of a given address, the status should return not only the amount of tokens staked, but also if there are any queued stake events for that address. This will require more work to be done in the querying logic, to trace the queued upcoming staking events. + +As an initial implementation, this can be implemented as a linear search over all queued staking events. However, for chains that need long epochs, they should eventually build additional support for nodes that support querying to be able to produce results in constant time. (This is do-able by maintaining an auxiliary hashmap for indexing upcoming staking events by address) + +**Step-3**: Adjust gas + +Currently gas represents the cost of executing a transaction when its done immediately. (Merging together costs of p2p overhead, state access overhead, and computational overhead) However, now a transaction can cause computation in a future block, namely at the epoch boundary. + +To handle this, we should initially include parameters for estimating the amount of future computation (denominated in gas), and add that as a flat charge needed for the message. +We leave it as out of scope for how to weight future computation versus current computation in gas pricing, and have it set such that the are weighted equally for now. + +## Consequences + +### Positive + +* Abstracts the proof of stake module that allows retaining the existing functionality +* Enables new features such as validator-set based threshold cryptography + +### Negative + +* Increases complexity of integrating more complex gas pricing mechanisms, as they now have to consider future execution costs as well. +* When epoch > 1, validators can no longer leave the network immediately, and must wait until an epoch boundary. diff --git a/versioned_docs/version-0.52/build/architecture/adr-040-storage-and-smt-state-commitments.md b/versioned_docs/version-0.52/build/architecture/adr-040-storage-and-smt-state-commitments.md new file mode 100644 index 000000000..8ca8952bf --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-040-storage-and-smt-state-commitments.md @@ -0,0 +1,289 @@ +# ADR 040: Storage and SMT State Commitments + +## Changelog + +* 2020-01-15: Draft + +## Status + +DRAFT Not Implemented + +## Abstract + +Sparse Merkle Tree ([SMT](https://osf.io/8mcnh/)) is a version of a Merkle Tree with various storage and performance optimizations. This ADR defines a separation of state commitments from data storage and the Cosmos SDK transition from IAVL to SMT. + +## Context + +Currently, Cosmos SDK uses IAVL for both state [commitments](https://cryptography.fandom.com/wiki/Commitment_scheme) and data storage. + +IAVL has effectively become an orphaned project within the Cosmos ecosystem and it's proven to be an inefficient state commitment data structure. +In the current design, IAVL is used for both data storage and as a Merkle Tree for state commitments. IAVL is meant to be a standalone Merkelized key/value database, however it's using a KV DB engine to store all tree nodes. So, each node is stored in a separate record in the KV DB. This causes many inefficiencies and problems: + +* Each object query requires a tree traversal from the root. Subsequent queries for the same object are cached on the Cosmos SDK level. +* Each edge traversal requires a DB query. +* Creating snapshots is [expensive](https://github.com/cosmos/cosmos-sdk/issues/7215#issuecomment-684804950). It takes about 30 seconds to export less than 100 MB of state (as of March 2020). +* Updates in IAVL may trigger tree reorganization and possible O(log(n)) hashes re-computation, which can become a CPU bottleneck. +* The node structure is pretty expensive - it contains a standard tree node elements (key, value, left and right element) and additional metadata such as height, version (which is not required by the Cosmos SDK). The entire node is hashed, and that hash is used as the key in the underlying database, [ref](https://github.com/cosmos/iavl/blob/master/docs/node/node.md +). + +Moreover, the IAVL project lacks support and a maintainer and we already see better and well-established alternatives. Instead of optimizing the IAVL, we are looking into other solutions for both storage and state commitments. + +## Decision + +We propose to separate the concerns of state commitment (**SC**), needed for consensus, and state storage (**SS**), needed for state machine. Finally we replace IAVL with [Celestia's SMT](https://github.com/lazyledger/smt). Celestia SMT is based on Diem (called jellyfish) design [*] - it uses a compute-optimised SMT by replacing subtrees with only default values with a single node (same approach is used by Ethereum2) and implements compact proofs. + +The storage model presented here doesn't deal with data structure nor serialization. It's a Key-Value database, where both key and value are binaries. The storage user is responsible for data serialization. + +### Decouple state commitment from storage + +Separation of storage and commitment (by the SMT) will allow the optimization of different components according to their usage and access patterns. + +`SC` (SMT) is used to commit to a data and compute Merkle proofs. `SS` is used to directly access data. To avoid collisions, both `SS` and `SC` will use a separate storage namespace (they could use the same database underneath). `SS` will store each record directly (mapping `(key, value)` as `key → value`). + +SMT is a merkle tree structure: we don't store keys directly. For every `(key, value)` pair, `hash(key)` is used as leaf path (we hash a key to uniformly distribute leaves in the tree) and `hash(value)` as the leaf contents. The tree structure is specified in more depth [below](#smt-for-state-commitment). + +For data access we propose 2 additional KV buckets (implemented as namespaces for the key-value pairs, sometimes called [column family](https://github.com/facebook/rocksdb/wiki/Terminology)): + +1. B1: `key → value`: the principal object storage, used by a state machine, behind the Cosmos SDK `KVStore` interface: provides direct access by key and allows prefix iteration (KV DB backend must support it). +2. B2: `hash(key) → key`: a reverse index to get a key from an SMT path. Internally the SMT will store `(key, value)` as `prefix || hash(key) || hash(value)`. So, we can get an object value by composing `hash(key) → B2 → B1`. +3. We could use more buckets to optimize the app usage if needed. + +We propose to use a KV database for both `SS` and `SC`. The store interface will allow to use the same physical DB backend for both `SS` and `SC` as well two separate DBs. The latter option allows for the separation of `SS` and `SC` into different hardware units, providing support for more complex setup scenarios and improving overall performance: one can use different backends (eg RocksDB and Badger) as well as independently tuning the underlying DB configuration. + +### Requirements + +State Storage requirements: + +* range queries +* quick (key, value) access +* creating a snapshot +* historical versioning +* pruning (garbage collection) + +State Commitment requirements: + +* fast updates +* tree path should be short +* query historical commitment proofs using ICS-23 standard +* pruning (garbage collection) + +### SMT for State Commitment + +A Sparse Merkle tree is based on the idea of a complete Merkle tree of an intractable size. The assumption here is that as the size of the tree is intractable, there would only be a few leaf nodes with valid data blocks relative to the tree size, rendering a sparse tree. + +The full specification can be found at [Celestia](https://github.com/celestiaorg/celestia-specs/blob/ec98170398dfc6394423ee79b00b71038879e211/src/specs/data_structures.md#sparse-merkle-tree). In summary: + +* The SMT consists of a binary Merkle tree, constructed in the same fashion as described in [Certificate Transparency (RFC-6962)](https://tools.ietf.org/html/rfc6962), but using as the hashing function SHA-2-256 as defined in [FIPS 180-4](https://doi.org/10.6028/NIST.FIPS.180-4). +* Leaves and internal nodes are hashed differently: the one-byte `0x00` is prepended for leaf nodes while `0x01` is prepended for internal nodes. +* Default values are given to leaf nodes with empty leaves. +* While the above rule is sufficient to pre-compute the values of intermediate nodes that are roots of empty subtrees, a further simplification is to extend this default value to all nodes that are roots of empty subtrees. The 32-byte zero is used as the default value. This rule takes precedence over the above one. +* An internal node that is the root of a subtree that contains exactly one non-empty leaf is replaced by that leaf's leaf node. + +### Snapshots for storage sync and state versioning + +Below, with simple _snapshot_ we refer to a database snapshot mechanism, not to a _ABCI snapshot sync_. The latter will be referred as _snapshot sync_ (which will directly use DB snapshot as described below). + +Database snapshot is a view of DB state at a certain time or transaction. It's not a full copy of a database (it would be too big). Usually a snapshot mechanism is based on a _copy on write_ and it allows DB state to be efficiently delivered at a certain stage. +Some DB engines support snapshotting. Hence, we propose to reuse that functionality for the state sync and versioning (described below). We limit the supported DB engines to ones which efficiently implement snapshots. In a final section we discuss the evaluated DBs. + +One of the Stargate core features is a _snapshot sync_ delivered in the `/snapshot` package. It provides a way to trustlessly sync a blockchain without repeating all transactions from the genesis. This feature is implemented in Cosmos SDK and requires storage support. Currently IAVL is the only supported backend. It works by streaming to a client a snapshot of a `SS` at a certain version together with a header chain. + +A new database snapshot will be created in every `EndBlocker` and identified by a block height. The `root` store keeps track of the available snapshots to offer `SS` at a certain version. The `root` store implements the `RootStore` interface described below. In essence, `RootStore` encapsulates a `Committer` interface. `Committer` has a `Commit`, `SetPruning`, `GetPruning` functions which will be used for creating and removing snapshots. The `rootStore.Commit` function creates a new snapshot and increments the version on each call, and checks if it needs to remove old versions. We will need to update the SMT interface to implement the `Committer` interface. +NOTE: `Commit` must be called exactly once per block. Otherwise we risk going out of sync for the version number and block height. +NOTE: For the Cosmos SDK storage, we may consider splitting that interface into `Committer` and `PruningCommitter` - only the multiroot should implement `PruningCommitter` (cache and prefix store don't need pruning). + +Number of historical versions for `abci.QueryRequest` and state sync snapshots is part of a node configuration, not a chain configuration (configuration implied by the blockchain consensus). A configuration should allow to specify number of past blocks and number of past blocks modulo some number (eg: 100 past blocks and one snapshot every 100 blocks for past 2000 blocks). Archival nodes can keep all past versions. + +Pruning old snapshots is effectively done by a database. Whenever we update a record in `SC`, SMT won't update nodes - instead it creates new nodes on the update path, without removing the old one. Since we are snapshotting each block, we need to change that mechanism to immediately remove orphaned nodes from the database. This is a safe operation - snapshots will keep track of the records and make it available when accessing past versions. + +To manage the active snapshots we will either use a DB _max number of snapshots_ option (if available), or we will remove DB snapshots in the `EndBlocker`. The latter option can be done efficiently by identifying snapshots with block height and calling a store function to remove past versions. + +#### Accessing old state versions + +One of the functional requirements is to access old state. This is done through `abci.QueryRequest` structure. The version is specified by a block height (so we query for an object by a key `K` at block height `H`). The number of old versions supported for `abci.QueryRequest` is configurable. Accessing an old state is done by using available snapshots. +`abci.QueryRequest` doesn't need old state of `SC` unless the `prove=true` parameter is set. The SMT merkle proof must be included in the `abci.QueryResponse` only if both `SC` and `SS` have a snapshot for requested version. + +Moreover, Cosmos SDK could provide a way to directly access a historical state. However, a state machine shouldn't do that - since the number of snapshots is configurable, it would lead to nondeterministic execution. + +We positively [validated](https://github.com/cosmos/cosmos-sdk/discussions/8297) a versioning and snapshot mechanism for querying old state with regards to the database we evaluated. + +### State Proofs + +For any object stored in State Store (SS), we have corresponding object in `SC`. A proof for object `V` identified by a key `K` is a branch of `SC`, where the path corresponds to the key `hash(K)`, and the leaf is `hash(K, V)`. + +### Rollbacks + +We need to be able to process transactions and roll-back state updates if a transaction fails. This can be done in the following way: during transaction processing, we keep all state change requests (writes) in a `CacheWrapper` abstraction (as it's done today). Once we finish the block processing, in the `Endblocker`, we commit a root store - at that time, all changes are written to the SMT and to the `SS` and a snapshot is created. + +### Committing to an object without saving it + +We identified use-cases, where modules will need to save an object commitment without storing an object itself. Sometimes clients are receiving complex objects, and they have no way to prove a correctness of that object without knowing the storage layout. For those use cases it would be easier to commit to the object without storing it directly. + +### Refactor MultiStore + +The Stargate `/store` implementation (store/v1) adds an additional layer in the SDK store construction - the `MultiStore` structure. The multistore exists to support the modularity of the Cosmos SDK - each module is using its own instance of IAVL, but in the current implementation, all instances share the same database. The latter indicates, however, that the implementation doesn't provide true modularity. Instead it causes problems related to race condition and atomic DB commits (see: [\#6370](https://github.com/cosmos/cosmos-sdk/issues/6370) and [discussion](https://github.com/cosmos/cosmos-sdk/discussions/8297#discussioncomment-757043)). + +We propose to reduce the multistore concept from the SDK, and to use a single instance of `SC` and `SS` in a `RootStore` object. To avoid confusion, we should rename the `MultiStore` interface to `RootStore`. The `RootStore` will have the following interface; the methods for configuring tracing and listeners are omitted for brevity. + +```go +// Used where read-only access to versions is needed. +type BasicRootStore interface { + Store + GetKVStore(StoreKey) KVStore + CacheRootStore() CacheRootStore +} + +// Used as the main app state, replacing CommitMultiStore. +type CommitRootStore interface { + BasicRootStore + Committer + Snapshotter + + GetVersion(uint64) (BasicRootStore, error) + SetInitialVersion(uint64) error + + ... // Trace and Listen methods +} + +// Replaces CacheMultiStore for branched state. +type CacheRootStore interface { + BasicRootStore + Write() + + ... // Trace and Listen methods +} + +// Example of constructor parameters for the concrete type. +type RootStoreConfig struct { + Upgrades *StoreUpgrades + InitialVersion uint64 + + ReservePrefix(StoreKey, StoreType) +} +``` + + + + +In contrast to `MultiStore`, `RootStore` doesn't allow to dynamically mount sub-stores or provide an arbitrary backing DB for individual sub-stores. + +NOTE: modules will be able to use a special commitment and their own DBs. For example: a module which will use ZK proofs for state can store and commit this proof in the `RootStore` (usually as a single record) and manage the specialized store privately or using the `SC` low level interface. + +#### Compatibility support + +To ease the transition to this new interface for users, we can create a shim which wraps a `CommitMultiStore` but provides a `CommitRootStore` interface, and expose functions to safely create and access the underlying `CommitMultiStore`. + +The new `RootStore` and supporting types can be implemented in a `store/v2alpha1` package to avoid breaking existing code. + +#### Merkle Proofs and IBC + +Currently, an IBC (v1.0) Merkle proof path consists of two elements (`["", ""]`), with each key corresponding to a separate proof. These are each verified according to individual [ICS-23 specs](https://github.com/cosmos/ibc-go/blob/f7051429e1cf833a6f65d51e6c3df1609290a549/modules/core/23-commitment/types/merkle.go#L17), and the result hash of each step is used as the committed value of the next step, until a root commitment hash is obtained. +The root hash of the proof for `""` is hashed with the `""` to validate against the App Hash. + +This is not compatible with the `RootStore`, which stores all records in a single Merkle tree structure, and won't produce separate proofs for the store- and record-key. Ideally, the store-key component of the proof could just be omitted, and updated to use a "no-op" spec, so only the record-key is used. However, because the IBC verification code hardcodes the `"ibc"` prefix and applies it to the SDK proof as a separate element of the proof path, this isn't possible without a breaking change. Breaking this behavior would severely impact the Cosmos ecosystem which already widely adopts the IBC module. Requesting an update of the IBC module across the chains is a time consuming effort and not easily feasible. + +As a workaround, the `RootStore` will have to use two separate SMTs (they could use the same underlying DB): one for IBC state and one for everything else. A simple Merkle map that reference these SMTs will act as a Merkle Tree to create a final App hash. The Merkle map is not stored in a DBs - it's constructed in the runtime. The IBC substore key must be `"ibc"`. + +The workaround can still guarantee atomic syncs: the [proposed DB backends](#evaluated-kv-databases) support atomic transactions and efficient rollbacks, which will be used in the commit phase. + +The presented workaround can be used until the IBC module is fully upgraded to supports single-element commitment proofs. + +### Optimization: compress module key prefixes + +We consider a compression of prefix keys by creating a mapping from module key to an integer, and serializing the integer using varint coding. Varint coding assures that different values don't have common byte prefix. For Merkle Proofs we can't use prefix compression - so it should only apply for the `SS` keys. Moreover, the prefix compression should be only applied for the module namespace. More precisely: + +* each module has it's own namespace; +* when accessing a module namespace we create a KVStore with embedded prefix; +* that prefix will be compressed only when accessing and managing `SS`. + +We need to assure that the codes won't change. We can fix the mapping in a static variable (provided by an app) or SS state under a special key. + +TODO: need to make decision about the key compression. + +## Optimization: SS key compression + +Some objects may be saved with key, which contains a Protobuf message type. Such keys are long. We could save a lot of space if we can map Protobuf message types in varints. + +TODO: finalize this or move to another ADR. + +## Migration + +Using the new store will require a migration. 2 Migrations are proposed: + +1. Genesis export -- it will reset the blockchain history. +2. In place migration: we can reuse `UpgradeKeeper.SetUpgradeHandler` to provide the migration logic: + +```go +app.UpgradeKeeper.SetUpgradeHandler("adr-40", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) { + + storev2.Migrate(iavlstore, v2.store) + + // RunMigrations returns the VersionMap + // with the updated module ConsensusVersions + return app.mm.RunMigrations(ctx, vm) +}) +``` + +The `Migrate` function will read all entries from a store/v1 DB and save them to the AD-40 combined KV store. +Cache layer should not be used and the operation must finish with a single Commit call. + +Inserting records to the `SC` (SMT) component is the bottleneck. Unfortunately SMT doesn't support batch transactions. +Adding batch transactions to `SC` layer is considered as a feature after the main release. + +## Consequences + +### Backwards Compatibility + +This ADR doesn't introduce any Cosmos SDK level API changes. + +We change the storage layout of the state machine, a storage hard fork and network upgrade is required to incorporate these changes. SMT provides a merkle proof functionality, however it is not compatible with ICS23. Updating the proofs for ICS23 compatibility is required. + +### Positive + +* Decoupling state from state commitment introduce better engineering opportunities for further optimizations and better storage patterns. +* Performance improvements. +* Joining SMT based camp which has wider and proven adoption than IAVL. Example projects which decided on SMT: Ethereum2, Diem (Libra), Trillan, Tezos, Celestia. +* Multistore removal fixes a longstanding issue with the current MultiStore design. +* Simplifies merkle proofs - all modules, except IBC, have only one pass for merkle proof. + +### Negative + +* Storage migration +* LL SMT doesn't support pruning - we will need to add and test that functionality. +* `SS` keys will have an overhead of a key prefix. This doesn't impact `SC` because all keys in `SC` have same size (they are hashed). + +### Neutral + +* Deprecating IAVL, which is one of the core proposals of Cosmos Whitepaper. + +## Alternative designs + +Most of the alternative designs were evaluated in [state commitments and storage report](https://paper.dropbox.com/published/State-commitments-and-storage-review--BDvA1MLwRtOx55KRihJ5xxLbBw-KeEB7eOd11pNrZvVtqUgL3h). + +Ethereum research published [Verkle Trie](https://dankradfeist.de/ethereum/2021/06/18/verkle-trie-for-eth1.html) - an idea of combining polynomial commitments with merkle tree in order to reduce the tree height. This concept has a very good potential, but we think it's too early to implement it. The current, SMT based design could be easily updated to the Verkle Trie once other research implement all necessary libraries. The main advantage of the design described in this ADR is the separation of state commitments from the data storage and designing a more powerful interface. + +## Further Discussions + +### Evaluated KV Databases + +We verified existing databases KV databases for evaluating snapshot support. The following databases provide efficient snapshot mechanism: Badger, RocksDB, [Pebble](https://github.com/cockroachdb/pebble). Databases which don't provide such support or are not production ready: boltdb, leveldb, goleveldb, membdb, lmdb. + +### RDBMS + +Use of RDBMS instead of simple KV store for state. Use of RDBMS will require a Cosmos SDK API breaking change (`KVStore` interface) and will allow better data extraction and indexing solutions. Instead of saving an object as a single blob of bytes, we could save it as record in a table in the state storage layer, and as a `hash(key, protobuf(object))` in the SMT as outlined above. To verify that an object registered in RDBMS is same as the one committed to SMT, one will need to load it from RDBMS, marshal using protobuf, hash and do SMT search. + +### Off Chain Store + +We were discussing use case where modules can use a support database, which is not automatically committed. Module will responsible for having a sound storage model and can optionally use the feature discussed in __Committing to an object without saving it_ section. + +## References + +* [IAVL What's Next?](https://github.com/cosmos/cosmos-sdk/issues/7100) +* [IAVL overview](https://docs.google.com/document/d/16Z_hW2rSAmoyMENO-RlAhQjAG3mSNKsQueMnKpmcBv0/edit#heading=h.yd2th7x3o1iv) of it's state v0.15 +* [State commitments and storage report](https://paper.dropbox.com/published/State-commitments-and-storage-review--BDvA1MLwRtOx55KRihJ5xxLbBw-KeEB7eOd11pNrZvVtqUgL3h) +* [Celestia (LazyLedger) SMT](https://github.com/lazyledger/smt) +* Facebook Diem (Libra) SMT [design](https://developers.diem.com/papers/jellyfish-merkle-tree/2021-01-14.pdf) +* [Trillian Revocation Transparency](https://github.com/google/trillian/blob/master/docs/papers/RevocationTransparency.pdf), [Trillian Verifiable Data Structures](https://github.com/google/trillian/blob/master/docs/papers/VerifiableDataStructures.pdf). +* Design and implementation [discussion](https://github.com/cosmos/cosmos-sdk/discussions/8297). +* [How to Upgrade IBC Chains and their Clients](https://github.com/cosmos/ibc-go/blob/main/docs/docs/01-ibc/05-upgrades/01-quick-guide.md) +* [ADR-40 Effect on IBC](https://github.com/cosmos/ibc-go/discussions/256) diff --git a/versioned_docs/version-0.52/build/architecture/adr-041-in-place-store-migrations.md b/versioned_docs/version-0.52/build/architecture/adr-041-in-place-store-migrations.md new file mode 100644 index 000000000..e9fc374bc --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-041-in-place-store-migrations.md @@ -0,0 +1,167 @@ +# ADR 041: In-Place Store Migrations + +## Changelog + +* 17.02.2021: Initial Draft + +## Status + +Accepted + +## Abstract + +This ADR introduces a mechanism to perform in-place state store migrations during chain software upgrades. + +## Context + +When a chain upgrade introduces state-breaking changes inside modules, the current procedure consists of exporting the whole state into a JSON file (via the `simd genesis export` command), running migration scripts on the JSON file (`simd genesis migrate` command), clearing the stores (`simd unsafe-reset-all` command), and starting a new chain with the migrated JSON file as new genesis (optionally with a custom initial block height). An example of such a procedure can be seen [in the Cosmos Hub 3->4 migration guide](https://github.com/cosmos/gaia/blob/v4.0.3/docs/migration/cosmoshub-3.md#upgrade-procedure). + +This procedure is cumbersome for multiple reasons: + +* The procedure takes time. It can take hours to run the `export` command, plus some additional hours to run `InitChain` on the fresh chain using the migrated JSON. +* The exported JSON file can be heavy (~100MB-1GB), making it difficult to view, edit and transfer, which in turn introduces additional work to solve these problems (such as [streaming genesis](https://github.com/cosmos/cosmos-sdk/issues/6936)). + +## Decision + +We propose a migration procedure based on modifying the KV store in-place without involving the JSON export-process-import flow described above. + +### Module `ConsensusVersion` + +We introduce a new method on the `AppModule` interface: + +```go +type AppModule interface { + // --snip-- + ConsensusVersion() uint64 +} +``` + +This methods returns an `uint64` which serves as state-breaking version of the module. It MUST be incremented on each consensus-breaking change introduced by the module. To avoid potential errors with default values, the initial version of a module MUST be set to 1. In the Cosmos SDK, version 1 corresponds to the modules in the v0.41 series. + +### Module-Specific Migration Functions + +For each consensus-breaking change introduced by the module, a migration script from ConsensusVersion `N` to version `N+1` MUST be registered in the `Configurator` using its newly-added `RegisterMigration` method. All modules receive a reference to the configurator in their `RegisterServices` method on `AppModule`, and this is where the migration functions should be registered. The migration functions should be registered in increasing order. + +```go +func (am AppModule) RegisterServices(cfg module.Configurator) { + // --snip-- + cfg.RegisterMigration(types.ModuleName, 1, func(ctx sdk.Context) error { + // Perform in-place store migrations from ConsensusVersion 1 to 2. + }) + cfg.RegisterMigration(types.ModuleName, 2, func(ctx sdk.Context) error { + // Perform in-place store migrations from ConsensusVersion 2 to 3. + }) + // etc. +} +``` + +For example, if the new ConsensusVersion of a module is `N` , then `N-1` migration functions MUST be registered in the configurator. + +In the Cosmos SDK, the migration functions are handled by each module's keeper, because the keeper holds the `sdk.StoreKey` used to perform in-place store migrations. To not overload the keeper, a `Migrator` wrapper is used by each module to handle the migration functions: + +```go +// Migrator is a struct for handling in-place store migrations. +type Migrator struct { + BaseKeeper +} +``` + +Migration functions should live inside the `migrations/` folder of each module, and be called by the Migrator's methods. We propose the format `Migrate{M}to{N}` for method names. + +```go +// Migrate1to2 migrates from version 1 to 2. +func (m Migrator) Migrate1to2(ctx sdk.Context) error { + return v2bank.MigrateStore(ctx, m.keeper.storeKey) // v043bank is package `x/bank/migrations/v2`. +} +``` + +Each module's migration functions are specific to the module's store evolutions, and are not described in this ADR. An example of x/bank store key migrations after the introduction of ADR-028 length-prefixed addresses can be seen in this [store.go code](https://github.com/cosmos/cosmos-sdk/blob/36f68eb9e041e20a5bb47e216ac5eb8b91f95471/x/bank/legacy/v043/store.go#L41-L62). + +### Tracking Module Versions in `x/upgrade` + +We introduce a new prefix store in `x/upgrade`'s store. This store will track each module's current version, it can be modelized as a `map[string]uint64` of module name to module ConsensusVersion, and will be used when running the migrations (see next section for details). The key prefix used is `0x1`, and the key/value format is: + +```text +0x2 | {bytes(module_name)} => BigEndian(module_consensus_version) +``` + +The initial state of the store is set from `app.go`'s `InitChainer` method. + +The UpgradeHandler signature needs to be updated to take a `VersionMap`, as well as return an upgraded `VersionMap` and an error: + +```diff +- type UpgradeHandler func(ctx sdk.Context, plan Plan) ++ type UpgradeHandler func(ctx sdk.Context, plan Plan, versionMap VersionMap) (VersionMap, error) +``` + +To apply an upgrade, we query the `VersionMap` from the `x/upgrade` store and pass it into the handler. The handler runs the actual migration functions (see next section), and if successful, returns an updated `VersionMap` to be stored in state. + +```diff +func (k UpgradeKeeper) ApplyUpgrade(ctx sdk.Context, plan types.Plan) { + // --snip-- +- handler(ctx, plan) ++ updatedVM, err := handler(ctx, plan, k.GetModuleVersionMap(ctx)) // k.GetModuleVersionMap() fetches the VersionMap stored in state. ++ if err != nil { ++ return err ++ } ++ ++ // Set the updated consensus versions to state ++ k.SetModuleVersionMap(ctx, updatedVM) +} +``` + +A gRPC query endpoint to query the `VersionMap` stored in `x/upgrade`'s state will also be added, so that app developers can double-check the `VersionMap` before the upgrade handler runs. + +### Running Migrations + +Once all the migration handlers are registered inside the configurator (which happens at startup), running migrations can happen by calling the `RunMigrations` method on `module.Manager`. This function will loop through all modules, and for each module: + +* Get the old ConsensusVersion of the module from its `VersionMap` argument (let's call it `M`). +* Fetch the new ConsensusVersion of the module from the `ConsensusVersion()` method on `AppModule` (call it `N`). +* If `N>M`, run all registered migrations for the module sequentially `M -> M+1 -> M+2...` until `N`. + * There is a special case where there is no ConsensusVersion for the module, as this means that the module has been newly added during the upgrade. In this case, no migration function is run, and the module's current ConsensusVersion is saved to `x/upgrade`'s store. + +If a required migration is missing (e.g. if it has not been registered in the `Configurator`), then the `RunMigrations` function will error. + +In practice, the `RunMigrations` method should be called from inside an `UpgradeHandler`. + +```go +app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) { + return app.mm.RunMigrations(ctx, vm) +}) +``` + +Assuming a chain upgrades at block `n`, the procedure should run as follows: + +* the old binary will halt in `BeginBlock` when starting block `N`. In its store, the ConsensusVersions of the old binary's modules are stored. +* the new binary will start at block `N`. The UpgradeHandler is set in the new binary, so will run at `BeginBlock` of the new binary. Inside `x/upgrade`'s `ApplyUpgrade`, the `VersionMap` will be retrieved from the (old binary's) store, and passed into the `RunMigrations` function, migrating all module stores in-place before the modules' own `BeginBlock`s. + +## Consequences + +### Backwards Compatibility + +This ADR introduces a new method `ConsensusVersion()` on `AppModule`, which all modules need to implement. It also alters the UpgradeHandler function signature. As such, it is not backwards-compatible. + +While modules MUST register their migration functions when bumping ConsensusVersions, running those scripts using an upgrade handler is optional. An application may perfectly well decide to not call the `RunMigrations` inside its upgrade handler, and continue using the legacy JSON migration path. + +### Positive + +* Perform chain upgrades without manipulating JSON files. +* While no benchmark has been made yet, it is probable that in-place store migrations will take less time than JSON migrations. The main reason supporting this claim is that both the `simd genesis export` command on the old binary and the `InitChain` function on the new binary will be skipped. + +### Negative + +* Module developers MUST correctly track consensus-breaking changes in their modules. If a consensus-breaking change is introduced in a module without its corresponding `ConsensusVersion()` bump, then the `RunMigrations` function won't detect the migration, and the chain upgrade might be unsuccessful. Documentation should clearly reflect this. + +### Neutral + +* The Cosmos SDK will continue to support JSON migrations via the existing `simd genesis export` and `simd genesis migrate` commands. +* The current ADR does not allow creating, renaming or deleting stores, only modifying existing store keys and values. The Cosmos SDK already has the `StoreLoader` for those operations. + +## Further Discussions + +## References + +* Initial discussion: https://github.com/cosmos/cosmos-sdk/discussions/8429 +* Implementation of `ConsensusVersion` and `RunMigrations`: https://github.com/cosmos/cosmos-sdk/pull/8485 +* Issue discussing `x/upgrade` design: https://github.com/cosmos/cosmos-sdk/issues/8514 diff --git a/versioned_docs/version-0.52/build/architecture/adr-042-group-module.md b/versioned_docs/version-0.52/build/architecture/adr-042-group-module.md new file mode 100644 index 000000000..df1db86ca --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-042-group-module.md @@ -0,0 +1,277 @@ +# ADR 042: Group Module + +## Changelog + +* 2020/04/09: Initial Draft + +## Status + +Draft + +## Abstract + +This ADR defines the `x/group` module which allows the creation and management of on-chain multi-signature accounts and enables voting for message execution based on configurable decision policies. + +## Context + +The legacy amino multi-signature mechanism of the Cosmos SDK has certain limitations: + +* Key rotation is not possible, although this can be solved with [account rekeying](adr-034-account-rekeying.md). +* Thresholds can't be changed. +* UX is cumbersome for non-technical users ([#5661](https://github.com/cosmos/cosmos-sdk/issues/5661)). +* It requires `legacy_amino` sign mode ([#8141](https://github.com/cosmos/cosmos-sdk/issues/8141)). + +While the group module is not meant to be a total replacement for the current multi-signature accounts, it provides a solution to the limitations described above, with a more flexible key management system where keys can be added, updated or removed, as well as configurable thresholds. +It's meant to be used with other access control modules such as [`x/feegrant`](./adr-029-fee-grant-module.md) and [`x/authz`](adr-030-authz-module.md) to simplify key management for individuals and organizations. + +## Decision + +We propose merging the `x/group` module with its supporting ORM/Table Store package ([#7098](https://github.com/cosmos/cosmos-sdk/issues/7098)) into the Cosmos SDK and continuing development here. There will be a dedicated ADR for the ORM package. + +### Group + +A group is a composition of accounts with associated weights. It is not +an account and doesn't have a balance. It doesn't in and of itself have any +sort of voting or decision weight. +Group members can create proposals and vote on them through group accounts using different decision policies. + +It has an `admin` account which can manage members in the group, update the group +metadata and set a new admin. + +```protobuf +message GroupInfo { + + // group_id is the unique ID of this group. + uint64 group_id = 1; + + // admin is the account address of the group's admin. + string admin = 2; + + // metadata is any arbitrary metadata to attached to the group. + bytes metadata = 3; + + // version is used to track changes to a group's membership structure that + // would break existing proposals. Whenever a member weight has changed, + // or any member is added or removed, the version is incremented and will + // invalidate all proposals from older versions. + uint64 version = 4; + + // total_weight is the sum of the group members' weights. + string total_weight = 5; +} +``` + +```protobuf +message GroupMember { + + // group_id is the unique ID of the group. + uint64 group_id = 1; + + // member is the member data. + Member member = 2; +} + +// Member represents a group member with an account address, +// non-zero weight and metadata. +message Member { + + // address is the member's account address. + string address = 1; + + // weight is the member's voting weight that should be greater than 0. + string weight = 2; + + // metadata is any arbitrary metadata to attached to the member. + bytes metadata = 3; +} +``` + +### Group Account + +A group account is an account associated with a group and a decision policy. +A group account does have a balance. + +Group accounts are abstracted from groups because a single group may have +multiple decision policies for different types of actions. Managing group +membership separately from decision policies results in the least overhead +and keeps membership consistent across different policies. The pattern that +is recommended is to have a single master group account for a given group, +and then to create separate group accounts with different decision policies +and delegate the desired permissions from the master account to +those "sub-accounts" using the [`x/authz` module](adr-030-authz-module.md). + +```protobuf +message GroupAccountInfo { + + // address is the group account address. + string address = 1; + + // group_id is the ID of the Group the GroupAccount belongs to. + uint64 group_id = 2; + + // admin is the account address of the group admin. + string admin = 3; + + // metadata is any arbitrary metadata of this group account. + bytes metadata = 4; + + // version is used to track changes to a group's GroupAccountInfo structure that + // invalidates active proposal from old versions. + uint64 version = 5; + + // decision_policy specifies the group account's decision policy. + google.protobuf.Any decision_policy = 6 [(cosmos_proto.accepts_interface) = "cosmos.group.v1.DecisionPolicy"]; +} +``` + +Similarly to a group admin, a group account admin can update its metadata, decision policy or set a new group account admin. + +A group account can also be an admin or a member of a group. +For instance, a group admin could be another group account which could "elects" the members or it could be the same group that elects itself. + +### Decision Policy + +A decision policy is the mechanism by which members of a group can vote on +proposals. + +All decision policies should have a minimum and maximum voting window. +The minimum voting window is the minimum duration that must pass in order +for a proposal to potentially pass, and it may be set to 0. The maximum voting +window is the maximum time that a proposal may be voted on and executed if +it reached enough support before it is closed. +Both of these values must be less than a chain-wide max voting window parameter. + +We define the `DecisionPolicy` interface that all decision policies must implement: + +```go +type DecisionPolicy interface { + codec.ProtoMarshaler + + ValidateBasic() error + GetTimeout() types.Duration + Allow(tally Tally, totalPower string, votingDuration time.Duration) (DecisionPolicyResult, error) + Validate(g GroupInfo) error +} + +type DecisionPolicyResult struct { + Allow bool + Final bool +} +``` + +#### Threshold decision policy + +A threshold decision policy defines a minimum support votes (_yes_), based on a tally +of voter weights, for a proposal to pass. For +this decision policy, abstain and veto are treated as no support (_no_). + +```protobuf +message ThresholdDecisionPolicy { + + // threshold is the minimum weighted sum of support votes for a proposal to succeed. + string threshold = 1; + + // voting_period is the duration from submission of a proposal to the end of voting period + // Within this period, votes and exec messages can be submitted. + google.protobuf.Duration voting_period = 2 [(gogoproto.nullable) = false]; +} +``` + +### Proposal + +Any member of a group can submit a proposal for a group account to decide upon. +A proposal consists of a set of `sdk.Msg`s that will be executed if the proposal +passes as well as any metadata associated with the proposal. These `sdk.Msg`s get validated as part of the `Msg/CreateProposal` request validation. They should also have their signer set as the group account. + +Internally, a proposal also tracks: + +* its current `Status`: submitted, closed or aborted +* its `Result`: unfinalized, accepted or rejected +* its `VoteState` in the form of a `Tally`, which is calculated on new votes and when executing the proposal. + +```protobuf +// Tally represents the sum of weighted votes. +message Tally { + option (gogoproto.goproto_getters) = false; + + // yes_count is the weighted sum of yes votes. + string yes_count = 1; + + // no_count is the weighted sum of no votes. + string no_count = 2; + + // abstain_count is the weighted sum of abstainers. + string abstain_count = 3; + + // veto_count is the weighted sum of vetoes. + string veto_count = 4; +} +``` + +### Voting + +Members of a group can vote on proposals. There are four choices to choose while voting - yes, no, abstain and veto. Not +all decision policies will support them. Votes can contain some optional metadata. +In the current implementation, the voting window begins as soon as a proposal +is submitted. + +Voting internally updates the proposal `VoteState` as well as `Status` and `Result` if needed. + +### Executing Proposals + +Proposals will not be automatically executed by the chain in this current design, +but rather a user must submit a `Msg/Exec` transaction to attempt to execute the +proposal based on the current votes and decision policy. A future upgrade could +automate this and have the group account (or a fee granter) pay. + +#### Changing Group Membership + +In the current implementation, updating a group or a group account after submitting a proposal will make it invalid. It will simply fail if someone calls `Msg/Exec` and will eventually be garbage collected. + +### Notes on current implementation + +This section outlines the current implementation used in the proof of concept of the group module but this could be subject to changes and iterated on. + +#### ORM + +The [ORM package](https://github.com/cosmos/cosmos-sdk/discussions/9156) defines tables, sequences and secondary indexes which are used in the group module. + +Groups are stored in state as part of a `groupTable`, the `group_id` being an auto-increment integer. Group members are stored in a `groupMemberTable`. + +Group accounts are stored in a `groupAccountTable`. The group account address is generated based on an auto-increment integer which is used to derive the group module `RootModuleKey` into a `DerivedModuleKey`, as stated in [ADR-033](adr-033-protobuf-inter-module-comm.md#modulekeys-and-moduleids). The group account is added as a new `ModuleAccount` through `x/auth`. + +Proposals are stored as part of the `proposalTable` using the `Proposal` type. The `proposal_id` is an auto-increment integer. + +Votes are stored in the `voteTable`. The primary key is based on the vote's `proposal_id` and `voter` account address. + +#### ADR-033 to route proposal messages + +Inter-module communication introduced by [ADR-033](adr-033-protobuf-inter-module-comm.md) can be used to route a proposal's messages using the `DerivedModuleKey` corresponding to the proposal's group account. + +## Consequences + +### Positive + +* Improved UX for multi-signature accounts allowing key rotation and custom decision policies. + +### Negative + +### Neutral + +* It uses ADR 033 so it will need to be implemented within the Cosmos SDK, but this doesn't imply necessarily any large refactoring of existing Cosmos SDK modules. +* The current implementation of the group module uses the ORM package. + +## Further Discussions + +* Convergence of `/group` and `x/gov` as both support proposals and voting: https://github.com/cosmos/cosmos-sdk/discussions/9066 +* `x/group` possible future improvements: + * Execute proposals on submission (https://github.com/regen-network/regen-ledger/issues/288) + * Withdraw a proposal (https://github.com/regen-network/cosmos-modules/issues/41) + * Make `Tally` more flexible and support non-binary choices + +## References + +* Initial specification: + * https://gist.github.com/aaronc/b60628017352df5983791cad30babe56#group-module + * [#5236](https://github.com/cosmos/cosmos-sdk/pull/5236) +* Proposal to add `x/group` into the Cosmos SDK: [#7633](https://github.com/cosmos/cosmos-sdk/issues/7633) diff --git a/versioned_docs/version-0.52/build/architecture/adr-043-nft-module.md b/versioned_docs/version-0.52/build/architecture/adr-043-nft-module.md new file mode 100644 index 000000000..40c21cd3c --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-043-nft-module.md @@ -0,0 +1,349 @@ +# ADR 43: NFT Module + +## Changelog + +* 2021-05-01: Initial Draft +* 2021-07-02: Review updates +* 2022-06-15: Add batch operation +* 2022-11-11: Remove strict validation of classID and tokenID + +## Status + +PROPOSED + +## Abstract + +This ADR defines the `x/nft` module which is a generic implementation of NFTs, roughly "compatible" with ERC721. **Applications using the `x/nft` module must implement the following functions**: + +* `MsgNewClass` - Receive the user's request to create a class, and call the `NewClass` of the `x/nft` module. +* `MsgUpdateClass` - Receive the user's request to update a class, and call the `UpdateClass` of the `x/nft` module. +* `MsgMintNFT` - Receive the user's request to mint an NFT, and call the `MintNFT` of the `x/nft` module. +* `BurnNFT` - Receive the user's request to burn an NFT, and call the `BurnNFT` of the `x/nft` module. +* `UpdateNFT` - Receive the user's request to update an NFT, and call the `UpdateNFT` of the `x/nft` module. + +## Context + +NFTs are more than just crypto art, which is very helpful for accruing value to the Cosmos ecosystem. As a result, Cosmos Hub should implement NFT functions and enable a unified mechanism for storing and sending the ownership representative of NFTs as discussed in https://github.com/cosmos/cosmos-sdk/discussions/9065. + +As discussed in [#9065](https://github.com/cosmos/cosmos-sdk/discussions/9065), several potential solutions can be considered: + +* irismod/nft and modules/incubator/nft +* CW721 +* DID NFTs +* interNFT + +Since functions/use cases of NFTs are tightly connected with their logic, it is almost impossible to support all the NFTs' use cases in one Cosmos SDK module by defining and implementing different transaction types. + +Considering generic usage and compatibility of interchain protocols including IBC and Gravity Bridge, it is preferred to have a generic NFT module design which handles the generic NFTs logic. +This design idea can enable composability that application-specific functions should be managed by other modules on Cosmos Hub or on other Zones by importing the NFT module. + +The current design is based on the work done by [IRISnet team](https://github.com/irisnet/irismod/tree/master/modules/nft) and an older implementation in the [Cosmos repository](https://github.com/cosmos/modules/tree/master/incubator/nft). + +## Decision + +We create a `x/nft` module, which contains the following functionality: + +* Store NFTs and track their ownership. +* Expose `Keeper` interface for composing modules to transfer, mint and burn NFTs. +* Expose external `Message` interface for users to transfer ownership of their NFTs. +* Query NFTs and their supply information. + +The proposed module is a base module for NFT app logic. Its goal is to provide a common layer for storage, basic transfer functionality and IBC. The module should not be used as a standalone. +Instead an app should create a specialized module to handle app specific logic (eg: NFT ID construction, royalty), user level minting and burning. Moreover an app specialized module should handle auxiliary data to support the app logic (eg indexes, ORM, business data). + +All data carried over IBC must be part of the `NFT` or `Class` type described below. The app specific NFT data should be encoded in `NFT.data` for cross-chain integrity. Other objects related to NFT, which are not important for integrity can be part of the app specific module. + +### Types + +We propose two main types: + +* `Class` -- describes NFT class. We can think about it as a smart contract address. +* `NFT` -- object representing unique, non-fungible asset. Each NFT is associated with a class. + +#### Class + +NFT **Class** is comparable to an ERC-721 smart contract (provides description of a smart contract), under which a collection of NFTs can be created and managed. + +```protobuf +message Class { + string id = 1; + string name = 2; + string symbol = 3; + string description = 4; + string uri = 5; + string uri_hash = 6; + google.protobuf.Any data = 7; +} +``` + +* `id` is used as the primary index for storing the class; _required_ +* `name` is a descriptive name of the NFT class; _optional_ +* `symbol` is the symbol usually shown on exchanges for the NFT class; _optional_ +* `description` is a detailed description of the NFT class; _optional_ +* `uri` is a URI for the class metadata stored off chain. It should be a JSON file that contains metadata about the NFT class and NFT data schema ([OpenSea example](https://docs.opensea.io/docs/contract-level-metadata)); _optional_ +* `uri_hash` is a hash of the document pointed by URI; _optional_ +* `data` is app specific metadata of the class; _optional_ + +#### NFT + +We define a general model for `NFT` as follows. + +```protobuf +message NFT { + string class_id = 1; + string id = 2; + string uri = 3; + string uri_hash = 4; + google.protobuf.Any data = 10; +} +``` + +* `class_id` is the identifier of the NFT class where the NFT belongs; _required_ +* `id` is an identifier of the NFT, unique within the scope of its class. It is specified by the creator of the NFT and may be expanded to use DID in the future. `class_id` combined with `id` uniquely identifies an NFT and is used as the primary index for storing the NFT; _required_ + + ```text + {class_id}/{id} --> NFT (bytes) + ``` + +* `uri` is a URI for the NFT metadata stored off chain. Should point to a JSON file that contains metadata about this NFT (Ref: [ERC721 standard and OpenSea extension](https://docs.opensea.io/docs/metadata-standards)); _required_ +* `uri_hash` is a hash of the document pointed by uri; _optional_ +* `data` is an app specific data of the NFT. Can be used by composing modules to specify additional properties of the NFT; _optional_ + +This ADR doesn't specify values that `data` can take; however, best practices recommend upper-level NFT modules clearly specify their contents. Although the value of this field doesn't provide the additional context required to manage NFT records, which means that the field can technically be removed from the specification, the field's existence allows basic informational/UI functionality. + +### `Keeper` Interface + +```go +type Keeper interface { + NewClass(ctx sdk.Context,class Class) + UpdateClass(ctx sdk.Context,class Class) + + Mint(ctx sdk.Context,nft NFT，receiver sdk.AccAddress) // updates totalSupply + BatchMint(ctx sdk.Context, tokens []NFT,receiver sdk.AccAddress) error + + Burn(ctx sdk.Context, classId string, nftId string) // updates totalSupply + BatchBurn(ctx sdk.Context, classID string, nftIDs []string) error + + Update(ctx sdk.Context, nft NFT) + BatchUpdate(ctx sdk.Context, tokens []NFT) error + + Transfer(ctx sdk.Context, classId string, nftId string, receiver sdk.AccAddress) + BatchTransfer(ctx sdk.Context, classID string, nftIDs []string, receiver sdk.AccAddress) error + + GetClass(ctx sdk.Context, classId string) Class + GetClasses(ctx sdk.Context) []Class + + GetNFT(ctx sdk.Context, classId string, nftId string) NFT + GetNFTsOfClassByOwner(ctx sdk.Context, classId string, owner sdk.AccAddress) []NFT + GetNFTsOfClass(ctx sdk.Context, classId string) []NFT + + GetOwner(ctx sdk.Context, classId string, nftId string) sdk.AccAddress + GetBalance(ctx sdk.Context, classId string, owner sdk.AccAddress) uint64 + GetTotalSupply(ctx sdk.Context, classId string) uint64 +} +``` + +Other business logic implementations should be defined in composing modules that import `x/nft` and use its `Keeper`. + +### `Msg` Service + +```protobuf +service Msg { + rpc Send(MsgSend) returns (MsgSendResponse); +} + +message MsgSend { + string class_id = 1; + string id = 2; + string sender = 3; + string receiver = 4; +} +message MsgSendResponse {} +``` + +`MsgSend` can be used to transfer the ownership of an NFT to another address. + +The implementation outline of the server is as follows: + +```go +type msgServer struct{ + k Keeper +} + +func (m msgServer) Send(ctx context.Context, msg *types.MsgSend) (*types.MsgSendResponse, error) { + // check current ownership + assertEqual(msg.Sender, m.k.GetOwner(msg.ClassId, msg.Id)) + + // transfer ownership + m.k.Transfer(msg.ClassId, msg.Id, msg.Receiver) + + return &types.MsgSendResponse{}, nil +} +``` + +The query service methods for the `x/nft` module are: + +```protobuf +service Query { + // Balance queries the number of NFTs of a given class owned by the owner, same as balanceOf in ERC721 + rpc Balance(QueryBalanceRequest) returns (QueryBalanceResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/balance/{owner}/{class_id}"; + } + + // Owner queries the owner of the NFT based on its class and id, same as ownerOf in ERC721 + rpc Owner(QueryOwnerRequest) returns (QueryOwnerResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/owner/{class_id}/{id}"; + } + + // Supply queries the number of NFTs from the given class, same as totalSupply of ERC721. + rpc Supply(QuerySupplyRequest) returns (QuerySupplyResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/supply/{class_id}"; + } + + // NFTs queries all NFTs of a given class or owner,choose at least one of the two, similar to tokenByIndex in ERC721Enumerable + rpc NFTs(QueryNFTsRequest) returns (QueryNFTsResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/nfts"; + } + + // NFT queries an NFT based on its class and id. + rpc NFT(QueryNFTRequest) returns (QueryNFTResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/nfts/{class_id}/{id}"; + } + + // Class queries an NFT class based on its id + rpc Class(QueryClassRequest) returns (QueryClassResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/classes/{class_id}"; + } + + // Classes queries all NFT classes + rpc Classes(QueryClassesRequest) returns (QueryClassesResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/classes"; + } +} + +// QueryBalanceRequest is the request type for the Query/Balance RPC method +message QueryBalanceRequest { + string class_id = 1; + string owner = 2; +} + +// QueryBalanceResponse is the response type for the Query/Balance RPC method +message QueryBalanceResponse { + uint64 amount = 1; +} + +// QueryOwnerRequest is the request type for the Query/Owner RPC method +message QueryOwnerRequest { + string class_id = 1; + string id = 2; +} + +// QueryOwnerResponse is the response type for the Query/Owner RPC method +message QueryOwnerResponse { + string owner = 1; +} + +// QuerySupplyRequest is the request type for the Query/Supply RPC method +message QuerySupplyRequest { + string class_id = 1; +} + +// QuerySupplyResponse is the response type for the Query/Supply RPC method +message QuerySupplyResponse { + uint64 amount = 1; +} + +// QueryNFTsRequest is the request type for the Query/NFTs RPC method +message QueryNFTsRequest { + string class_id = 1; + string owner = 2; + cosmos.base.query.v1beta1.PageRequest pagination = 3; +} + +// QueryNFTsResponse is the response type for the Query/NFTs RPC methods +message QueryNFTsResponse { + repeated cosmos.nft.v1beta1.NFT nfts = 1; + cosmos.base.query.v1beta1.PageResponse pagination = 2; +} + +// QueryNFTRequest is the request type for the Query/NFT RPC method +message QueryNFTRequest { + string class_id = 1; + string id = 2; +} + +// QueryNFTResponse is the response type for the Query/NFT RPC method +message QueryNFTResponse { + cosmos.nft.v1beta1.NFT nft = 1; +} + +// QueryClassRequest is the request type for the Query/Class RPC method +message QueryClassRequest { + string class_id = 1; +} + +// QueryClassResponse is the response type for the Query/Class RPC method +message QueryClassResponse { + cosmos.nft.v1beta1.Class class = 1; +} + +// QueryClassesRequest is the request type for the Query/Classes RPC method +message QueryClassesRequest { + // pagination defines an optional pagination for the request. + cosmos.base.query.v1beta1.PageRequest pagination = 1; +} + +// QueryClassesResponse is the response type for the Query/Classes RPC method +message QueryClassesResponse { + repeated cosmos.nft.v1beta1.Class classes = 1; + cosmos.base.query.v1beta1.PageResponse pagination = 2; +} +``` + +### Interoperability + +Interoperability is all about reusing assets between modules and chains. The former one is achieved by ADR-33: Protobuf client - server communication. At the time of writing ADR-33 is not finalized. The latter is achieved by IBC. Here we will focus on the IBC side. +IBC is implemented per module. Here, we aligned that NFTs will be recorded and managed in the x/nft. This requires creation of a new IBC standard and implementation of it. + +For IBC interoperability, NFT custom modules MUST use the NFT object type understood by the IBC client. So, for x/nft interoperability, custom NFT implementations (example: x/cryptokitty) should use the canonical x/nft module and proxy all NFT balance keeping functionality to x/nft or else re-implement all functionality using the NFT object type understood by the IBC client. In other words: x/nft becomes the standard NFT registry for all Cosmos NFTs (example: x/cryptokitty will register a kitty NFT in x/nft and use x/nft for book keeping). This was [discussed](https://github.com/cosmos/cosmos-sdk/discussions/9065#discussioncomment-873206) in the context of using x/bank as a general asset balance book. Not using x/nft will require implementing another module for IBC. + +## Consequences + +### Backward Compatibility + +No backward incompatibilities. + +### Forward Compatibility + +This specification conforms to the ERC-721 smart contract specification for NFT identifiers. Note that ERC-721 defines uniqueness based on (contract address, uint256 tokenId), and we conform to this implicitly because a single module is currently aimed to track NFT identifiers. Note: use of the (mutable) data field to determine uniqueness is not safe.s + +### Positive + +* NFT identifiers available on Cosmos Hub. +* Ability to build different NFT modules for the Cosmos Hub, e.g., ERC-721. +* NFT module which supports interoperability with IBC and other cross-chain infrastructures like Gravity Bridge + +### Negative + +* New IBC app is required for x/nft +* CW721 adapter is required + +### Neutral + +* Other functions need more modules. For example, a custody module is needed for NFT trading function, a collectible module is needed for defining NFT properties. + +## Further Discussions + +For other kinds of applications on the Hub, more app-specific modules can be developed in the future: + +* `x/nft/custody`: custody of NFTs to support trading functionality. +* `x/nft/marketplace`: selling and buying NFTs using sdk.Coins. +* `x/fractional`: a module to split an ownership of an asset (NFT or other assets) for multiple stakeholder. `x/group` should work for most of the cases. + +Other networks in the Cosmos ecosystem could design and implement their own NFT modules for specific NFT applications and use cases. + +## References + +* Initial discussion: https://github.com/cosmos/cosmos-sdk/discussions/9065 +* x/nft: initialize module: https://github.com/cosmos/cosmos-sdk/pull/9174 +* [ADR 033](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-033-protobuf-inter-module-comm.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-044-protobuf-updates-guidelines.md b/versioned_docs/version-0.52/build/architecture/adr-044-protobuf-updates-guidelines.md new file mode 100644 index 000000000..93e28d2d1 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-044-protobuf-updates-guidelines.md @@ -0,0 +1,129 @@ +# ADR 044: Guidelines for Updating Protobuf Definitions + +## Changelog + +* 28.06.2021: Initial Draft +* 02.12.2021: Add `Since:` comment for new fields +* 21.07.2022: Remove the rule of no new `Msg` in the same proto version. + +## Status + +Draft + +## Abstract + +This ADR provides guidelines and recommended practices when updating Protobuf definitions. These guidelines are targeting module developers. + +## Context + +The Cosmos SDK maintains a set of [Protobuf definitions](https://github.com/cosmos/cosmos-sdk/tree/main/proto/cosmos). It is important to correctly design Protobuf definitions to avoid any breaking changes within the same version. The reasons are to not break tooling (including indexers and explorers), wallets and other third-party integrations. + +When making changes to these Protobuf definitions, the Cosmos SDK currently only follows [Buf's](https://docs.buf.build/) recommendations. We noticed however that Buf's recommendations might still result in breaking changes in the SDK in some cases. For example: + +* Adding fields to `Msg`s. Adding fields is a not a Protobuf spec-breaking operation. However, when adding new fields to `Msg`s, the unknown field rejection will throw an error when sending the new `Msg` to an older node. +* Marking fields as `reserved`. Protobuf proposes the `reserved` keyword for removing fields without the need to bump the package version. However, by doing so, client backwards compatibility is broken as Protobuf doesn't generate anything for `reserved` fields. See [#9446](https://github.com/cosmos/cosmos-sdk/issues/9446) for more details on this issue. + +Moreover, module developers often face other questions around Protobuf definitions such as "Can I rename a field?" or "Can I deprecate a field?" This ADR aims to answer all these questions by providing clear guidelines about allowed updates for Protobuf definitions. + +## Decision + +We decide to keep [Buf's](https://docs.buf.build/) recommendations with the following exceptions: + +* `UNARY_RPC`: the Cosmos SDK currently does not support streaming RPCs. +* `COMMENT_FIELD`: the Cosmos SDK allows fields with no comments. +* `SERVICE_SUFFIX`: we use the `Query` and `Msg` service naming convention, which doesn't use the `-Service` suffix. +* `PACKAGE_VERSION_SUFFIX`: some packages, such as `cosmos.crypto.ed25519`, don't use a version suffix. +* `RPC_REQUEST_STANDARD_NAME`: Requests for the `Msg` service don't have the `-Request` suffix to keep backwards compatibility. + +On top of Buf's recommendations we add the following guidelines that are specific to the Cosmos SDK. + +### Updating Protobuf Definition Without Bumping Version + +#### 1. Module developers MAY add new Protobuf definitions + +Module developers MAY add new `message`s, new `Service`s, new `rpc` endpoints, and new fields to existing messages. This recommendation follows the Protobuf specification, but is added in this document for clarity, as the SDK requires one additional change. + +The SDK requires the Protobuf comment of the new addition to contain one line with the following format: + +```protobuf +// Since: cosmos-sdk {, ...} +``` + +Where each `version` denotes a minor ("0.45") or patch ("0.44.5") version from which the field is available. This will greatly help client libraries, who can optionally use reflection or custom code generation to show/hide these fields depending on the targeted node version. + +As examples, the following comments are valid: + +```protobuf +// Since: cosmos-sdk 0.44 + +// Since: cosmos-sdk 0.42.11, 0.44.5 +``` + +and the following ones are NOT valid: + +```protobuf +// Since cosmos-sdk v0.44 + +// since: cosmos-sdk 0.44 + +// Since: cosmos-sdk 0.42.11 0.44.5 + +// Since: Cosmos SDK 0.42.11, 0.44.5 +``` + +#### 2. Fields MAY be marked as `deprecated`, and nodes MAY implement a protocol-breaking change for handling these fields + +Protobuf supports the [`deprecated` field option](https://protobuf.dev/programming-guides/proto2/), and this option MAY be used on any field, including `Msg` fields. If a node handles a Protobuf message with a non-empty deprecated field, the node MAY change its behavior upon processing it, even in a protocol-breaking way. When possible, the node MUST handle backwards compatibility without breaking the consensus (unless we increment the proto version). + +As an example, the Cosmos SDK v0.42 to v0.43 update contained two Protobuf-breaking changes, listed below. Instead of bumping the package versions from `v1beta1` to `v1`, the SDK team decided to follow this guideline, by reverting the breaking changes, marking those changes as deprecated, and modifying the node implementation when processing messages with deprecated fields. More specifically: + +* The Cosmos SDK recently removed support for [time-based software upgrades](https://github.com/cosmos/cosmos-sdk/pull/8849). As such, the `time` field has been marked as deprecated in `cosmos.upgrade.v1beta1.Plan`. Moreover, the node will reject any proposal containing an upgrade Plan whose `time` field is non-empty. +* The Cosmos SDK now supports [governance split votes](./adr-037-gov-split-vote.md). When querying for votes, the returned `cosmos.gov.v1beta1.Vote` message has its `option` field (used for 1 vote option) deprecated in favor of its `options` field (allowing multiple vote options). Whenever possible, the SDK still populates the deprecated `option` field, that is, if and only if the `len(options) == 1` and `options[0].Weight == 1.0`. + +#### 3. Fields MUST NOT be renamed + +Whereas the official Protobuf recommendations do not prohibit renaming fields, as it does not break the Protobuf binary representation, the SDK explicitly forbids renaming fields in Protobuf structs. The main reason for this choice is to avoid introducing breaking changes for clients, which often rely on hard-coded fields from generated types. Moreover, renaming fields will lead to client-breaking JSON representations of Protobuf definitions, used in REST endpoints and in the CLI. + +### Incrementing Protobuf Package Version + +TODO, needs architecture review. Some topics: + +* Bumping versions frequency +* When bumping versions, should the Cosmos SDK support both versions? + * i.e. v1beta1 -> v1, should we have two folders in the Cosmos SDK, and handlers for both versions? +* mention ADR-023 Protobuf naming + +## Consequences + +> This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the "positive" ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and project in the future. + +### Backwards Compatibility + +> All ADRs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The ADR must explain how the author proposes to deal with these incompatibilities. ADR submissions without a sufficient backwards compatibility treatise may be rejected outright. + +### Positive + +* less pain to tool developers +* more compatibility in the ecosystem +* ... + +### Negative + +{negative consequences} + +### Neutral + +* more rigor in Protobuf review + +## Further Discussions + +This ADR is still in the DRAFT stage, and the "Incrementing Protobuf Package Version" will be filled in once we make a decision on how to correctly do it. + +## Test Cases [optional] + +Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable. + +## References + +* [#9445](https://github.com/cosmos/cosmos-sdk/issues/9445) Release proto definitions v1 +* [#9446](https://github.com/cosmos/cosmos-sdk/issues/9446) Address v1beta1 proto breaking changes diff --git a/versioned_docs/version-0.52/build/architecture/adr-045-check-delivertx-middlewares.md b/versioned_docs/version-0.52/build/architecture/adr-045-check-delivertx-middlewares.md new file mode 100644 index 000000000..60172977c --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-045-check-delivertx-middlewares.md @@ -0,0 +1,312 @@ +# ADR 045: BaseApp `{Check,Deliver}Tx` as Middlewares + +## Changelog + +* 20.08.2021: Initial draft. +* 07.12.2021: Update `tx.Handler` interface ([\#10693](https://github.com/cosmos/cosmos-sdk/pull/10693)). +* 17.05.2022: ADR is abandoned, as middlewares are deemed too hard to reason about. + +## Status + +ABANDONED. Replacement is being discussed in [#11955](https://github.com/cosmos/cosmos-sdk/issues/11955). + +## Abstract + +This ADR replaces the current BaseApp `runTx` and antehandlers design with a middleware-based design. + +## Context + +BaseApp's implementation of ABCI `{Check,Deliver}Tx()` and its own `Simulate()` method call the `runTx` method under the hood, which first runs antehandlers, then executes `Msg`s. However, the [transaction Tips](https://github.com/cosmos/cosmos-sdk/issues/9406) and [refunding unused gas](https://github.com/cosmos/cosmos-sdk/issues/2150) use cases require custom logic to be run after the `Msg`s execution. There is currently no way to achieve this. + +An naive solution would be to add post-`Msg` hooks to BaseApp. However, the Cosmos SDK team thinks in parallel about the bigger picture of making app wiring simpler ([#9181](https://github.com/cosmos/cosmos-sdk/discussions/9182)), which includes making BaseApp more lightweight and modular. + +## Decision + +We decide to transform Baseapp's implementation of ABCI `{Check,Deliver}Tx` and its own `Simulate` methods to use a middleware-based design. + +The two following interfaces are the base of the middleware design, and are defined in `types/tx`: + +```go +type Handler interface { + CheckTx(ctx context.Context, req Request, checkReq RequestCheckTx) (Response, ResponseCheckTx, error) + DeliverTx(ctx context.Context, req Request) (Response, error) + SimulateTx(ctx context.Context, req Request (Response, error) +} + +type Middleware func(Handler) Handler +``` + +where we define the following arguments and return types: + +```go +type Request struct { + Tx sdk.Tx + TxBytes []byte +} + +type Response struct { + GasWanted uint64 + GasUsed uint64 + // MsgResponses is an array containing each Msg service handler's response + // type, packed in an Any. This will get proto-serialized into the `Data` field + // in the ABCI Check/DeliverTx responses. + MsgResponses []*codectypes.Any + Log string + Events []abci.Event +} + +type RequestCheckTx struct { + Type abci.CheckTxType +} + +type ResponseCheckTx struct { + Priority int64 +} +``` + +Please note that because CheckTx handles separate logic related to mempool priotization, its signature is different than DeliverTx and SimulateTx. + +BaseApp holds a reference to a `tx.Handler`: + +```go +type BaseApp struct { + // other fields + txHandler tx.Handler +} +``` + +Baseapp's ABCI `{Check,Deliver}Tx()` and `Simulate()` methods simply call `app.txHandler.{Check,Deliver,Simulate}Tx()` with the relevant arguments. For example, for `DeliverTx`: + +```go +func (app *BaseApp) DeliverTx(req abci.RequestDeliverTx) abci.ResponseDeliverTx { + var abciRes abci.ResponseDeliverTx + ctx := app.getContextForTx(runTxModeDeliver, req.Tx) + res, err := app.txHandler.DeliverTx(ctx, tx.Request{TxBytes: req.Tx}) + if err != nil { + abciRes = sdkerrors.ResponseDeliverTx(err, uint64(res.GasUsed), uint64(res.GasWanted), app.trace) + return abciRes + } + + abciRes, err = convertTxResponseToDeliverTx(res) + if err != nil { + return sdkerrors.ResponseDeliverTx(err, uint64(res.GasUsed), uint64(res.GasWanted), app.trace) + } + + return abciRes +} + +// convertTxResponseToDeliverTx converts a tx.Response into a abci.ResponseDeliverTx. +func convertTxResponseToDeliverTx(txRes tx.Response) (abci.ResponseDeliverTx, error) { + data, err := makeABCIData(txRes) + if err != nil { + return abci.ResponseDeliverTx{}, nil + } + + return abci.ResponseDeliverTx{ + Data: data, + Log: txRes.Log, + Events: txRes.Events, + }, nil +} + +// makeABCIData generates the Data field to be sent to ABCI Check/DeliverTx. +func makeABCIData(txRes tx.Response) ([]byte, error) { + return proto.Marshal(&sdk.TxMsgData{MsgResponses: txRes.MsgResponses}) +} +``` + +The implementations are similar for `BaseApp.CheckTx` and `BaseApp.Simulate`. + +`baseapp.txHandler`'s three methods' implementations can obviously be monolithic functions, but for modularity we propose a middleware composition design, where a middleware is simply a function that takes a `tx.Handler`, and returns another `tx.Handler` wrapped around the previous one. + +### Implementing a Middleware + +In practice, middlewares are created by Go function that takes as arguments some parameters needed for the middleware, and returns a `tx.Middleware`. + +For example, for creating an arbitrary `MyMiddleware`, we can implement: + +```go +// myTxHandler is the tx.Handler of this middleware. Note that it holds a +// reference to the next tx.Handler in the stack. +type myTxHandler struct { + // next is the next tx.Handler in the middleware stack. + next tx.Handler + // some other fields that are relevant to the middleware can be added here +} + +// NewMyMiddleware returns a middleware that does this and that. +func NewMyMiddleware(arg1, arg2) tx.Middleware { + return func (txh tx.Handler) tx.Handler { + return myTxHandler{ + next: txh, + // optionally, set arg1, arg2... if they are needed in the middleware + } + } +} + +// Assert myTxHandler is a tx.Handler. +var _ tx.Handler = myTxHandler{} + +func (h myTxHandler) CheckTx(ctx context.Context, req Request, checkReq RequestcheckTx) (Response, ResponseCheckTx, error) { + // CheckTx specific pre-processing logic + + // run the next middleware + res, checkRes, err := txh.next.CheckTx(ctx, req, checkReq) + + // CheckTx specific post-processing logic + + return res, checkRes, err +} + +func (h myTxHandler) DeliverTx(ctx context.Context, req Request) (Response, error) { + // DeliverTx specific pre-processing logic + + // run the next middleware + res, err := txh.next.DeliverTx(ctx, tx, req) + + // DeliverTx specific post-processing logic + + return res, err +} + +func (h myTxHandler) SimulateTx(ctx context.Context, req Request) (Response, error) { + // SimulateTx specific pre-processing logic + + // run the next middleware + res, err := txh.next.SimulateTx(ctx, tx, req) + + // SimulateTx specific post-processing logic + + return res, err +} +``` + +### Composing Middlewares + +While BaseApp simply holds a reference to a `tx.Handler`, this `tx.Handler` itself is defined using a middleware stack. The Cosmos SDK exposes a base (i.e. innermost) `tx.Handler` called `RunMsgsTxHandler`, which executes messages. + +Then, the app developer can compose multiple middlewares on top on the base `tx.Handler`. Each middleware can run pre-and-post-processing logic around its next middleware, as described in the section above. Conceptually, as an example, given the middlewares `A`, `B`, and `C` and the base `tx.Handler` `H` the stack looks like: + +```text +A.pre + B.pre + C.pre + H # The base tx.handler, for example `RunMsgsTxHandler` + C.post + B.post +A.post +``` + +We define a `ComposeMiddlewares` function for composing middlewares. It takes the base handler as first argument, and middlewares in the "outer to inner" order. For the above stack, the final `tx.Handler` is: + +```go +txHandler := middleware.ComposeMiddlewares(H, A, B, C) +``` + +The middleware is set in BaseApp via its `SetTxHandler` setter: + +```go +// simapp/app.go + +txHandler := middleware.ComposeMiddlewares(...) +app.SetTxHandler(txHandler) +``` + +The app developer can define their own middlewares, or use the Cosmos SDK's pre-defined middlewares from `middleware.NewDefaultTxHandler()`. + +### Middlewares Maintained by the Cosmos SDK + +While the app developer can define and compose the middlewares of their choice, the Cosmos SDK provides a set of middlewares that caters for the ecosystem's most common use cases. These middlewares are: + +| Middleware | Description | +| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| RunMsgsTxHandler | This is the base `tx.Handler`. It replaces the old baseapp's `runMsgs`, and executes a transaction's `Msg`s. | +| TxDecoderMiddleware | This middleware takes in transaction raw bytes, and decodes them into a `sdk.Tx`. It replaces the `baseapp.txDecoder` field, so that BaseApp stays as thin as possible. Since most middlewares read the contents of the `sdk.Tx`, the TxDecoderMiddleware should be run first in the middleware stack. | +| {Antehandlers} | Each antehandler is converted to its own middleware. These middlewares perform signature verification, fee deductions and other validations on the incoming transaction. | +| IndexEventsTxMiddleware | This is a simple middleware that chooses which events to index in Tendermint. Replaces `baseapp.indexEvents` (which unfortunately still exists in baseapp too, because it's used to index Begin/EndBlock events) | +| RecoveryTxMiddleware | This index recovers from panics. It replaces baseapp.runTx's panic recovery described in [ADR-022](./adr-022-custom-panic-handling.md). | +| GasTxMiddleware | This replaces the [`Setup`](https://github.com/cosmos/cosmos-sdk/blob/v0.43.0/x/auth/ante/setup.go) Antehandler. It sets a GasMeter on sdk.Context. Note that before, GasMeter was set on sdk.Context inside the antehandlers, and there was some mess around the fact that antehandlers had their own panic recovery system so that the GasMeter could be read by baseapp's recovery system. Now, this mess is all removed: one middleware sets GasMeter, another one handles recovery. | + +### Similarities and Differences between Antehandlers and Middlewares + +The middleware-based design builds upon the existing antehandlers design described in [ADR-010](./adr-010-modular-antehandler.md). Even though the final decision of ADR-010 was to go with the "Simple Decorators" approach, the middleware design is actually very similar to the other [Decorator Pattern](./adr-010-modular-antehandler.md#decorator-pattern) proposal, also used in [weave](https://github.com/iov-one/weave). + +#### Similarities with Antehandlers + +* Designed as chaining/composing small modular pieces. +* Allow code reuse for `{Check,Deliver}Tx` and for `Simulate`. +* Set up in `app.go`, and easily customizable by app developers. +* Order is important. + +#### Differences with Antehandlers + +* The Antehandlers are run before `Msg` execution, whereas middlewares can run before and after. +* The middleware approach uses separate methods for `{Check,Deliver,Simulate}Tx`, whereas the antehandlers pass a `simulate bool` flag and uses the `sdkCtx.Is{Check,Recheck}Tx()` flags to determine in which transaction mode we are. +* The middleware design lets each middleware hold a reference to the next middleware, whereas the antehandlers pass a `next` argument in the `AnteHandle` method. +* The middleware design use Go's standard `context.Context`, whereas the antehandlers use `sdk.Context`. + +## Consequences + +### Backwards Compatibility + +Since this refactor removes some logic away from BaseApp and into middlewares, it introduces API-breaking changes for app developers. Most notably, instead of creating an antehandler chain in `app.go`, app developers need to create a middleware stack: + +```diff +- anteHandler, err := ante.NewAnteHandler( +- ante.HandlerOptions{ +- AccountKeeper: app.AccountKeeper, +- BankKeeper: app.BankKeeper, +- SignModeHandler: encodingConfig.TxConfig.SignModeHandler(), +- FeegrantKeeper: app.FeeGrantKeeper, +- SigGasConsumer: ante.DefaultSigVerificationGasConsumer, +- }, +-) ++txHandler, err := authmiddleware.NewDefaultTxHandler(authmiddleware.TxHandlerOptions{ ++ Debug: app.Trace(), ++ IndexEvents: indexEvents, ++ LegacyRouter: app.legacyRouter, ++ MsgServiceRouter: app.msgSvcRouter, ++ LegacyAnteHandler: anteHandler, ++ TxDecoder: encodingConfig.TxConfig.TxDecoder, ++}) +if err != nil { + panic(err) +} +- app.SetAnteHandler(anteHandler) ++ app.SetTxHandler(txHandler) +``` + +Other more minor API breaking changes will also be provided in the CHANGELOG. As usual, the Cosmos SDK will provide a release migration document for app developers. + +This ADR does not introduce any state-machine-, client- or CLI-breaking changes. + +### Positive + +* Allow custom logic to be run before an after `Msg` execution. This enables the [tips](https://github.com/cosmos/cosmos-sdk/issues/9406) and [gas refund](https://github.com/cosmos/cosmos-sdk/issues/2150) uses cases, and possibly other ones. +* Make BaseApp more lightweight, and defer complex logic to small modular components. +* Separate paths for `{Check,Deliver,Simulate}Tx` with different returns types. This allows for improved readability (replace `if sdkCtx.IsRecheckTx() && !simulate {...}` with separate methods) and more flexibility (e.g. returning a `priority` in `ResponseCheckTx`). + +### Negative + +* It is hard to understand at first glance the state updates that would occur after a middleware runs given the `sdk.Context` and `tx`. A middleware can have an arbitrary number of nested middleware being called within its function body, each possibly doing some pre- and post-processing before calling the next middleware on the chain. Thus to understand what a middleware is doing, one must also understand what every other middleware further along the chain is also doing, and the order of middlewares matters. This can get quite complicated to understand. +* API-breaking changes for app developers. + +### Neutral + +No neutral consequences. + +## Further Discussions + +* [#9934](https://github.com/cosmos/cosmos-sdk/discussions/9934) Decomposing BaseApp's other ABCI methods into middlewares. +* Replace `sdk.Tx` interface with the concrete protobuf Tx type in the `tx.Handler` methods signature. + +## Test Cases + +We update the existing baseapp and antehandlers tests to use the new middleware API, but keep the same test cases and logic, to avoid introducing regressions. Existing CLI tests will also be left untouched. + +For new middlewares, we introduce unit tests. Since middlewares are purposefully small, unit tests suit well. + +## References + +* Initial discussion: https://github.com/cosmos/cosmos-sdk/issues/9585 +* Implementation: [#9920 BaseApp refactor](https://github.com/cosmos/cosmos-sdk/pull/9920) and [#10028 Antehandlers migration](https://github.com/cosmos/cosmos-sdk/pull/10028) diff --git a/versioned_docs/version-0.52/build/architecture/adr-046-module-params.md b/versioned_docs/version-0.52/build/architecture/adr-046-module-params.md new file mode 100644 index 000000000..b0b60863f --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-046-module-params.md @@ -0,0 +1,184 @@ +# ADR 046: Module Params + +## Changelog + +* Sep 22, 2021: Initial Draft + +## Status + +ACCEPTED + +## Abstract + +This ADR describes an alternative approach to how Cosmos SDK modules use, interact, +and store their respective parameters. + +## Context + +Currently, in the Cosmos SDK, modules that require the use of parameters use the +`x/params` module. The `x/params` works by having modules define parameters, +typically via a simple `Params` structure, and registering that structure in +the `x/params` module via a unique `Subspace` that belongs to the respective +registering module. The registering module then has unique access to its respective +`Subspace`. Through this `Subspace`, the module can get and set its `Params` +structure. + +In addition, the Cosmos SDK's `x/gov` module has direct support for changing +parameters on-chain via a `ParamChangeProposal` governance proposal type, where +stakeholders can vote on suggested parameter changes. + +There are various tradeoffs to using the `x/params` module to manage individual +module parameters. Namely, managing parameters essentially comes for "free" in +that developers only need to define the `Params` struct, the `Subspace`, and the +various auxiliary functions, e.g. `ParamSetPairs`, on the `Params` type. However, +there are some notable drawbacks. These drawbacks include the fact that parameters +are serialized in state via JSON which is extremely slow. In addition, parameter +changes via `ParamChangeProposal` governance proposals have no way of reading from +or writing to state. In other words, it is currently not possible to have any +state transitions in the application during an attempt to change param(s). + +## Decision + +We will build off of the alignment of `x/gov` and `x/authz` work per +[#9810](https://github.com/cosmos/cosmos-sdk/pull/9810). Namely, module developers +will create one or more unique parameter data structures that must be serialized +to state. The Param data structures must implement `sdk.Msg` interface with respective +Protobuf Msg service method which will validate and update the parameters with all +necessary changes. The `x/gov` module via the work done in +[#9810](https://github.com/cosmos/cosmos-sdk/pull/9810), will dispatch Param +messages, which will be handled by Protobuf Msg services. + +Note, it is up to developers to decide how to structure their parameters and +the respective `sdk.Msg` messages. Consider the parameters currently defined in +`x/auth` using the `x/params` module for parameter management: + +```protobuf +message Params { + uint64 max_memo_characters = 1; + uint64 tx_sig_limit = 2; + uint64 tx_size_cost_per_byte = 3; + uint64 sig_verify_cost_ed25519 = 4; + uint64 sig_verify_cost_secp256k1 = 5; +} +``` + +Developers can choose to either create a unique data structure for every field in +`Params` or they can create a single `Params` structure as outlined above in the +case of `x/auth`. + +In the former, `x/params`, approach, a `sdk.Msg` would need to be created for every single +field along with a handler. This can become burdensome if there are a lot of +parameter fields. In the latter case, there is only a single data structure and +thus only a single message handler, however, the message handler might have to be +more sophisticated in that it might need to understand what parameters are being +changed vs what parameters are untouched. + +Params change proposals are made using the `x/gov` module. Execution is done through +`x/authz` authorization to the root `x/gov` module's account. + +Continuing to use `x/auth`, we demonstrate a more complete example: + +```go +type Params struct { + MaxMemoCharacters uint64 + TxSigLimit uint64 + TxSizeCostPerByte uint64 + SigVerifyCostED25519 uint64 + SigVerifyCostSecp256k1 uint64 +} + +type MsgUpdateParams struct { + MaxMemoCharacters uint64 + TxSigLimit uint64 + TxSizeCostPerByte uint64 + SigVerifyCostED25519 uint64 + SigVerifyCostSecp256k1 uint64 +} + +type MsgUpdateParamsResponse struct {} + +func (ms msgServer) UpdateParams(goCtx context.Context, msg *types.MsgUpdateParams) (*types.MsgUpdateParamsResponse, error) { + ctx := sdk.UnwrapSDKContext(goCtx) + + // verification logic... + + // persist params + params := ParamsFromMsg(msg) + ms.SaveParams(ctx, params) + + return &types.MsgUpdateParamsResponse{}, nil +} + +func ParamsFromMsg(msg *types.MsgUpdateParams) Params { + // ... +} +``` + +A gRPC `Service` query should also be provided, for example: + +```protobuf +service Query { + // ... + + rpc Params(QueryParamsRequest) returns (QueryParamsResponse) { + option (google.api.http).get = "/cosmos//v1beta1/params"; + } +} + +message QueryParamsResponse { + Params params = 1 [(gogoproto.nullable) = false]; +} +``` + +## Consequences + +As a result of implementing the module parameter methodology, we gain the ability +for module parameter changes to be stateful and extensible to fit nearly every +application's use case. We will be able to emit events (and trigger hooks registered +to that events using the work proposed in [event hooks](https://github.com/cosmos/cosmos-sdk/discussions/9656)), +call other Msg service methods or perform migration. +In addition, there will be significant gains in performance when it comes to reading +and writing parameters from and to state, especially if a specific set of parameters +are read on a consistent basis. + +However, this methodology will require developers to implement more types and +Msg service metohds which can become burdensome if many parameters exist. In addition, +developers are required to implement persistence logics of module parameters. +However, this should be trivial. + +### Backwards Compatibility + +The new method for working with module parameters is naturally not backwards +compatible with the existing `x/params` module. However, the `x/params` will +remain in the Cosmos SDK and will be marked as deprecated with no additional +functionality being added apart from potential bug fixes. Note, the `x/params` +module may be removed entirely in a future release. + +### Positive + +* Module parameters are serialized more efficiently +* Modules are able to react on parameters changes and perform additional actions. +* Special events can be emitted, allowing hooks to be triggered. + +### Negative + +* Module parameters becomes slightly more burdensome for module developers: + * Modules are now responsible for persisting and retrieving parameter state + * Modules are now required to have unique message handlers to handle parameter + changes per unique parameter data structure. + +### Neutral + +* Requires [#9810](https://github.com/cosmos/cosmos-sdk/pull/9810) to be reviewed + and merged. + + + +## References + +* https://github.com/cosmos/cosmos-sdk/pull/9810 +* https://github.com/cosmos/cosmos-sdk/issues/9438 +* https://github.com/cosmos/cosmos-sdk/discussions/9913 diff --git a/versioned_docs/version-0.52/build/architecture/adr-047-extend-upgrade-plan.md b/versioned_docs/version-0.52/build/architecture/adr-047-extend-upgrade-plan.md new file mode 100644 index 000000000..3500bb334 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-047-extend-upgrade-plan.md @@ -0,0 +1,253 @@ +# ADR 047: Extend Upgrade Plan + +## Changelog + +* Nov, 23, 2021: Initial Draft +* May, 16, 2023: Proposal ABANDONED. `pre_run` and `post_run` are not necessary anymore and adding the `artifacts` brings minor benefits. + +## Status + +ABANDONED + +## Abstract + +This ADR expands the existing x/upgrade `Plan` proto message to include new fields for defining pre-run and post-run processes within upgrade tooling. +It also defines a structure for providing downloadable artifacts involved in an upgrade. + +## Context + +The `upgrade` module in conjunction with Cosmovisor are designed to facilitate and automate a blockchain's transition from one version to another. + +Users submit a software upgrade governance proposal containing an upgrade `Plan`. +The [Plan](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/proto/cosmos/upgrade/v1beta1/upgrade.proto#L12) currently contains the following fields: + +* `name`: A short string identifying the new version. +* `height`: The chain height at which the upgrade is to be performed. +* `info`: A string containing information about the upgrade. + +The `info` string can be anything. +However, Cosmovisor will try to use the `info` field to automatically download a new version of the blockchain executable. +For the auto-download to work, Cosmovisor expects it to be either a stringified JSON object (with a specific structure defined through documentation), or a URL that will return such JSON. +The JSON object identifies URLs used to download the new blockchain executable for different platforms (OS and Architecture, e.g. "linux/amd64"). +Such a URL can either return the executable file directly or can return an archive containing the executable and possibly other assets. + +If the URL returns an archive, it is decompressed into `{DAEMON_HOME}/cosmovisor/{upgrade name}`. +Then, if `{DAEMON_HOME}/cosmovisor/{upgrade name}/bin/{DAEMON_NAME}` does not exist, but `{DAEMON_HOME}/cosmovisor/{upgrade name}/{DAEMON_NAME}` does, the latter is copied to the former. +If the URL returns something other than an archive, it is downloaded to `{DAEMON_HOME}/cosmovisor/{upgrade name}/bin/{DAEMON_NAME}`. + +If an upgrade height is reached and the new version of the executable version isn't available, Cosmovisor will stop running. + +Both `DAEMON_HOME` and `DAEMON_NAME` are [environment variables used to configure Cosmovisor](https://github.com/cosmos/cosmos-sdk/blob/cosmovisor/v1.0.0/cosmovisor/README.md#command-line-arguments-and-environment-variables). + +Currently, there is no mechanism that makes Cosmovisor run a command after the upgraded chain has been restarted. + +The current upgrade process has this timeline: + +1. An upgrade governance proposal is submitted and approved. +1. The upgrade height is reached. +1. The `x/upgrade` module writes the `upgrade_info.json` file. +1. The chain halts. +1. Cosmovisor backs up the data directory (if set up to do so). +1. Cosmovisor downloads the new executable (if not already in place). +1. Cosmovisor executes the `${DAEMON_NAME} pre-upgrade`. +1. Cosmovisor restarts the app using the new version and same args originally provided. + +## Decision + +### Protobuf Updates + +We will update the `x/upgrade.Plan` message for providing upgrade instructions. +The upgrade instructions will contain a list of artifacts available for each platform. +It allows for the definition of a pre-run and post-run commands. +These commands are not consensus guaranteed; they will be executed by Cosmosvisor (or other) during its upgrade handling. + +```protobuf +message Plan { + // ... (existing fields) + + UpgradeInstructions instructions = 6; +} +``` + +The new `UpgradeInstructions instructions` field MUST be optional. + +```protobuf +message UpgradeInstructions { + string pre_run = 1; + string post_run = 2; + repeated Artifact artifacts = 3; + string description = 4; +} +``` + +All fields in the `UpgradeInstructions` are optional. + +* `pre_run` is a command to run prior to the upgraded chain restarting. + If defined, it will be executed after halting and downloading the new artifact but before restarting the upgraded chain. + The working directory this command runs from MUST be `{DAEMON_HOME}/cosmovisor/{upgrade name}`. + This command MUST behave the same as the current [pre-upgrade](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/docs/migrations/pre-upgrade.md) command. + It does not take in any command-line arguments and is expected to terminate with the following exit codes: + + | Exit status code | How it is handled in Cosmosvisor | + |------------------|---------------------------------------------------------------------------------------------------------------------| + | `0` | Assumes `pre-upgrade` command executed successfully and continues the upgrade. | + | `1` | Default exit code when `pre-upgrade` command has not been implemented. | + | `30` | `pre-upgrade` command was executed but failed. This fails the entire upgrade. | + | `31` | `pre-upgrade` command was executed but failed. But the command is retried until exit code `1` or `30` are returned. | + If defined, then the app supervisors (e.g. Cosmovisor) MUST NOT run `app pre-run`. +* `post_run` is a command to run after the upgraded chain has been started. If defined, this command MUST be only executed at most once by an upgrading node. + The output and exit code SHOULD be logged but SHOULD NOT affect the running of the upgraded chain. + The working directory this command runs from MUST be `{DAEMON_HOME}/cosmovisor/{upgrade name}`. +* `artifacts` define items to be downloaded. + It SHOULD have only one entry per platform. +* `description` contains human-readable information about the upgrade and might contain references to external resources. + It SHOULD NOT be used for structured processing information. + +```protobuf +message Artifact { + string platform = 1; + string url = 2; + string checksum = 3; + string checksum_algo = 4; +} +``` + +* `platform` is a required string that SHOULD be in the format `{OS}/{CPU}`, e.g. `"linux/amd64"`. + The string `"any"` SHOULD also be allowed. + An `Artifact` with a `platform` of `"any"` SHOULD be used as a fallback when a specific `{OS}/{CPU}` entry is not found. + That is, if an `Artifact` exists with a `platform` that matches the system's OS and CPU, that should be used; + otherwise, if an `Artifact` exists with a `platform` of `any`, that should be used; + otherwise no artifact should be downloaded. +* `url` is a required URL string that MUST conform to [RFC 1738: Uniform Resource Locators](https://www.ietf.org/rfc/rfc1738.txt). + A request to this `url` MUST return either an executable file or an archive containing either `bin/{DAEMON_NAME}` or `{DAEMON_NAME}`. + The URL should not contain checksum - it should be specified by the `checksum` attribute. +* `checksum` is a checksum of the expected result of a request to the `url`. + It is not required, but is recommended. + If provided, it MUST be a hex encoded checksum string. + Tools utilizing these `UpgradeInstructions` MUST fail if a `checksum` is provided but is different from the checksum of the result returned by the `url`. +* `checksum_algo` is a string identify the algorithm used to generate the `checksum`. + Recommended algorithms: `sha256`, `sha512`. + Algorithms also supported (but not recommended): `sha1`, `md5`. + If a `checksum` is provided, a `checksum_algo` MUST also be provided. + +A `url` is not required to contain a `checksum` query parameter. +If the `url` does contain a `checksum` query parameter, the `checksum` and `checksum_algo` fields MUST also be populated, and their values MUST match the value of the query parameter. +For example, if the `url` is `"https://example.com?checksum=md5:d41d8cd98f00b204e9800998ecf8427e"`, then the `checksum` field must be `"d41d8cd98f00b204e9800998ecf8427e"` and the `checksum_algo` field must be `"md5"`. + +### Upgrade Module Updates + +If an upgrade `Plan` does not use the new `UpgradeInstructions` field, existing functionality will be maintained. +The parsing of the `info` field as either a URL or `binaries` JSON will be deprecated. +During validation, if the `info` field is used as such, a warning will be issued, but not an error. + +We will update the creation of the `upgrade-info.json` file to include the `UpgradeInstructions`. + +We will update the optional validation available via CLI to account for the new `Plan` structure. +We will add the following validation: + +1. If `UpgradeInstructions` are provided: + 1. There MUST be at least one entry in `artifacts`. + 1. All of the `artifacts` MUST have a unique `platform`. + 1. For each `Artifact`, if the `url` contains a `checksum` query parameter: + 1. The `checksum` query parameter value MUST be in the format of `{checksum_algo}:{checksum}`. + 1. The `{checksum}` from the query parameter MUST equal the `checksum` provided in the `Artifact`. + 1. The `{checksum_algo}` from the query parameter MUST equal the `checksum_algo` provided in the `Artifact`. +1. The following validation is currently done using the `info` field. We will apply similar validation to the `UpgradeInstructions`. + For each `Artifact`: + 1. The `platform` MUST have the format `{OS}/{CPU}` or be `"any"`. + 1. The `url` field MUST NOT be empty. + 1. The `url` field MUST be a proper URL. + 1. A `checksum` MUST be provided either in the `checksum` field or as a query parameter in the `url`. + 1. If the `checksum` field has a value and the `url` also has a `checksum` query parameter, the two values MUST be equal. + 1. The `url` MUST return either a file or an archive containing either `bin/{DAEMON_NAME}` or `{DAEMON_NAME}`. + 1. If a `checksum` is provided (in the field or as a query param), the checksum of the result of the `url` MUST equal the provided checksum. + +Downloading of an `Artifact` will happen the same way that URLs from `info` are currently downloaded. + +### Cosmovisor Updates + +If the `upgrade-info.json` file does not contain any `UpgradeInstructions`, existing functionality will be maintained. + +We will update Cosmovisor to look for and handle the new `UpgradeInstructions` in `upgrade-info.json`. +If the `UpgradeInstructions` are provided, we will do the following: + +1. The `info` field will be ignored. +1. The `artifacts` field will be used to identify the artifact to download based on the `platform` that Cosmovisor is running in. +1. If a `checksum` is provided (either in the field or as a query param in the `url`), and the downloaded artifact has a different checksum, the upgrade process will be interrupted and Cosmovisor will exit with an error. +1. If a `pre_run` command is defined, it will be executed at the same point in the process where the `app pre-upgrade` command would have been executed. + It will be executed using the same environment as other commands run by Cosmovisor. +1. If a `post_run` command is defined, it will be executed after executing the command that restarts the chain. + It will be executed in a background process using the same environment as the other commands. + Any output generated by the command will be logged. + Once complete, the exit code will be logged. + +We will deprecate the use of the `info` field for anything other than human readable information. +A warning will be logged if the `info` field is used to define the assets (either by URL or JSON). + +The new upgrade timeline is very similar to the current one. Changes are in bold: + +1. An upgrade governance proposal is submitted and approved. +1. The upgrade height is reached. +1. The `x/upgrade` module writes the `upgrade_info.json` file **(now possibly with `UpgradeInstructions`)**. +1. The chain halts. +1. Cosmovisor backs up the data directory (if set up to do so). +1. Cosmovisor downloads the new executable (if not already in place). +1. Cosmovisor executes **the `pre_run` command if provided**, or else the `${DAEMON_NAME} pre-upgrade` command. +1. Cosmovisor restarts the app using the new version and same args originally provided. +1. **Cosmovisor immediately runs the `post_run` command in a detached process.** + +## Consequences + +### Backwards Compatibility + +Since the only change to existing definitions is the addition of the `instructions` field to the `Plan` message, and that field is optional, there are no backwards incompatibilities with respects to the proto messages. +Additionally, current behavior will be maintained when no `UpgradeInstructions` are provided, so there are no backwards incompatibilities with respects to either the upgrade module or Cosmovisor. + +### Forwards Compatibility + +In order to utilize the `UpgradeInstructions` as part of a software upgrade, both of the following must be true: + +1. The chain must already be using a sufficiently advanced version of the Cosmos SDK. +1. The chain's nodes must be using a sufficiently advanced version of Cosmovisor. + +### Positive + +1. The structure for defining artifacts is clearer since it is now defined in the proto instead of in documentation. +1. Availability of a pre-run command becomes more obvious. +1. A post-run command becomes possible. + +### Negative + +1. The `Plan` message becomes larger. This is negligible because A) the `x/upgrades` module only stores at most one upgrade plan, and B) upgrades are rare enough that the increased gas cost isn't a concern. +1. There is no option for providing a URL that will return the `UpgradeInstructions`. +1. The only way to provide multiple assets (executables and other files) for a platform is to use an archive as the platform's artifact. + +### Neutral + +1. Existing functionality of the `info` field is maintained when the `UpgradeInstructions` aren't provided. + +## Further Discussions + +1. [Draft PR #10032 Comment](https://github.com/cosmos/cosmos-sdk/pull/10032/files?authenticity_token=pLtzpnXJJB%2Fif2UWiTp9Td3MvRrBF04DvjSuEjf1azoWdLF%2BSNymVYw9Ic7VkqHgNLhNj6iq9bHQYnVLzMXd4g%3D%3D&file-filters%5B%5D=.go&file-filters%5B%5D=.proto#r698708349): + Consider different names for `UpgradeInstructions instructions` (either the message type or field name). +1. [Draft PR #10032 Comment](https://github.com/cosmos/cosmos-sdk/pull/10032/files?authenticity_token=pLtzpnXJJB%2Fif2UWiTp9Td3MvRrBF04DvjSuEjf1azoWdLF%2BSNymVYw9Ic7VkqHgNLhNj6iq9bHQYnVLzMXd4g%3D%3D&file-filters%5B%5D=.go&file-filters%5B%5D=.proto#r754655072): + 1. Consider putting the `string platform` field inside `UpgradeInstructions` and make `UpgradeInstructions` a repeated field in `Plan`. + 1. Consider using a `oneof` field in the `Plan` which could either be `UpgradeInstructions` or else a URL that should return the `UpgradeInstructions`. + 1. Consider allowing `info` to either be a JSON serialized version of `UpgradeInstructions` or else a URL that returns that. +1. [Draft PR #10032 Comment](https://github.com/cosmos/cosmos-sdk/pull/10032/files?authenticity_token=pLtzpnXJJB%2Fif2UWiTp9Td3MvRrBF04DvjSuEjf1azoWdLF%2BSNymVYw9Ic7VkqHgNLhNj6iq9bHQYnVLzMXd4g%3D%3D&file-filters%5B%5D=.go&file-filters%5B%5D=.proto#r755462876): + Consider not including the `UpgradeInstructions.description` field, using the `info` field for that purpose instead. +1. [Draft PR #10032 Comment](https://github.com/cosmos/cosmos-sdk/pull/10032/files?authenticity_token=pLtzpnXJJB%2Fif2UWiTp9Td3MvRrBF04DvjSuEjf1azoWdLF%2BSNymVYw9Ic7VkqHgNLhNj6iq9bHQYnVLzMXd4g%3D%3D&file-filters%5B%5D=.go&file-filters%5B%5D=.proto#r754643691): + Consider allowing multiple artifacts to be downloaded for any given `platform` by adding a `name` field to the `Artifact` message. +1. [PR #10502 Comment](https://github.com/cosmos/cosmos-sdk/pull/10602#discussion_r781438288) + Allow the new `UpgradeInstructions` to be provided via URL. +1. [PR #10502 Comment](https://github.com/cosmos/cosmos-sdk/pull/10602#discussion_r781438288) + Allow definition of a `signer` for assets (as an alternative to using a `checksum`). + +## References + +* [Current upgrade.proto](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/proto/cosmos/upgrade/v1beta1/upgrade.proto) +* [Upgrade Module README](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/x/upgrade/spec/README.md) +* [Cosmovisor README](https://github.com/cosmos/cosmos-sdk/blob/cosmovisor/v1.0.0/cosmovisor/README.md) +* [Pre-upgrade README](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/docs/migrations/pre-upgrade.md) +* [Draft/POC PR #10032](https://github.com/cosmos/cosmos-sdk/pull/10032) +* [RFC 1738: Uniform Resource Locators](https://www.ietf.org/rfc/rfc1738.txt) diff --git a/versioned_docs/version-0.52/build/architecture/adr-048-consensus-fees.md b/versioned_docs/version-0.52/build/architecture/adr-048-consensus-fees.md new file mode 100644 index 000000000..194748a98 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-048-consensus-fees.md @@ -0,0 +1,205 @@ +# ADR 048: Multi Tire Gas Price System + +## Changelog + +* Dec 1, 2021: Initial Draft + +## Status + +Rejected + +## Abstract + +This ADR describes a flexible mechanism to maintain a consensus level gas prices, in which one can choose a multi-tier gas price system or EIP-1559 like one through configuration. + +## Context + +Currently, each validator configures it's own `minimal-gas-prices` in `app.yaml`. But setting a proper minimal gas price is critical to protect network from dos attack, and it's hard for all the validators to pick a sensible value, so we propose to maintain a gas price in consensus level. + +Since tendermint 0.34.20 has supported mempool prioritization, we can take advantage of that to implement more sophisticated gas fee system. + +## Multi-Tier Price System + +We propose a multi-tier price system on consensus to provide maximum flexibility: + +* Tier 1: a constant gas price, which could only be modified occasionally through governance proposal. +* Tier 2: a dynamic gas price which is adjusted according to previous block load. +* Tier 3: a dynamic gas price which is adjusted according to previous block load at a higher speed. + +The gas price of higher tier should bigger than the lower tier. + +The transaction fees are charged with the exact gas price calculated on consensus. + +The parameter schema is like this: + +```protobuf +message TierParams { + uint32 priority = 1 // priority in tendermint mempool + Coin initial_gas_price = 2 // + uint32 parent_gas_target = 3 // the target saturation of block + uint32 change_denominator = 4 // decides the change speed + Coin min_gas_price = 5 // optional lower bound of the price adjustment + Coin max_gas_price = 6 // optional upper bound of the price adjustment +} + +message Params { + repeated TierParams tiers = 1; +} +``` + +### Extension Options + +We need to allow user to specify the tier of service for the transaction, to support it in an extensible way, we add an extension option in `AuthInfo`: + +```protobuf +message ExtensionOptionsTieredTx { + uint32 fee_tier = 1 +} +``` + +The value of `fee_tier` is just the index to the `tiers` parameter list. + +We also change the semantic of existing `fee` field of `Tx`, instead of charging user the exact `fee` amount, we treat it as a fee cap, while the actual amount of fee charged is decided dynamically. If the `fee` is smaller than dynamic one, the transaction won't be included in current block and ideally should stay in the mempool until the consensus gas price drop. The mempool can eventually prune old transactions. + +### Tx Prioritization + +Transactions are prioritized based on the tier, the higher the tier, the higher the priority. + +Within the same tier, follow the default Tendermint order (currently FIFO). Be aware of that the mempool tx ordering logic is not part of consensus and can be modified by malicious validator. + +This mechanism can be easily composed with prioritization mechanisms: + +* we can add extra tiers out of a user control: + * Example 1: user can set tier 0, 10 or 20, but the protocol will create tiers 0, 1, 2 ... 29. For example IBC transactions will go to tier `user_tier + 5`: if user selected tier 1, then the transaction will go to tier 15. + * Example 2: we can reserve tier 4, 5, ... only for special transaction types. For example, tier 5 is reserved for evidence tx. So if submits a bank.Send transaction and set tier 5, it will be delegated to tier 3 (the max tier level available for any transaction). + * Example 3: we can enforce that all transactions of a specific type will go to specific tier. For example, tier 100 will be reserved for evidence transactions and all evidence transactions will always go to that tier. + +### `min-gas-prices` + +Deprecate the current per-validator `min-gas-prices` configuration, since it would confusing for it to work together with the consensus gas price. + +### Adjust For Block Load + +For tier 2 and tier 3 transactions, the gas price is adjusted according to previous block load, the logic could be similar to EIP-1559: + +```python +def adjust_gas_price(gas_price, parent_gas_used, tier): + if parent_gas_used == tier.parent_gas_target: + return gas_price + elif parent_gas_used > tier.parent_gas_target: + gas_used_delta = parent_gas_used - tier.parent_gas_target + gas_price_delta = max(gas_price * gas_used_delta // tier.parent_gas_target // tier.change_speed, 1) + return gas_price + gas_price_delta + else: + gas_used_delta = parent_gas_target - parent_gas_used + gas_price_delta = gas_price * gas_used_delta // parent_gas_target // tier.change_speed + return gas_price - gas_price_delta +``` + +### Block Segment Reservation + +Ideally we should reserve block segments for each tier, so the lower tiered transactions won't be completely squeezed out by higher tier transactions, which will force user to use higher tier, and the system degraded to a single tier. + +We need help from tendermint to implement this. + +## Implementation + +We can make each tier's gas price strategy fully configurable in protocol parameters, while providing a sensible default one. + +Pseudocode in python-like syntax: + +```python +interface TieredTx: + def tier(self) -> int: + pass + +def tx_tier(tx): + if isinstance(tx, TieredTx): + return tx.tier() + else: + # default tier for custom transactions + return 0 + # NOTE: we can add more rules here per "Tx Prioritization" section + +class TierParams: + 'gas price strategy parameters of one tier' + priority: int # priority in tendermint mempool + initial_gas_price: Coin + parent_gas_target: int + change_speed: Decimal # 0 means don't adjust for block load. + +class Params: + 'protocol parameters' + tiers: List[TierParams] + +class State: + 'consensus state' + # total gas used in last block, None when it's the first block + parent_gas_used: Optional[int] + # gas prices of last block for all tiers + gas_prices: List[Coin] + +def begin_block(): + 'Adjust gas prices' + for i, tier in enumerate(Params.tiers): + if State.parent_gas_used is None: + # initialized gas price for the first block + State.gas_prices[i] = tier.initial_gas_price + else: + # adjust gas price according to gas used in previous block + State.gas_prices[i] = adjust_gas_price(State.gas_prices[i], State.parent_gas_used, tier) + +def mempoolFeeTxHandler_checkTx(ctx, tx): + # the minimal-gas-price configured by validator, zero in deliver_tx context + validator_price = ctx.MinGasPrice() + consensus_price = State.gas_prices[tx_tier(tx)] + min_price = max(validator_price, consensus_price) + + # zero means infinity for gas price cap + if tx.gas_price() > 0 and tx.gas_price() < min_price: + return 'insufficient fees' + return next_CheckTx(ctx, tx) + +def txPriorityHandler_checkTx(ctx, tx): + res, err := next_CheckTx(ctx, tx) + # pass priority to tendermint + res.Priority = Params.tiers[tx_tier(tx)].priority + return res, err + +def end_block(): + 'Update block gas used' + State.parent_gas_used = block_gas_meter.consumed() +``` + +### Dos attack protection + +To fully saturate the blocks and prevent other transactions from executing, attacker need to use transactions of highest tier, the cost would be significantly higher than the default tier. + +If attacker spam with lower tier transactions, user can mitigate by sending higher tier transactions. + +## Consequences + +### Backwards Compatibility + +* New protocol parameters. +* New consensus states. +* New/changed fields in transaction body. + +### Positive + +* The default tier keeps the same predictable gas price experience for client. +* The higher tier's gas price can adapt to block load. +* No priority conflict with custom priority based on transaction types, since this proposal only occupy three priority levels. +* Possibility to compose different priority rules with tiers + +### Negative + +* Wallets & tools need to update to support the new `tier` parameter, and semantic of `fee` field is changed. + +### Neutral + +## References + +* https://eips.ethereum.org/EIPS/eip-1559 + +* https://iohk.io/en/blog/posts/2021/11/26/network-traffic-and-tiered-pricing diff --git a/versioned_docs/version-0.52/build/architecture/adr-049-state-sync-hooks.md b/versioned_docs/version-0.52/build/architecture/adr-049-state-sync-hooks.md new file mode 100644 index 000000000..50c551ea2 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-049-state-sync-hooks.md @@ -0,0 +1,173 @@ +# ADR 049: State Sync Hooks + +## Changelog + +* Jan 19, 2022: Initial Draft +* Apr 29, 2022: Safer extension snapshotter interface + +## Status + +Implemented + +## Abstract + +This ADR outlines a hooks-based mechanism for application modules to provide additional state (outside of the IAVL tree) to be used +during state sync. + +## Context + +New clients use state-sync to download snapshots of module state from peers. Currently, the snapshot consists of a +stream of `SnapshotStoreItem` and `SnapshotIAVLItem`, which means that application modules that define their state outside of the IAVL +tree cannot include their state as part of the state-sync process. + +Note, Even though the module state data is outside of the tree, for determinism we require that the hash of the external data should +be posted in the IAVL tree. + +## Decision + +A simple proposal based on our existing implementation is that, we can add two new message types: `SnapshotExtensionMeta` +and `SnapshotExtensionPayload`, and they are appended to the existing multi-store stream with `SnapshotExtensionMeta` +acting as a delimiter between extensions. As the chunk hashes should be able to ensure data integrity, we don't need +a delimiter to mark the end of the snapshot stream. + +Besides, we provide `Snapshotter` and `ExtensionSnapshotter` interface for modules to implement snapshotters, which will handle both taking +snapshot and the restoration. Each module could have multiple snapshotters, and for modules with additional state, they should +implement `ExtensionSnapshotter` as extension snapshotters. When setting up the application, the snapshot `Manager` should call +`RegisterExtensions([]ExtensionSnapshotter…)` to register all the extension snapshotters. + +```protobuf +// SnapshotItem is an item contained in a rootmulti.Store snapshot. +// On top of the existing SnapshotStoreItem and SnapshotIAVLItem, we add two new options for the item. +message SnapshotItem { + // item is the specific type of snapshot item. + oneof item { + SnapshotStoreItem store = 1; + SnapshotIAVLItem iavl = 2 [(gogoproto.customname) = "IAVL"]; + SnapshotExtensionMeta extension = 3; + SnapshotExtensionPayload extension_payload = 4; + } +} + +// SnapshotExtensionMeta contains metadata about an external snapshotter. +// One module may need multiple snapshotters, so each module may have multiple SnapshotExtensionMeta. +message SnapshotExtensionMeta { + // the name of the ExtensionSnapshotter, and it is registered to snapshotter manager when setting up the application + // name should be unique for each ExtensionSnapshotter as we need to alphabetically order their snapshots to get + // deterministic snapshot stream. + string name = 1; + // this is used by each ExtensionSnapshotter to decide the format of payloads included in SnapshotExtensionPayload message + // it is used within the snapshotter/namespace, not global one for all modules + uint32 format = 2; +} + +// SnapshotExtensionPayload contains payloads of an external snapshotter. +message SnapshotExtensionPayload { + bytes payload = 1; +} +``` + +When we create a snapshot stream, the `multistore` snapshot is always placed at the beginning of the binary stream, and other extension snapshots are alphabetically ordered by the name of the corresponding `ExtensionSnapshotter`. + +The snapshot stream would look like as follows: + +```go +// multi-store snapshot +{SnapshotStoreItem | SnapshotIAVLItem, ...} +// extension1 snapshot +SnapshotExtensionMeta +{SnapshotExtensionPayload, ...} +// extension2 snapshot +SnapshotExtensionMeta +{SnapshotExtensionPayload, ...} +``` + +We add an `extensions` field to snapshot `Manager` for extension snapshotters. The `multistore` snapshotter is a special one and it doesn't need a name because it is always placed at the beginning of the binary stream. + +```go +type Manager struct { + store *Store + multistore types.Snapshotter + extensions map[string]types.ExtensionSnapshotter + mtx sync.Mutex + operation operation + chRestore chan<- io.ReadCloser + chRestoreDone <-chan restoreDone + restoreChunkHashes [][]byte + restoreChunkIndex uint32 +} +``` + +For extension snapshotters that implement the `ExtensionSnapshotter` interface, their names should be registered to the snapshot `Manager` by +calling `RegisterExtensions` when setting up the application. The snapshotters will handle both taking snapshot and restoration. + +```go +// RegisterExtensions register extension snapshotters to manager +func (m *Manager) RegisterExtensions(extensions ...types.ExtensionSnapshotter) error +``` + +On top of the existing `Snapshotter` interface for the `multistore`, we add `ExtensionSnapshotter` interface for the extension snapshotters. Three more function signatures: `SnapshotFormat()`, `SupportedFormats()` and `SnapshotName()` are added to `ExtensionSnapshotter`. + +```go +// ExtensionPayloadReader read extension payloads, +// it returns io.EOF when reached either end of stream or the extension boundaries. +type ExtensionPayloadReader = func() ([]byte, error) + +// ExtensionPayloadWriter is a helper to write extension payloads to underlying stream. +type ExtensionPayloadWriter = func([]byte) error + +// ExtensionSnapshotter is an extension Snapshotter that is appended to the snapshot stream. +// ExtensionSnapshotter has an unique name and manages it's own internal formats. +type ExtensionSnapshotter interface { + // SnapshotName returns the name of snapshotter, it should be unique in the manager. + SnapshotName() string + + // SnapshotFormat returns the default format used to take a snapshot. + SnapshotFormat() uint32 + + // SupportedFormats returns a list of formats it can restore from. + SupportedFormats() []uint32 + + // SnapshotExtension writes extension payloads into the underlying protobuf stream. + SnapshotExtension(height uint64, payloadWriter ExtensionPayloadWriter) error + + // RestoreExtension restores an extension state snapshot, + // the payload reader returns `io.EOF` when reached the extension boundaries. + RestoreExtension(height uint64, format uint32, payloadReader ExtensionPayloadReader) error + +} +``` + +## Consequences + +As a result of this implementation, we are able to create snapshots of binary chunk stream for the state that we maintain outside of the IAVL Tree, CosmWasm blobs for example. And new clients are able to fetch snapshots of state for all modules that have implemented the corresponding interface from peer nodes. + + +### Backwards Compatibility + +This ADR introduces new proto message types, add an `extensions` field in snapshot `Manager`, and add new `ExtensionSnapshotter` interface, so this is not backwards compatible if we have extensions. + +But for applications that does not have the state data outside of the IAVL tree for any module, the snapshot stream is backwards-compatible. + +### Positive + +* State maintained outside of IAVL tree like CosmWasm blobs can create snapshots by implementing extension snapshotters, and being fetched by new clients via state-sync. + +### Negative + +### Neutral + +* All modules that maintain state outside of IAVL tree need to implement `ExtensionSnapshotter` and the snapshot `Manager` need to call `RegisterExtensions` when setting up the application. + +## Further Discussions + +While an ADR is in the DRAFT or PROPOSED stage, this section should contain a summary of issues to be solved in future iterations (usually referencing comments from a pull-request discussion). +Later, this section can optionally list ideas or improvements the author or reviewers found during the analysis of this ADR. + +## Test Cases [optional] + +Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable. + +## References + +* https://github.com/cosmos/cosmos-sdk/pull/10961 +* https://github.com/cosmos/cosmos-sdk/issues/7340 diff --git a/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual-annex1.md b/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual-annex1.md new file mode 100644 index 000000000..64b143bf5 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual-annex1.md @@ -0,0 +1,358 @@ +# ADR 050: SIGN_MODE_TEXTUAL: Annex 1 Value Renderers + +## Changelog + +* Dec 06, 2021: Initial Draft +* Feb 07, 2022: Draft read and concept-ACKed by the Ledger team. +* Dec 01, 2022: Remove `Object: ` prefix on Any header screen. +* Dec 13, 2022: Sign over bytes hash when bytes length > 32. +* Mar 27, 2023: Update `Any` value renderer to omit message header screen. + +## Status + +Accepted. Implementation started. Small value renderers details still need to be polished. + +## Abstract + +This Annex describes value renderers, which are used for displaying Protobuf values in a human-friendly way using a string array. + +## Value Renderers + +Value Renderers describe how values of different Protobuf types should be encoded as a string array. Value renderers can be formalized as a set of bijective functions `func renderT(value T) []string`, where `T` is one of the below Protobuf types for which this spec is defined. + +### Protobuf `number` + +* Applies to: + * protobuf numeric integer types (`int{32,64}`, `uint{32,64}`, `sint{32,64}`, `fixed{32,64}`, `sfixed{32,64}`) + * strings whose `customtype` is `github.com/cosmos/cosmos-sdk/types.Int` or `github.com/cosmos/cosmos-sdk/types.Dec` + * bytes whose `customtype` is `github.com/cosmos/cosmos-sdk/types.Int` or `github.com/cosmos/cosmos-sdk/types.Dec` +* Trailing decimal zeroes are always removed +* Formatting with `'`s for every three integral digits. +* Usage of `.` to denote the decimal delimiter. + +#### Examples + +* `1000` (uint64) -> `1'000` +* `"1000000.00"` (string representing a Dec) -> `1'000'000` +* `"1000000.10"` (string representing a Dec) -> `1'000'000.1` + +### `coin` + +* Applies to `cosmos.base.v1beta1.Coin`. +* Denoms are converted to `display` denoms using `Metadata` (if available). **This requires a state query**. The definition of `Metadata` can be found in the [bank protobuf definition](https://buf.build/cosmos/cosmos-sdk/docs/main:cosmos.bank.v1beta1#cosmos.bank.v1beta1.Metadata). If the `display` field is empty or nil, then we do not perform any denom conversion. +* Amounts are converted to `display` denom amounts and rendered as `number`s above + * We do not change the capitalization of the denom. In practice, `display` denoms are stored in lowercase in state (e.g. `10 atom`), however they are often showed in UPPERCASE in everyday life (e.g. `10 ATOM`). Value renderers keep the case used in state, but we may recommend chains changing the denom metadata to be uppercase for better user display. +* One space between the denom and amount (e.g. `10 atom`). +* In the future, IBC denoms could maybe be converted to DID/IIDs, if we can find a robust way for doing this (ex. `cosmos:cosmos:hub:bank:denom:atom`) + +#### Examples + +* `1000000000uatom` -> `["1'000 atom"]`, because atom is the metadata's display denom. + +### `coins` + +* an array of `coin` is display as the concatenation of each `coin` encoded as the specification above, the joined together with the delimiter `", "` (a comma and a space, no quotes around). +* the list of coins is ordered by unicode code point of the display denom: `A-Z` < `a-z`. For example, the string `aAbBcC` would be sorted `ABCabc`. + * if the coins list had 0 items in it then it'll be rendered as `zero` + +### Example + +* `["3cosm", "2000000uatom"]` -> `2 atom, 3 COSM` (assuming the display denoms are `atom` and `COSM`) +* `["10atom", "20Acoin"]` -> `20 Acoin, 10 atom` (assuming the display denoms are `atom` and `Acoin`) +* `[]` -> `zero` + +### `repeated` + +* Applies to all `repeated` fields, except `cosmos.tx.v1beta1.TxBody#Messages`, which has a particular encoding (see [ADR-050](./adr-050-sign-mode-textual.md)). +* A repeated type has the following template: + +``` +: + (/): + + (/): + +End of . +``` + +where: + +* `field_name` is the Protobuf field name of the repeated field +* `field_kind`: + * if the type of the repeated field is a message, `field_kind` is the message name + * if the type of the repeated field is an enum, `field_kind` is the enum name + * in any other case, `field_kind` is the protobuf primitive type (e.g. "string" or "bytes") +* `int` is the length of the array +* `index` is one based index of the repeated field + +#### Examples + +Given the proto definition: + +```protobuf +message AllowedMsgAllowance { + repeated string allowed_messages = 1; +} +``` + +and initializing with: + +```go +x := []AllowedMsgAllowance{"cosmos.bank.v1beta1.MsgSend", "cosmos.gov.v1.MsgVote"} +``` + +we have the following value-rendered encoding: + +``` +Allowed messages: 2 strings +Allowed messages (1/2): cosmos.bank.v1beta1.MsgSend +Allowed messages (2/2): cosmos.gov.v1.MsgVote +End of Allowed messages +``` + +### `message` + +* Applies to all Protobuf messages that do not have a custom encoding. +* Field names follow [sentence case](https://en.wiktionary.org/wiki/sentence_case) + * replace each `_` with a space + * capitalize first letter of the sentence +* Field names are ordered by their Protobuf field number +* Screen title is the field name, and screen content is the value. +* Nesting: + * if a field contains a nested message, we value-render the underlying message using the template: + + ``` + : <1st line of value-rendered message> + > // Notice the `>` prefix. + ``` + + * `>` character is used to denote nesting. For each additional level of nesting, add `>`. + +#### Examples + +Given the following Protobuf messages: + +```protobuf +enum VoteOption { + VOTE_OPTION_UNSPECIFIED = 0; + VOTE_OPTION_YES = 1; + VOTE_OPTION_ABSTAIN = 2; + VOTE_OPTION_NO = 3; + VOTE_OPTION_NO_WITH_VETO = 4; +} + +message WeightedVoteOption { + VoteOption option = 1; + string weight = 2 [(cosmos_proto.scalar) = "cosmos.Dec"]; +} + +message Vote { + uint64 proposal_id = 1; + string voter = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + reserved 3; + repeated WeightedVoteOption options = 4; +} +``` + +we get the following encoding for the `Vote` message: + +``` +Vote object +> Proposal id: 4 +> Voter: cosmos1abc...def +> Options: 2 WeightedVoteOptions +> Options (1/2): WeightedVoteOption object +>> Option: VOTE_OPTION_YES +>> Weight: 0.7 +> Options (2/2): WeightedVoteOption object +>> Option: VOTE_OPTION_NO +>> Weight: 0.3 +> End of Options +``` + +### Enums + +* Show the enum variant name as string. + +#### Examples + +See example above with `message Vote{}`. + +### `google.protobuf.Any` + +* Applies to `google.protobuf.Any` +* Rendered as: + +``` + +> +``` + +There is however one exception: when the underlying message is a Protobuf message that does not have a custom encoding, then the message header screen is omitted, and one level of indentation is removed. + +Messages that have a custom encoding, including `google.protobuf.Timestamp`, `google.protobuf.Duration`, `google.protobuf.Any`, `cosmos.base.v1beta1.Coin`, and messages that have an app-defined custom encoding, will preserve their header and indentation level. + +#### Examples + +Message header screen is stripped, one-level of indentation removed: +``` +/cosmos.gov.v1.Vote +> Proposal id: 4 +> Vote: cosmos1abc...def +> Options: 2 WeightedVoteOptions +> Options (1/2): WeightedVoteOption object +>> Option: Yes +>> Weight: 0.7 +> Options (2/2): WeightedVoteOption object +>> Option: No +>> Weight: 0.3 +> End of Options +``` + +Message with custom encoding: +``` +/cosmos.base.v1beta1.Coin +> 10uatom +``` + +### `google.protobuf.Timestamp` + +Rendered using [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339) (a +simplification of ISO 8601), which is the current recommendation for portable +time values. The rendering always uses "Z" (UTC) as the timezone. It uses only +the necessary fractional digits of a second, omitting the fractional part +entirely if the timestamp has no fractional seconds. (The resulting timestamps +are not automatically sortable by standard lexicographic order, but we favor +the legibility of the shorter string.) + +#### Examples + +The timestamp with 1136214245 seconds and 700000000 nanoseconds is rendered +as `2006-01-02T15:04:05.7Z`. +The timestamp with 1136214245 seconds and zero nanoseconds is rendered +as `2006-01-02T15:04:05Z`. + +### `google.protobuf.Duration` + +The duration proto expresses a raw number of seconds and nanoseconds. +This will be rendered as longer time units of days, hours, and minutes, +plus any remaining seconds, in that order. +Leading and trailing zero-quantity units will be omitted, but all +units in between nonzero units will be shown, e.g. ` 3 days, 0 hours, 0 minutes, 5 seconds`. + +Even longer time units such as months or years are imprecise. +Weeks are precise, but not commonly used - `91 days` is more immediately +legible than `13 weeks`. Although `days` can be problematic, +e.g. noon to noon on subsequent days can be 23 or 25 hours depending on +daylight savings transitions, there is significant advantage in using +strict 24-hour days over using only hours (e.g. `91 days` vs `2184 hours`). + +When nanoseconds are nonzero, they will be shown as fractional seconds, +with only the minimum number of digits, e.g `0.5 seconds`. + +A duration of exactly zero is shown as `0 seconds`. + +Units will be given as singular (no trailing `s`) when the quantity is exactly one, +and will be shown in plural otherwise. + +Negative durations will be indicated with a leading minus sign (`-`). + +Examples: + +* `1 day` +* `30 days` +* `-1 day, 12 hours` +* `3 hours, 0 minutes, 53.025 seconds` + +### bytes + +* Bytes of length shorter or equal to 35 are rendered in hexadecimal, all capital letters, without the `0x` prefix. +* Bytes of length greater than 35 are hashed using SHA256. The rendered text is `SHA-256=`, followed by the 32-byte hash, in hexadecimal, all capital letters, without the `0x` prefix. +* The hexadecimal string is finally separated into groups of 4 digits, with a space `' '` as separator. If the bytes length is odd, the 2 remaining hexadecimal characters are at the end. + +The number 35 was chosen because it is the longest length where the hashed-and-prefixed representation is longer than the original data directly formatted, using the 3 rules above. More specifically: +- a 35-byte array will have 70 hex characters, plus 17 space characters, resulting in 87 characters. +- byte arrays starting from length 36 will be hashed to 32 bytes, which is 64 hex characters plus 15 spaces, and with the `SHA-256=` prefix, it takes 87 characters. +Also, secp256k1 public keys have length 33, so their Textual representation is not their hashed value, which we would like to avoid. + +Note: Data longer than 35 bytes are not rendered in a way that can be inverted. See ADR-050's [section about invertability](./adr-050-sign-mode-textual.md#invertible-rendering) for a discussion. + +#### Examples + +Inputs are displayed as byte arrays. + +* `[0]`: `00` +* `[0,1,2]`: `0001 02` +* `[0,1,2,..,34]`: `0001 0203 0405 0607 0809 0A0B 0C0D 0E0F 1011 1213 1415 1617 1819 1A1B 1C1D 1E1F 2021 22` +* `[0,1,2,..,35]`: `SHA-256=5D7E 2D9B 1DCB C85E 7C89 0036 A2CF 2F9F E7B6 6554 F2DF 08CE C6AA 9C0A 25C9 9C21` + +### address bytes + +We currently use `string` types in protobuf for addresses so this may not be needed, but if any address bytes are used in sign mode textual they should be rendered with bech32 formatting + +### strings + +Strings are rendered as-is. + +### Default Values + +* Default Protobuf values for each field are skipped. + +#### Example + +```protobuf +message TestData { + string signer = 1; + string metadata = 2; +} +``` + +```go +myTestData := TestData{ + Signer: "cosmos1abc" +} +``` + +We get the following encoding for the `TestData` message: + +``` +TestData object +> Signer: cosmos1abc +``` + +### bool + +Boolean values are rendered as `True` or `False`. + +### [ABANDONED] Custom `msg_title` instead of Msg `type_url` + +_This paragraph is in the Annex for informational purposes only, and will be removed in a next update of the ADR._ + +
+ Click to see abandoned idea. + +* all protobuf messages to be used with `SIGN_MODE_TEXTUAL` CAN have a short title associated with them that can be used in format strings whenever the type URL is explicitly referenced via the `cosmos.msg.v1.textual.msg_title` Protobuf message option. +* if this option is not specified for a Msg, then the Protobuf fully qualified name will be used. + +```protobuf +message MsgSend { + option (cosmos.msg.v1.textual.msg_title) = "bank send coins"; +} +``` + +* they MUST be unique per message, per chain + +#### Examples + +* `cosmos.gov.v1.MsgVote` -> `governance v1 vote` + +#### Best Pratices + +We recommend to use this option only for `Msg`s whose Protobuf fully qualified name can be hard to understand. As such, the two examples above (`MsgSend` and `MsgVote`) are not good examples to be used with `msg_title`. We still allow `msg_title` for chains who might have `Msg`s with complex or non-obvious names. + +In those cases, we recommend to drop the version (e.g. `v1`) in the string if there's only one version of the module on chain. This way, the bijective mapping can figure out which message each string corresponds to. If multiple Protobuf versions of the same module exist on the same chain, we recommend keeping the first `msg_title` with version, and the second `msg_title` with version (e.g. `v2`): + +* `mychain.mymodule.v1.MsgDo` -> `mymodule do something` +* `mychain.mymodule.v2.MsgDo` -> `mymodule v2 do something` + +
diff --git a/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual-annex2.md b/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual-annex2.md new file mode 100644 index 000000000..e27981047 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual-annex2.md @@ -0,0 +1,122 @@ +# ADR 050: SIGN_MODE_TEXTUAL: Annex 2 XXX + +## Changelog + +* Oct 3, 2022: Initial Draft + +## Status + +DRAFT + +## Abstract + +This annex provides normative guidance on how devices should render a +`SIGN_MODE_TEXTUAL` document. + +## Context + +`SIGN_MODE_TEXTUAL` allows a legible version of a transaction to be signed +on a hardware security device, such as a Ledger. Early versions of the +design rendered transactions directly to lines of ASCII text, but this +proved awkward from its in-band signaling, and for the need to display +Unicode text within the transaction. + +## Decision + +`SIGN_MODE_TEXTUAL` renders to an abstract representation, leaving it +up to device-specific software how to present this representation given the +capabilities, limitations, and conventions of the device. + +We offer the following normative guidance: + +1. The presentation should be as legible as possible to the user, given +the capabilities of the device. If legibility could be sacrificed for other +properties, we would recommend just using some other signing mode. +Legibility should focus on the common case - it is okay for unusual cases +to be less legible. + +2. The presentation should be invertible if possible without substantial +sacrifice of legibility. Any change to the rendered data should result +in a visible change to the presentation. This extends the integrity of the +signing to user-visible presentation. + +3. The presentation should follow normal conventions of the device, +without sacrificing legibility or invertibility. + +As an illustration of these principles, here is an example algorithm +for presentation on a device which can display a single 80-character +line of printable ASCII characters: + +* The presentation is broken into lines, and each line is presented in +sequence, with user controls for going forward or backward a line. + +* Expert mode screens are only presented if the device is in expert mode. + +* Each line of the screen starts with a number of `>` characters equal +to the screen's indentation level, followed by a `+` character if this +isn't the first line of the screen, followed by a space if either a +`>` or a `+` has been emitted, +or if this header is followed by a `>`, `+`, or space. + +* If the line ends with whitespace or an `@` character, an additional `@` +character is appended to the line. + +* The following ASCII control characters or backslash (`\`) are converted +to a backslash followed by a letter code, in the manner of string literals +in many languages: + + * a: U+0007 alert or bell + * b: U+0008 backspace + * f: U+000C form feed + * n: U+000A line feed + * r: U+000D carriage return + * t: U+0009 horizontal tab + * v: U+000B vertical tab + * `\`: U+005C backslash + +* All other ASCII control characters, plus non-ASCII Unicode code points, +are shown as either: + + * `\u` followed by 4 uppercase hex characters for code points + in the basic multilingual plane (BMP). + + * `\U` followed by 8 uppercase hex characters for other code points. + +* The screen will be broken into multiple lines to fit the 80-character +limit, considering the above transformations in a way that attempts to +minimize the number of lines generated. Expanded control or Unicode characters +are never split across lines. + +Example output: + +``` +An introductory line. +key1: 123456 +key2: a string that ends in whitespace @ +key3: a string that ends in a single ampersand - @@ + >tricky key4<: note the leading space in the presentation +introducing an aggregate +> key5: false +> key6: a very long line of text, please co\u00F6perate and break into +>+ multiple lines. +> Can we do further nesting? +>> You bet we can! +``` + +The inverse mapping gives us the only input which could have +generated this output (JSON notation for string data): + +``` +Indent Text +------ ---- +0 "An introductory line." +0 "key1: 123456" +0 "key2: a string that ends in whitespace " +0 "key3: a string that ends in a single ampersand - @" +0 ">tricky key4<: note the leading space in the presentation" +0 "introducing an aggregate" +1 "key5: false" +1 "key6: a very long line of text, please cooperate and break into multiple lines." +1 "Can we do further nesting?" +2 "You bet we can!" +``` diff --git a/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual.md b/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual.md new file mode 100644 index 000000000..8eec8e42b --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-050-sign-mode-textual.md @@ -0,0 +1,370 @@ +# ADR 050: SIGN_MODE_TEXTUAL + +## Changelog + +* Dec 06, 2021: Initial Draft. +* Feb 07, 2022: Draft read and concept-ACKed by the Ledger team. +* May 16, 2022: Change status to Accepted. +* Aug 11, 2022: Require signing over tx raw bytes. +* Sep 07, 2022: Add custom `Msg`-renderers. +* Sep 18, 2022: Structured format instead of lines of text +* Nov 23, 2022: Specify CBOR encoding. +* Dec 01, 2022: Link to examples in separate JSON file. +* Dec 06, 2022: Re-ordering of envelope screens. +* Dec 14, 2022: Mention exceptions for invertability. +* Jan 23, 2023: Switch Screen.Text to Title+Content. +* Mar 07, 2023: Change SignDoc from array to struct containing array. +* Mar 20, 2023: Introduce a spec version initialized to 0. + +## Status + +Accepted. Implementation started. Small value renderers details still need to be polished. + +Spec version: 0. + +## Abstract + +This ADR specifies SIGN_MODE_TEXTUAL, a new string-based sign mode that is targeted at signing with hardware devices. + +## Context + +Protobuf-based SIGN_MODE_DIRECT was introduced in [ADR-020](./adr-020-protobuf-transaction-encoding.md) and is intended to replace SIGN_MODE_LEGACY_AMINO_JSON in most situations, such as mobile wallets and CLI keyrings. However, the [Ledger](https://www.ledger.com/) hardware wallet is still using SIGN_MODE_LEGACY_AMINO_JSON for displaying the sign bytes to the user. Hardware wallets cannot transition to SIGN_MODE_DIRECT as: + +* SIGN_MODE_DIRECT is binary-based and thus not suitable for display to end-users. Technically, hardware wallets could simply display the sign bytes to the user. But this would be considered as blind signing, and is a security concern. +* hardware cannot decode the protobuf sign bytes due to memory constraints, as the Protobuf definitions would need to be embedded on the hardware device. + +In an effort to remove Amino from the SDK, a new sign mode needs to be created for hardware devices. [Initial discussions](https://github.com/cosmos/cosmos-sdk/issues/6513) propose a text-based sign mode, which this ADR formally specifies. + +## Decision + +In SIGN_MODE_TEXTUAL, a transaction is rendered into a textual representation, +which is then sent to a secure device or subsystem for the user to review and sign. +Unlike `SIGN_MODE_DIRECT`, the transmitted data can be simply decoded into legible text +even on devices with limited processing and display. + +The textual representation is a sequence of _screens_. +Each screen is meant to be displayed in its entirety (if possible) even on a small device like a Ledger. +A screen is roughly equivalent to a short line of text. +Large screens can be displayed in several pieces, +much as long lines of text are wrapped, +so no hard guidance is given, though 40 characters is a good target. +A screen is used to display a single key/value pair for scalar values +(or composite values with a compact notation, such as `Coins`) +or to introduce or conclude a larger grouping. + +The text can contain the full range of Unicode code points, including control characters and nul. +The device is responsible for deciding how to display characters it cannot render natively. +See [annex 2](./adr-050-sign-mode-textual-annex2.md) for guidance. + +Screens have a non-negative indentation level to signal composite or nested structures. +Indentation level zero is the top level. +Indentation is displayed via some device-specific mechanism. +Message quotation notation is an appropriate model, such as +leading `>` characters or vertical bars on more capable displays. + +Some screens are marked as _expert_ screens, +meant to be displayed only if the viewer chooses to opt in for the extra detail. +Expert screens are meant for information that is rarely useful, +or needs to be present only for signature integrity (see below). + +### Invertible Rendering + +We require that the rendering of the transaction be invertible: +there must be a parsing function such that for every transaction, +when rendered to the textual representation, +parsing that representation yields a proto message equivalent +to the original under proto equality. + +Note that this inverse function does not need to perform correct +parsing or error signaling for the whole domain of textual data. +Merely that the range of valid transactions be invertible under +the composition of rendering and parsing. + +Note that the existence of an inverse function ensures that the +rendered text contains the full information of the original transaction, +not a hash or subset. + +We make an exception for invertibility for data which are too large to +meaningfully display, such as byte strings longer than 32 bytes. We may then +selectively render them with a cryptographically-strong hash. In these cases, +it is still computationally infeasible to find a different transaction which +has the same rendering. However, we must ensure that the hash computation is +simple enough to be reliably executed independently, so at least the hash is +itself reasonably verifiable when the raw byte string is not. + +### Chain State + +The rendering function (and parsing function) may depend on the current chain state. +This is useful for reading parameters, such as coin display metadata, +or for reading user-specific preferences such as language or address aliases. +Note that if the observed state changes between signature generation +and the transaction's inclusion in a block, the delivery-time rendering +might differ. If so, the signature will be invalid and the transaction +will be rejected. + +### Signature and Security + +For security, transaction signatures should have three properties: + +1. Given the transaction, signatures, and chain state, it must be possible to validate that the signatures matches the transaction, +to verify that the signers must have known their respective secret keys. + +2. It must be computationally infeasible to find a substantially different transaction for which the given signatures are valid, given the same chain state. + +3. The user should be able to give informed consent to the signed data via a simple, secure device with limited display capabilities. + +The correctness and security of `SIGN_MODE_TEXTUAL` is guaranteed by demonstrating an inverse function from the rendering to transaction protos. +This means that it is impossible for a different protocol buffer message to render to the same text. + +### Transaction Hash Malleability + +When client software forms a transaction, the "raw" transaction (`TxRaw`) is serialized as a proto +and a hash of the resulting byte sequence is computed. +This is the `TxHash`, and is used by various services to track the submitted transaction through its lifecycle. +Various misbehavior is possible if one can generate a modified transaction with a different TxHash +but for which the signature still checks out. + +SIGN_MODE_TEXTUAL prevents this transaction malleability by including the TxHash as an expert screen +in the rendering. + +### SignDoc + +The SignDoc for `SIGN_MODE_TEXTUAL` is formed from a data structure like: + +```go +type Screen struct { + Title string // possibly size limited to, advised to 64 characters + Content string // possibly size limited to, advised to 255 characters + Indent uint8 // size limited to something small like 16 or 32 + Expert bool +} + +type SignDocTextual struct { + Screens []Screen +} +``` + +We do not plan to use protobuf serialization to form the sequence of bytes +that will be transmitted and signed, in order to keep the decoder simple. +We will use [CBOR](https://cbor.io) ([RFC 8949](https://www.rfc-editor.org/rfc/rfc8949.html)) instead. +The encoding is defined by the following CDDL ([RFC 8610](https://www.rfc-editor.org/rfc/rfc8610)): + +``` +;;; CDDL (RFC 8610) Specification of SignDoc for SIGN_MODE_TEXTUAL. +;;; Must be encoded using CBOR deterministic encoding (RFC 8949, section 4.2.1). + +;; A Textual document is a struct containing one field: an array of screens. +sign_doc = { + screens_key: [* screen], +} + +;; The key is an integer to keep the encoding small. +screens_key = 1 + +;; A screen consists of a text string, an indentation, and the expert flag, +;; represented as an integer-keyed map. All entries are optional +;; and MUST be omitted from the encoding if empty, zero, or false. +;; Text defaults to the empty string, indent defaults to zero, +;; and expert defaults to false. +screen = { + ? title_key: tstr, + ? content_key: tstr, + ? indent_key: uint, + ? expert_key: bool, +} + +;; Keys are small integers to keep the encoding small. +title_key = 1 +content_key = 2 +indent_key = 3 +expert_key = 4 +``` + +Defining the sign_doc as directly an array of screens has also been considered. However, given the possibility of future iterations of this specification, using a single-keyed struct has been chosen over the former proposal, as structs allow for easier backwards-compatibility. + +## Details + +In the examples that follow, screens will be shown as lines of text, +indentation is indicated with a leading '>', +and expert screens are marked with a leading `*`. + +### Encoding of the Transaction Envelope + +We define "transaction envelope" as all data in a transaction that is not in the `TxBody.Messages` field. Transaction envelope includes fee, signer infos and memo, but don't include `Msg`s. `//` denotes comments and are not shown on the Ledger device. + +``` +Chain ID: +Account number: +Sequence: +Address: +*Public Key: +This transaction has Message(s) // Pluralize "Message" only when int>1 +> Message (/): // See value renderers for Any rendering. +End of Message +Memo: // Skipped if no memo set. +Fee: // See value renderers for coins rendering. +*Fee payer: // Skipped if no fee_payer set. +*Fee granter: // Skipped if no fee_granter set. +Tip: // Skippted if no tip. +Tipper: +*Gas Limit: +*Timeout Height: // Skipped if no timeout_height set. +*Other signer: SignerInfo // Skipped if the transaction only has 1 signer. +*> Other signer (/): +*End of other signers +*Extension options: Any: // Skipped if no body extension options +*> Extension options (/): +*End of extension options +*Non critical extension options: Any: // Skipped if no body non critical extension options +*> Non critical extension options (/): +*End of Non critical extension options +*Hash of raw bytes: // Hex encoding of bytes defined, to prevent tx hash malleability. +``` + +### Encoding of the Transaction Body + +Transaction Body is the `Tx.TxBody.Messages` field, which is an array of `Any`s, where each `Any` packs a `sdk.Msg`. Since `sdk.Msg`s are widely used, they have a slightly different encoding than usual array of `Any`s (Protobuf: `repeated google.protobuf.Any`) described in Annex 1. + +``` +This transaction has message: // Optional 's' for "message" if there's is >1 sdk.Msgs. +// For each Msg, print the following 2 lines: +Msg (/): // E.g. Msg (1/2): bank v1beta1 send coins + +End of transaction messages +``` + +#### Example + +Given the following Protobuf message: + +```protobuf +message Grant { + google.protobuf.Any authorization = 1 [(cosmos_proto.accepts_interface) = "cosmos.authz.v1beta1.Authorization"]; + google.protobuf.Timestamp expiration = 2 [(gogoproto.stdtime) = true, (gogoproto.nullable) = false]; +} + +message MsgGrant { + option (cosmos.msg.v1.signer) = "granter"; + + string granter = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + string grantee = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; +} +``` + +and a transaction containing 1 such `sdk.Msg`, we get the following encoding: + +``` +This transaction has 1 message: +Msg (1/1): authz v1beta1 grant +Granter: cosmos1abc...def +Grantee: cosmos1ghi...jkl +End of transaction messages +``` + +### Custom `Msg` Renderers + +Application developers may choose to not follow default renderer value output for their own `Msg`s. In this case, they can implement their own custom `Msg` renderer. This is similar to [EIP4430](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-4430.md), where the smart contract developer chooses the description string to be shown to the end user. + +This is done by setting the `cosmos.msg.textual.v1.expert_custom_renderer` Protobuf option to a non-empty string. This option CAN ONLY be set on a Protobuf message representing transaction message object (implementing `sdk.Msg` interface). + +```protobuf +message MsgFooBar { + // Optional comments to describe in human-readable language the formatting + // rules of the custom renderer. + option (cosmos.msg.textual.v1.expert_custom_renderer) = ""; + + // proto fields +} +``` + +When this option is set on a `Msg`, a registered function will transform the `Msg` into an array of one or more strings, which MAY use the key/value format (described in point #3) with the expert field prefix (described in point #5) and arbitrary indentation (point #6). These strings MAY be rendered from a `Msg` field using a default value renderer, or they may be generated from several fields using custom logic. + +The `` is a string convention chosen by the application developer and is used to identify the custom `Msg` renderer. For example, the documentation or specification of this custom algorithm can reference this identifier. This identifier CAN have a versioned suffix (e.g. `_v1`) to adapt for future changes (which would be consensus-breaking). We also recommend adding Protobuf comments to describe in human language the custom logic used. + +Moreover, the renderer must provide 2 functions: one for formatting from Protobuf to string, and one for parsing string to Protobuf. These 2 functions are provided by the application developer. To satisfy point #1, the parse function MUST be the inverse of the formatting function. This property will not be checked by the SDK at runtime. However, we strongly recommend the application developer to include a comprehensive suite in their app repo to test invertibility, as to not introduce security bugs. + +### Require signing over the `TxBody` and `AuthInfo` raw bytes + +Recall that the transaction bytes merklelized on chain are the Protobuf binary serialization of [TxRaw](https://buf.build/cosmos/cosmos-sdk/docs/main:cosmos.tx.v1beta1#cosmos.tx.v1beta1.TxRaw), which contains the `body_bytes` and `auth_info_bytes`. Moreover, the transaction hash is defined as the SHA256 hash of the `TxRaw` bytes. We require that the user signs over these bytes in SIGN_MODE_TEXTUAL, more specifically over the following string: + +``` +*Hash of raw bytes: +``` + +where: + +* `++` denotes concatenation, +* `HEX` is the hexadecimal representation of the bytes, all in capital letters, no `0x` prefix, +* and `len()` is encoded as a Big-Endian uint64. + +This is to prevent transaction hash malleability. The point #1 about invertiblity assures that transaction `body` and `auth_info` values are not malleable, but the transaction hash still might be malleable with point #1 only, because the SIGN_MODE_TEXTUAL strings don't follow the byte ordering defined in `body_bytes` and `auth_info_bytes`. Without this hash, a malicious validator or exchange could intercept a transaction, modify its transaction hash _after_ the user signed it using SIGN_MODE_TEXTUAL (by tweaking the byte ordering inside `body_bytes` or `auth_info_bytes`), and then submit it to Tendermint. + +By including this hash in the SIGN_MODE_TEXTUAL signing payload, we keep the same level of guarantees as [SIGN_MODE_DIRECT](./adr-020-protobuf-transaction-encoding.md). + +These bytes are only shown in expert mode, hence the leading `*`. + +## Updates to the current specification + +The current specification is not set in stone, and future iterations are to be expected. We distinguish two categories of updates to this specification: + +1. Updates that require changes of the hardware device embedded application. +2. Updates that only modify the envelope and the value renderers. + +Updates in the 1st category include changes of the `Screen` struct or its corresponding CBOR encoding. This type of updates require a modification of the hardware signer application, to be able to decode and parse the new types. Backwards-compatibility must also be guaranteed, so that the new hardware application works with existing versions of the SDK. These updates require the coordination of multiple parties: SDK developers, hardware application developers (currently: Zondax), and client-side developers (e.g. CosmJS). Furthermore, a new submission of the hardware device application may be necessary, which, depending on the vendor, can take some time. As such, we recommend to avoid this type of updates as much as possible. + +Updates in the 2nd category include changes to any of the value renderers or to the transaction envelope. For example, the ordering of fields in the envelope can be swapped, or the timestamp formatting can be modified. Since SIGN_MODE_TEXTUAL sends `Screen`s to the hardware device, this type of change do not need a hardware wallet application update. They are however state-machine-breaking, and must be documented as such. They require the coordination of SDK developers with client-side developers (e.g. CosmJS), so that the updates are released on both sides close to each other in time. + +We define a spec version, which is an integer that must be incremented on each update of either category. This spec version will be exposed by the SDK's implementation, and can be communicated to clients. For example, SDK v0.50 might use the spec version 1, and SDK v0.51 might use 2; thanks to this versioning, clients can know how to craft SIGN_MODE_TEXTUAL transactions based on the target SDK version. + +The current spec version is defined in the "Status" section, on the top of this document. It is initialized to `0` to allow flexibility in choosing how to define future versions, as it would allow adding a field either in the SignDoc Go struct or in Protobuf in a backwards-compatible way. + +## Additional Formatting by the Hardware Device + +See [annex 2](./adr-050-sign-mode-textual-annex2.md). + +## Examples + +1. A minimal MsgSend: [see transaction](https://github.com/cosmos/cosmos-sdk/blob/094abcd393379acbbd043996024d66cd65246fb1/tx/textual/internal/testdata/e2e.json#L2-L70). +2. A transaction with a bit of everything: [see transaction](https://github.com/cosmos/cosmos-sdk/blob/094abcd393379acbbd043996024d66cd65246fb1/tx/textual/internal/testdata/e2e.json#L71-L270). + +The examples below are stored in a JSON file with the following fields: + +* `proto`: the representation of the transaction in ProtoJSON, +* `screens`: the transaction rendered into SIGN_MODE_TEXTUAL screens, +* `cbor`: the sign bytes of the transaction, which is the CBOR encoding of the screens. + +## Consequences + +### Backwards Compatibility + +SIGN_MODE_TEXTUAL is purely additive, and doesn't break any backwards compatibility with other sign modes. + +### Positive + +* Human-friendly way of signing in hardware devices. +* Once SIGN_MODE_TEXTUAL is shipped, SIGN_MODE_LEGACY_AMINO_JSON can be deprecated and removed. On the longer term, once the ecosystem has totally migrated, Amino can be totally removed. + +### Negative + +* Some fields are still encoded in non-human-readable ways, such as public keys in hexadecimal. +* New ledger app needs to be released, still unclear + +### Neutral + +* If the transaction is complex, the string array can be arbitrarily long, and some users might just skip some screens and blind sign. + +## Further Discussions + +* Some details on value renderers need to be polished, see [Annex 1](./adr-050-sign-mode-textual-annex1.md). +* Are ledger apps able to support both SIGN_MODE_LEGACY_AMINO_JSON and SIGN_MODE_TEXTUAL at the same time? +* Open question: should we add a Protobuf field option to allow app developers to overwrite the textual representation of certain Protobuf fields and message? This would be similar to Ethereum's [EIP4430](https://github.com/ethereum/EIPs/pull/4430), where the contract developer decides on the textual representation. +* Internationalization. + +## References + +* [Annex 1](./adr-050-sign-mode-textual-annex1.md) + +* Initial discussion: https://github.com/cosmos/cosmos-sdk/issues/6513 +* Living document used in the working group: https://hackmd.io/fsZAO-TfT0CKmLDtfMcKeA?both +* Working group meeting notes: https://hackmd.io/7RkGfv_rQAaZzEigUYhcXw +* Ethereum's "Described Transactions" https://github.com/ethereum/EIPs/pull/4430 diff --git a/versioned_docs/version-0.52/build/architecture/adr-053-go-module-refactoring.md b/versioned_docs/version-0.52/build/architecture/adr-053-go-module-refactoring.md new file mode 100644 index 000000000..d15c39019 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-053-go-module-refactoring.md @@ -0,0 +1,110 @@ +# ADR 053: Go Module Refactoring + +## Changelog + +* 2022-04-27: First Draft + +## Status + +PROPOSED + +## Abstract + +The current SDK is built as a single monolithic go module. This ADR describes +how we refactor the SDK into smaller independently versioned go modules +for ease of maintenance. + +## Context + +Go modules impose certain requirements on software projects with respect to +stable version numbers (anything above 0.x) in that [any API breaking changes +necessitate a major version](https://go.dev/doc/modules/release-workflow#breaking) +increase which technically creates a new go module +(with a v2, v3, etc. suffix). + +[Keeping modules API compatible](https://go.dev/blog/module-compatibility) in +this way requires a fair amount of fair thought and discipline. + +The Cosmos SDK is a fairly large project which originated before go modules +came into existence and has always been under a v0.x release even though +it has been used in production for years now, not because it isn't production +quality software, but rather because the API compatibility guarantees required +by go modules are fairly complex to adhere to with such a large project. +Up to now, it has generally been deemed more important to be able to break the +API if needed rather than require all users update all package import paths +to accommodate breaking changes causing v2, v3, etc. releases. This is in +addition to the other complexities related to protobuf generated code that will +be addressed in a separate ADR. + +Nevertheless, the desire for semantic versioning has been [strong in the +community](https://github.com/cosmos/cosmos-sdk/discussions/10162) and the +single go module release process has made it very hard to +release small changes to isolated features in a timely manner. Release cycles +often exceed six months which means small improvements done in a day or +two get bottle-necked by everything else in the monolithic release cycle. + +## Decision + +To improve the current situation, the SDK is being refactored into multiple +go modules within the current repository. There has been a [fair amount of +debate](https://github.com/cosmos/cosmos-sdk/discussions/10582#discussioncomment-1813377) +as to how to do this, with some developers arguing for larger vs smaller +module scopes. There are pros and cons to both approaches (which will be +discussed below in the [Consequences](#consequences) section), but the +approach being adopted is the following: + +* a go module should generally be scoped to a specific coherent set of +functionality (such as math, errors, store, etc.) +* when code is removed from the core SDK and moved to a new module path, every +effort should be made to avoid API breaking changes in the existing code using +aliases and wrapper types (as done in https://github.com/cosmos/cosmos-sdk/pull/10779 +and https://github.com/cosmos/cosmos-sdk/pull/11788) +* new go modules should be moved to a standalone domain (`cosmossdk.io`) before +being tagged as `v1.0.0` to accommodate the possibility that they may be +better served by a standalone repository in the future +* all go modules should follow the guidelines in https://go.dev/blog/module-compatibility +before `v1.0.0` is tagged and should make use of `internal` packages to limit +the exposed API surface +* the new go module's API may deviate from the existing code where there are +clear improvements to be made or to remove legacy dependencies (for instance on +amino or gogo proto), as long the old package attempts +to avoid API breakage with aliases and wrappers +* care should be taken when simply trying to turn an existing package into a +new go module: https://github.com/golang/go/wiki/Modules#is-it-possible-to-add-a-module-to-a-multi-module-repository. +In general, it seems safer to just create a new module path (appending v2, v3, etc. +if necessary), rather than trying to make an old package a new module. + +## Consequences + +### Backwards Compatibility + +If the above guidelines are followed to use aliases or wrapper types pointing +in existing APIs that point back to the new go modules, there should be no or +very limited breaking changes to existing APIs. + +### Positive + +* standalone pieces of software will reach `v1.0.0` sooner +* new features to specific functionality will be released sooner + +### Negative + +* there will be more go module versions to update in the SDK itself and +per-project, although most of these will hopefully be indirect + +### Neutral + +## Further Discussions + +Further discussions are occurring in primarily in +https://github.com/cosmos/cosmos-sdk/discussions/10582 and within +the Cosmos SDK Framework Working Group. + +## References + +* https://go.dev/doc/modules/release-workflow +* https://go.dev/blog/module-compatibility +* https://github.com/cosmos/cosmos-sdk/discussions/10162 +* https://github.com/cosmos/cosmos-sdk/discussions/10582 +* https://github.com/cosmos/cosmos-sdk/pull/10779 +* https://github.com/cosmos/cosmos-sdk/pull/11788 diff --git a/versioned_docs/version-0.52/build/architecture/adr-054-semver-compatible-modules.md b/versioned_docs/version-0.52/build/architecture/adr-054-semver-compatible-modules.md new file mode 100644 index 000000000..6710ff8f0 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-054-semver-compatible-modules.md @@ -0,0 +1,731 @@ +# ADR 054: Semver Compatible SDK Modules + +## Changelog + +* 2022-04-27: First draft + +## Status + +DRAFT + +## Abstract + +In order to move the Cosmos SDK to a system of decoupled semantically versioned +modules which can be composed in different combinations (ex. staking v3 with +bank v1 and distribution v2), we need to reassess how we organize the API surface +of modules to avoid problems with go semantic import versioning and +circular dependencies. This ADR explores various approaches we can take to +addressing these issues. + +## Context + +There has been [a fair amount of desire](https://github.com/cosmos/cosmos-sdk/discussions/10162) +in the community for semantic versioning in the SDK and there has been significant +movement to splitting SDK modules into [standalone go modules](https://github.com/cosmos/cosmos-sdk/issues/11899). +Both of these will ideally allow the ecosystem to move faster because we won't +be waiting for all dependencies to update synchronously. For instance, we could +have 3 versions of the core SDK compatible with the latest 2 releases of +CosmWasm as well as 4 different versions of staking . This sort of setup would +allow early adopters to aggressively integrate new versions, while allowing +more conservative users to be selective about which versions they're ready for. + +In order to achieve this, we need to solve the following problems: + +1. because of the way [go semantic import versioning](https://research.swtch.com/vgo-import) (SIV) + works, moving to SIV naively will actually make it harder to achieve these goals +2. circular dependencies between modules need to be broken to actually release + many modules in the SDK independently +3. pernicious minor version incompatibilities introduced through correctly + [evolving protobuf schemas](https://protobuf.dev/programming-guides/proto3/#updating) + without correct [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) + +Note that all the following discussion assumes that the proto file versioning and state machine versioning of a module +are distinct in that: + +* proto files are maintained in a non-breaking way (using something + like [buf breaking](https://docs.buf.build/breaking/overview) + to ensure all changes are backwards compatible) +* proto file versions get bumped much less frequently, i.e. we might maintain `cosmos.bank.v1` through many versions + of the bank module state machine +* state machine breaking changes are more common and ideally this is what we'd want to semantically version with + go modules, ex. `x/bank/v2`, `x/bank/v3`, etc. + +### Problem 1: Semantic Import Versioning Compatibility + +Consider we have a module `foo` which defines the following `MsgDoSomething` and that we've released its state +machine in go module `example.com/foo`: + +```protobuf +package foo.v1; + +message MsgDoSomething { + string sender = 1; + uint64 amount = 2; +} + +service Msg { + DoSomething(MsgDoSomething) returns (MsgDoSomethingResponse); +} +``` + +Now consider that we make a revision to this module and add a new `condition` field to `MsgDoSomething` and also +add a new validation rule on `amount` requiring it to be non-zero, and that following go semantic versioning we +release the next state machine version of `foo` as `example.com/foo/v2`. + +```protobuf +// Revision 1 +package foo.v1; + +message MsgDoSomething { + string sender = 1; + + // amount must be a non-zero integer. + uint64 amount = 2; + + // condition is an optional condition on doing the thing. + // + // Since: Revision 1 + Condition condition = 3; +} +``` + +Approaching this naively, we would generate the protobuf types for the initial +version of `foo` in `example.com/foo/types` and we would generate the protobuf +types for the second version in `example.com/foo/v2/types`. + +Now let's say we have a module `bar` which talks to `foo` using this keeper +interface which `foo` provides: + +```go +type FooKeeper interface { + DoSomething(MsgDoSomething) error +} +``` + +#### Scenario A: Backward Compatibility: Newer Foo, Older Bar + +Imagine we have a chain which uses both `foo` and `bar` and wants to upgrade to +`foo/v2`, but the `bar` module has not upgraded to `foo/v2`. + +In this case, the chain will not be able to upgrade to `foo/v2` until `bar` +has upgraded its references to `example.com/foo/types.MsgDoSomething` to +`example.com/foo/v2/types.MsgDoSomething`. + +Even if `bar`'s usage of `MsgDoSomething` has not changed at all, the upgrade +will be impossible without this change because `example.com/foo/types.MsgDoSomething` +and `example.com/foo/v2/types.MsgDoSomething` are fundamentally different +incompatible structs in the go type system. + +#### Scenario B: Forward Compatibility: Older Foo, Newer Bar + +Now let's consider the reverse scenario, where `bar` upgrades to `foo/v2` +by changing the `MsgDoSomething` reference to `example.com/foo/v2/types.MsgDoSomething` +and releases that as `bar/v2` with some other changes that a chain wants. +The chain, however, has decided that it thinks the changes in `foo/v2` are too +risky and that it'd prefer to stay on the initial version of `foo`. + +In this scenario, it is impossible to upgrade to `bar/v2` without upgrading +to `foo/v2` even if `bar/v2` would have worked 100% fine with `foo` other +than changing the import path to `MsgDoSomething` (meaning that `bar/v2` +doesn't actually use any new features of `foo/v2`). + +Now because of the way go semantic import versioning works, we are locked +into either using `foo` and `bar` OR `foo/v2` and `bar/v2`. We cannot have +`foo` + `bar/v2` OR `foo/v2` + `bar`. The go type system doesn't allow this +even if both versions of these modules are otherwise compatible with each +other. + +#### Naive Mitigation + +A naive approach to fixing this would be to not regenerate the protobuf types +in `example.com/foo/v2/types` but instead just update `example.com/foo/types` +to reflect the changes needed for `v2` (adding `condition` and requiring +`amount` to be non-zero). Then we could release a patch of `example.com/foo/types` +with this update and use that for `foo/v2`. But this change is state machine +breaking for `v1`. It requires changing the `ValidateBasic` method to reject +the case where `amount` is zero, and it adds the `condition` field which +should be rejected based +on [ADR 020 unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering). +So adding these changes as a patch on `v1` is actually incorrect based on semantic +versioning. Chains that want to stay on `v1` of `foo` should not +be importing these changes because they are incorrect for `v1.` + +### Problem 2: Circular dependencies + +None of the above approaches allow `foo` and `bar` to be separate modules +if for some reason `foo` and `bar` depend on each other in different ways. +For instance, we can't have `foo` import `bar/types` while `bar` imports +`foo/types`. + +We have several cases of circular module dependencies in the SDK +(ex. staking, distribution and slashing) that are legitimate from a state machine +perspective. Without separating the API types out somehow, there would be +no way to independently semantically version these modules without some other +mitigation. + +### Problem 3: Handling Minor Version Incompatibilities + +Imagine that we solve the first two problems but now have a scenario where +`bar/v2` wants the option to use `MsgDoSomething.condition` which only `foo/v2` +supports. If `bar/v2` works with `foo` `v1` and sets `condition` to some non-nil +value, then `foo` will silently ignore this field resulting in a silent logic +possibly dangerous logic error. If `bar/v2` were able to check whether `foo` was +on `v1` or `v2` and dynamically, it could choose to only use `condition` when +`foo/v2` is available. Even if `bar/v2` were able to perform this check, however, +how do we know that it is always performing the check properly. Without +some sort of +framework-level [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), +it is hard to know whether these pernicious hard to detect bugs are getting into +our app and a client-server layer such as [ADR 033: Inter-Module Communication](./adr-033-protobuf-inter-module-comm.md) +may be needed to do this. + +## Solutions + +### Approach A) Separate API and State Machine Modules + +One solution (first proposed in https://github.com/cosmos/cosmos-sdk/discussions/10582) is to isolate all protobuf +generated code into a separate module +from the state machine module. This would mean that we could have state machine +go modules `foo` and `foo/v2` which could use a types or API go module say +`foo/api`. This `foo/api` go module would be perpetually on `v1.x` and only +accept non-breaking changes. This would then allow other modules to be +compatible with either `foo` or `foo/v2` as long as the inter-module API only +depends on the types in `foo/api`. It would also allow modules `foo` and `bar` +to depend on each other in that both of them could depend on `foo/api` and +`bar/api` without `foo` directly depending on `bar` and vice versa. + +This is similar to the naive mitigation described above except that it separates +the types into separate go modules which in and of itself could be used to +break circular module dependencies. It has the same problems as the naive solution, +otherwise, which we could rectify by: + +1. removing all state machine breaking code from the API module (ex. `ValidateBasic` and any other interface methods) +2. embedding the correct file descriptors for unknown field filtering in the binary + +#### Migrate all interface methods on API types to handlers + +To solve 1), we need to remove all interface implementations from generated +types and instead use a handler approach which essentially means that given +a type `X`, we have some sort of resolver which allows us to resolve interface +implementations for that type (ex. `sdk.Msg` or `authz.Authorization`). For +example: + +```go +func (k Keeper) DoSomething(msg MsgDoSomething) error { + var validateBasicHandler ValidateBasicHandler + err := k.resolver.Resolve(&validateBasicHandler, msg) + if err != nil { + return err + } + + err = validateBasicHandler.ValidateBasic() + ... +} +``` + +In the case of some methods on `sdk.Msg`, we could replace them with declarative +annotations. For instance, `GetSigners` can already be replaced by the protobuf +annotation `cosmos.msg.v1.signer`. In the future, we may consider some sort +of protobuf validation framework (like https://github.com/bufbuild/protoc-gen-validate +but more Cosmos-specific) to replace `ValidateBasic`. + +#### Pinned FileDescriptor's + +To solve 2), state machine modules must be able to specify what the version of +the protobuf files was that they were built against. For instance if the API +module for `foo` upgrades to `foo/v2`, the original `foo` module still needs +a copy of the original protobuf files it was built with so that ADR 020 +unknown field filtering will reject `MsgDoSomething` when `condition` is +set. + +The simplest way to do this may be to embed the protobuf `FileDescriptor`s into +the module itself so that these `FileDescriptor`s are used at runtime rather +than the ones that are built into the `foo/api` which may be different. Using +[buf build](https://docs.buf.build/build/usage#output-format), [go embed](https://pkg.go.dev/embed), +and a build script we can probably come up with a solution for embedding +`FileDescriptor`s into modules that is fairly straightforward. + +#### Potential limitations to generated code + +One challenge with this approach is that it places heavy restrictions on what +can go in API modules and requires that most of this is state machine breaking. +All or most of the code in the API module would be generated from protobuf +files, so we can probably control this with how code generation is done, but +it is a risk to be aware of. + +For instance, we do code generation for the ORM that in the future could +contain optimizations that are state machine breaking. We +would either need to ensure very carefully that the optimizations aren't +actually state machine breaking in generated code or separate this generated code +out from the API module into the state machine module. Both of these mitigations +are potentially viable but the API module approach does require an extra level +of care to avoid these sorts of issues. + +#### Minor Version Incompatibilities + +This approach in and of itself does little to address any potential minor +version incompatibilities and the +requisite [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering). +Likely some sort of client-server routing layer which does this check such as +[ADR 033: Inter-Module communication](./adr-033-protobuf-inter-module-comm.md) +is required to make sure that this is done properly. We could then allow +modules to perform a runtime check given a `MsgClient`, ex: + +```go +func (k Keeper) CallFoo() error { + if k.interModuleClient.MinorRevision(k.fooMsgClient) >= 2 { + k.fooMsgClient.DoSomething(&MsgDoSomething{Condition: ...}) + } else { + ... + } +} +``` + +To do the unknown field filtering itself, the ADR 033 router would need to use +the [protoreflect API](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect) +to ensure that no fields unknown to the receiving module are set. This could +result in an undesirable performance hit depending on how complex this logic is. + +### Approach B) Changes to Generated Code + +An alternate approach to solving the versioning problem is to change how protobuf code is generated and move modules +mostly or completely in the direction of inter-module communication as described +in [ADR 033](./adr-033-protobuf-inter-module-comm.md). +In this paradigm, a module could generate all the types it needs internally - including the API types of other modules - +and talk to other modules via a client-server boundary. For instance, if `bar` needs to talk to `foo`, it could +generate its own version of `MsgDoSomething` as `bar/internal/foo/v1.MsgDoSomething` and just pass this to the +inter-module router which would somehow convert it to the version which foo needs (ex. `foo/internal.MsgDoSomething`). + +Currently, two generated structs for the same protobuf type cannot exist in the same go binary without special +build flags (see https://protobuf.dev/reference/go/faq/#fix-namespace-conflict). +A relatively simple mitigation to this issue would be to set up the protobuf code to not register protobuf types +globally if they are generated in an `internal/` package. This will require modules to register their types manually +with the app-level level protobuf registry, this is similar to what modules already do with the `InterfaceRegistry` +and amino codec. + +If modules _only_ do ADR 033 message passing then a naive and non-performant solution for +converting `bar/internal/foo/v1.MsgDoSomething` +to `foo/internal.MsgDoSomething` would be marshaling and unmarshaling in the ADR 033 router. This would break down if +we needed to expose protobuf types in `Keeper` interfaces because the whole point is to try to keep these types +`internal/` so that we don't end up with all the import version incompatibilities we've described above. However, +because of the issue with minor version incompatibilities and the need +for [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), +sticking with the `Keeper` paradigm instead of ADR 033 may be unviable to begin with. + +A more performant solution (that could maybe be adapted to work with `Keeper` interfaces) would be to only expose +getters and setters for generated types and internally store data in memory buffers which could be passed from +one implementation to another in a zero-copy way. + +For example, imagine this protobuf API with only getters and setters is exposed for `MsgSend`: + +```go +type MsgSend interface { + proto.Message + GetFromAddress() string + GetToAddress() string + GetAmount() []v1beta1.Coin + SetFromAddress(string) + SetToAddress(string) + SetAmount([]v1beta1.Coin) +} + +func NewMsgSend() MsgSend { return &msgSendImpl{memoryBuffers: ...} } +``` + +Under the hood, `MsgSend` could be implemented based on some raw memory buffer in the same way +that [Cap'n Proto](https://capnproto.org) +and [FlatBuffers](https://google.github.io/flatbuffers/) so that we could convert between one version of `MsgSend` +and another without serialization (i.e. zero-copy). This approach would have the added benefits of allowing zero-copy +message passing to modules written in other languages such as Rust and accessed through a VM or FFI. It could also make +unknown field filtering in inter-module communication simpler if we require that all new fields are added in sequential +order, ex. just checking that no field `> 5` is set. + +Also, we wouldn't have any issues with state machine breaking code on generated types because all the generated +code used in the state machine would actually live in the state machine module itself. Depending on how interface +types and protobuf `Any`s are used in other languages, however, it may still be desirable to take the handler +approach described in approach A. Either way, types implementing interfaces would still need to be registered +with an `InterfaceRegistry` as they are now because there would be no way to retrieve them via the global registry. + +In order to simplify access to other modules using ADR 033, a public API module (maybe even one +[remotely generated by Buf](https://docs.buf.build/bsr/remote-generation/go)) could be used by client modules instead +of requiring to generate all client types internally. + +The big downsides of this approach are that it requires big changes to how people use protobuf types and would be a +substantial rewrite of the protobuf code generator. This new generated code, however, could still be made compatible +with +the [`google.golang.org/protobuf/reflect/protoreflect`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect) +API in order to work with all standard golang protobuf tooling. + +It is possible that the naive approach of marshaling/unmarshaling in the ADR 033 router is an acceptable intermediate +solution if the changes to the code generator are seen as too complex. However, since all modules would likely need +to migrate to ADR 033 anyway with this approach, it might be better to do this all at once. + +### Approach C) Don't address these issues + +If the above solutions are seen as too complex, we can also decide not to do anything explicit to enable better module +version compatibility, and break circular dependencies. + +In this case, when developers are confronted with the issues described above they can require dependencies to update in +sync (what we do now) or attempt some ad-hoc potentially hacky solution. + +One approach is to ditch go semantic import versioning (SIV) altogether. Some people have commented that go's SIV +(i.e. changing the import path to `foo/v2`, `foo/v3`, etc.) is too restrictive and that it should be optional. The +golang maintainers disagree and only officially support semantic import versioning. We could, however, take the +contrarian perspective and get more flexibility by using 0.x-based versioning basically forever. + +Module version compatibility could then be achieved using go.mod replace directives to pin dependencies to specific +compatible 0.x versions. For instance if we knew `foo` 0.2 and 0.3 were both compatible with `bar` 0.3 and 0.4, we +could use replace directives in our go.mod to stick to the versions of `foo` and `bar` we want. This would work as +long as the authors of `foo` and `bar` avoid incompatible breaking changes between these modules. + +Or, if developers choose to use semantic import versioning, they can attempt the naive solution described above +and would also need to use special tags and replace directives to make sure that modules are pinned to the correct +versions. + +Note, however, that all of these ad-hoc approaches, would be vulnerable to the minor version compatibility issues +described above unless [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) +is properly addressed. + +### Approach D) Avoid protobuf generated code in public APIs + +An alternative approach would be to avoid protobuf generated code in public module APIs. This would help avoid the +discrepancy between state machine versions and client API versions at the module to module boundaries. It would mean +that we wouldn't do inter-module message passing based on ADR 033, but rather stick to the existing keeper approach +and take it one step further by avoiding any protobuf generated code in the keeper interface methods. + +Using this approach, our `foo.Keeper.DoSomething` method wouldn't have the generated `MsgDoSomething` struct (which +comes from the protobuf API), but instead positional parameters. Then in order for `foo/v2` to support the `foo/v1` +keeper it would simply need to implement both the v1 and v2 keeper APIs. The `DoSomething` method in v2 could have the +additional `condition` parameter, but this wouldn't be present in v1 at all so there would be no danger of a client +accidentally setting this when it isn't available. + +So this approach would avoid the challenge around minor version incompatibilities because the existing module keeper +API would not get new fields when they are added to protobuf files. + +Taking this approach, however, would likely require making all protobuf generated code internal in order to prevent +it from leaking into the keeper API. This means we would still need to modify the protobuf code generator to not +register `internal/` code with the global registry, and we would still need to manually register protobuf +`FileDescriptor`s (this is probably true in all scenarios). It may, however, be possible to avoid needing to refactor +interface methods on generated types to handlers. + +Also, this approach doesn't address what would be done in scenarios where modules still want to use the message router. +Either way, we probably still want a way to pass messages from one module to another router safely even if it's just for +use cases like `x/gov`, `x/authz`, CosmWasm, etc. That would still require most of the things outlined in approach (B), +although we could advise modules to prefer keepers for communicating with other modules. + +The biggest downside of this approach is probably that it requires a strict refactoring of keeper interfaces to avoid +generated code leaking into the API. This may result in cases where we need to duplicate types that are already defined +in proto files and then write methods for converting between the golang and protobuf version. This may end up in a lot +of unnecessary boilerplate and that may discourage modules from actually adopting it and achieving effective version +compatibility. Approaches (A) and (B), although heavy handed initially, aim to provide a system which once adopted +more or less gives the developer version compatibility for free with minimal boilerplate. Approach (D) may not be able +to provide such a straightforward system since it requires a golang API to be defined alongside a protobuf API in a +way that requires duplication and differing sets of design principles (protobuf APIs encourage additive changes +while golang APIs would forbid it). + +Other downsides to this approach are: + +* no clear roadmap to supporting modules in other languages like Rust +* doesn't get us any closer to proper object capability security (one of the goals of ADR 033) +* ADR 033 needs to be done properly anyway for the set of use cases which do need it + +## Decision + +The latest **DRAFT** proposal is: + +1. we are alignment on adopting [ADR 033](./adr-033-protobuf-inter-module-comm.md) not just as an addition to the + framework, but as a core replacement to the keeper paradigm entirely. +2. the ADR 033 inter-module router will accommodate any variation of approach (A) or (B) given the following rules: + a. if the client type is the same as the server type then pass it directly through, + b. if both client and server use the zero-copy generated code wrappers (which still need to be defined), then pass + the memory buffers from one wrapper to the other, or + c. marshal/unmarshal types between client and server. + +This approach will allow for both maximal correctness and enable a clear path to enabling modules within in other +languages, possibly executed within a WASM VM. + +### Minor API Revisions + +To declare minor API revisions of proto files, we propose the following guidelines (which were already documented +in [cosmos.app.v1alpha module options](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/app/v1alpha1/module.proto): + +* proto packages which are revised from their initial version (considered revision `0`) should include a `package` +* comment in some .proto file containing the test `Revision N` at the start of a comment line where `N` is the current +revision number. +* all fields, messages, etc. added in a version beyond the initial revision should add a comment at the start of a +comment line of the form `Since: Revision N` where `N` is the non-zero revision it was added. + +It is advised that there is a 1:1 correspondence between a state machine module and versioned set of proto files +which are versioned either as a buf module a go API module or both. If the buf schema registry is used, the version of +this buf module should always be `1.N` where `N` corresponds to the package revision. Patch releases should be used when +only documentation comments are updated. It is okay to include proto packages named `v2`, `v3`, etc. in this same +`1.N` versioned buf module (ex. `cosmos.bank.v2`) as long as all these proto packages consist of a single API intended +to be served by a single SDK module. + +### Introspecting Minor API Revisions + +In order for modules to introspect the minor API revision of peer modules, we propose adding the following method +to `cosmossdk.io/core/intermodule.Client`: + +```go +ServiceRevision(ctx context.Context, serviceName string) uint64 +``` + +Modules could all this using the service name statically generated by the go grpc code generator: + +```go +intermoduleClient.ServiceRevision(ctx, bankv1beta1.Msg_ServiceDesc.ServiceName) +``` + +In the future, we may decide to extend the code generator used for protobuf services to add a field +to client types which does this check more concisely, ex: + +```go +package bankv1beta1 + +type MsgClient interface { + Send(context.Context, MsgSend) (MsgSendResponse, error) + ServiceRevision(context.Context) uint64 +} +``` + +### Unknown Field Filtering + +To correctly perform [unknown field filtering](./adr-020-protobuf-transaction-encoding.md#unknown-field-filtering), +the inter-module router can do one of the following: + +* use the `protoreflect` API for messages which support that +* for gogo proto messages, marshal and use the existing `codec/unknownproto` code +* for zero-copy messages, do a simple check on the highest set field number (assuming we can require that fields are + adding consecutively in increasing order) + +### `FileDescriptor` Registration + +Because a single go binary may contain different versions of the same generated protobuf code, we cannot rely on the +global protobuf registry to contain the correct `FileDescriptor`s. Because `appconfig` module configuration is itself +written in protobuf, we would like to load the `FileDescriptor`s for a module before loading a module itself. So we +will provide ways to register `FileDescriptor`s at module registration time before instantiation. We propose the +following `cosmossdk.io/core/appmodule.Option` constructors for the various cases of how `FileDescriptor`s may be +packaged: + +```go +package appmodule + +// this can be used when we are using google.golang.org/protobuf compatible generated code +// Ex: +// ProtoFiles(bankv1beta1.File_cosmos_bank_v1beta1_module_proto) +func ProtoFiles(file []protoreflect.FileDescriptor) Option {} + +// this can be used when we are using gogo proto generated code. +func GzippedProtoFiles(file [][]byte) Option {} + +// this can be used when we are using buf build to generated a pinned file descriptor +func ProtoImage(protoImage []byte) Option {} +``` + +This approach allows us to support several ways protobuf files might be generated: + +* proto files generated internally to a module (use `ProtoFiles`) +* the API module approach with pinned file descriptors (use `ProtoImage`) +* gogo proto (use `GzippedProtoFiles`) + +### Module Dependency Declaration + +One risk of ADR 033 is that dependencies are called at runtime which are not present in the loaded set of SDK modules. +Also we want modules to have a way to define a minimum dependency API revision that they require. Therefore, all +modules should declare their set of dependencies upfront. These dependencies could be defined when a module is +instantiated, but ideally we know what the dependencies are before instantiation and can statically look at an app +config and determine whether the set of modules. For example, if `bar` requires `foo` revision `>= 1`, then we +should be able to know this when creating an app config with two versions of `bar` and `foo`. + +We propose defining these dependencies in the proto options of the module config object itself. + +### Interface Registration + +We will also need to define how interface methods are defined on types that are serialized as `google.protobuf.Any`'s. +In light of the desire to support modules in other languages, we may want to think of solutions that will accommodate +other languages such as plugins described briefly in [ADR 033](./adr-033-protobuf-inter-module-comm.md#internal-methods). + +### Testing + +In order to ensure that modules are indeed with multiple versions of their dependencies, we plan to provide specialized +unit and integration testing infrastructure that automatically tests multiple versions of dependencies. + +#### Unit Testing + +Unit tests should be conducted inside SDK modules by mocking their dependencies. In a full ADR 033 scenario, +this means that all interaction with other modules is done via the inter-module router, so mocking of dependencies +means mocking their msg and query server implementations. We will provide both a test runner and fixture to make this +streamlined. The key thing that the test runner should do to test compatibility is to test all combinations of +dependency API revisions. This can be done by taking the file descriptors for the dependencies, parsing their comments +to determine the revisions various elements were added, and then created synthetic file descriptors for each revision +by subtracting elements that were added later. + +Here is a proposed API for the unit test runner and fixture: + +```go +package moduletesting + +import ( + "context" + "testing" + + "cosmossdk.io/core/intermodule" + "cosmossdk.io/depinject" + "google.golang.org/grpc" + "google.golang.org/protobuf/proto" + "google.golang.org/protobuf/reflect/protodesc" +) + +type TestFixture interface { + context.Context + intermodule.Client // for making calls to the module we're testing + BeginBlock() + EndBlock() +} + +type UnitTestFixture interface { + TestFixture + grpc.ServiceRegistrar // for registering mock service implementations +} + +type UnitTestConfig struct { + ModuleConfig proto.Message // the module's config object + DepinjectConfig depinject.Config // optional additional depinject config options + DependencyFileDescriptors []protodesc.FileDescriptorProto // optional dependency file descriptors to use instead of the global registry +} + +// Run runs the test function for all combinations of dependency API revisions. +func (cfg UnitTestConfig) Run(t *testing.T, f func(t *testing.T, f UnitTestFixture)) { + // ... +} +``` + +Here is an example for testing bar calling foo which takes advantage of conditional service revisions in the expected +mock arguments: + +```go +func TestBar(t *testing.T) { + UnitTestConfig{ModuleConfig: &foomodulev1.Module{}}.Run(t, func (t *testing.T, f moduletesting.UnitTestFixture) { + ctrl := gomock.NewController(t) + mockFooMsgServer := footestutil.NewMockMsgServer() + foov1.RegisterMsgServer(f, mockFooMsgServer) + barMsgClient := barv1.NewMsgClient(f) + if f.ServiceRevision(foov1.Msg_ServiceDesc.ServiceName) >= 1 { + mockFooMsgServer.EXPECT().DoSomething(gomock.Any(), &foov1.MsgDoSomething{ + ..., + Condition: ..., // condition is expected in revision >= 1 + }).Return(&foov1.MsgDoSomethingResponse{}, nil) + } else { + mockFooMsgServer.EXPECT().DoSomething(gomock.Any(), &foov1.MsgDoSomething{...}).Return(&foov1.MsgDoSomethingResponse{}, nil) + } + res, err := barMsgClient.CallFoo(f, &MsgCallFoo{}) + ... + }) +} +``` + +The unit test runner would make sure that no dependency mocks return arguments which are invalid for the service +revision being tested to ensure that modules don't incorrectly depend on functionality not present in a given revision. + +#### Integration Testing + +An integration test runner and fixture would also be provided which instead of using mocks would test actual module +dependencies in various combinations. Here is the proposed API: + +```go +type IntegrationTestFixture interface { + TestFixture +} + +type IntegrationTestConfig struct { + ModuleConfig proto.Message // the module's config object + DependencyMatrix map[string][]proto.Message // all the dependent module configs +} + +// Run runs the test function for all combinations of dependency modules. +func (cfg IntegationTestConfig) Run(t *testing.T, f func (t *testing.T, f IntegrationTestFixture)) { + // ... +} +``` + +And here is an example with foo and bar: + +```go +func TestBarIntegration(t *testing.T) { + IntegrationTestConfig{ + ModuleConfig: &barmodulev1.Module{}, + DependencyMatrix: map[string][]proto.Message{ + "runtime": []proto.Message{ // test against two versions of runtime + &runtimev1.Module{}, + &runtimev2.Module{}, + }, + "foo": []proto.Message{ // test against three versions of foo + &foomodulev1.Module{}, + &foomodulev2.Module{}, + &foomodulev3.Module{}, + } + } + }.Run(t, func (t *testing.T, f moduletesting.IntegrationTestFixture) { + barMsgClient := barv1.NewMsgClient(f) + res, err := barMsgClient.CallFoo(f, &MsgCallFoo{}) + ... + }) +} +``` + +Unlike unit tests, integration tests actually pull in other module dependencies. So that modules can be written +without direct dependencies on other modules and because golang has no concept of development dependencies, integration +tests should be written in separate go modules, ex. `example.com/bar/v2/test`. Because this paradigm uses go semantic +versioning, it is possible to build a single go module which imports 3 versions of bar and 2 versions of runtime and +can test these all together in the six various combinations of dependencies. + +## Consequences + +### Backwards Compatibility + +Modules which migrate fully to ADR 033 will not be compatible with existing modules which use the keeper paradigm. +As a temporary workaround we may create some wrapper types that emulate the current keeper interface to minimize +the migration overhead. + +### Positive + +* we will be able to deliver interoperable semantically versioned modules which should dramatically increase the + ability of the Cosmos SDK ecosystem to iterate on new features +* it will be possible to write Cosmos SDK modules in other languages in the near future + +### Negative + +* all modules will need to be refactored somewhat dramatically + +### Neutral + +* the `cosmossdk.io/core/appconfig` framework will play a more central role in terms of how modules are defined, this + is likely generally a good thing but does mean additional changes for users wanting to stick to the pre-depinject way + of wiring up modules +* `depinject` is somewhat less needed or maybe even obviated because of the full ADR 033 approach. If we adopt the + core API proposed in https://github.com/cosmos/cosmos-sdk/pull/12239, then a module would probably always instantiate + itself with a method `ProvideModule(appmodule.Service) (appmodule.AppModule, error)`. There is no complex wiring of + keeper dependencies in this scenario and dependency injection may not have as much of (or any) use case. + +## Further Discussions + +The decision described above is considered in draft mode and is pending final buy-in from the team and key stakeholders. +Key outstanding discussions if we do adopt that direction are: + +* how do module clients introspect dependency module API revisions +* how do modules determine a minor dependency module API revision requirement +* how do modules appropriately test compatibility with different dependency versions +* how to register and resolve interface implementations +* how do modules register their protobuf file descriptors depending on the approach they take to generated code (the + API module approach may still be viable as a supported strategy and would need pinned file descriptors) + +## References + +* https://github.com/cosmos/cosmos-sdk/discussions/10162 +* https://github.com/cosmos/cosmos-sdk/discussions/10582 +* https://github.com/cosmos/cosmos-sdk/discussions/10368 +* https://github.com/cosmos/cosmos-sdk/pull/11340 +* https://github.com/cosmos/cosmos-sdk/issues/11899 +* [ADR 020](./adr-020-protobuf-transaction-encoding.md) +* [ADR 033](./adr-033-protobuf-inter-module-comm.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-055-orm.md b/versioned_docs/version-0.52/build/architecture/adr-055-orm.md new file mode 100644 index 000000000..be7255f09 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-055-orm.md @@ -0,0 +1,113 @@ +# ADR 055: ORM + +## Changelog + +* 2022-04-27: First draft + +## Status + +ACCEPTED Implemented + +## Abstract + +In order to make it easier for developers to build Cosmos SDK modules and for clients to query, index and verify proofs +against state data, we have implemented an ORM (object-relational mapping) layer for the Cosmos SDK. + +## Context + +Historically modules in the Cosmos SDK have always used the key-value store directly and created various handwritten +functions for managing key format as well as constructing secondary indexes. This consumes a significant amount of +time when building a module and is error-prone. Because key formats are non-standard, sometimes poorly documented, +and subject to change, it is hard for clients to generically index, query and verify merkle proofs against state data. + +The known first instance of an "ORM" in the Cosmos ecosystem was in [weave](https://github.com/iov-one/weave/tree/master/orm). +A later version was built for [regen-ledger](https://github.com/regen-network/regen-ledger/tree/157181f955823149e1825263a317ad8e16096da4/orm) for +use in the group module and later [ported to the SDK](https://github.com/cosmos/cosmos-sdk/tree/35d3312c3be306591fcba39892223f1244c8d108/x/group/internal/orm) +just for that purpose. + +While these earlier designs made it significantly easier to write state machines, they still required a lot of manual +configuration, didn't expose state format directly to clients, and were limited in their support of different types +of index keys, composite keys, and range queries. + +Discussions about the design continued in https://github.com/cosmos/cosmos-sdk/discussions/9156 and more +sophisticated proofs of concept were created in https://github.com/allinbits/cosmos-sdk-poc/tree/master/runtime/orm +and https://github.com/cosmos/cosmos-sdk/pull/10454. + +## Decision + +These prior efforts culminated in the creation of the Cosmos SDK `orm` go module which uses protobuf annotations +for specifying ORM table definitions. This ORM is based on the new `google.golang.org/protobuf/reflect/protoreflect` +API and supports: + +* sorted indexes for all simple protobuf types (except `bytes`, `enum`, `float`, `double`) as well as `Timestamp` and `Duration` +* unsorted `bytes` and `enum` indexes +* composite primary and secondary keys +* unique indexes +* auto-incrementing `uint64` primary keys +* complex prefix and range queries +* paginated queries +* complete logical decoding of KV-store data + +Almost all the information needed to decode state directly is specified in .proto files. Each table definition specifies +an ID which is unique per .proto file and each index within a table is unique within that table. Clients then only need +to know the name of a module and the prefix ORM data for a specific .proto file within that module in order to decode +state data directly. This additional information will be exposed directly through app configs which will be explained +in a future ADR related to app wiring. + +The ORM makes optimizations around storage space by not repeating values in the primary key in the key value +when storing primary key records. For example, if the object `{"a":0,"b":1}` has the primary key `a`, it will +be stored in the key value store as `Key: '0', Value: {"b":1}` (with more efficient protobuf binary encoding). +Also, the generated code from https://github.com/cosmos/cosmos-proto does optimizations around the +`google.golang.org/protobuf/reflect/protoreflect` API to improve performance. + +A code generator is included with the ORM which creates type safe wrappers around the ORM's dynamic `Table` +implementation and is the recommended way for modules to use the ORM. + +The ORM tests provide a simplified bank module demonstration which illustrates: +* [ORM proto options](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/internal/testpb/bank.proto) +* [Generated Code](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/internal/testpb/bank.cosmos_orm.go) +* [Example Usage in a Module Keeper](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/model/ormdb/module_test.go) + +## Consequences + +### Backwards Compatibility + +State machine code that adopts the ORM will need migrations as the state layout is generally backwards incompatible. +These state machines will also need to migrate to https://github.com/cosmos/cosmos-proto at least for state data. + +### Positive + +* easier to build modules +* easier to add secondary indexes to state +* possible to write a generic indexer for ORM state +* easier to write clients that do state proofs +* possible to automatically write query layers rather than needing to manually implement gRPC queries + +### Negative + +* worse performance than handwritten keys (for now). See [Further Discussions](#further-discussions) +for potential improvements + +### Neutral + +## Further Discussions + +Further discussions will happen within the Cosmos SDK Framework Working Group. Current planned and ongoing work includes: + +* automatically generate client-facing query layer +* client-side query libraries that transparently verify light client proofs +* index ORM data to SQL databases +* improve performance by: + * optimizing existing reflection based code to avoid unnecessary gets when doing deletes & updates of simple tables + * more sophisticated code generation such as making fast path reflection even faster (avoiding `switch` statements), + or even fully generating code that equals handwritten performance + + +## References + +* https://github.com/iov-one/weave/tree/master/orm). +* https://github.com/regen-network/regen-ledger/tree/157181f955823149e1825263a317ad8e16096da4/orm +* https://github.com/cosmos/cosmos-sdk/tree/35d3312c3be306591fcba39892223f1244c8d108/x/group/internal/orm +* https://github.com/cosmos/cosmos-sdk/discussions/9156 +* https://github.com/allinbits/cosmos-sdk-poc/tree/master/runtime/orm +* https://github.com/cosmos/cosmos-sdk/pull/10454 diff --git a/versioned_docs/version-0.52/build/architecture/adr-057-app-wiring.md b/versioned_docs/version-0.52/build/architecture/adr-057-app-wiring.md new file mode 100644 index 000000000..d187cf2f8 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-057-app-wiring.md @@ -0,0 +1,369 @@ +# ADR 057: App Wiring + +## Changelog + +* 2022-05-04: Initial Draft +* 2022-08-19: Updates +* 2024-01-12: Updates + +## Status + +PROPOSED Implemented + +## Abstract + +In order to make it easier to build Cosmos SDK modules and apps, we propose a new app wiring system based on +dependency injection and declarative app configurations to replace the current `app.go` code. + +## Context + +A number of factors have made the SDK and SDK apps in their current state hard to maintain. A symptom of the current +state of complexity is [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/c3edbb22cab8678c35e21fe0253919996b780c01/simapp/app.go) +which contains almost 100 lines of imports and is otherwise over 600 lines of mostly boilerplate code that is +generally copied to each new project. (Not to mention the additional boilerplate which gets copied in `simapp/simd`.) + +The large amount of boilerplate needed to bootstrap an app has made it hard to release independently versioned go +modules for Cosmos SDK modules as described in [ADR 053: Go Module Refactoring](./adr-053-go-module-refactoring.md). + +In addition to being very verbose and repetitive, `app.go` also exposes a large surface area for breaking changes +as most modules instantiate themselves with positional parameters which forces breaking changes anytime a new parameter +(even an optional one) is needed. + +Several attempts were made to improve the current situation including [ADR 033: Internal-Module Communication](./adr-033-protobuf-inter-module-comm.md) +and [a proof-of-concept of a new SDK](https://github.com/allinbits/cosmos-sdk-poc). The discussions around these +designs led to the current solution described here. + +## Decision + +In order to improve the current situation, a new "app wiring" paradigm has been designed to replace `app.go` which +involves: + +* declaration configuration of the modules in an app which can be serialized to JSON or YAML +* a dependency-injection (DI) framework for instantiating apps from the that configuration + +### Dependency Injection + +When examining the code in `app.go` most of the code simply instantiates modules with dependencies provided either +by the framework (such as store keys) or by other modules (such as keepers). It is generally pretty obvious given +the context what the correct dependencies actually should be, so dependency-injection is an obvious solution. Rather +than making developers manually resolve dependencies, a module will tell the DI container what dependency it needs +and the container will figure out how to provide it. + +We explored several existing DI solutions in golang and felt that the reflection-based approach in [uber/dig](https://github.com/uber-go/dig) +was closest to what we needed but not quite there. Assessing what we needed for the SDK, we designed and built +the Cosmos SDK [depinject module](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/depinject), which has the following +features: + +* dependency resolution and provision through functional constructors, ex: `func(need SomeDep) (AnotherDep, error)` +* dependency injection `In` and `Out` structs which support `optional` dependencies +* grouped-dependencies (many-per-container) through the `ManyPerContainerType` tag interface +* module-scoped dependencies via `ModuleKey`s (where each module gets a unique dependency) +* one-per-module dependencies through the `OnePerModuleType` tag interface +* sophisticated debugging information and container visualization via GraphViz + +Here are some examples of how these would be used in an SDK module: + +* `StoreKey` could be a module-scoped dependency which is unique per module +* a module's `AppModule` instance (or the equivalent) could be a `OnePerModuleType` +* CLI commands could be provided with `ManyPerContainerType`s + +Note that even though dependency resolution is dynamic and based on reflection, which could be considered a pitfall +of this approach, the entire dependency graph should be resolved immediately on app startup and only gets resolved +once (except in the case of dynamic config reloading which is a separate topic). This means that if there are any +errors in the dependency graph, they will get reported immediately on startup so this approach is only slightly worse +than fully static resolution in terms of error reporting and much better in terms of code complexity. + +### Declarative App Config + +In order to compose modules into an app, a declarative app configuration will be used. This configuration is based off +of protobuf and its basic structure is very simple: + +```protobuf +package cosmos.app.v1; +message Config { + repeated ModuleConfig modules = 1; +} + +message ModuleConfig { + string name = 1; + google.protobuf.Any config = 2; +} +``` + +The configuration for every module is a protobuf message and modules will be identified and loaded based +on the protobuf type URL of their config object (ex. `cosmos.bank.module.v1.Module`). Modules are given a unique short `name` +to share resources across different versions of the same module which might have a different protobuf package +versions (ex. `cosmos.bank.module.v2.Module`). All module config objects should define the `cosmos.app.v1alpha1.module` +descriptor option which will provide additional useful metadata for the framework and which can also be indexed +in module registries. + +An example app config in YAML might look like this: + +```yaml +modules: + - name: baseapp + config: + "@type": cosmos.baseapp.module.v1.Module + begin_blockers: [staking, auth, bank] + end_blockers: [bank, auth, staking] + init_genesis: [bank, auth, staking] + - name: auth + config: + "@type": cosmos.auth.module.v1.Module + bech32_prefix: "foo" + - name: bank + config: + "@type": cosmos.bank.module.v1.Module + - name: staking + config: + "@type": cosmos.staking.module.v1.Module +``` + +In the above example, there is a hypothetical `baseapp` module which contains the information around ordering of +begin blockers, end blockers, and init genesis. Rather than lifting these concerns up to the module config layer, +they are themselves handled by a module which could allow a convenient way of swapping out different versions of +baseapp (for instance to target different versions of tendermint), without needing to change the rest of the config. +The `baseapp` module would then provide to the server framework (which sort of sits outside the ABCI app) an instance +of `abci.Application`. + +In this model, an app is *modules all the way down* and the dependency injection/app config layer is very much +protocol-agnostic and can adapt to even major breaking changes at the protocol layer. + +### Module & Protobuf Registration + +In order for the two components of dependency injection and declarative configuration to work together as described, +we need a way for modules to actually register themselves and provide dependencies to the container. + +One additional complexity that needs to be handled at this layer is protobuf registry initialization. Recall that +in both the current SDK `codec` and the proposed [ADR 054: Protobuf Semver Compatible Codegen](https://github.com/cosmos/cosmos-sdk/pull/11802), +protobuf types need to be explicitly registered. Given that the app config itself is based on protobuf and +uses protobuf `Any` types, protobuf registration needs to happen before the app config itself can be decoded. Because +we don't know which protobuf `Any` types will be needed a priori and modules themselves define those types, we need +to decode the app config in separate phases: + +1. parse app config JSON/YAML as raw JSON and collect required module type URLs (without doing proto JSON decoding) +2. build a [protobuf type registry](https://pkg.go.dev/google.golang.org/protobuf@v1.28.0/reflect/protoregistry) based + on file descriptors and types provided by each required module +3. decode the app config as proto JSON using the protobuf type registry + +Because in [ADR 054: Protobuf Semver Compatible Codegen](https://github.com/cosmos/cosmos-sdk/pull/11802), each module +might use `internal` generated code which is not registered with the global protobuf registry, this code should provide +an alternate way to register protobuf types with a type registry. In the same way that `.pb.go` files currently have a +`var File_foo_proto protoreflect.FileDescriptor` for the file `foo.proto`, generated code should have a new member +`var Types_foo_proto TypeInfo` where `TypeInfo` is an interface or struct with all the necessary info to register both +the protobuf generated types and file descriptor. + +So a module must provide dependency injection providers and protobuf types, and takes as input its module +config object which uniquely identifies the module based on its type URL. + +With this in mind, we define a global module register which allows module implementations to register themselves +with the following API: + +```go +// Register registers a module with the provided type name (ex. cosmos.bank.module.v1.Module) +// and the provided options. +func Register(configTypeName protoreflect.FullName, option ...Option) { ... } + +type Option { /* private methods */ } + +// Provide registers dependency injection provider functions which work with the +// cosmos-sdk container module. These functions can also accept an additional +// parameter for the module's config object. +func Provide(providers ...interface{}) Option { ... } + +// Types registers protobuf TypeInfo's with the protobuf registry. +func Types(types ...TypeInfo) Option { ... } +``` + +Ex: + +```go +func init() { + appconfig.Register("cosmos.bank.module.v1.Module", + appconfig.Types( + types.Types_tx_proto, + types.Types_query_proto, + types.Types_types_proto, + ), + appconfig.Provide( + provideBankModule, + ) + ) +} + +type Inputs struct { + container.In + + AuthKeeper auth.Keeper + DB ormdb.ModuleDB +} + +type Outputs struct { + Keeper bank.Keeper + AppModule appmodule.AppModule +} + +func ProvideBankModule(config *bankmodulev1.Module, Inputs) (Outputs, error) { ... } +``` + +Note that in this module, a module configuration object *cannot* register different dependency providers at runtime +based on the configuration. This is intentional because it allows us to know globally which modules provide which +dependencies, and it will also allow us to do code generation of the whole app initialization. This +can help us figure out issues with missing dependencies in an app config if the needed modules are loaded at runtime. +In cases where required modules are not loaded at runtime, it may be possible to guide users to the correct module if +through a global Cosmos SDK module registry. + +The `*appmodule.Handler` type referenced above is a replacement for the legacy `AppModule` framework, and +described in [ADR 063: Core Module API](./adr-063-core-module-api.md). + +### New `app.go` + +With this setup, `app.go` might now look something like this: + +```go +package main + +import ( + // Each go package which registers a module must be imported just for side-effects + // so that module implementations are registered. + _ "github.com/cosmos/cosmos-sdk/x/auth/module" + _ "cosmossdk.io/x/bank/module" + _ "cosmossdk.io/x/staking/module" + "github.com/cosmos/cosmos-sdk/core/app" +) + +// go:embed app.yaml +var appConfigYAML []byte + +func main() { + app.Run(app.LoadYAML(appConfigYAML)) +} +``` + +### Application to existing SDK modules + +So far we have described a system which is largely agnostic to the specifics of the SDK such as store keys, `AppModule`, +`BaseApp`, etc. Improvements to these parts of the framework that integrate with the general app wiring framework +defined here are described in [ADR 063: Core Module API](./adr-063-core-module-api.md). + +### Registration of Inter-Module Hooks + +Some modules define a hooks interface (ex. `StakingHooks`) which allows one module to call back into another module +when certain events happen. + +With the app wiring framework, these hooks interfaces can be defined as a `OnePerModuleType`s and then the module +which consumes these hooks can collect these hooks as a map of module name to hook type (ex. `map[string]FooHooks`). Ex: + +```go +func init() { + appconfig.RegisterModule( + &foomodulev1.Module{}, + appconfig.Invoke(InvokeSetFooHooks), + ... + ) +} +func InvokeSetFooHooks( + keeper *keeper.Keeper, + fooHooks map[string]FooHooks, +) error { + for k in sort.Strings(maps.Keys(fooHooks)) { + keeper.AddFooHooks(fooHooks[k]) + } +} +``` + +Optionally, the module consuming hooks can allow app's to define an order for calling these hooks based on module name +in its config object. + +An alternative way for registering hooks via reflection was considered where all keeper types are inspected to see if +they implement the hook interface by the modules exposing hooks. This has the downsides of: + +* needing to expose all the keepers of all modules to the module providing hooks, +* not allowing for encapsulating hooks on a different type which doesn't expose all keeper methods, +* harder to know statically which module expose hooks or are checking for them. + +With the approach proposed here, hooks registration will be obviously observable in `app.go` if `depinject` codegen +(described below) is used. + +### Code Generation + +> Not yet implemented + +The `depinject` framework will optionally allow the app configuration and dependency injection wiring to be code +generated. This will allow: + +* dependency injection wiring to be inspected as regular go code just like the existing `app.go`, +* dependency injection to be opt-in with manual wiring 100% still possible. + +Code generation requires that all providers and invokers and their parameters are exported and in non-internal packages. + +### Module Semantic Versioning + +When we start creating semantically versioned SDK modules that are in standalone go modules, a state machine breaking +change to a module should be handled as follows: + +* the semantic major version should be incremented, and +* a new semantically versioned module config protobuf type should be created. + +For instance, if we have the SDK module for bank in the go module `cosmossdk.io/x/bank` with the module config type +`cosmos.bank.module.v1.Module`, and we want to make a state machine breaking change to the module, we would: + +* create a new go module `cosmossdk.io/x/bank/v2`, +* with the module config protobuf type `cosmos.bank.module.v2.Module`. + +This *does not* mean that we need to increment the protobuf API version for bank. Both modules can support +`cosmos.bank.v1`, but `cosmossdk.io/x/bank/v2` will be a separate go module with a separate module config type. + +This practice will eventually allow us to use appconfig to load new versions of a module via a configuration change. + +Effectively, there should be a 1:1 correspondence between a semantically versioned go module and a +versioned module config protobuf type, and major versioning bumps should occur whenever state machine breaking changes +are made to a module. + +NOTE: SDK modules that are standalone go modules *should not* adopt semantic versioning until the concerns described in +[ADR 054: Module Semantic Versioning](./adr-054-semver-compatible-modules.md) are +addressed. The short-term solution for this issue was left somewhat unresolved. However, the easiest tactic is +likely to use a standalone API go module and follow the guidelines described in this comment: https://github.com/cosmos/cosmos-sdk/pull/11802#issuecomment-1406815181. For the time-being, it is recommended that +Cosmos SDK modules continue to follow tried and true [0-based versioning](https://0ver.org) until an officially +recommended solution is provided. This section of the ADR will be updated when that happens and for now, this section +should be considered as a design recommendation for future adoption of semantic versioning. + +## Consequences + +### Backwards Compatibility + +Modules which work with the new app wiring system do not need to drop their existing `AppModule` and `NewKeeper` +registration paradigms. These two methods can live side-by-side for as long as is needed. + +### Positive + +* wiring up new apps will be simpler, more succinct and less error-prone +* it will be easier to develop and test standalone SDK modules without needing to replicate all of simapp +* it may be possible to dynamically load modules and upgrade chains without needing to do a coordinated stop and binary + upgrade using this mechanism +* easier plugin integration +* dependency injection framework provides more automated reasoning about dependencies in the project, with a graph visualization. + +### Negative + +* it may be confusing when a dependency is missing although error messages, the GraphViz visualization, and global + module registration may help with that + +### Neutral + +* it will require work and education + +## Further Discussions + +The protobuf type registration system described in this ADR has not been implemented and may need to be reconsidered in +light of code generation. It may be better to do this type registration with a DI provider. + +## References + +* https://github.com/cosmos/cosmos-sdk/blob/c3edbb22cab8678c35e21fe0253919996b780c01/simapp/app.go +* https://github.com/allinbits/cosmos-sdk-poc +* https://github.com/uber-go/dig +* https://github.com/google/wire +* https://pkg.go.dev/github.com/cosmos/cosmos-sdk/container +* https://github.com/cosmos/cosmos-sdk/pull/11802 +* [ADR 063: Core Module API](./adr-063-core-module-api.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-058-auto-generated-cli.md b/versioned_docs/version-0.52/build/architecture/adr-058-auto-generated-cli.md new file mode 100644 index 000000000..b295ff4b2 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-058-auto-generated-cli.md @@ -0,0 +1,98 @@ +# ADR 058: Auto-Generated CLI + +## Changelog + +* 2022-05-04: Initial Draft + +## Status + +ACCEPTED Partially Implemented + +## Abstract + +In order to make it easier for developers to write Cosmos SDK modules, we provide infrastructure which automatically +generates CLI commands based on protobuf definitions. + +## Context + +Current Cosmos SDK modules generally implement a CLI command for every transaction and every query supported by the +module. These are handwritten for each command and essentially amount to providing some CLI flags or positional +arguments for specific fields in protobuf messages. + +In order to make sure CLI commands are correctly implemented as well as to make sure that the application works +in end-to-end scenarios, we do integration tests using CLI commands. While these tests are valuable on some-level, +they can be hard to write and maintain, and run slowly. [Some teams have contemplated](https://github.com/regen-network/regen-ledger/issues/1041) +moving away from CLI-style integration tests (which are really end-to-end tests) towards narrower integration tests +which exercise `MsgClient` and `QueryClient` directly. This might involve replacing the current end-to-end CLI +tests with unit tests as there still needs to be some way to test these CLI commands for full quality assurance. + +## Decision + +To make module development simpler, we provide infrastructure - in the new [`client/v2`](https://github.com/cosmos/cosmos-sdk/tree/main/client/v2) +go module - for automatically generating CLI commands based on protobuf definitions to either replace or complement +handwritten CLI commands. This will mean that when developing a module, it will be possible to skip both writing and +testing CLI commands as that can all be taken care of by the framework. + +The basic design for automatically generating CLI commands is to: + +* create one CLI command for each `rpc` method in a protobuf `Query` or `Msg` service +* create a CLI flag for each field in the `rpc` request type +* for `query` commands call gRPC and print the response as protobuf JSON or YAML (via the `-o`/`--output` flag) +* for `tx` commands, create a transaction and apply common transaction flags + +In order to make the auto-generated CLI as easy to use (or easier) than handwritten CLI, we need to do custom handling +of specific protobuf field types so that the input format is easy for humans: + +* `Coin`, `Coins`, `DecCoin`, and `DecCoins` should be input using the existing format (i.e. `1000uatom`) +* it should be possible to specify an address using either the bech32 address string or a named key in the keyring +* `Timestamp` and `Duration` should accept strings like `2001-01-01T00:00:00Z` and `1h3m` respectively +* pagination should be handled with flags like `--page-limit`, `--page-offset`, etc. +* it should be possible to customize any other protobuf type either via its message name or a `cosmos_proto.scalar` annotation + +At a basic level it should be possible to generate a command for a single `rpc` method as well as all the commands for +a whole protobuf `service` definition. It should be possible to mix and match auto-generated and handwritten commands. + +## Consequences + +### Backwards Compatibility + +Existing modules can mix and match auto-generated and handwritten CLI commands so it is up to them as to whether they +make breaking changes by replacing handwritten commands with slightly different auto-generated ones. + +For now the SDK will maintain the existing set of CLI commands for backwards compatibility but new commands will use +this functionality. + +### Positive + +* module developers will not need to write CLI commands +* module developers will not need to test CLI commands +* [lens](https://github.com/strangelove-ventures/lens) may benefit from this + +### Negative + +### Neutral + +## Further Discussions + +We would like to be able to customize: + +* short and long usage strings for commands +* aliases for flags (ex. `-a` for `--amount`) +* which fields are positional parameters rather than flags + +It is an [open discussion](https://github.com/cosmos/cosmos-sdk/pull/11725#issuecomment-1108676129) +as to whether these customizations options should line in: + +* the .proto files themselves, +* separate config files (ex. YAML), or +* directly in code + +Providing the options in .proto files would allow a dynamic client to automatically generate +CLI commands on the fly. However, that may pollute the .proto files themselves with information that is only relevant +for a small subset of users. + +## References + +* https://github.com/regen-network/regen-ledger/issues/1041 +* https://github.com/cosmos/cosmos-sdk/tree/main/client/v2 +* https://github.com/cosmos/cosmos-sdk/pull/11725#issuecomment-1108676129 diff --git a/versioned_docs/version-0.52/build/architecture/adr-059-test-scopes.md b/versioned_docs/version-0.52/build/architecture/adr-059-test-scopes.md new file mode 100644 index 000000000..cb1263443 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-059-test-scopes.md @@ -0,0 +1,252 @@ +# ADR 059: Test Scopes + +## Changelog + +* 2022-08-02: Initial Draft +* 2023-03-02: Add precision for integration tests +* 2023-03-23: Add precision for E2E tests + +## Status + +PROPOSED Partially Implemented + +## Abstract + +Recent work in the SDK aimed at breaking apart the monolithic root go module has highlighted +shortcomings and inconsistencies in our testing paradigm. This ADR clarifies a common +language for talking about test scopes and proposes an ideal state of tests at each scope. + +## Context + +[ADR-053: Go Module Refactoring](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-053-go-module-refactoring.md) expresses our desire for an SDK composed of many +independently versioned Go modules, and [ADR-057: App Wiring](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-057-app-wiring.md) offers a methodology +for breaking apart inter-module dependencies through the use of dependency injection. As +described in [EPIC: Separate all SDK modules into standalone go modules](https://github.com/cosmos/cosmos-sdk/issues/11899), module +dependencies are particularly complected in the test phase, where simapp is used as +the key test fixture in setting up and running tests. It is clear that the successful +completion of Phases 3 and 4 in that EPIC require the resolution of this dependency problem. + +In [EPIC: Unit Testing of Modules via Mocks](https://github.com/cosmos/cosmos-sdk/issues/12398) it was thought this Gordian knot could be +unwound by mocking all dependencies in the test phase for each module, but seeing how these +refactors were complete rewrites of test suites discussions began around the fate of the +existing integration tests. One perspective is that they ought to be thrown out, another is +that integration tests have some utility of their own and a place in the SDK's testing story. + +Another point of confusion has been the current state of CLI test suites, [x/auth](https://github.com/cosmos/cosmos-sdk/blob/0f7e56c6f9102cda0ca9aba5b6f091dbca976b5a/x/auth/client/testutil/suite.go#L44-L49) for +example. In code these are called integration tests, but in reality function as end to end +tests by starting up a tendermint node and full application. [EPIC: Rewrite and simplify +CLI tests](https://github.com/cosmos/cosmos-sdk/issues/12696) identifies the ideal state of CLI tests using mocks, but does not address the +place end to end tests may have in the SDK. + +From here we identify three scopes of testing, **unit**, **integration**, **e2e** (end to +end), seek to define the boundaries of each, their shortcomings (real and imposed), and their +ideal state in the SDK. + +### Unit tests + +Unit tests exercise the code contained in a single module (e.g. `/x/bank`) or package +(e.g. `/client`) in isolation from the rest of the code base. Within this we identify two +levels of unit tests, *illustrative* and *journey*. The definitions below lean heavily on +[The BDD Books - Formulation](https://leanpub.com/bddbooks-formulation) section 1.3. + +*Illustrative* tests exercise an atomic part of a module in isolation - in this case we +might do fixture setup/mocking of other parts of the module. + +Tests which exercise a whole module's function with dependencies mocked, are *journeys*. +These are almost like integration tests in that they exercise many things together but still +use mocks. + +Example 1 journey vs illustrative tests - depinject's BDD style tests, show how we can +rapidly build up many illustrative cases demonstrating behavioral rules without [very much code](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/binding_test.go) while maintaining high level readability. + +Example 2 [depinject table driven tests](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/provider_desc_test.go) + +Example 3 [Bank keeper tests](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/bank/keeper/keeper_test.go#L94-L105) - A mock implementation of `AccountKeeper` is supplied to the keeper constructor. + +#### Limitations + +Certain modules are tightly coupled beyond the test phase. A recent dependency report for +`bank -> auth` found 274 total usages of `auth` in `bank`, 50 of which are in +production code and 224 in test. This tight coupling may suggest that either the modules +should be merged, or refactoring is required to abstract references to the core types tying +the modules together. It could also indicate that these modules should be tested together +in integration tests beyond mocked unit tests. + +In some cases setting up a test case for a module with many mocked dependencies can be quite +cumbersome and the resulting test may only show that the mocking framework works as expected +rather than working as a functional test of interdependent module behavior. + +### Integration tests + +Integration tests define and exercise relationships between an arbitrary number of modules +and/or application subsystems. + +Wiring for integration tests is provided by `depinject` and some [helper code](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/testutil/sims/app_helpers.go#L95) starts up +a running application. A section of the running application may then be tested. Certain +inputs during different phases of the application life cycle are expected to produce +invariant outputs without too much concern for component internals. This type of black box +testing has a larger scope than unit testing. + +Example 1 [client/grpc_query_test/TestGRPCQuery](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/client/grpc_query_test.go#L111-L129) - This test is misplaced in `/client`, +but tests the life cycle of (at least) `runtime` and `bank` as they progress through +startup, genesis and query time. It also exercises the fitness of the client and query +server without putting bytes on the wire through the use of [QueryServiceTestHelper](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/baseapp/grpcrouter_helpers.go#L31). + +Example 2 `x/evidence` Keeper integration tests - Starts up an application composed of [8 +modules](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/evidence/testutil/app.yaml#L1) with [5 keepers](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/evidence/keeper/keeper_test.go#L101-L106) used in the integration test suite. One test in the suite +exercises [HandleEquivocationEvidence](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/evidence/keeper/infraction_test.go#L42) which contains many interactions with the staking +keeper. + +Example 3 - Integration suite app configurations may also be specified via golang (not +YAML as above) [statically](https://github.com/cosmos/cosmos-sdk/blob/release/v0.47.x/x/nft/testutil/app_config.go) or [dynamically](https://github.com/cosmos/cosmos-sdk/blob/8c23f6f957d1c0bedd314806d1ac65bea59b084c/tests/integration/bank/keeper/keeper_test.go#L129-L134). + +#### Limitations + +Setting up a particular input state may be more challenging since the application is +starting from a zero state. Some of this may be addressed by good test fixture +abstractions with testing of their own. Tests may also be more brittle, and larger +refactors could impact application initialization in unexpected ways with harder to +understand errors. This could also be seen as a benefit, and indeed the SDK's current +integration tests were helpful in tracking down logic errors during earlier stages +of app-wiring refactors. + +### Simulations + +Simulations (also called generative testing) are a special case of integration tests where +deterministically random module operations are executed against a running simapp, building +blocks on the chain until a specified height is reached. No *specific* assertions are +made for the state transitions resulting from module operations but any error will halt and +fail the simulation. Since `crisis` is included in simapp and the simulation runs +EndBlockers at the end of each block any module invariant violations will also fail +the simulation. + +Modules must implement [AppModuleSimulation.WeightedOperations](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/types/module/simulation.go#L31) to define their +simulation operations. Note that not all modules implement this which may indicate a +gap in current simulation test coverage. + +Modules not returning simulation operations: + +* `auth` +* `evidence` +* `mint` +* `params` + + +#### Limitations + +* May take a long time to run, 7-10 minutes per simulation in CI. +* Timeouts sometimes occur on apparent successes without any indication why. +* Useful error messages not provided on from CI, requiring a developer to run + the simulation locally to reproduce. + +### E2E tests + +End to end tests exercise the entire system as we understand it in as close an approximation +to a production environment as is practical. Presently these tests are located at +[tests/e2e](https://github.com/cosmos/cosmos-sdk/tree/main/tests/e2e) and rely on [testutil/network](https://github.com/cosmos/cosmos-sdk/tree/main/testutil/network) to start up an in-process Tendermint node. + +An application should be built as minimally as possible to exercise the desired functionality. +The SDK uses an application will only the required modules for the tests. The application developer is advised to use its own application for e2e tests. + +#### Limitations + +In general the limitations of end to end tests are orchestration and compute cost. +Scaffolding is required to start up and run a prod-like environment and the this +process takes much longer to start and run than unit or integration tests. + +Global locks present in Tendermint code cause stateful starting/stopping to sometimes hang +or fail intermittently when run in a CI environment. + +The scope of e2e tests has been complected with command line interface testing. + +## Decision + +We accept these test scopes and identify the following decisions points for each. + +| Scope | App Type | Mocks? | +| ----------- | ------------------- | ------ | +| Unit | None | Yes | +| Integration | integration helpers | Some | +| Simulation | minimal app | No | +| E2E | minimal app | No | + +The decision above is valid for the SDK. An application developer should test their application with their full application instead of the minimal app. + +### Unit Tests + +All modules must have mocked unit test coverage. + +Illustrative tests should outnumber journeys in unit tests. + +Unit tests should outnumber integration tests. + +Unit tests must not introduce additional dependencies beyond those already present in +production code. + +When module unit test introduction as per [EPIC: Unit testing of modules via mocks](https://github.com/cosmos/cosmos-sdk/issues/12398) +results in a near complete rewrite of an integration test suite the test suite should be +retained and moved to `/tests/integration`. We accept the resulting test logic +duplication but recommend improving the unit test suite through the addition of +illustrative tests. + +### Integration Tests + +All integration tests shall be located in `/tests/integration`, even those which do not +introduce extra module dependencies. + +To help limit scope and complexity, it is recommended to use the smallest possible number of +modules in application startup, i.e. don't depend on simapp. + +Integration tests should outnumber e2e tests. + +### Simulations + +Simulations shall use a minimal application (usually via app wiring). They are located under `/x/{moduleName}/simulation`. + +### E2E Tests + +Existing e2e tests shall be migrated to integration tests by removing the dependency on the +test network and in-process Tendermint node to ensure we do not lose test coverage. + +The e2e rest runner shall transition from in process Tendermint to a runner powered by +Docker via [dockertest](https://github.com/ory/dockertest). + +E2E tests exercising a full network upgrade shall be written. + +The CLI testing aspect of existing e2e tests shall be rewritten using the network mocking +demonstrated in [PR#12706](https://github.com/cosmos/cosmos-sdk/pull/12706). + +## Consequences + +### Positive + +* test coverage is increased +* test organization is improved +* reduced dependency graph size in modules +* simapp removed as a dependency from modules +* inter-module dependencies introduced in test code are removed +* reduced CI run time after transitioning away from in process Tendermint + +### Negative + +* some test logic duplication between unit and integration tests during transition +* test written using dockertest DX may be a bit worse + +### Neutral + +* some discovery required for e2e transition to dockertest + +## Further Discussions + +It may be useful if test suites could be run in integration mode (with mocked tendermint) or +with e2e fixtures (with real tendermint and many nodes). Integration fixtures could be used +for quicker runs, e2e fixures could be used for more battle hardening. + +A PoC `x/gov` was completed in PR [#12847](https://github.com/cosmos/cosmos-sdk/pull/12847) +is in progress for unit tests demonstrating BDD [Rejected]. +Observing that a strength of BDD specifications is their readability, and a con is the +cognitive load while writing and maintaining, current consensus is to reserve BDD use +for places in the SDK where complex rules and module interactions are demonstrated. +More straightforward or low level test cases will continue to rely on go table tests. + +Levels are network mocking in integration and e2e tests are still being worked on and formalized. diff --git a/versioned_docs/version-0.52/build/architecture/adr-060-abci-1.0.md b/versioned_docs/version-0.52/build/architecture/adr-060-abci-1.0.md new file mode 100644 index 000000000..d2d93a89e --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-060-abci-1.0.md @@ -0,0 +1,238 @@ +# ADR 60: ABCI 1.0 Integration (Phase I) + +## Changelog + +* 2022-08-10: Initial Draft (@alexanderbez, @tac0turtle) +* Nov 12, 2022: Update `PrepareProposal` and `ProcessProposal` semantics per the + initial implementation [PR](https://github.com/cosmos/cosmos-sdk/pull/13453) (@alexanderbez) + +## Status + +ACCEPTED + +## Abstract + +This ADR describes the initial adoption of [ABCI 1.0](https://github.com/tendermint/tendermint/blob/master/spec/abci%2B%2B/README.md), +the next evolution of ABCI, within the Cosmos SDK. ABCI 1.0 aims to provide +application developers with more flexibility and control over application and +consensus semantics, e.g. in-application mempools, in-process oracles, and +order-book style matching engines. + +## Context + +Tendermint will release ABCI 1.0. Notably, at the time of this writing, +Tendermint is releasing v0.37.0 which will include `PrepareProposal` and `ProcessProposal`. + +The `PrepareProposal` ABCI method is concerned with a block proposer requesting +the application to evaluate a series of transactions to be included in the next +block, defined as a slice of `TxRecord` objects. The application can either +accept, reject, or completely ignore some or all of these transactions. This is +an important consideration to make as the application can essentially define and +control its own mempool allowing it to define sophisticated transaction priority +and filtering mechanisms, by completely ignoring the `TxRecords` Tendermint +sends it, favoring its own transactions. This essentially means that the Tendermint +mempool acts more like a gossip data structure. + +The second ABCI method, `ProcessProposal`, is used to process the block proposer's +proposal as defined by `PrepareProposal`. It is important to note the following +with respect to `ProcessProposal`: + +* Execution of `ProcessProposal` must be deterministic. +* There must be coherence between `PrepareProposal` and `ProcessProposal`. In + other words, for any two correct processes *p* and *q*, if *q*'s Tendermint + calls `RequestProcessProposal` on *up*, *q*'s Application returns + ACCEPT in `ResponseProcessProposal`. + +It is important to note that in ABCI 1.0 integration, the application +is NOT responsible for locking semantics -- Tendermint will still be responsible +for that. In the future, however, the application will be responsible for locking, +which allows for parallel execution possibilities. + +## Decision + +We will integrate ABCI 1.0, which will be introduced in Tendermint +v0.37.0, in the next major release of the Cosmos SDK. We will integrate ABCI 1.0 +methods on the `BaseApp` type. We describe the implementations of the two methods +individually below. + +Prior to describing the implementation of the two new methods, it is important to +note that the existing ABCI methods, `CheckTx`, `DeliverTx`, etc, still exist and +serve the same functions as they do now. + +### `PrepareProposal` + +Prior to evaluating the decision for how to implement `PrepareProposal`, it is +important to note that `CheckTx` will still be executed and will be responsible +for evaluating transaction validity as it does now, with one very important +*additive* distinction. + +When executing transactions in `CheckTx`, the application will now add valid +transactions, i.e. passing the AnteHandler, to its own mempool data structure. +In order to provide a flexible approach to meet the varying needs of application +developers, we will define both a mempool interface and a data structure utilizing +Golang generics, allowing developers to focus only on transaction +ordering. Developers requiring absolute full control can implement their own +custom mempool implementation. + +We define the general mempool interface as follows (subject to change): + +```go +type Mempool interface { + // Insert attempts to insert a Tx into the app-side mempool returning + // an error upon failure. + Insert(sdk.Context, sdk.Tx) error + + // Select returns an Iterator over the app-side mempool. If txs are specified, + // then they shall be incorporated into the Iterator. The Iterator must + // closed by the caller. + Select(sdk.Context, [][]byte) Iterator + + // CountTx returns the number of transactions currently in the mempool. + CountTx() int + + // Remove attempts to remove a transaction from the mempool, returning an error + // upon failure. + Remove(sdk.Tx) error +} + +// Iterator defines an app-side mempool iterator interface that is as minimal as +// possible. The order of iteration is determined by the app-side mempool +// implementation. +type Iterator interface { + // Next returns the next transaction from the mempool. If there are no more + // transactions, it returns nil. + Next() Iterator + + // Tx returns the transaction at the current position of the iterator. + Tx() sdk.Tx +} +``` + +We will define an implementation of `Mempool`, defined by `nonceMempool`, that +will cover most basic application use-cases. Namely, it will prioritize transactions +by transaction sender, allowing for multiple transactions from the same sender. + +The default app-side mempool implementation, `nonceMempool`, will operate on a +single skip list data structure. Specifically, transactions with the lowest nonce +globally are prioritized. Transactions with the same nonce are prioritized by +sender address. + +```go +type nonceMempool struct { + txQueue *huandu.SkipList +} +``` + +Previous discussions1 have come to the agreement that Tendermint will +perform a request to the application, via `RequestPrepareProposal`, with a certain +amount of transactions reaped from Tendermint's local mempool. The exact amount +of transactions reaped will be determined by a local operator configuration. +This is referred to as the "one-shot approach" seen in discussions. + +When Tendermint reaps transactions from the local mempool and sends them to the +application via `RequestPrepareProposal`, the application will have to evaluate +the transactions. Specifically, it will need to inform Tendermint if it should +reject and or include each transaction. Note, the application can even *replace* +transactions entirely with other transactions. + +When evaluating transactions from `RequestPrepareProposal`, the application will +ignore *ALL* transactions sent to it in the request and instead reap up to +`RequestPrepareProposal.max_tx_bytes` from it's own mempool. + +Since an application can technically insert or inject transactions on `Insert` +during `CheckTx` execution, it is recommended that applications ensure transaction +validity when reaping transactions during `PrepareProposal`. However, what validity +exactly means is entirely determined by the application. + +The Cosmos SDK will provide a default `PrepareProposal` implementation that simply +select up to `MaxBytes` *valid* transactions. + +However, applications can override this default implementation with their own +implementation and set that on `BaseApp` via `SetPrepareProposal`. + + +### `ProcessProposal` + +The `ProcessProposal` ABCI method is relatively straightforward. It is responsible +for ensuring validity of the proposed block containing transactions that were +selected from the `PrepareProposal` step. However, how an application determines +validity of a proposed block depends on the application and its varying use cases. +For most applications, simply calling the `AnteHandler` chain would suffice, but +there could easily be other applications that need more control over the validation +process of the proposed block, such as ensuring txs are in a certain order or +that certain transactions are included. While this theoretically could be achieved +with a custom `AnteHandler` implementation, it's not the cleanest UX or the most +efficient solution. + +Instead, we will define an additional ABCI interface method on the existing +`Application` interface, similar to the existing ABCI methods such as `BeginBlock` +or `EndBlock`. This new interface method will be defined as follows: + +```go +ProcessProposal(sdk.Context, abci.ProcessProposalRequest) error {} +``` + +Note, we must call `ProcessProposal` with a new internal branched state on the +`Context` argument as we cannot simply just use the existing `checkState` because +`BaseApp` already has a modified `checkState` at this point. So when executing +`ProcessProposal`, we create a similar branched state, `processProposalState`, +off of `deliverState`. Note, the `processProposalState` is never committed and +is completely discarded after `ProcessProposal` finishes execution. + +The Cosmos SDK will provide a default implementation of `ProcessProposal` in which +all transactions are validated using the CheckTx flow, i.e. the AnteHandler, and +will always return ACCEPT unless any transaction cannot be decoded. + +### `DeliverTx` + +Since transactions are not truly removed from the app-side mempool during +`PrepareProposal`, since `ProcessProposal` can fail or take multiple rounds and +we do not want to lose transactions, we need to finally remove the transaction +from the app-side mempool during `DeliverTx` since during this phase, the +transactions are being included in the proposed block. + +Alternatively, we can keep the transactions as truly being removed during the +reaping phase in `PrepareProposal` and add them back to the app-side mempool in +case `ProcessProposal` fails. + +## Consequences + +### Backwards Compatibility + +ABCI 1.0 is naturally not backwards compatible with prior versions of the Cosmos SDK +and Tendermint. For example, an application that requests `RequestPrepareProposal` +to the same application that does not speak ABCI 1.0 will naturally fail. + +However, in the first phase of the integration, the existing ABCI methods as we +know them today will still exist and function as they currently do. + +### Positive + +* Applications now have full control over transaction ordering and priority. +* Lays the groundwork for the full integration of ABCI 1.0, which will unlock more + app-side use cases around block construction and integration with the Tendermint + consensus engine. + +### Negative + +* Requires that the "mempool", as a general data structure that collects and stores + uncommitted transactions will be duplicated between both Tendermint and the + Cosmos SDK. +* Additional requests between Tendermint and the Cosmos SDK in the context of + block execution. Albeit, the overhead should be negligible. +* Not backwards compatible with previous versions of Tendermint and the Cosmos SDK. + +## Further Discussions + +It is possible to design the app-side implementation of the `Mempool[T MempoolTx]` +in many different ways using different data structures and implementations. All +of which have different tradeoffs. The proposed solution keeps things simple +and covers cases that would be required for most basic applications. There are +tradeoffs that can be made to improve performance of reaping and inserting into +the provided mempool implementation. + +## References + +* https://github.com/tendermint/tendermint/blob/master/spec/abci%2B%2B/README.md +* [1] https://github.com/tendermint/tendermint/issues/7750#issuecomment-1076806155 +* [2] https://github.com/tendermint/tendermint/issues/7750#issuecomment-1075717151 diff --git a/versioned_docs/version-0.52/build/architecture/adr-061-liquid-staking.md b/versioned_docs/version-0.52/build/architecture/adr-061-liquid-staking.md new file mode 100644 index 000000000..81ce2f485 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-061-liquid-staking.md @@ -0,0 +1,82 @@ +# ADR ADR-061: Liquid Staking + +## Changelog + +* 2022-09-10: Initial Draft (@zmanian) + +## Status + +ACCEPTED + +## Abstract + +Add a semi-fungible liquid staking primitive to the default Cosmos SDK staking module. This upgrades proof of stake to enable safe designs with lower overall monetary issuance and integration with numerous liquid staking protocols like Stride, Persistence, Quicksilver, Lido etc. + +## Context + +The original release of the Cosmos Hub featured the implementation of a ground breaking proof of stake mechanism featuring delegation, slashing, in protocol reward distribution and adaptive issuance. This design was state of the art for 2016 and has been deployed without major changes by many L1 blockchains. + +As both Proof of Stake and blockchain use cases have matured, this design has aged poorly and should no longer be considered a good baseline Proof of Stake issuance. In the world of application specific blockchains, there cannot be a one size fits all blockchain but the Cosmos SDK does endeavour to provide a good baseline implementation and one that is suitable for the Cosmos Hub. + +The most important deficiency of the legacy staking design is that it composes poorly with on chain protocols for trading, lending, derivatives that are referred to collectively as DeFi. The legacy staking implementation starves these applications of liquidity by increasing the risk free rate adaptively. It basically makes DeFi and staking security somewhat incompatible. + +The Osmosis team has adopted the idea of Superfluid and Interfluid staking where assets that are participating in DeFi applications can also be used in proof of stake. This requires tight integration with an enshrined set of DeFi applications and thus is unsuitable for the Cosmos SDK. + +It's also important to note that Interchain Accounts are available in the default IBC implementation and can be used to [rehypothecate](https://www.investopedia.com/terms/h/hypothecation.asp#toc-what-is-rehypothecation) delegations. Thus liquid staking is already possible and these changes merely improve the UX of liquid staking. Centralized exchanges also rehypothecate staked assets, posing challenges for decentralization. This ADR takes the position that adoption of in-protocol liquid staking is the preferable outcome and provides new levers to incentivize decentralization of stake. + +These changes to the staking module have been in development for more than a year and have seen substantial industry adoption who plan to build staking UX. The internal economics at Informal team has also done a review of the impacts of these changes and this review led to the development of the exempt delegation system. This system provides governance with a tuneable parameter for modulating the risks of principal agent problem called the exemption factor. + +## Decision + +We implement the semi-fungible liquid staking system and exemption factor system within the cosmos sdk. Though registered as fungible assets, these tokenized shares have extremely limited fungibility, only among the specific delegation record that was created when shares were tokenized. These assets can be used for OTC trades but composability with DeFi is limited. The primary expected use case is improving the user experience of liquid staking providers. + +A new governance parameter is introduced that defines the ratio of exempt to issued tokenized shares. This is called the exemption factor. A larger exemption factor allows more tokenized shares to be issued for a smaller amount of exempt delegations. If governance is comfortable with how the liquid staking market is evolving, it makes sense to increase this value. + +Min self delegation is removed from the staking system with the expectation that it will be replaced by the exempt delegations system. The exempt delegation system allows multiple accounts to demonstrate economic alignment with the validator operator as team members, partners etc. without co-mingling funds. Delegation exemption will likely be required to grow the validators' business under widespread adoption of liquid staking once governance has adjusted the exemption factor. + +When shares are tokenized, the underlying shares are transferred to a module account and rewards go to the module account for the TokenizedShareRecord. + +There is no longer a mechanism to override the validators vote for TokenizedShares. + + +### `MsgTokenizeShares` + +The MsgTokenizeShares message is used to create tokenize delegated tokens. This message can be executed by any delegator who has positive amount of delegation and after execution the specific amount of delegation disappear from the account and share tokens are provided. Share tokens are denominated in the validator and record id of the underlying delegation. + +A user may tokenize some or all of their delegation. + +They will receive shares with the denom of `cosmosvaloper1xxxx/5` where 5 is the record id for the validator operator. + +MsgTokenizeShares fails if the account is a VestingAccount. Users will have to move vested tokens to a new account and endure the unbonding period. We view this as an acceptable tradeoff vs. the complex book keeping required to track vested tokens. + +The total amount of outstanding tokenized shares for the validator is checked against the sum of exempt delegations multiplied by the exemption factor. If the tokenized shares exceeds this limit, execution fails. + +MsgTokenizeSharesResponse provides the number of tokens generated and their denom. + + +### `MsgRedeemTokensforShares` + +The MsgRedeemTokensforShares message is used to redeem the delegation from share tokens. This message can be executed by any user who owns share tokens. After execution delegations will appear to the user. + +### `MsgTransferTokenizeShareRecord` + +The MsgTransferTokenizeShareRecord message is used to transfer the ownership of rewards generated from the tokenized amount of delegation. The tokenize share record is created when a user tokenize his/her delegation and deleted when the full amount of share tokens are redeemed. + +This is designed to work with liquid staking designs that do not redeem the tokenized shares and may instead want to keep the shares tokenized. + + +### `MsgExemptDelegation` + +The MsgExemptDelegation message is used to exempt a delegation to a validator. If the exemption factor is greater than 0, this will allow more delegation shares to be issued from the validator. + +This design allows the chain to force an amount of self-delegation by validators participating in liquid staking schemes. + +## Consequences + +### Backwards Compatibility + +By setting the exemption factor to zero, this module works like legacy staking. The only substantial change is the removal of min-self-bond and without any tokenized shares, there is no incentive to exempt delegation. + +### Positive + +This approach should enable integration with liquid staking providers and improved user experience. It provides a pathway to security under non-exponential issuance policies in the baseline staking module. diff --git a/versioned_docs/version-0.52/build/architecture/adr-062-collections-state-layer.md b/versioned_docs/version-0.52/build/architecture/adr-062-collections-state-layer.md new file mode 100644 index 000000000..8d3a7b61c --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-062-collections-state-layer.md @@ -0,0 +1,117 @@ +# ADR 062: Collections, a simplified storage layer for cosmos-sdk modules. + +## Changelog + +* 30/11/2022: PROPOSED + +## Status + +PROPOSED - Implemented + +## Abstract + +We propose a simplified module storage layer which leverages golang generics to allow module developers to handle module +storage in a simple and straightforward manner, whilst offering safety, extensibility and standardisation. + +## Context + +Module developers are forced into manually implementing storage functionalities in their modules, those functionalities include +but are not limited to: + +- Defining key to bytes formats. +- Defining value to bytes formats. +- Defining secondary indexes. +- Defining query methods to expose outside to deal with storage. +- Defining local methods to deal with storage writing. +- Dealing with genesis imports and exports. +- Writing tests for all the above. + + +This brings in a lot of problems: +- It blocks developers from focusing on the most important part: writing business logic. +- Key to bytes formats are complex and their definition is error-prone, for example: + - how do I format time to bytes in such a way that bytes are sorted? + - how do I ensure when I don't have namespace collisions when dealing with secondary indexes? +- The lack of standardisation makes life hard for clients, and the problem is exacerbated when it comes to providing proofs for objects present in state. Clients are forced to maintain a list of object paths to gather proofs. + +### Current Solution: ORM + +The current SDK proposed solution to this problem is [ORM](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-055-orm.md). +Whilst ORM offers a lot of good functionality aimed at solving these specific problems, it has some downsides: +- It requires migrations. +- It uses the newest protobuf golang API, whilst the SDK still mainly uses gogoproto. +- Integrating ORM into a module would require the developer to deal with two different golang frameworks (golang protobuf + gogoproto) representing the same API objects. +- It has a high learning curve, even for simple storage layers as it requires developers to have knowledge around protobuf options, custom cosmos-sdk storage extensions, and tooling download. Then after this they still need to learn the code-generated API. + +### CosmWasm Solution: cw-storage-plus + +The collections API takes inspiration from [cw-storage-plus](https://docs.cosmwasm.com/docs/smart-contracts/state/cw-plus), +which has demonstrated to be a powerful tool for dealing with storage in CosmWasm contracts. +It's simple, does not require extra tooling, it makes it easy to deal with complex storage structures (indexes, snapshot, etc). +The API is straightforward and explicit. + +## Decision + +We propose to port the `collections` API, whose implementation lives in [NibiruChain/collections](https://github.com/NibiruChain/collections) to cosmos-sdk. + +Collections implements four different storage handlers types: + +- `Map`: which deals with simple `key=>object` mappings. +- `KeySet`: which acts as a `Set` and only retains keys and no object (usecase: allow-lists). +- `Item`: which always contains only one object (usecase: Params) +- `Sequence`: which implements a simple always increasing number (usecase: Nonces) +- `IndexedMap`: builds on top of `Map` and `KeySet` and allows to create relationships with `Objects` and `Objects` secondary keys. + +All the collection APIs build on top of the simple `Map` type. + +Collections is fully generic, meaning that anything can be used as `Key` and `Value`. It can be a protobuf object or not. + +Collections types, in fact, delegate the duty of serialisation of keys and values to a secondary collections API component called `ValueEncoders` and `KeyEncoders`. + +`ValueEncoders` take care of converting a value to bytes (relevant only for `Map`). And offers a plug and play layer which allows us to change how we encode objects, +which is relevant for swapping serialisation frameworks and enhancing performance. +`Collections` already comes in with default `ValueEncoders`, specifically for: protobuf objects, special SDK types (sdk.Int, sdk.Dec). + +`KeyEncoders` take care of converting keys to bytes, `collections` already comes in with some default `KeyEncoders` for some privimite golang types +(uint64, string, time.Time, ...) and some widely used sdk types (sdk.Acc/Val/ConsAddress, sdk.Int/Dec, ...). +These default implementations also offer safety around proper lexicographic ordering and namespace-collision. + +Examples of the collections API can be found here: +- introduction: https://github.com/NibiruChain/collections/tree/main/examples +- usage in nibiru: [x/oracle](https://github.com/NibiruChain/nibiru/blob/master/x/oracle/keeper/keeper.go#L32), [x/epoch](https://github.com/NibiruChain/nibiru/blob/4566d9f6d22807abbd78c01454664d64f6e108e0/x/epochs/keeper/epoch.go) +- cosmos-sdk's x/staking migrated: https://github.com/testinginprod/cosmos-sdk/pull/22 + + +## Consequences + +### Backwards Compatibility + +The design of `ValueEncoders` and `KeyEncoders` allows modules to retain the same `byte(key)=>byte(value)` mappings, making +the upgrade to the new storage layer non-state breaking. + + +### Positive + +- ADR aimed at removing code from the SDK rather than adding it. Migrating just `x/staking` to collections would yield to a net decrease in LOC (even considering the addition of collections itself). +- Simplifies and standardises storage layers across modules in the SDK. +- Does not require to have to deal with protobuf. +- It's pure golang code. +- Generalisation over `KeyEncoders` and `ValueEncoders` allows us to not tie ourself to the data serialisation framework. +- `KeyEncoders` and `ValueEncoders` can be extended to provide schema reflection. + +### Negative + +- Golang generics are not as battle-tested as other Golang features, despite being used in production right now. +- Collection types instantiation needs to be improved. + +### Neutral + +{neutral consequences} + +## Further Discussions + +- Automatic genesis import/export (not implemented because of API breakage) +- Schema reflection + + +## References diff --git a/versioned_docs/version-0.52/build/architecture/adr-063-core-module-api.md b/versioned_docs/version-0.52/build/architecture/adr-063-core-module-api.md new file mode 100644 index 000000000..74212b298 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-063-core-module-api.md @@ -0,0 +1,564 @@ +# ADR 063: Core Module API + +## Changelog + +* 2022-08-18 First Draft +* 2022-12-08 First Draft +* 2023-01-24 Updates + +## Status + +ACCEPTED Partially Implemented + +## Abstract + +A new core API is proposed as a way to develop cosmos-sdk applications that will eventually replace the existing +`AppModule` and `sdk.Context` frameworks a set of core services and extension interfaces. This core API aims to: + +* be simpler +* more extensible +* more stable than the current framework +* enable deterministic events and queries, +* support event listeners +* [ADR 033: Protobuf-based Inter-Module Communication](./adr-033-protobuf-inter-module-comm.md) clients. + +## Context + +Historically modules have exposed their functionality to the framework via the `AppModule` and `AppModuleBasic` +interfaces which have the following shortcomings: + +* both `AppModule` and `AppModuleBasic` need to be defined and registered which is counter-intuitive +* apps need to implement the full interfaces, even parts they don't need (although there are workarounds for this), +* interface methods depend heavily on unstable third party dependencies, in particular Comet, +* legacy required methods have littered these interfaces for far too long + +In order to interact with the state machine, modules have needed to do a combination of these things: + +* get store keys from the app +* call methods on `sdk.Context` which contains more or less the full set of capability available to modules. + +By isolating all the state machine functionality into `sdk.Context`, the set of functionalities available to +modules are tightly coupled to this type. If there are changes to upstream dependencies (such as Comet) +or new functionalities are desired (such as alternate store types), the changes need impact `sdk.Context` and all +consumers of it (basically all modules). Also, all modules now receive `context.Context` and need to convert these +to `sdk.Context`'s with a non-ergonomic unwrapping function. + +Any breaking changes to these interfaces, such as ones imposed by third-party dependencies like Comet, have the +side effect of forcing all modules in the ecosystem to update in lock-step. This means it is almost impossible to have +a version of the module which can be run with 2 or 3 different versions of the SDK or 2 or 3 different versions of +another module. This lock-step coupling slows down overall development within the ecosystem and causes updates to +components to be delayed longer than they would if things were more stable and loosely coupled. + +## Decision + +The `core` API proposes a set of core APIs that modules can rely on to interact with the state machine and expose their +functionalities to it that are designed in a principled way such that: + +* tight coupling of dependencies and unrelated functionalities is minimized or eliminated +* APIs can have long-term stability guarantees +* the SDK framework is extensible in a safe and straightforward way + +The design principles of the core API are as follows: + +* everything that a module wants to interact with in the state machine is a service +* all services coordinate state via `context.Context` and don't try to recreate the "bag of variables" approach of `sdk.Context` +* all independent services are isolated in independent packages with minimal APIs and minimal dependencies +* the core API should be minimalistic and designed for long-term support (LTS) +* a "runtime" module will implement all the "core services" defined by the core API and can handle all module + functionalities exposed by core extension interfaces +* other non-core and/or non-LTS services can be exposed by specific versions of runtime modules or other modules +following the same design principles, this includes functionality that interacts with specific non-stable versions of +third party dependencies such as Comet +* the core API doesn't implement *any* functionality, it just defines types +* go stable API compatibility guidelines are followed: https://go.dev/blog/module-compatibility + +A "runtime" module is any module which implements the core functionality of composing an ABCI app, which is currently +handled by `BaseApp` and the `ModuleManager`. Runtime modules which implement the core API are *intentionally* separate +from the core API in order to enable more parallel versions and forks of the runtime module than is possible with the +SDK's current tightly coupled `BaseApp` design while still allowing for a high degree of composability and +compatibility. + +Modules which are built only against the core API don't need to know anything about which version of runtime, +`BaseApp` or Comet in order to be compatible. Modules from the core mainline SDK could be easily composed +with a forked version of runtime with this pattern. + +This design is intended to enable matrices of compatible dependency versions. Ideally a given version of any module +is compatible with multiple versions of the runtime module and other compatible modules. This will allow dependencies +to be selectively updated based on battle-testing. More conservative projects may want to update some dependencies +slower than more fast moving projects. + +### Core Services + +The following "core services" are defined by the core API. All valid runtime module implementations should provide +implementations of these services to modules via both [dependency injection](./adr-057-app-wiring.md) and +manual wiring. The individual services described below are all bundled in a convenient `appmodule.Service` +"bundle service" so that for simplicity modules can declare a dependency on a single service. + +#### Store Services + +Store services will be defined in the `cosmossdk.io/core/store` package. + +The generic `store.KVStore` interface is the same as current SDK `KVStore` interface. Store keys have been refactored +into store services which, instead of expecting the context to know about stores, invert the pattern and allow +retrieving a store from a generic context. There are three store services for the three types of currently supported +stores - regular kv-store, memory, and transient: + +```go +type KVStoreService interface { + OpenKVStore(context.Context) KVStore +} + +type MemoryStoreService interface { + OpenMemoryStore(context.Context) KVStore +} +type TransientStoreService interface { + OpenTransientStore(context.Context) KVStore +} +``` + +Modules can use these services like this: + +```go +func (k msgServer) Send(ctx context.Context, msg *types.MsgSend) (*types.MsgSendResponse, error) { + store := k.kvStoreSvc.OpenKVStore(ctx) +} +``` + +Just as with the current runtime module implementation, modules will not need to explicitly name these store keys, +but rather the runtime module will choose an appropriate name for them and modules just need to request the +type of store they need in their dependency injection (or manual) constructors. + +#### Event Service + +The event `Service` will be defined in the `cosmossdk.io/core/event` package. + +The event `Service` allows modules to emit typed and legacy untyped events: + +```go +package event + +type Service interface { + // EmitProtoEvent emits events represented as a protobuf message (as described in ADR 032). + // + // Callers SHOULD assume that these events may be included in consensus. These events + // MUST be emitted deterministically and adding, removing or changing these events SHOULD + // be considered state-machine breaking. + EmitProtoEvent(ctx context.Context, event protoiface.MessageV1) error + + // EmitKVEvent emits an event based on an event and kv-pair attributes. + // + // These events will not be part of consensus and adding, removing or changing these events is + // not a state-machine breaking change. + EmitKVEvent(ctx context.Context, eventType string, attrs ...KVEventAttribute) error + + // EmitProtoEventNonConsensus emits events represented as a protobuf message (as described in ADR 032), without + // including it in blockchain consensus. + // + // These events will not be part of consensus and adding, removing or changing events is + // not a state-machine breaking change. + EmitProtoEventNonConsensus(ctx context.Context, event protoiface.MessageV1) error +} +``` + +Typed events emitted with `EmitProto` should be assumed to be part of blockchain consensus (whether they are part of +the block or app hash is left to the runtime to specify). + +Events emitted by `EmitKVEvent` and `EmitProtoEventNonConsensus` are not considered to be part of consensus and cannot be observed +by other modules. If there is a client-side need to add events in patch releases, these methods can be used. + +#### Logger + +A logger (`cosmossdk.io/log`) must be supplied using `depinject`, and will +be made available for modules to use via `depinject.In`. +Modules using it should follow the current pattern in the SDK by adding the module name before using it. + +```go +type ModuleInputs struct { + depinject.In + + Logger log.Logger +} + +func ProvideModule(in ModuleInputs) ModuleOutputs { + keeper := keeper.NewKeeper( + in.logger, + ) +} + +func NewKeeper(logger log.Logger) Keeper { + return Keeper{ + logger: logger.With(log.ModuleKey, "x/"+types.ModuleName), + } +} +``` + +``` + +### Core `AppModule` extension interfaces + + +Modules will provide their core services to the runtime module via extension interfaces built on top of the +`cosmossdk.io/core/appmodule.AppModule` tag interface. This tag interface requires only two empty methods which +allow `depinject` to identify implementers as `depinject.OnePerModule` types and as app module implementations: + +```go +type AppModule interface { + depinject.OnePerModuleType + + // IsAppModule is a dummy method to tag a struct as implementing an AppModule. + IsAppModule() +} +``` + +Other core extension interfaces will be defined in `cosmossdk.io/core` should be supported by valid runtime +implementations. + +#### `MsgServer` and `QueryServer` registration + +`MsgServer` and `QueryServer` registration is done by implementing the `HasServices` extension interface: + +```go +type HasServices interface { + AppModule + + RegisterServices(grpc.ServiceRegistrar) +} + +``` + +Because of the `cosmos.msg.v1.service` protobuf option, required for `Msg` services, the same `ServiceRegitrar` can be +used to register both `Msg` and query services. + +#### Genesis + +The genesis `Handler` functions - `DefaultGenesis`, `ValidateGenesis`, `InitGenesis` and `ExportGenesis` - are specified +against the `GenesisSource` and `GenesisTarget` interfaces which will abstract over genesis sources which may be a single +JSON object or collections of JSON objects that can be efficiently streamed. + +```go +// GenesisSource is a source for genesis data in JSON format. It may abstract over a +// single JSON object or separate files for each field in a JSON object that can +// be streamed over. Modules should open a separate io.ReadCloser for each field that +// is required. When fields represent arrays they can efficiently be streamed +// over. If there is no data for a field, this function should return nil, nil. It is +// important that the caller closes the reader when done with it. +type GenesisSource = func(field string) (io.ReadCloser, error) + +// GenesisTarget is a target for writing genesis data in JSON format. It may +// abstract over a single JSON object or JSON in separate files that can be +// streamed over. Modules should open a separate io.WriteCloser for each field +// and should prefer writing fields as arrays when possible to support efficient +// iteration. It is important the caller closers the writer AND checks the error +// when done with it. It is expected that a stream of JSON data is written +// to the writer. +type GenesisTarget = func(field string) (io.WriteCloser, error) +``` + +All genesis objects for a given module are expected to conform to the semantics of a JSON object. +Each field in the JSON object should be read and written separately to support streaming genesis. +The [ORM](./adr-055-orm.md) and [collections](./adr-062-collections-state-layer.md) both support +streaming genesis and modules using these frameworks generally do not need to write any manual +genesis code. + +To support genesis, modules should implement the `HasGenesis` extension interface: + +```go +type HasGenesis interface { + AppModule + + // DefaultGenesis writes the default genesis for this module to the target. + DefaultGenesis(GenesisTarget) error + + // ValidateGenesis validates the genesis data read from the source. + ValidateGenesis(GenesisSource) error + + // InitGenesis initializes module state from the genesis source. + InitGenesis(context.Context, GenesisSource) error + + // ExportGenesis exports module state to the genesis target. + ExportGenesis(context.Context, GenesisTarget) error +} +``` + +#### Pre Blockers + +Modules that have functionality that runs before BeginBlock and should implement the `HasPreBlocker` interfaces: + +```go +type HasPreBlocker interface { + AppModule + PreBlock(context.Context) error +} +``` + +#### Begin and End Blockers + +Modules that have functionality that runs before transactions (begin blockers) or after transactions +(end blockers) should implement the `HasBeginBlocker` and/or `HasEndBlocker` interfaces: + +```go +type HasBeginBlocker interface { + AppModule + BeginBlock(context.Context) error +} + +type HasEndBlocker interface { + AppModule + EndBlock(context.Context) error +} +``` + +The `BeginBlock` and `EndBlock` methods will take a `context.Context`, because: + +* most modules don't need Comet information other than `BlockInfo` so we can eliminate dependencies on specific +Comet versions +* for the few modules that need Comet block headers and/or return validator updates, specific versions of the +runtime module will provide specific functionality for interacting with the specific version(s) of Comet +supported + +In order for `BeginBlock`, `EndBlock` and `InitGenesis` to send back validator updates and retrieve full Comet +block headers, the runtime module for a specific version of Comet could provide services like this: + +```go +type ValidatorUpdateService interface { + SetValidatorUpdates(context.Context, []abci.ValidatorUpdate) +} +``` + +Header Service defines a way to get header information about a block. This information is generalized for all implementations: + +```go + +type Service interface { + HeaderInfo(context.Context) Info +} + +type Info struct { + Height int64 // Height returns the height of the block + Hash []byte // Hash returns the hash of the block header + Time time.Time // Time returns the time of the block + ChainID string // ChainId returns the chain ID of the block +} +``` + +Comet Service provides a way to get comet specific information: + +```go +type Service interface { + GetCometInfo(context.Context) Info +} + +type CometInfo struct { + Evidence []abci.Misbehavior // Misbehavior returns the misbehavior of the block + // ValidatorsHash returns the hash of the validators + // For Comet, it is the hash of the next validators + ValidatorsHash []byte + ProposerAddress []byte // ProposerAddress returns the address of the block proposer + DecidedLastCommit abci.CommitInfo // DecidedLastCommit returns the last commit info +} +``` + +If a user would like to provide a module other information they would need to implement another service like: + +```go +type RollKit Interface { + ... +} +``` + +We know these types will change at the Comet level and that also a very limited set of modules actually need this +functionality, so they are intentionally kept out of core to keep core limited to the necessary, minimal set of stable +APIs. + +#### Remaining Parts of AppModule + +The current `AppModule` framework handles a number of additional concerns which aren't addressed by this core API. +These include: + +* gas +* block headers +* upgrades +* registration of gogo proto and amino interface types +* cobra query and tx commands +* gRPC gateway +* crisis module invariants +* simulations + +Additional `AppModule` extension interfaces either inside or outside of core will need to be specified to handle +these concerns. + +In the case of gogo proto and amino interfaces, the registration of these generally should happen as early +as possible during initialization and in [ADR 057: App Wiring](./adr-057-app-wiring.md), protobuf type registration +happens before dependency injection (although this could alternatively be done dedicated DI providers). + +gRPC gateway registration should probably be handled by the runtime module, but the core API shouldn't depend on gRPC +gateway types as 1) we are already using an older version and 2) it's possible the framework can do this registration +automatically in the future. So for now, the runtime module should probably provide some sort of specific type for doing +this registration ex: + +```go +type GrpcGatewayInfo struct { + Handlers []GrpcGatewayHandler +} + +type GrpcGatewayHandler func(ctx context.Context, mux *runtime.ServeMux, client QueryClient) error +``` + +which modules can return in a provider: + +```go +func ProvideGrpcGateway() GrpcGatewayInfo { + return GrpcGatewayinfo { + Handlers: []Handler {types.RegisterQueryHandlerClient} + } +} +``` + +Crisis module invariants and simulations are subject to potential redesign and should be managed with types +defined in the crisis and simulation modules respectively. + +Extension interface for CLI commands will be provided via the `cosmossdk.io/client/v2` module and its +[autocli](./adr-058-auto-generated-cli.md) framework. + +#### Example Usage + +Here is an example of setting up a hypothetical `foo` v2 module which uses the [ORM](./adr-055-orm.md) for its state +management and genesis. + +```go + +type Keeper struct { + db orm.ModuleDB + evtSrv event.Service +} + +func (k Keeper) RegisterServices(r grpc.ServiceRegistrar) { + foov1.RegisterMsgServer(r, k) + foov1.RegisterQueryServer(r, k) +} + +func (k Keeper) BeginBlock(context.Context) error { + return nil +} + +func ProvideApp(config *foomodulev2.Module, evtSvc event.EventService, db orm.ModuleDB) (Keeper, appmodule.AppModule){ + k := &Keeper{db: db, evtSvc: evtSvc} + return k, k +} +``` + +### Runtime Compatibility Version + +The `core` module will define a static integer var, `cosmossdk.io/core.RuntimeCompatibilityVersion`, which is +a minor version indicator of the core module that is accessible at runtime. Correct runtime module implementations +should check this compatibility version and return an error if the current `RuntimeCompatibilityVersion` is higher +than the version of the core API that this runtime version can support. When new features are added to the `core` +module API that runtime modules are required to support, this version should be incremented. + +### Runtime Modules + +The initial `runtime` module will simply be created within the existing `github.com/cosmos/cosmos-sdk` go module +under the `runtime` package. This module will be a small wrapper around the existing `BaseApp`, `sdk.Context` and +module manager and follow the Cosmos SDK's existing [0-based versioning](https://0ver.org). To move to semantic +versioning as well as runtime modularity, new officially supported runtime modules will be created under the +`cosmossdk.io/runtime` prefix. For each supported consensus engine a semantically-versioned go module should be created +with a runtime implementation for that consensus engine. For example: + +* `cosmossdk.io/runtime/comet` +* `cosmossdk.io/runtime/comet/v2` +* `cosmossdk.io/runtime/rollkit` +* etc. + +These runtime modules should attempt to be semantically versioned even if the underlying consensus engine is not. Also, +because a runtime module is also a first class Cosmos SDK module, it should have a protobuf module config type. +A new semantically versioned module config type should be created for each of these runtime module such that there is a +1:1 correspondence between the go module and module config type. This is the same practice should be followed for every +semantically versioned Cosmos SDK module as described in [ADR 057: App Wiring](./adr-057-app-wiring.md). + +Currently, `github.com/cosmos/cosmos-sdk/runtime` uses the protobuf config type `cosmos.app.runtime.v1alpha1.Module`. +When we have a standalone v1 comet runtime, we should use a dedicated protobuf module config type such as +`cosmos.runtime.comet.v1.Module1`. When we release v2 of the comet runtime (`cosmossdk.io/runtime/comet/v2`) we should +have a corresponding `cosmos.runtime.comet.v2.Module` protobuf type. + +In order to make it easier to support different consensus engines that support the same core module functionality as +described in this ADR, a common go module should be created with shared runtime components. The easiest runtime components +to share initially are probably the message/query router, inter-module client, service register, and event router. +This common runtime module should be created initially as the `cosmossdk.io/runtime/common` go module. + +When this new architecture has been implemented, the main dependency for a Cosmos SDK module would be +`cosmossdk.io/core` and that module should be able to be used with any supported consensus engine (to the extent +that it does not explicitly depend on consensus engine specific functionality such as Comet's block headers). An +app developer would then be able to choose which consensus engine they want to use by importing the corresponding +runtime module. The current `BaseApp` would be refactored into the `cosmossdk.io/runtime/comet` module, the router +infrastructure in `baseapp/` would be refactored into `cosmossdk.io/runtime/common` and support ADR 033, and eventually +a dependency on `github.com/cosmos/cosmos-sdk` would no longer be required. + +In short, modules would depend primarily on `cosmossdk.io/core`, and each `cosmossdk.io/runtime/{consensus-engine}` +would implement the `cosmossdk.io/core` functionality for that consensus engine. + +On additional piece that would need to be resolved as part of this architecture is how runtimes relate to the server. +Likely it would make sense to modularize the current server architecture so that it can be used with any runtime even +if that is based on a consensus engine besides Comet. This means that eventually the Comet runtime would need to +encapsulate the logic for starting Comet and the ABCI app. + +### Testing + +A mock implementation of all services should be provided in core to allow for unit testing of modules +without needing to depend on any particular version of runtime. Mock services should +allow tests to observe service behavior or provide a non-production implementation - for instance memory +stores can be used to mock stores. + +For integration testing, a mock runtime implementation should be provided that allows composing different app modules +together for testing without a dependency on runtime or Comet. + +## Consequences + +### Backwards Compatibility + +Early versions of runtime modules should aim to support as much as possible modules built with the existing +`AppModule`/`sdk.Context` framework. As the core API is more widely adopted, later runtime versions may choose to +drop support and only support the core API plus any runtime module specific APIs (like specific versions of Comet). + +The core module itself should strive to remain at the go semantic version `v1` as long as possible and follow design +principles that allow for strong long-term support (LTS). + +Older versions of the SDK can support modules built against core with adaptors that convert wrap core `AppModule` +implementations in implementations of `AppModule` that conform to that version of the SDK's semantics as well +as by providing service implementations by wrapping `sdk.Context`. + +### Positive + +* better API encapsulation and separation of concerns +* more stable APIs +* more framework extensibility +* deterministic events and queries +* event listeners +* inter-module msg and query execution support +* more explicit support for forking and merging of module versions (including runtime) + +### Negative + +### Neutral + +* modules will need to be refactored to use this API +* some replacements for `AppModule` functionality still need to be defined in follow-ups + (type registration, commands, invariants, simulations) and this will take additional design work + +## Further Discussions + +* gas +* block headers +* upgrades +* registration of gogo proto and amino interface types +* cobra query and tx commands +* gRPC gateway +* crisis module invariants +* simulations + +## References + +* [ADR 033: Protobuf-based Inter-Module Communication](./adr-033-protobuf-inter-module-comm.md) +* [ADR 057: App Wiring](./adr-057-app-wiring.md) +* [ADR 055: ORM](./adr-055-orm.md) +* [ADR 028: Public Key Addresses](./adr-028-public-key-addresses.md) +* [Keeping Your Modules Compatible](https://go.dev/blog/module-compatibility) diff --git a/versioned_docs/version-0.52/build/architecture/adr-064-abci-2.0.md b/versioned_docs/version-0.52/build/architecture/adr-064-abci-2.0.md new file mode 100644 index 000000000..d73cfcc85 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-064-abci-2.0.md @@ -0,0 +1,473 @@ +# ADR 64: ABCI 2.0 Integration (Phase II) + +## Changelog + +* 2023-01-17: Initial Draft (@alexanderbez) +* 2023-04-06: Add upgrading section (@alexanderbez) +* 2023-04-10: Simplify vote extension state persistence (@alexanderbez) +* 2023-07-07: Revise vote extension state persistence (@alexanderbez) +* 2023-08-24: Revise vote extension power calculations and staking interface (@davidterpay) + +## Status + +ACCEPTED + +## Abstract + +This ADR outlines the continuation of the efforts to implement ABCI++ in the Cosmos +SDK outlined in [ADR 060: ABCI 1.0 (Phase I)](adr-060-abci-1.0.md). + +Specifically, this ADR outlines the design and implementation of ABCI 2.0, which +includes `ExtendVote`, `VerifyVoteExtension` and `FinalizeBlock`. + +## Context + +ABCI 2.0 continues the promised updates from ABCI++, specifically three additional +ABCI methods that the application can implement in order to gain further control, +insight and customization of the consensus process, unlocking many novel use-cases +that were previously not possible. We describe these three new methods below: + +### `ExtendVote` + +This method allows each validator process to extend the pre-commit phase of the +CometBFT consensus process. Specifically, it allows the application to perform +custom business logic that extends the pre-commit vote and supply additional data +as part of the vote, although they are signed separately by the same key. + +The data, called vote extension, will be broadcast and received together with the +vote it is extending, and will be made available to the application in the next +height. Specifically, the proposer of the next block will receive the vote extensions +in `RequestPrepareProposal.local_last_commit.votes`. + +If the application does not have vote extension information to provide, it +returns a 0-length byte array as its vote extension. + +**NOTE**: + +* Although each validator process submits its own vote extension, ONLY the *proposer* + of the *next* block will receive all the vote extensions included as part of the + pre-commit phase of the previous block. This means only the proposer will + implicitly have access to all the vote extensions, via `RequestPrepareProposal`, + and that not all vote extensions may be included, since a validator does not + have to wait for all pre-commits, only 2/3. +* The pre-commit vote is signed independently from the vote extension. + +### `VerifyVoteExtension` + +This method allows validators to validate the vote extension data attached to +each pre-commit message it receives. If the validation fails, the whole pre-commit +message will be deemed invalid and ignored by CometBFT. + +CometBFT uses `VerifyVoteExtension` when validating a pre-commit vote. Specifically, +for a pre-commit, CometBFT will: + +* Reject the message if it doesn't contain a signed vote AND a signed vote extension +* Reject the message if the vote's signature OR the vote extension's signature fails to verify +* Reject the message if `VerifyVoteExtension` was rejected by the app + +Otherwise, CometBFT will accept the pre-commit message. + +Note, this has important consequences on liveness, i.e., if vote extensions repeatedly +cannot be verified by correct validators, CometBFT may not be able to finalize +a block even if sufficiently many (+2/3) validators send pre-commit votes for +that block. Thus, `VerifyVoteExtension` should be used with special care. + +CometBFT recommends that an application that detects an invalid vote extension +SHOULD accept it in `ResponseVerifyVoteExtension` and ignore it in its own logic. + +### `FinalizeBlock` + +This method delivers a decided block to the application. The application must +execute the transactions in the block deterministically and update its state +accordingly. Cryptographic commitments to the block and transaction results, +returned via the corresponding parameters in `ResponseFinalizeBlock`, are +included in the header of the next block. CometBFT calls it when a new block +is decided. + +In other words, `FinalizeBlock` encapsulates the current ABCI execution flow of +`BeginBlock`, one or more `DeliverTx`, and `EndBlock` into a single ABCI method. +CometBFT will no longer execute requests for these legacy methods and instead +will just simply call `FinalizeBlock`. + +## Decision + +We will discuss changes to the Cosmos SDK to implement ABCI 2.0 in two distinct +phases, `VoteExtensions` and `FinalizeBlock`. + +### `VoteExtensions` + +Similarly for `PrepareProposal` and `ProcessProposal`, we propose to introduce +two new handlers that an application can implement in order to provide and verify +vote extensions. + +We propose the following new handlers for applications to implement: + +```go +type ExtendVoteHandler func(sdk.Context, abci.ExtendVoteRequest) abci.ExtendVoteResponse +type VerifyVoteExtensionHandler func(sdk.Context, abci.RequestVerifyVoteExtension) abci.ResponseVerifyVoteExtension +``` + +An ephemeral context and state will be supplied to both handlers. The +context will contain relevant metadata such as the block height and block hash. +The state will be a cached version of the committed state of the application and +will be discarded after the execution of the handler, this means that both handlers +get a fresh state view and no changes made to it will be written. + +If an application decides to implement `ExtendVoteHandler`, it must return a +non-nil `ResponseExtendVote.VoteExtension`. + +Recall, an implementation of `ExtendVoteHandler` does NOT need to be deterministic, +however, given a set of vote extensions, `VerifyVoteExtensionHandler` must be +deterministic, otherwise the chain may suffer from liveness faults. In addition, +recall CometBFT proceeds in rounds for each height, so if a decision cannot be +made about a block proposal at a given height, CometBFT will proceed to the +next round and thus will execute `ExtendVote` and `VerifyVoteExtension` again for +the new round for each validator until 2/3 valid pre-commits can be obtained. + +Given the broad scope of potential implementations and use-cases of vote extensions, +and how to verify them, most applications should choose to implement the handlers +through a single handler type, which can have any number of dependencies injected +such as keepers. In addition, this handler type could contain some notion of +volatile vote extension state management which would assist in vote extension +verification. This state management could be ephemeral or could be some form of +on-disk persistence. + +Example: + +```go +// VoteExtensionHandler implements an Oracle vote extension handler. +type VoteExtensionHandler struct { + cdc Codec + mk MyKeeper + state VoteExtState // This could be a map or a DB connection object +} + +// ExtendVoteHandler can do something with h.mk and possibly h.state to create +// a vote extension, such as fetching a series of prices for supported assets. +func (h VoteExtensionHandler) ExtendVoteHandler(ctx sdk.Context, req abci.ExtendVoteRequest) abci.ExtendVoteResponse { + prices := GetPrices(ctx, h.mk.Assets()) + bz, err := EncodePrices(h.cdc, prices) + if err != nil { + panic(fmt.Errorf("failed to encode prices for vote extension: %w", err)) + } + + // store our vote extension at the given height + // + // NOTE: Vote extensions can be overridden since we can timeout in a round. + SetPrices(h.state, req, bz) + + return abci.ExtendVoteResponse{VoteExtension: bz} +} + +// VerifyVoteExtensionHandler can do something with h.state and req to verify +// the req.VoteExtension field, such as ensuring the provided oracle prices are +// within some valid range of our prices. +func (h VoteExtensionHandler) VerifyVoteExtensionHandler(ctx sdk.Context, req abci.RequestVerifyVoteExtension) abci.ResponseVerifyVoteExtension { + prices, err := DecodePrices(h.cdc, req.VoteExtension) + if err != nil { + log("failed to decode vote extension", "err", err) + return abci.ResponseVerifyVoteExtension{Status: REJECT} + } + + if err := ValidatePrices(h.state, req, prices); err != nil { + log("failed to validate vote extension", "prices", prices, "err", err) + return abci.ResponseVerifyVoteExtension{Status: REJECT} + } + + // store updated vote extensions at the given height + // + // NOTE: Vote extensions can be overridden since we can timeout in a round. + SetPrices(h.state, req, req.VoteExtension) + + return abci.ResponseVerifyVoteExtension{Status: ACCEPT} +} +``` + +#### Vote Extension Propagation & Verification + +As mentioned previously, vote extensions for height `H` are only made available +to the proposer at height `H+1` during `PrepareProposal`. However, in order to +make vote extensions useful, all validators should have access to the agreed upon +vote extensions at height `H` during `H+1`. + +Since CometBFT includes all the vote extension signatures in `RequestPrepareProposal`, +we propose that the proposing validator manually "inject" the vote extensions +along with their respective signatures via a special transaction, `VoteExtsTx`, +into the block proposal during `PrepareProposal`. The `VoteExtsTx` will be +populated with a single `ExtendedCommitInfo` object which is received directly +from `RequestPrepareProposal`. + +For convention, the `VoteExtsTx` transaction should be the first transaction in +the block proposal, although chains can implement their own preferences. For +safety purposes, we also propose that the proposer itself verify all the vote +extension signatures it receives in `RequestPrepareProposal`. + +A validator, upon a `RequestProcessProposal`, will receive the injected `VoteExtsTx` +which includes the vote extensions along with their signatures. If no such transaction +exists, the validator MUST REJECT the proposal. + +When a validator inspects a `VoteExtsTx`, it will evaluate each `SignedVoteExtension`. +For each signed vote extension, the validator will generate the signed bytes and +verify the signature. At least 2/3 valid signatures, based on voting power, must +be received in order for the block proposal to be valid, otherwise the validator +MUST REJECT the proposal. + +In order to have the ability to validate signatures, `BaseApp` must have access +to the `x/staking` module, since this module stores an index from consensus +address to public key. However, we will avoid a direct dependency on `x/staking` +and instead rely on an interface instead. In addition, the Cosmos SDK will expose +a default signature verification method which applications can use: + +```go +type ValidatorStore interface { + GetPubKeyByConsAddr(context.Context, sdk.ConsAddress) (cmtprotocrypto.PublicKey, error) +} + +// ValidateVoteExtensions is a function that an application can execute in +// ProcessProposal to verify vote extension signatures. +func (app *BaseApp) ValidateVoteExtensions(ctx sdk.Context, currentHeight int64, extCommit abci.ExtendedCommitInfo) error { + votingPower := 0 + totalVotingPower := 0 + + for _, vote := range extCommit.Votes { + totalVotingPower += vote.Validator.Power + + if !vote.SignedLastBlock || len(vote.VoteExtension) == 0 { + continue + } + + valConsAddr := sdk.ConsAddress(vote.Validator.Address) + pubKeyProto, err := valStore.GetPubKeyByConsAddr(ctx, valConsAddr) + if err != nil { + return fmt.Errorf("failed to get public key for validator %s: %w", valConsAddr, err) + } + + if len(vote.ExtensionSignature) == 0 { + return fmt.Errorf("received a non-empty vote extension with empty signature for validator %s", valConsAddr) + } + + cmtPubKey, err := cryptoenc.PubKeyFromProto(pubKeyProto) + if err != nil { + return fmt.Errorf("failed to convert validator %X public key: %w", valConsAddr, err) + } + + cve := cmtproto.CanonicalVoteExtension{ + Extension: vote.VoteExtension, + Height: currentHeight - 1, // the vote extension was signed in the previous height + Round: int64(extCommit.Round), + ChainId: app.GetChainID(), + } + + extSignBytes, err := cosmosio.MarshalDelimited(&cve) + if err != nil { + return fmt.Errorf("failed to encode CanonicalVoteExtension: %w", err) + } + + if !cmtPubKey.VerifySignature(extSignBytes, vote.ExtensionSignature) { + return errors.New("received vote with invalid signature") + } + + votingPower += vote.Validator.Power + } + + if (votingPower / totalVotingPower) < threshold { + return errors.New("not enough voting power for the vote extensions") + } + + return nil +} +``` + +Once at least 2/3 signatures, by voting power, are received and verified, the +validator can use the vote extensions to derive additional data or come to some +decision based on the vote extensions. + +> NOTE: It is very important to state, that neither the vote propagation technique +> nor the vote extension verification mechanism described above is required for +> applications to implement. In other words, a proposer is not required to verify +> and propagate vote extensions along with their signatures nor are proposers +> required to verify those signatures. An application can implement it's own +> PKI mechanism and use that to sign and verify vote extensions. + +#### Vote Extension Persistence + +In certain contexts, it may be useful or necessary for applications to persist +data derived from vote extensions. In order to facilitate this use case, we propose +to allow app developers to define a pre-Blocker hook which will be called +at the very beginning of `FinalizeBlock`, i.e. before `BeginBlock` (see below). + +Note, we cannot allow applications to directly write to the application state +during `ProcessProposal` because during replay, CometBFT will NOT call `ProcessProposal`, +which would result in an incomplete state view. + +```go +func (a MyApp) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) error { + voteExts := GetVoteExtensions(ctx, req.Txs) + + // Process and perform some compute on vote extensions, storing any resulting + // state. + if err a.processVoteExtensions(ctx, voteExts); if err != nil { + return err + } +} +``` + +### `FinalizeBlock` + +The existing ABCI methods `BeginBlock`, `DeliverTx`, and `EndBlock` have existed +since the dawn of ABCI-based applications. Thus, applications, tooling, and developers +have grown used to these methods and their use-cases. Specifically, `BeginBlock` +and `EndBlock` have grown to be pretty integral and powerful within ABCI-based +applications. E.g. an application might want to run distribution and inflation +related operations prior to executing transactions and then have staking related +changes to happen after executing all transactions. + +We propose to keep `BeginBlock` and `EndBlock` within the SDK's core module +interfaces only so application developers can continue to build against existing +execution flows. However, we will remove `BeginBlock`, `DeliverTx` and `EndBlock` +from the SDK's `BaseApp` implementation and thus the ABCI surface area. + +What will then exist is a single `FinalizeBlock` execution flow. Specifically, in +`FinalizeBlock` we will execute the application's `BeginBlock`, followed by +execution of all the transactions, finally followed by execution of the application's +`EndBlock`. + +Note, we will still keep the existing transaction execution mechanics within +`BaseApp`, but all notions of `DeliverTx` will be removed, i.e. `deliverState` +will be replace with `finalizeState`, which will be committed on `Commit`. + +However, there are current parameters and fields that exist in the existing +`BeginBlock` and `EndBlock` ABCI types, such as votes that are used in distribution +and byzantine validators used in evidence handling. These parameters exist in the +`FinalizeBlock` request type, and will need to be passed to the application's +implementations of `BeginBlock` and `EndBlock`. + +This means the Cosmos SDK's core module interfaces will need to be updated to +reflect these parameters. The easiest and most straightforward way to achieve +this is to just pass `RequestFinalizeBlock` to `BeginBlock` and `EndBlock`. +Alternatively, we can create dedicated proxy types in the SDK that reflect these +legacy ABCI types, e.g. `LegacyBeginBlockRequest` and `LegacyEndBlockRequest`. Or, +we can come up with new types and names altogether. + +```go +func (app *BaseApp) FinalizeBlock(req abci.RequestFinalizeBlock) (*abci.ResponseFinalizeBlock, error) { + ctx := ... + + if app.preBlocker != nil { + ctx := app.finalizeBlockState.ctx + rsp, err := app.preBlocker(ctx, req) + if err != nil { + return nil, err + } + if rsp.ConsensusParamsChanged { + app.finalizeBlockState.ctx = ctx.WithConsensusParams(app.GetConsensusParams(ctx)) + } + } + beginBlockResp, err := app.beginBlock(req) + appendBlockEventAttr(beginBlockResp.Events, "begin_block") + + txExecResults := make([]abci.ExecTxResult, 0, len(req.Txs)) + for _, tx := range req.Txs { + result := app.runTx(runTxModeFinalize, tx) + txExecResults = append(txExecResults, result) + } + + endBlockResp, err := app.endBlock(app.finalizeBlockState.ctx) + appendBlockEventAttr(beginBlockResp.Events, "end_block") + + return abci.ResponseFinalizeBlock{ + TxResults: txExecResults, + Events: joinEvents(beginBlockResp.Events, endBlockResp.Events), + ValidatorUpdates: endBlockResp.ValidatorUpdates, + ConsensusParamUpdates: endBlockResp.ConsensusParamUpdates, + AppHash: nil, + } +} +``` + +#### Events + +Many tools, indexers and ecosystem libraries rely on the existence `BeginBlock` +and `EndBlock` events. Since CometBFT now only exposes `FinalizeBlockEvents`, we +find that it will still be useful for these clients and tools to still query for +and rely on existing events, especially since applications will still define +`BeginBlock` and `EndBlock` implementations. + +In order to facilitate existing event functionality, we propose that all `BeginBlock` +and `EndBlock` events have a dedicated `EventAttribute` with `key=block` and +`value=begin_block|end_block`. The `EventAttribute` will be appended to each event +in both `BeginBlock` and `EndBlock` events`. + + +### Upgrading + +CometBFT defines a consensus parameter, [`VoteExtensionsEnableHeight`](https://docs.cometbft.com/v1.0/spec/abci/abci++_app_requirements#featureparamsvoteextensionsenableheight), +which specifies the height at which vote extensions are enabled and **required**. +If the value is set to zero, which is the default, then vote extensions are +disabled and an application is not required to implement and use vote extensions. + +However, if the value `H` is positive, at all heights greater than the configured +height `H` vote extensions must be present (even if empty). When the configured +height `H` is reached, `PrepareProposal` will not include vote extensions yet, +but `ExtendVote` and `VerifyVoteExtension` will be called. Then, when reaching +height `H+1`, `PrepareProposal` will include the vote extensions from height `H`. + +It is very important to note, for all heights after H: + +* Vote extensions CANNOT be disabled +* They are mandatory, i.e. all pre-commit messages sent MUST have an extension + attached (even if empty) + +When an application updates to the Cosmos SDK version with CometBFT v0.38 support, +in the upgrade handler it must ensure to set the consensus parameter +`VoteExtensionsEnableHeight` to the correct value. E.g. if an application is set +to perform an upgrade at height `H`, then the value of `VoteExtensionsEnableHeight` +should be set to any value `>=H+1`. This means that at the upgrade height, `H`, +vote extensions will not be enabled yet, but at height `H+1` they will be enabled. + +## Consequences + +### Backwards Compatibility + +ABCI 2.0 is naturally not backwards compatible with prior versions of the Cosmos SDK +and CometBFT. For example, an application that requests `RequestFinalizeBlock` +to the same application that does not speak ABCI 2.0 will naturally fail. + +In addition, `BeginBlock`, `DeliverTx` and `EndBlock` will be removed from the +application ABCI interfaces and along with the inputs and outputs being modified +in the module interfaces. + +### Positive + +* `BeginBlock` and `EndBlock` semantics remain, so burden on application developers + should be limited. +* Less communication overhead as multiple ABCI requests are condensed into a single + request. +* Sets the groundwork for optimistic execution. +* Vote extensions allow for an entirely new set of application primitives to be + developed, such as in-process price oracles and encrypted mempools. + +### Negative + +* Some existing Cosmos SDK core APIs may need to be modified and thus broken. +* Signature verification in `ProcessProposal` of 100+ vote extension signatures + will add significant performance overhead to `ProcessProposal`. Granted, the + signature verification process can happen concurrently using an error group + with `GOMAXPROCS` goroutines. + +### Neutral + +* Having to manually "inject" vote extensions into the block proposal during + `PrepareProposal` is an awkward approach and takes up block space unnecessarily. +* The requirement of `ResetProcessProposalState` can create a footgun for + application developers if they're not careful, but this is necessary in order + for applications to be able to commit state from vote extension computation. + +## Further Discussions + +Future discussions include design and implementation of ABCI 3.0, which is a +continuation of ABCI++ and the general discussion of optimistic execution. + +## References + +* [ADR 060: ABCI 1.0 (Phase I)](adr-060-abci-1.0.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-065-store-v2.md b/versioned_docs/version-0.52/build/architecture/adr-065-store-v2.md new file mode 100644 index 000000000..b1377555e --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-065-store-v2.md @@ -0,0 +1,292 @@ +# ADR-065: Store V2 + +## Changelog + +* Feb 14, 2023: Initial Draft (@alexanderbez) +* Dec 21, 2023: Updates after implementation (@alexanderbez) + +## Status + +ACCEPTED + +## Abstract + +The storage and state primitives that Cosmos SDK based applications have used have +by and large not changed since the launch of the inaugural Cosmos Hub. The demands +and needs of Cosmos SDK based applications, from both developer and client UX +perspectives, have evolved and outgrown the ecosystem since these primitives +were first introduced. + +Over time as these applications have gained significant adoption, many critical +shortcomings and flaws have been exposed in the state and storage primitives of +the Cosmos SDK. + +In order to keep up with the evolving demands and needs of both clients and developers, +a major overhaul to these primitives are necessary. + +## Context + +The Cosmos SDK provides application developers with various storage primitives +for dealing with application state. Specifically, each module contains its own +merkle commitment data structure -- an IAVL tree. In this data structure, a module +can store and retrieve key-value pairs along with Merkle commitments, i.e. proofs, +to those key-value pairs indicating that they do or do not exist in the global +application state. This data structure is the base layer `KVStore`. + +In addition, the SDK provides abstractions on top of this Merkle data structure. +Namely, a root multi-store (RMS) is a collection of each module's `KVStore`. +Through the RMS, the application can serve queries and provide proofs to clients +in addition to provide a module access to its own unique `KVStore` though the use +of `StoreKey`, which is an OCAP primitive. + +There are further layers of abstraction that sit between the RMS and the underlying +IAVL `KVStore`. A `GasKVStore` is responsible for tracking gas IO consumption for +state machine reads and writes. A `CacheKVStore` is responsible for providing a +way to cache reads and buffer writes to make state transitions atomic, e.g. +transaction execution or governance proposal execution. + +There are a few critical drawbacks to these layers of abstraction and the overall +design of storage in the Cosmos SDK: + +* Since each module has its own IAVL `KVStore`, commitments are not [atomic](https://github.com/cosmos/cosmos-sdk/issues/14625) + * Note, we can still allow modules to have their own IAVL `KVStore`, but the + IAVL library will need to support the ability to pass a DB instance as an + argument to various IAVL APIs. +* Since IAVL is responsible for both state storage and commitment, running an + archive node becomes increasingly expensive as disk space grows exponentially. +* As the size of a network increases, various performance bottlenecks start to + emerge in many areas such as query performance, network upgrades, state + migrations, and general application performance. +* Developer UX is poor as it does not allow application developers to experiment + with different types of approaches to storage and commitments, along with the + complications of many layers of abstractions referenced above. + +See the [Storage Discussion](https://github.com/cosmos/cosmos-sdk/discussions/13545) for more information. + +## Alternatives + +There was a previous attempt to refactor the storage layer described in [ADR-040](./adr-040-storage-and-smt-state-commitments.md). +However, this approach mainly stems on the short comings of IAVL and various performance +issues around it. While there was a (partial) implementation of [ADR-040](./adr-040-storage-and-smt-state-commitments.md), +it was never adopted for a variety of reasons, such as the reliance on using an +SMT, which was more in a research phase, and some design choices that couldn't +be fully agreed upon, such as the snap-shotting mechanism that would result in +massive state bloat. + +## Decision + +We propose to build upon some of the great ideas introduced in [ADR-040](./adr-040-storage-and-smt-state-commitments.md), +while being a bit more flexible with the underlying implementations and overall +less intrusive. Specifically, we propose to: + +* Separate the concerns of state commitment (**SC**), needed for consensus, and + state storage (**SS**), needed for state machine and clients. +* Reduce layers of abstractions necessary between the RMS and underlying stores. +* Remove unnecessary store types and implementations such as `CacheKVStore`. +* Simplify the branching logic. +* Ensure the `RootStore` interface remains as lightweight as possible. +* Allow application developers to easily swap out SS and SC backends. + +Furthermore, we will keep IAVL as the default [SC](https://cryptography.fandom.com/wiki/Commitment_scheme) +backend for the time being. While we might not fully settle on the use of IAVL in +the long term, we do not have strong empirical evidence to suggest a better +alternative. Given that the SDK provides interfaces for stores, it should be sufficient +to change the backing commitment store in the future should evidence arise to +warrant a better alternative. However there is promising work being done to IAVL +that should result in significant performance improvement [1,2]. + +Note, we will provide applications with the ability to use IAVL v1 and IAVL v2 as +either SC backend, with the latter showing extremely promising performance improvements +over IAVL v0 and v1, at the cost of a state migration. + +### Separating SS and SC + +By separating SS and SC, it will allow for us to optimize against primary use cases +and access patterns to state. Specifically, The SS layer will be responsible for +direct access to data in the form of (key, value) pairs, whereas the SC layer (e.g. IAVL) +will be responsible for committing to data and providing Merkle proofs. + +#### State Commitment (SC) + +A foremost design goal is that SC backends should be easily swappable, i.e. not +necessarily IAVL. To this end, the scope of SC has been reduced, it must only: + +* Provide a stateful root app hash for height h resulting from applying a batch + of key-value set/deletes to height h-1. +* Fulfill (though not necessarily provide) historical proofs for all heights < h. +* Provide an API for snapshot create/restore to fulfill state sync requests. + +An SC implementation may choose not to provide historical proofs past height h - n (n can be 0) +due to the time and space constraints, but since store v2 defines an API for historical +proofs there should be at least one configuration of a given SC backend which +supports this. + +#### State Storage (SS) + +The goal of SS is to provide a modular storage backend, i.e. multiple implementations, +to facilitate storing versioned raw key/value pairs in a fast embedded database. +The responsibility and functions of SS include the following: + +* Provided fast and efficient queries for versioned raw key/value pairs +* Provide versioned CRUD operations +* Provide versioned batching functionality +* Provide versioned iteration (forward and reverse) functionality +* Provide pruning functionality + +All of the functionality provided by an SS backend should work under a versioned +scheme, i.e. a user should be able to get, store, and iterate over keys for the latest +and historical versions efficiently and a store key, which is used for name-spacing +purposes. + +We propose to have three defaulting SS backends for applications to choose from: + +* RocksDB + * CGO based + * Usage of User-Defined Timestamps as a built-in versioning mechanism +* PebbleDB + * Native + * Manual implementation of MVCC keys for versioning +* SQLite + * CGO based + * Single table for all state + +Since operators might want pruning strategies to differ in SS compared to SC, +e.g. having a very tight pruning strategy in SC while having a looser pruning +strategy for SS, we propose to introduce an additional pruning configuration, +with parameters that are identical to what exists in the SDK today, and allow +operators to control the pruning strategy of the SS layer independently of the +SC layer. + +Note, the SC pruning strategy must be congruent with the operator's state sync +configuration. This is so as to allow state sync snapshots to execute successfully, +otherwise, a snapshot could be triggered on a height that is not available in SC. + +#### State Sync + +The state sync process should be largely unaffected by the separation of the SC +and SS layers. However, if a node syncs via state sync, the SS layer of the node +will not have the state synced height available, since the IAVL import process is +not setup in way to easily allow direct key/value insertion. + +We propose a simple `SnapshotManager` that consumes and produces snapshots. SC +backends will be responsible for providing a snapshot of the state at a given +height and both SS and SC consume snapshots to restore state. + +#### RootStore + +We will define a `RootStore` interface and default implementation that will be +the primary interface for the application to interact with. The `RootStore` will +be responsible for housing SS and SC backends. Specifically, a `RootStore` will +provide the following functionality: + +* Manage commitment of state (both SS and SC) +* Provide modules access to state +* Query delegation (i.e. get a value for a tuple) +* Providing commitment proofs + +#### Store Keys + +Naturally, if a single SC tree is used in all RootStore implementations, then the +notion of a store key becomes entirely useless. However, we cannot dictate or +predicate how all applications will implement their RooStore (if they choose to). + +Since an app can choose to have multiple SC trees, we need to keep the notion of +store keys. Unlike store v1, we represent store keys as simple strings as opposed +to concrete types to provide OCAP functionality. The store key strings act to +solely provide key prefixing/namespacing functionality for modules. + +#### Proofs + +Since the SS layer is naturally a storage layer only, without any commitments +to (key, value) pairs, it cannot provide Merkle proofs to clients during queries. + +So providing inclusion and exclusion proofs, via a `CommitmentOp` type, will be +the responsibility of the SC backend. Retrieving proofs will be done through the +a `RootStore`, which will internally route the request to the SC backend. + +#### Commitment + +Before ABCI 2.0, specifically before `FinalizeBlock` was introduced, the flow of state +commitment in BaseApp was defined by writes being written to the `RootMultiStore` +and then a single Commit call on the `RootMultiStore` during the ABCI Commit method. + +With the advent of ABCI 2.0, the commitment flow has now changed to `WorkingHash` being +called during `FinalizeBlock` and then Commit being called on ABCI Commit. Note, +`WorkingHash` does not actually commit state to disk, but rather computes an +uncommitted work-in-progress hash, which is returned in `FinalizeBlock`. Then, +during the ABCI Commit phase, the state is finally flushed to disk. + +In store v2, we must respect this flow. Thus, a caller is expected to call `WorkingHash` +during `FinalizeBlock`, which takes the latest changeset in the `RootStore`, +writes that to the SC tree in a single batch and returns a hash. Finally, during +the ABCI Commit phase, we call `Commit` on the `RootStore` which commits the SC +tree and flushes the changeset to the SS backend. + +## Consequences + +As a result of a new store V2 package, we should expect to see improved performance +for queries and transactions due to the separation of concerns. We should also +expect to see improved developer UX around experimentation of commitment schemes +and storage backends for further performance, in addition to a reduced amount of +abstraction around KVStores making operations such as caching and state branching +more intuitive. + +However, due to the proposed design, there are drawbacks around providing state +proofs for historical queries. + +### Backwards Compatibility + +This ADR proposes changes to the storage implementation in the Cosmos SDK through +an entirely new package. Interfaces may be borrowed and extended from existing +types that exist in `store`, but no existing implementations or interfaces will +be broken or modified. + +### Positive + +* Improved performance of independent SS and SC layers +* Reduced layers of abstraction making storage primitives easier to understand +* Atomic commitments for SC +* Redesign of storage types and interfaces will allow for greater experimentation + such as different physical storage backends and different commitment schemes + for different application modules + +### Negative + +* Providing proofs for historical state is challenging + +### Neutral + +* Removal of OCAP-based store keys in favor of simple strings for state retrieval + and name-spacing. We consider this neutral as removal of OCAP functionality can + be seen as a negative, however, we're simply moving the OCAP functionality upstream + to the KVStore service. The SS and SC layers shouldn't have to concern themselves + with OCAP responsibilities. +* Keeping IAVL as the primary commitment data structure, although drastic + performance improvements are being made + +## Further Discussions + +### Module Storage Control + +Many modules store secondary indexes that are typically solely used to support +client queries, but are actually not needed for the state machine's state +transitions. What this means is that these indexes technically have no reason to +exist in the SC layer at all, as they take up unnecessary space. It is worth +exploring what an API would look like to allow modules to indicate what (key, value) +pairs they want to be persisted in the SC layer, implicitly indicating the SS +layer as well, as opposed to just persisting the (key, value) pair only in the +SS layer. + +### Historical State Proofs + +It is not clear what the importance or demand is within the community of providing +commitment proofs for historical state. While solutions can be devised such as +rebuilding trees on the fly based on state snapshots, it is not clear what the +performance implications are for such solutions. + +## References + +* [1] https://github.com/cosmos/iavl/pull/676 +* [2] https://github.com/cosmos/iavl/pull/664 +* [3] https://github.com/cosmos/cosmos-sdk/issues/14990 +* [4] https://docs.google.com/document/d/e/2PACX-1vSCFfXZm2vsRsACOPoxGqysMaUg7jY833LwR3YyjA1S3FNHfXRiJor-qLjzx833TavLXLPSIcFZJhyh/pub diff --git a/versioned_docs/version-0.52/build/architecture/adr-067-simulator-v2.md b/versioned_docs/version-0.52/build/architecture/adr-067-simulator-v2.md new file mode 100644 index 000000000..ba3be1229 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-067-simulator-v2.md @@ -0,0 +1,194 @@ +# ADR 067: Simulator v2 + +## Changelog + +* June 01, 2023: Initial Draft (@alexanderbez) + +## Status + +DRAFT + +## Abstract + +The Cosmos SDK simulator is a tool that allows developers to test the entirety +of their application's state machine through the use of pseudo-randomized "operations", +which represent transactions. The simulator also provides primitives that ensures +there are no non-determinism issues and that the application's state machine can +be successfully exported and imported using randomized state. + +The simulator has played an absolutely critical role in the development and testing +of the Cosmos Hub and all the releases of the Cosmos SDK after the launch of the +Cosmos Hub. Since the Hub, the simulator has relatively not changed much, so it's +overdue for a revamp. + +## Context + +The current simulator, `x/simulation`, acts as a semi-fuzz testing suite that takes +in an integer that represents a seed into a PRNG. The PRNG is used to generate a +sequence of "operations" that are meant to reflect transactions that an application's +state machine can process. Through the use of the PRNG, all aspects of block production +and consumption are randomized. This includes a block's proposer, the validators +who both sign and miss the block, along with the transaction operations themselves. + +Each Cosmos SDK module defines a set of simulation operations that _attempt_ to +produce valid transactions, e.g. `x/distribution/simulation/operations.go`. These +operations can sometimes fail depending on the accumulated state of the application +within that simulation run. The simulator will continue to generate operations +until it has reached a certain number of operations or until it has reached a +fatal state, reporting results. This gives the ability for application developers +to reliably execute full range application simulation and fuzz testing against +their application. + +However, there are a few major drawbacks. Namely, with the advent of ABCI++, specifically +`FinalizeBlock`, the internal workings of the simulator no longer comply with how +an application would actually perform. Specifically, operations are executed +_after_ `FinalizeBlock`, whereas they should be executed _within_ `FinalizeBlock`. + +Additionally, the simulator is not very extensible. Developers should be able to +easily define and extend the following: + +* Consistency or validity predicates (what are known as invariants today) +* Property tests of state before and after a block is simulated + +In addition, we also want to achieve the following: + +* Consolidated weight management, i.e. define weights within the simulator itself + via a config and not defined in each module +* Observability of the simulator's execution, i.e. have easy to understand output/logs + with the ability to pipe those logs into some external sink +* Smart replay, i.e. the ability to not only rerun a simulation from a seed, but + also the ability to replay from an arbitrary breakpoint +* Run a simulation based off of real network state + +## Decision + +Instead of refactoring the existing simulator, `x/simulation`, we propose to create +a new package in the root of the Cosmos SDK, `simulator`, that will be the new +simulation framework. The simulator will more accurately reflect the complete +lifecycle of an ABCI application. + +Specifically, we propose a similar implementation and use of a `simulator.Manager`, +that exists today, that is responsible for managing the execution of a simulation. +The manager will wrap an ABCI application and will be responsible for the following: + +* Populating the application's mempool with a set of pseudo-random transactions + before each block, some of which may contain invalid messages. +* Selecting transactions and a random proposer to execute `PrepareProposal`. +* Executing `ProcessProposal`, `FinalizeBlock` and `Commit`. +* Executing a set of validity predicates before and after each block. +* Maintaining a CPU and RAM profile of the simulation execution. +* Allowing a simulation to stop and resume from a given block height. +* Simulation liveness of each validator per-block. + +From an application developer's perspective, they will only need to provide the +modules to be used in the simulator and the manager will take care of the rest. +In addition, they will not need to write their own simulation test(s), e.g. +non-determinism, multi-seed, etc..., as the manager will provide these as well. + +```go +type Manager struct { + app sdk.Application + mempool sdk.Mempool + rng rand.Rand + // ... +} +``` + +### Configuration + +The simulator's testing input will be driven by a configuration file, as opposed +to CLI arguments. This will allow for more extensibility and ease of use along with +the ability to have shared configuration files across multiple simulations. + +### Execution + +As alluded to previously, after the execution of each block, the manager will +generate a series of pseudo-random transactions and attempt to insert them into +the mempool via `BaseApp#CheckTx`. During the ABCI lifecycle of a block, this +mempool will be used to seed the transactions into a block proposal as it would +in a real network. This allows us to not only test the state machine, but also +test the ABCI lifecycle of a block. + +Statistics, such as total blocks and total failed proposals, will be collected, +logged and written to output after the full or partial execution of a simulation. +The output destination of these statistics will be configurable. + +```go +func (s *Simulator) SimulateBlock() { + rProposer := s.SelectRandomProposer() + rTxs := s.SelectTxs() + + prepareResp, err := s.app.PrepareProposal(&abci.PrepareProposalRequest{Txs: rTxs}) + // handle error + + processResp, err := s.app.ProcessProposal(&abci.ProcessProposalRequest{ + Txs: prepareResp.Txs, + // ... + }) + // handle error + + // execute liveness matrix... + + _, err = s.app.FinalizeBlock(...) + // handle error + + _, err = s.app.Commit(...) + // handle error +} +``` + +Note, some applications do not define or need their own app-side mempool, so we +propose that `SelectTxs` mimic CometBFT and just return FIFO-ordered transactions +from an ad-hoc simulator mempool. In the case where an application does define +its own mempool, it will simply ignore what is provided in `RequestPrepareProposal`. + +### Profiling + +The manager will be responsible for collecting CPU and RAM profiles of the simulation +execution. We propose to use [Pyroscope](https://pyroscope.io/docs/golang/) to +capture profiles and export them to a local file and via an HTTP endpoint. This +can be disabled or enabled by configuration. + +### Breakpoints + +Via configuration, a caller can express a height-based breakpoint that will allow +the simulation to stop and resume from a given height. This will allow for debugging +of CPU, RAM, and state. + +### Validity Predicates + +We propose to provide the ability for an application to provide the simulator a +set of validity predicates, i.e. invariant checkers, that will be executed before +and after each block. This will allow for the application to assert that certain +state invariants are held before and after each block. Note, as a consequence of +this, we propose to remove the existing notion of invariants from module production +execution paths and deprecate their usage altogether. + +```go +type Manager struct { + // ... + preBlockValidator func(sdk.Context) error + postBlockValidator func(sdk.Context) error +} +``` + +## Consequences + +### Backwards Compatibility + +The new simulator package will not naturally not be backwards compatible with the +existing `x/simulation` module. However, modules will still be responsible for +providing pseudo-random transactions to the simulator. + +### Positive + +* Providing more intuitive and cleaner APIs for application developers +* More closely resembling the true lifecycle of an ABCI application + +### Negative + +* Breaking current Cosmos SDK module APIs for transaction generation + +## References + +* [Osmosis Simulation ADR](https://github.com/osmosis-labs/osmosis/blob/main/simulation/ADR.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-068-preblock.md b/versioned_docs/version-0.52/build/architecture/adr-068-preblock.md new file mode 100644 index 000000000..86692c412 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-068-preblock.md @@ -0,0 +1,61 @@ +# ADR 068: Preblock + +## Changelog + +* Sept 13, 2023: Initial Draft + +## Status + +DRAFT + +## Abstract + +Introduce `PreBlock`, which runs before begin blocker other modules, and allows to modify consensus parameters, and the changes are visible to the following state machine logics. + +## Context + +When upgrading to sdk 0.47, the storage format for consensus parameters changed, but in the migration block, `ctx.ConsensusParams()` is always `nil`, because it fails to load the old format using new code, it's supposed to be migrated by the `x/upgrade` module first, but unfortunately, the migration happens in `BeginBlocker` handler, which runs after the `ctx` is initialized. +When we try to solve this, we find the `x/upgrade` module can't modify the context to make the consensus parameters visible for the other modules, the context is passed by value, and sdk team want to keep it that way, that's good for isolations between modules. + +## Alternatives + +The first alternative solution introduced a `MigrateModuleManager`, which only includes the `x/upgrade` module right now, and baseapp will run their `BeginBlocker`s before the other modules, and reload context's consensus parameters in between. + +## Decision + +Suggested this new lifecycle method. + +### `PreBlocker` + +There are two semantics around the new lifecycle method: + +- It runs before the `BeginBlocker` of all modules +- It can modify consensus parameters in storage, and signal the caller through the return value. + +When it returns `ConsensusParamsChanged=true`, the caller must refresh the consensus parameter in the finalize context: +``` +app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) +``` + +The new ctx must be passed to all the other lifecycle methods. + + +## Consequences + +### Backwards Compatibility + +### Positive + +### Negative + +### Neutral + +## Further Discussions + +## Test Cases + +## References +* [1] https://github.com/cosmos/cosmos-sdk/issues/16494 +* [2] https://github.com/cosmos/cosmos-sdk/pull/16583 +* [3] https://github.com/cosmos/cosmos-sdk/pull/17421 +* [4] https://github.com/cosmos/cosmos-sdk/pull/17713 diff --git a/versioned_docs/version-0.52/build/architecture/adr-069-gov-improvements.md b/versioned_docs/version-0.52/build/architecture/adr-069-gov-improvements.md new file mode 100644 index 000000000..1ef6971c7 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-069-gov-improvements.md @@ -0,0 +1,224 @@ +# ADR 069: `x/gov` modularity, multiple choice and optimistic proposals + +## Changelog + +* 2023-11-17: Initial draft (@julienrbrt, @tac0turtle) + +## Status + +ACCEPTED - Implemented + +## Abstract + +Governance is an important aspect of Cosmos SDK chains. + +This ADR aimed to extend the `x/gov` module functionalities by adding two different kinds of proposals, as well as making `x/gov` more composable and extendable. + +Those two types are, namely: multiple choice proposals and optimistic proposals. + +## Context + +`x/gov` is the center of Cosmos governance, and has already been improved from its first version `v1beta1`, with a second version [`v1`][5]. +This second iteration of gov unlocked many possibilities by letting governance proposals contain any number of proposals. +The last addition of gov has been expedited proposals (proposals that have a shorter voting period and a higher quorum, approval threshold). + +The community requested ([1], [4]) two additional proposals for improving governance choices. Those proposals would be useful when having protocol decisions made on specific choices or simplifying regular proposals that do not require high community involvement. + +Additionally, the SDK should allow chains to customize the tallying method of proposals (if they want to count the votes in another way). Currently, the Cosmos SDK counts votes proportionally to the voting power/stake. However, custom tallying could allow counting votes with a quadratic function instead. + +## Decision + +`x/gov` will integrate these functions and extract helpers and interfaces for extending the `x/gov` module capabilities. + +### Proposals + +Currently, all proposals are [`v1.Proposal`][5]. Optimistic and multiple choice proposals require a different tally logic, but the rest of the proposal stays the same to not create other proposal types, `v1.Proposal` will have an extra field: + +```protobuf +// ProposalType enumerates the valid proposal types. +// All proposal types are v1.Proposal which have different voting periods or tallying logic. +enum ProposalType { + // PROPOSAL_TYPE_UNSPECIFIED defines no proposal type, which fallback to PROPOSAL_TYPE_STANDARD. + PROPOSAL_TYPE_UNSPECIFIED = 0; + // PROPOSAL_TYPE_STANDARD defines the type for a standard proposal. + PROPOSAL_TYPE_STANDARD = 1; + // PROPOSAL_TYPE_MULTIPLE_CHOICE defines the type for a multiple choice proposal. + PROPOSAL_TYPE_MULTIPLE_CHOICE = 2; + // PROPOSAL_TYPE_OPTIMISTIC defines the type for an optimistic proposal. + PROPOSAL_TYPE_OPTIMISTIC = 3; + // PROPOSAL_TYPE_EXPEDITED defines the type for an expedited proposal. + PROPOSAL_TYPE_EXPEDITED = 4; +} +``` + +Note, that expedited becomes a proposal type itself instead of a boolean on the `v1.Proposal` struct. + +> An expedited proposal is by design a standard proposal with a quicker voting period and higher threshold. When an expedited proposal fails, it gets converted to a standard proposal. + +An expedited optimistic proposal and an expedited multiple choice proposal do not make sense based on the definition above and is a proposal type instead of a proposal characteristic. + +#### Optimistic Proposal + +An optimistic proposal is a proposal that passes unless a threshold a NO votes is reached. + +Voter can only vote NO on the proposal. If the NO threshold is reached, the optimistic proposal is converted to a standard proposal. + +Two governance parameters will be in added [`v1.Params`][5] to support optimistic proposals: + +```protobuf +// optimistic_authorized_addreses is an optional governance parameter that limits the authorized accounts that can submit optimistic proposals +repeated string optimistic_authorized_addreses = 17 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + +// Optimistic rejected threshold defines at which percentage of NO votes, the optimistic proposal should fail and be converted to a standard proposal. +string optimistic_rejected_threshold = 18 [(cosmos_proto.scalar) = "cosmos.Dec"]; +``` + +#### Multiple Choice Proposal + +A multiple choice proposal is a proposal where the voting options can be defined by the proposer. + +The number of voting options will be limited to a maximum of 4. +A new vote option `SPAM` will be added and distinguished from those voting options. `SPAM` will be used to mark a proposal as spam and is explained further below. + +Multiple choice proposals, contrary to any other proposal type, cannot have messages to execute. They are only text proposals. + +Submitting a new multiple choice proposal will use a different message than the [`v1.MsgSubmitProposal`][5]. This is done in order to simplify the proposal submission and allow defining the voting options directly. + + +```protobuf +message MsgSubmitMultipleChoiceProposal { + repeated cosmos.base.v1beta1.Coin initial_deposit = 1; + string proposer = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + string metadata = 3; + string title = 4; + string summary = 5; + string option_one = 6; + string option_two = 7; + string option_three = 8; + string option_four = 9; +} +``` + +Voters can only vote on the defined options in the proposal. + +To maintain compatibility with the existing endpoints, the voting options will not be stored in the proposal itself and each option will be mapped to [`v1.VoteOption`][5]. A multiple choice proposal will be stored as a [`v1.Proposal`][5]. A query will be available for multiple choice proposal types to get the voting options. + +### Votes + +As mentioned above [multiple choice proposal](#multiple-choice-proposal) will introduce an additional vote option: `SPAM`. + +This vote option will be supported by all proposal types. +At the end of the voting period, if a proposal is voted as `SPAM`, it fails and its deposit is burned. + +`SPAM` differs from the `No with Veto` vote as its threshold is dynamic. +A proposal is marked as `SPAM` when the total of weighted votes for all options is lower than the amount of weighted vote on `SPAM` +(`spam` > `option_one + option_two + option_three + option_four` = proposal marked as spam). +This allows clear spam proposals to be marked as spam easily, even with low participation from validators. + +To avoid voters wrongfully voting down a proposal as `SPAM`, voters will be slashed `x`% (default 0%) of their voting stake if they voted `SPAM` on a proposal that wasn't a spam proposal. The parameter allows to incentivise voters to only vote `SPAM` on actual spam proposals and not use `SPAM` as a way to vote `No with Veto` with a different threshold. + +This leads to the addition of the following governance parameter in [`v1.Params`][5]: + +```protobuf +// burn_spam_amount defines the percentage of the voting stake that will be burned if a voter votes SPAM on a proposal that is not marked as SPAM. +string burn_spam_amount = 8 [(cosmos_proto.scalar) = "cosmos.Dec"]; +``` + +Additionally, the current vote options will be aliased to better accommodate the multiple choice proposal: + +```protobuf +// VoteOption enumerates the valid vote options for a given governance proposal. +enum VoteOption { + option allow_alias = true; + + // VOTE_OPTION_UNSPECIFIED defines a no-op vote option. + VOTE_OPTION_UNSPECIFIED = 0; + // VOTE_OPTION_ONE defines the first proposal vote option. + VOTE_OPTION_ONE = 1; + // VOTE_OPTION_YES defines the yes proposal vote option. + VOTE_OPTION_YES = 1; + // VOTE_OPTION_TWO defines the second proposal vote option. + VOTE_OPTION_TWO = 2; + // VOTE_OPTION_ABSTAIN defines the abstain proposal vote option. + VOTE_OPTION_ABSTAIN = 2; + // VOTE_OPTION_THREE defines the third proposal vote option. + VOTE_OPTION_THREE = 3; + // VOTE_OPTION_NO defines the no proposal vote option. + VOTE_OPTION_NO = 3; + // VOTE_OPTION_FOUR defines the fourth proposal vote option. + VOTE_OPTION_FOUR = 4; + // VOTE_OPTION_NO_WITH_VETO defines the no with veto proposal vote option. + VOTE_OPTION_NO_WITH_VETO = 4; + // VOTE_OPTION_SPAM defines the spam proposal vote option. + VOTE_OPTION_SPAM = 5; +} +``` + +The order does not change for a standard proposal (1 = yes, 2 = abstain, 3 = no, 4 = no with veto as it was) and the aliased enum can be used interchangeably. + +Updating vote options means updating [`v1.TallyResult`][5] as well. + +#### Tally + +Due to the vote option change, each proposal can have the same tallying method. + +However, chains may want to change the tallying function (weighted vote per voting power) of `x/gov` for a different algorithm (using a quadratic function on the voter stake, for instance). + +The custom tallying function can be passed to the `x/gov` keeper config: + +```go +type CalculateVoteResultsAndVotingPowerFn func( + ctx context.Context, + keeper Keeper, + proposalID uint64, + validators map[string]v1.ValidatorGovInfo, +) (totalVoterPower math.LegacyDec, results map[v1.VoteOption]math.LegacyDec, err error) +``` + +## Consequences + +Changing voting possibilities has a direct consequence for the clients. Clients, like Keplr or Mintscan, need to implement logic for multiple choice proposals. + +That logic consists of querying multiple choice proposals vote mapping their vote options. + +### Backwards Compatibility + +Legacy proposals (`v1beta1`) endpoints will not be supporting the new proposal types. + +Voting on a gov v1 proposal having a different type than [`standard` or `expedited`](#proposals) via the `v1beta1` will not be supported. +This is already the case for the expedited proposals. + +### Positive + +* Extended governance features +* Extended governance customization + +### Negative + +* Increase gov wiring complexity + +### Neutral + +* Increases the number of parameters available + +## Further Discussions + +This ADR starts the `x/gov` overhaul for the `cosmossdk.io/x/gov` v1.0.0 release. +Further internal improvements of `x/gov` will happen soon after, in order to simplify its state management and making gov calculation in a more "lazy"-fashion. + +Those improvements may change the tallying api. + +* https://github.com/cosmos/cosmos-sdk/issues/16270 + +## References + +* [https://github.com/cosmos/cosmos-sdk/issues/16270][1] +* [https://github.com/cosmos/cosmos-sdk/issues/17781][2] +* [https://github.com/cosmos/cosmos-sdk/issues/14403][3] +* [https://github.com/decentralists/DAO/issues/28][4] + +[1]: https://grants.osmosis.zone/blog/rfp-cosmos-sdk-governance-module-improvements +[2]: https://github.com/cosmos/cosmos-sdk/issues/17781 +[3]: https://github.com/cosmos/cosmos-sdk/issues/14403 +[4]: https://github.com/decentralists/DAO/issues/28 +[5]: https://buf.build/cosmos/cosmos-sdk/docs/main:cosmos.gov.v1 diff --git a/versioned_docs/version-0.52/build/architecture/adr-070-unordered-transactions.md b/versioned_docs/version-0.52/build/architecture/adr-070-unordered-transactions.md new file mode 100644 index 000000000..abcf63e76 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-070-unordered-transactions.md @@ -0,0 +1,337 @@ +# ADR 070: Unordered Transactions + +## Changelog + +* Dec 4, 2023: Initial Draft (@yihuang, @tac0turtle, @alexanderbez) +* Jan 30, 2024: Include section on deterministic transaction encoding + +## Status + +ACCEPTED + +## Abstract + +We propose a way to do replay-attack protection without enforcing the order of +transactions, without requiring the use of nonces. In this way, we can support +un-ordered transaction inclusion. + +## Context + +As of today, the nonce value (account sequence number) prevents replay-attack and +ensures the transactions from the same sender are included into blocks and executed +in sequential order. However it makes it tricky to send many transactions from the +same sender concurrently in a reliable way. IBC relayer and crypto exchanges are +typical examples of such use cases. + +## Decision + +We propose to add a boolean field `unordered` to transaction body to mark "un-ordered" +transactions. + +Un-ordered transactions will bypass the nonce rules and follow the rules described +below instead, in contrary, the default ordered transactions are not impacted by +this proposal, they'll follow the nonce rules the same as before. + +When an un-ordered transaction is included into a block, the transaction hash is +recorded in a dictionary. New transactions are checked against this dictionary for +duplicates, and to prevent the dictionary grow indefinitely, the transaction must +specify `timeout_timestamp` for expiration, so it's safe to removed it from the +dictionary after it's expired. + +The dictionary can be simply implemented as an in-memory golang map, a preliminary +analysis shows that the memory consumption won't be too big, for example `32M = 32 * 1024 * 1024` +can support 1024 blocks where each block contains 1024 unordered transactions. For +safety, we should limit the range of `timeout_timestamp` to prevent very long expiration, +and limit the size of the dictionary. + +### Transaction Format + +```protobuf +message TxBody { + ... + + bool unordered = 4; +} +``` + +### Replay Protection + +In order to provide replay protection, a user should ensure that the transaction's +TTL value is relatively short-lived but long enough to provide enough time to be +included in a block, e.g. ~10 minutes. + +We facilitate this by storing the transaction's hash in a durable map, `UnorderedTxManager`, +to prevent duplicates, i.e. replay attacks. Upon transaction ingress during `CheckTx`, +we check if the transaction's hash exists in this map or if the TTL value is stale, +i.e. before the current block time. If so, we reject it. Upon inclusion in a block +during `DeliverTx`, the transaction's hash is set in the map along with it's TTL +value. + +This map is evaluated at the end of each block, e.g. ABCI `Commit`, and all stale +transactions, i.e. transactions's TTL value who's now beyond the committed block, +are purged from the map. + +An important point to note is that in theory, it may be possible to submit an unordered +transaction twice, or multiple times, before the transaction is included in a block. +However, we'll note a few important layers of protection and mitigation: + +* Assuming CometBFT is used as the underlying consensus engine and a non-noop mempool + is used, CometBFT will reject the duplicate for you. +* For applications that leverage ABCI++, `ProcessProposal` should evaluate and reject + malicious proposals with duplicate transactions. +* For applications that leverage their own application mempool, their mempool should + reject the duplicate for you. +* Finally, worst case if the duplicate transaction is somehow selected for a block + proposal, 2nd and all further attempts to evaluate it, will fail during `DeliverTx`, + so worst case you just end up filling up block space with a duplicate transaction. + +```golang +type TxHash [32]byte + +const PurgeLoopSleepMS = 500 + +// UnorderedTxManager contains the tx hash dictionary for duplicates checking, +// and expire them when block production progresses. +type UnorderedTxManager struct { + // blockCh defines a channel to receive newly committed block time + blockCh chan time.Time + + mu sync.RWMutex + // txHashes defines a map from tx hash -> TTL value, which is used for duplicate + // checking and replay protection, as well as purging the map when the TTL is + // expired. + txHashes map[TxHash]time.Time +} + +func NewUnorderedTxManager() *UnorderedTxManager { + m := &UnorderedTxManager{ + blockCh: make(chan time.Time, 16), + txHashes: make(map[TxHash]time.Time), + } + + return m +} + +func (m *UnorderedTxManager) Start() { + go m.purgeLoop() +} + +func (m *UnorderedTxManager) Close() error { + close(m.blockCh) + m.blockCh = nil + return nil +} + +func (m *UnorderedTxManager) Contains(hash TxHash) bool{ + m.mu.RLock() + defer m.mu.RUnlock() + + _, ok := m.txHashes[hash] + return ok +} + +func (m *UnorderedTxManager) Size() int { + m.mu.RLock() + defer m.mu.RUnlock() + + return len(m.txHashes) +} + +func (m *UnorderedTxManager) Add(hash TxHash, expire time.Time) { + m.mu.Lock() + defer m.mu.Unlock() + + m.txHashes[hash] = expire +} + +// OnNewBlock send the latest block time to the background purge loop, which +// should be called in ABCI Commit event. +func (m *UnorderedTxManager) OnNewBlock(blockTime time.Time) { + m.blockCh <- blockTime +} + +// expiredTxs returns expired tx hashes based on the provided block time. +func (m *UnorderedTxManager) expiredTxs(blockTime time.Time) []TxHash { + m.mu.RLock() + defer m.mu.RUnlock() + + var result []TxHash + for txHash, expire := range m.txHashes { + if blockTime.After(expire) { + result = append(result, txHash) + } + } + + return result +} + +func (m *UnorderedTxManager) purge(txHashes []TxHash) { + m.mu.Lock() + defer m.mu.Unlock() + + for _, txHash := range txHashes { + delete(m.txHashes, txHash) + } +} + + +// purgeLoop removes expired tx hashes in the background +func (m *UnorderedTxManager) purgeLoop() error { + for { + latestTime, ok := m.batchReceive() + if !ok { + // channel closed + return + } + + hashes := m.expiredTxs(latestTime) + if len(hashes) > 0 { + m.purge(hashes) + } + } +} + + +// channelBatchRecv try to exhaust the channel buffer when it's not empty, +// and block when it's empty. +func channelBatchRecv[T any](ch <-chan *T) []*T { + item := <-ch // block if channel is empty + if item == nil { + // channel is closed + return nil + } + + remaining := len(ch) + result := make([]*T, 0, remaining+1) + result = append(result, item) + for i := 0; i < remaining; i++ { + result = append(result, <-ch) + } + + return result +} +``` + +### AnteHandler Decorator + +In order to facilitate bypassing nonce verification, we have to modify the existing +`IncrementSequenceDecorator` AnteHandler decorator to skip the nonce verification +when the transaction is marked as un-ordered. + +```golang +func (isd IncrementSequenceDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (sdk.Context, error) { + if tx.UnOrdered() { + return next(ctx, tx, simulate) + } + + // ... +} +``` + +In addition, we need to introduce a new decorator to perform the un-ordered transaction +verification and map lookup. + +```golang +const ( + // DefaultMaxTimeoutDuration defines the default maximum duration an un-ordered transaction + // can set. + DefaultMaxTimeoutDuration = time.Minute * 40 +) + +type DedupTxDecorator struct { + m *UnorderedTxManager + maxTimeoutDuration time.Time +} + +func (d *DedupTxDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (sdk.Context, error) { + // only apply to un-ordered transactions + if !tx.UnOrdered() { + return next(ctx, tx, simulate) + } + + headerInfo := d.env.HeaderService.HeaderInfo(ctx) + timeoutTimestamp := unorderedTx.GetTimeoutTimeStamp() + if timeoutTimestamp.IsZero() || timeoutTimestamp.Unix() == 0 { + return ctx, errorsmod.Wrap(sdkerrors.ErrInvalidRequest, "unordered transaction must have timeout_timestamp set") + } + if timeoutTimestamp.Before(headerInfo.Time) { + return ctx, errorsmod.Wrap(sdkerrors.ErrInvalidRequest, "unordered transaction has a timeout_timestamp that has already passed") + } + if timeoutTimestamp.After(headerInfo.Time.Add(d.maxTimeoutDuration)) { + return ctx, errorsmod.Wrapf(sdkerrors.ErrInvalidRequest, "unordered tx ttl exceeds %s", d.maxTimeoutDuration.String()) + } + + // in order to create a deterministic hash based on the tx, we need to hash the contents of the tx with signature + // Get a Buffer from the pool + buf := bufPool.Get().(*bytes.Buffer) + // Make sure to reset the buffer + buf.Reset() + + // Use the buffer + for _, msg := range tx.GetMsgs() { + // loop through the messages and write them to the buffer + // encoding the msg to bytes makes it deterministic within the state machine. + // Malleability is not a concern here because the state machine will encode the transaction deterministically. + bz, err := proto.Marshal(msg) + if err != nil { + return ctx, errorsmod.Wrap(sdkerrors.ErrInvalidRequest, "failed to marshal message") + } + + buf.Write(bz) + } + + // check for duplicates + // check for duplicates + if d.txManager.Contains(txHash) { + return ctx, errorsmod.Wrap(sdkerrors.ErrInvalidRequest, "tx %X is duplicated") + } + + if d.env.TransactionService.ExecMode(ctx) == transaction.ExecModeFinalize { + // a new tx included in the block, add the hash to the unordered tx manager + d.txManager.Add(txHash, ttl) + } + + return next(ctx, tx, simulate) +} +``` + +### Transaction Hashes + +It is absolutely vital that transaction hashes are deterministic, i.e. transaction +encoding is not malleable. If a given transaction, which is otherwise valid, can +be encoded to produce different hashes, which reflect the same valid transaction, +then a duplicate unordered transaction can be submitted and included in a block. + +In order to prevent this, the decoded transaction contents is taken. Starting with the content of the transaction we marshal the transaction in order to prevent a client reordering the transaction. Next we include the gas and timeout timestamp as part of the identifier. All these fields are signed over in the transaction payload. If one of them changes the signature will not match the transaction. + +### State Management + +On start up, the node needs to ensure the TxManager's state contains all un-expired +transactions that have been committed to the chain. This is critical since if the +state is not properly initialized, the node will not reject duplicate transactions +and thus will not provide replay protection, and will likely get an app hash mismatch error. + +We propose to write all un-expired unordered transactions from the TxManager's to +file on disk. On start up, the node will read this file and re-populate the TxManager's +map. The write to file will happen when the node gracefully shuts down on `Close()`. + +Note, this is not a perfect solution, in the context of store v1. With store v2, +we can omit explicit file handling altogether and simply write the all the transactions +to non-consensus state, i.e State Storage (SS). + +Alternatively, we can write all the transactions to consensus state. + +## Consequences + +### Positive + +* Support un-ordered and concurrent transaction inclusion. + +### Negative + +* Requires additional storage overhead and management of processed unordered + transactions that exist outside of consensus state. + +## References + +* https://github.com/cosmos/cosmos-sdk/issues/13009 diff --git a/versioned_docs/version-0.52/build/architecture/adr-073-indexer.md b/versioned_docs/version-0.52/build/architecture/adr-073-indexer.md new file mode 100644 index 000000000..c751c9b24 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-073-indexer.md @@ -0,0 +1,139 @@ +# ADR 073: Built-in In-process Indexer + +## Changelog + +* 2024-06-03: Initial draft (@aaronc) + +## Status + +Accepted Not Implemented + +## Abstract + +This ADR proposes developing a built-in query indexer framework for Cosmos SDK applications that leverages `collections` and `orm` schemas to index on-chain state and events into a PostgreSQL database, or another database if applications prefer. This indexer should be designed to be run in-process with the Cosmos SDK node with guaranteed delivery and provide a full-featured query interface for clients. + +## Context + +Historically in Cosmos SDK applications, the needs of clients have been intertwined with the needs of state machine +logic. While the state machine logic itself may only require certain pieces of state data, if client applications +need additional data, this data has usually been added to the state. One example is adding additional secondary +indexes and query endpoints that are only needed for client queries, such as the bank reverse denom index. This +can add to state bloat and complicate the state machine logic. + +While Cosmos SDK applications try their best to support client needs, the query experience provided by the gRPC +queries provided by modules is always sub-par compared to a real database dedicated to queries. On-chain queries +tend to be slow, and it is impossible to optimize query performance by adding additional indexes without burdening +on-chain state. Most sophisticated client applications build their own off-chain indexers to provide the queries they +need, but these are often highly application specific and not reusable. The Cosmos SDK has attempted to provide some +additional infrastructure to support indexers such as [ADR 038: State Listening](./adr-038-state-listening.md), but +this still requires the creation of lots of custom infrastructure. + +Now that the [collections](./adr-062-collections-state-layer.md) and [orm](./adr-055-orm.md) frameworks exist and many SDK modules have been migrated to use them, we can do better by providing a built-in indexer that is easy to deploy, handles 90% of client query needs out of the box, and is easy to extend for the remaining 10% of needs. + +In considering the design of built-in support, in discussions we have considered the needs of UI application developers, node operators and application and module developers. The following user stories have been identified as _ideal_ features, with the understanding that the SDK can likely only provide a subset of these features, but should aim for the best balance. + +When considering the needs of developers building UI applications, the following desirable features have been identified: + +* U1: quickly querying the current on-chain state with any sort of filtering, sorting and pagination the client desires with the ability to optimize queries with custom indexes. +* U2: data should be available in client friendly formats, for instance even if addresses are stored as bytes in state, clients should be available to view and query bech32 encoded addresses. +* U3: state that was pruned from on-chain state to save space but is still otherwise valid (such as completed proposals and votes) should be queryable. +* U4: a full log of changes for each entity should be available, including: + * U4.1: semantic information about each change (i.e. events), i.e. not just balance went up or down, but why (transfer, withdraw rewards, etc.) + * U4.2: links to the block, transaction and message which produced the change + * U4.3: the ability to see the version of the entity at the time of the change +* U5: the query index should be consistent, meaning that UI applications should not need to be concerned with the possibility that the index has missed indexing some blocks or events + +When considering the needs of node operators, the following desirable features have been identified: + +* N1: nodes shouldn't _need_ to store indexes or events that are only needed for client queries, i.e. it should be possible to run a lean node +* N2: it should be possible to quickly spin up a node with the query index without tons of custom infrastructure and coding, i.e. this should be strictly configuration change and a bit of devops. It should be possible to configure the index to contain just state, just events, or both. +* N3: it should be possible to build up an almost complete query index starting from any point in time without necessarily needing to replay a full chain's history (with the understanding that some historical data may be missing) + +Finally, when considering the needs of application and module developers, the following desirable features have been identified: + +* A1: enabling support for query indexing should require minimal tweaking to app boilerplate and configurable by node operators (off by default). +* A2: it should be possible to index applications built with older versions of the SDK (including v0.47 and v0.50, possibly earlier) without major changes. +* A3: module developers should mostly just need to use the collections and orm frameworks to get their data indexed, with minimal additional work. +* A4: there should be hooks available to module developers for extending the built-in indexing functionality. +* A5: the built-in indexer framework should allow newer modules to entirely or mostly skip building custom gRPC queries to satisfy client needs and modules should no longer need to add indexes only for client use, i.e. the indexer framework should essentially be a complete replacement for the gRPC query system (with inter-module query needs being considered a separate concern). + +## Decision + +We have decided to build a built-in query indexer for Cosmos SDK applications that is structured as two components: + +1. a state decoder that takes data from `collections` and `orm` and provides an indexable version of that data to an actual indexer +2. a PostgreSQL-based indexer implementation that can be run in-process with the Cosmos SDK node + +This infrastructure should be built upon the existing [ADR 038: State Listening](./adr-038-state-listening.md) functionality as much as possible and integrating it into existing apps should require minimal changes. These components should work with any SDK application that is built against v0.47 or v0.50 and possibly earlier versions of the SDK. + +### State Decoder + +The state decoder framework should expose a way for modules using `collections` or `orm` to expose their state schemas so that the state decoder can take data exposed by state listening and decode it into logical packets which can consumed by an indexer. It should define an interface that an indexer implements to consume these packets. This framework should be designed to run in-process within a Cosmos SDK node with guaranteed delivery and consistency (satisfying `U5`). While concurrency should be used to optimize performance, there should be a guarantee that if a block is committed, that it was also indexed. This framework should also allow indexers to consume block, transaction, and event data and optionally index these. + +At its core, the state decoder framework shouldn't make any assumptions about what database the indexer is targeting. While a built-in indexer will be provided (discussed below), a developer may choose to write an indexer targeting another database and the state decoder framework should be flexible enough to support this. + +The state decoder should provide hooks for handling custom data types (to support `U2`), in particular addresses so that indexers can store these in their bech32 string format when they are actually stored as bytes in state. + +Some changes to `collections` will be needed in order to expose for decoding functionality and saner naming of map keys and values. These can likely be made in a non-breaking way and any features that allow for better key and value naming would be opt-in. For both `collections` and `orm`, we will need a lightweight way to expose these schemas on `AppModule`s. + +To support `U3`, `collections` and `orm` can add "prune" methods that allow the indexer framework to distinguish pruning from deletion so that the query index could for instance retain historical proposals and votes from `x/gov` while these get deleted in state. Alternatively, configuration flags could be used to instruct the indexer to retain certain types of data - these could be configured at the module or node level. + +In order to support indexing from any height (`N3`), the state decoder will need the ability to read the full state at the height at which indexing started and also keep track of which blocks have been indexed. + +### PostgreSQL Indexer + +PostgreSQL has been chosen as the target database for the built-in indexer component because it is widely used, has a rich feature set and is a favorite choice in the open-source community. It is easy to deploy and there are fully managed hosting services that can be used. In addition, the PostgreSQL community has a number of mature frameworks that expose a complete REST or GraphQL query interface for clients with zero code such as [PostgREST](https://postgrest.org/), [PostGraphile](https://www.graphile.org/postgraphile/), [Hasura](https://hasura.io) and [Supabase](https://supabase.com). By combining a PostgreSQL database with one of these "Backend as a Service" frameworks, Web 2.0 client applications can basically be built without any special backend code other than the database schema itself. We can take advantage of this functionality to provide a full-featured query interface for web clients with minimal effort as soon as we can get the data into PostgreSQL. + +The PostgreSQL indexer should provide sane default mappings of all `collections` and `orm` types to SQL tables. It should also allow for hooks in modules to write migrations in the SQL schema whenever there are migrations in a module's state so that these can stay in sync. + +Blocks, transactions and events should be stored as rows in PostgreSQL tables when this is enabled by the node operator. This data should come directly from the state decoder framework without needing to go through Comet. + +For a full batteries included, client friendly query experience, a GraphQL endpoint should be exposed in the HTTP server for any PostgreSQL database that has the [Supabase pg_graphql](https://github.com/supabase/pg_graphql) extension enabled. `pg_graphql` will expose rich GraphQL queries for all PostgreSQL tables with zero code that support filtering, pagination, sorting and traversing foreign key references. (Support for defining foreign keys with `collections` and `orm` could be added in the future to take advantage of this). In addition, a [GraphiQL](https://github.com/graphql/graphiql) query explorer endpoint can be exposed to simplify client development. + +With this setup, a node operator would only need to 1) setup a PostgreSQL database with the `pg_graphql` extension and 2) enable the query indexer in the configuration in order to provide a full-featured query experience to clients. Because PostgreSQL is a full-featured database, node operators can enable any sort of custom indexes or views that are needed for their specific application with no need for this to affect the state machine or any other nodes. + + +## Alternatives + +The following alternatives were considered: + +* support any SQL database not just PostgreSQL using a framework like [GORM](https://gorm.io/). While this would be more flexible, it would be slower, require heavy usage of golang reflection and might limit how much we can take advantage of PostgreSQL's unique features for little benefit (the assumption being that most users would choose PostgreSQL anyway and or be happy enough that we made that choice). +* don't support any specific database, but just build the decoder framework. While this would simplify our efforts in the short-term, it still doesn't provide a full-featured solution and requires others to build out the key infrastructure similar to [ADR 038](adr-038-state-listening.md). This limbo state would not allow the SDK to definitely make key optimizations to state layout and simple the task of module development in a definitive way by providing a full replacement for gRPC client queries. +* target a database with full historical query support like [Datomic](https://www.datomic.com). This requires a bunch of custom infrastructure and would expose a powerful, yet unfamiliar query language to users. +* advocate an event sourcing design and build an event sourcing based indexer which would recompute state based on the event log. This is also discussed more below and is considered a complementary idea that can provide better support for historical change logs. Considering event sourcing as a full alternative to a state-based indexer, however, would require a lot of module refactoring, likely custom code, and wouldn't take advantage of the work we've already done in supporting state schemas through `collections` and `orm`. +* build a full-featured out-of-process indexer based on ADR 038 and changelog files. This was another design initially considered, but it requires additional infrastructure and processes. In particular, it also requires a full decodable schema for `collections` which at the moment is fairly complex. It is easier to use the `collections` schemas already in the binary to do indexing rather than create a whole schema definition language for a separate process to consume. Also we want to provide a more batteries-included experience for users and in particular satisfy `N2`. If creating a full query index is easier, it makes everyone's life easier. +* build a GraphQL client on top of the existing state store. This was considered, but it would be slow and not provide the full-featured query experience that a real database can provide. It would still require client only indexes in state, it would be hard to configure custom indexes and views, and would likely require building or reusing a full query planner. In the end, using a real database is easier to build and provides a better experience for clients. + +## Consequences + +### Backwards Compatibility + +We believe that these features can be built to target SDK versions v0.47, v0.50 and possibly earlier without breaking changes. + +### Positive + +Considering the user stories identified in the context section, we believe that the proposed design can meet all the user stories identified, except `U4`. Overall, the proposed design should provide a full query experience that is in most ways better than what is provided by the existing gRPC query infrastructure, is easy to deploy and manage, and easy to extend for custom needs. It also simplifies the job of writing a module because module developers mostly do not need to worry about writing query endpoints or other client concerns besides making sure that the design and naming of `collections` and `orm` schemas is client friendly. + +Also, because we are separating the design into decoder and indexer components, it should be possible to write indexers targeting other databases besides PostgreSQL using the decoder framework. While the built-in PostgreSQL indexer should provide a good battery-included experience for most users, this design also supports users wanting to target other databases. + +### Negative + +If module developers choose to deprecate support for some or all gRPC queries then this will be a breaking change for clients. The resulting query experience should be better, but it will require significant changes. This ADR doesn't advocate deprecating these queries, but it could encourage developers to go in that direction. To mitigate this concern, we encourage module authors to consider the client developer impact as they make decisions about deleting existing indexes and queries. + +Also, this design does impose the requirement that module developers use `collections` and `orm` to get their data indexed. While we believe that these frameworks are the best way to structure state in the SDK, some developers may not want to refactor their code to use them. This design does not provide a way to index state that is not structured with `collections` or `orm`, although that support could be added in the future to the decoder framework. + +Furthermore, this design does require that node operators that want to support the new infrastructure run a PostgreSQL database. While we believe that this will actually be better from an infrastructure perspective - PostgreSQL should be able to serve queries significantly faster than the state machine can - it does require additional infrastructure and knowledge for query node operators. + +### Neutral + +Regarding `U4`, if event indexing is enabled, then it could be argued that `U4.1` is met, but whether this is _actually_ met depends heavily on how well modules structure their events. `U4.2` and `U4.3` likely require a full archive node which is out of scope of this design. One alternative which satisfies `U4.2` and `U4.3` would be targeting a database with historical data, such as [Datomic](https://www.datomic.com). However, this requires some pretty custom infrastructure and exposes a query interface which is unfamiliar for most users. Also, if events aren't properly structured, `U4.1` still really isn't met. A simpler alternative would be for module developers to follow an event sourcing design more closely so that historical state for any entity could be derived from the history of events. This event sourcing could even be done in client applications themselves by querying all the events relevant to an entity (such as a balance). This topic may be covered in more detail in a separate document in the future and may come down to best practices combined, with maybe a bit of framework support. However, in general satisfying `U4` (other than support event indexing) is mostly concerned out of the scope of the design, because it is either much more complex from an infrastructure perspective (full archive node or custom database like Datomic) or easy to solve with good event design. + +## Further Discussions + +Further discussions can take place on GitHub as needed. + +## References + +* [ADR 038: State Listening](./adr-038-state-listening.md) +* [ADR 055: ORM](./adr-055-orm.md) +* [ADR 062: Collections](./adr-062-collections-state-layer.md) diff --git a/versioned_docs/version-0.52/build/architecture/adr-074-implicit-msg-signers.md b/versioned_docs/version-0.52/build/architecture/adr-074-implicit-msg-signers.md new file mode 100644 index 000000000..37853cc2d --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-074-implicit-msg-signers.md @@ -0,0 +1,131 @@ +# ADR 074: Messages with Implicit Signers + +## Changelog + +* 2024-06-10: Initial draft + +## Status + +PROPOSED Not Implemented + +## Abstract + +This ADR introduces a new `MsgV2` standard where the signer of the message is implied by the +credentials of the party sending it, and unlike the current design not part of the message body. +This can be used for both simple inter-module message passing and simpler messages in transactions. + +## Context + +Historically operations in the SDK have been modelled with the `sdk.Msg` interface and +the account signing the message has to be explicitly extracted from the body of `Msg`s. +Originally this was via a `GetSigners` method on the `sdk.Msg` interface which returned +instances of `sdk.AccAddress` which itself relied on a global variable for decoding +the addresses from bech32 strings. This was a messy situation. In addition, the implementation +for `GetSigners` was different for each `Msg` type and clients would need to do a custom +implementation for each `Msg` type. These were improved somewhat with the introduction of +the `cosmos.msg.v1.signer` protobuf option which allowed for a more standardised way of +defining who the signer of a message was and its implementation in the `x/tx` module which +extracts signers dynamically and allowed removing the dependency on the global bech32 +configuration. + +Still this design introduces a fair amount of complexity. For instance, inter-module message +passing ([ADR 033](./adr-033-protobuf-inter-module-comm.md)) has been in discussion for years +without much progress and one of the main blockers is figuring out how to properly authenticate +messages in a performant and consistent way. With embedded message signers there will always need +to be a step of extracting the signer and then checking with the module sending is actually +authorized to perform the operation. With dynamic signer extraction, although the system is +more consistent, more performance overhead is introduced. In any case why should an inter-module +message passing system need to do so much conversion, parsing, etc. just to check if a message +is authenticated? In addition, we have the complexity where modules can actually have many valid +addresses. How are we to accommodate this? Should there be a lookup into `x/auth` to check if an +address belongs to a module or not? All of these thorny questions are delaying the delivery of +inter-module message passing because we do not want an implementation that is overly complex. +There are many use cases for inter-module message passing which are still relevant, the most +immediate of which is a more robust denom management system in `x/bank` `v2` which is being explored +in [ADR 071](https://github.com/cosmos/cosmos-sdk/pull/20316). + +## Alternatives + +Alternatives that have been considered are extending the current `x/tx` signer extraction system +to inter-module message passing as defined in [ADR 033](./adr-033-protobuf-inter-module-comm.md). + +## Decision + +We have decided to introduce a new `MsgV2` standard whereby the signer of the message is implied +by the credentials of the party sending it. These messages will be distinct from the existing messages +and define new semantics with the understanding that signers are implicit. + +In the case of messages passed internally by a module or `x/account` instance, the signer of a message +will simply be the main root address of the module or account sending the message. An interface for +safely passing such messages to the message router will need to be defined. + +In the case of messages passed externally in transactions, `MsgV2` instances will need to be wrapped +in a `MsgV2` envelope: +```protobuf +message MsgV2 { + string signer = 1; + google.protobuf.Any msg = 2; +} +``` + +Because the `cosmos.msg.v1.signer` annotation is required currently, `MsgV2` types should set the message option +`cosmos.msg.v2.is_msg` to `true` instead. + +Here is an example comparing a v1 an v2 message: +```protobuf +// v1 +message MsgSendV1 { + option (cosmos.msg.v1.signer) = "from_address"; + string from_address = 1 ; + string to_address = 2; + repeated Coin amount = 3; +} + +// v2 +message MsgSendV2 { + option (cosmos.msg.v2.is_msg) = true; + // from address is implied by the signer + string to_address = 1; + repeated Coin amount = 2; +} +``` + +Modules defining handlers for `MsgV2` instances will need to extract the sender from the `context.Context` that is +passed in. An interface in `core` which will be present on the `appmodule.Environment` will be defined for this purpose: +```go +type GetSenderService interface { + GetSender(ctx context.Context) []byte +} +``` + +Sender addresses that are returned by the service will be simple `[]byte` slices and any bech32 conversion will be +done by the framework. + +## Consequences + +### Backwards Compatibility + +This design does not depreciate the existing method of embedded signers in `Msg`s and is totally compatible with it. + +### Positive + +* Allows for a simple inter-module communication design which can be used soon for the `bank` `v2` redesign. +* Allows for simpler client implementations for messages in the future. + +### Negative + +* There will be two message designs and developers will need to pick between them. + +### Neutral + +## Further Discussions + +Two possible directions that have been proposed are: +1. allowing for the omission of the `cosmos.msg.v2.is_msg` option and assuming any `Msg`s registered that do not include `cosmos.msg.v1.signer` are `MsgV2` instances. The pitfall is that this could be incorrect if `Msg` v1 behavior is actually decided but the user forgot the `cosmos.msg.v1.signer` option. +2. allow `Msg` v1 instances to be wrapped in a `MsgV2` envelope as well to simplify things client-side. In this scenario we would need to either a) check that the signer in the envelope and the signer in the message are the same or b) allow the signer in the message to be empty and then set it inside the state machine before it reaches the module. While this may be easier for some clients, it may introduce unexpected behavior with Ledger signing via Amino JSON or SIGN_MODE_TEXTUAL. + +Both of these are seem as quality of life improvements for some users, but not strictly necessary and could have some pitfalls so further discussion is needed. + +## References + +* [ADR 033](./adr-033-protobuf-inter-module-comm.md) \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/architecture/adr-template.md b/versioned_docs/version-0.52/build/architecture/adr-template.md new file mode 100644 index 000000000..04b0450c6 --- /dev/null +++ b/versioned_docs/version-0.52/build/architecture/adr-template.md @@ -0,0 +1,83 @@ +# ADR {ADR-NUMBER}: {TITLE} + +## Changelog + +* {date}: {changelog} + +## Status + +{DRAFT | PROPOSED} Not Implemented + +> Please have a look at the [PROCESS](./PROCESS.md#adr-status) page. +> Use DRAFT if the ADR is in a draft stage (draft PR) or PROPOSED if it's in review. + +## Abstract + +> "If you can't explain it simply, you don't understand it well enough." Provide +> a simplified and layman-accessible explanation of the ADR. +> A short (~200 word) description of the issue being addressed. + +## Context + +> This section describes the forces at play, including technological, political, +> social, and project local. These forces are probably in tension, and should be +> called out as such. The language in this section is value-neutral. It is simply +> describing facts. It should clearly explain the problem and motivation that the +> proposal aims to resolve. +> {context body} + +## Alternatives + +> This section describes alternative designs to the chosen design. This section +> is important and if an adr does not have any alternatives then it should be +> considered that the ADR was not thought through. + +## Decision + +> This section describes our response to these forces. It is stated in full +> sentences, with active voice. "We will ..." +> {decision body} + +## Consequences + +> This section describes the resulting context, after applying the decision. All +> consequences should be listed here, not just the "positive" ones. A particular +> decision may have positive, negative, and neutral consequences, but all of them +> affect the team and project in the future. + +### Backwards Compatibility + +> All ADRs that introduce backwards incompatibilities must include a section +> describing these incompatibilities and their severity. The ADR must explain +> how the author proposes to deal with these incompatibilities. ADR submissions +> without a sufficient backwards compatibility treatise may be rejected outright. + +### Positive + +> {positive consequences} + +### Negative + +> {negative consequences} + +### Neutral + +> {neutral consequences} + +## Further Discussions + +> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a +> summary of issues to be solved in future iterations (usually referencing comments +> from a pull-request discussion). +> +> Later, this section can optionally list ideas or improvements the author or +> reviewers found during the analysis of this ADR. + +## Test Cases [optional] + +Test cases for an implementation are mandatory for ADRs that are affecting consensus +changes. Other ADRs can choose to include links to test cases if applicable. + +## References + +* {reference link} diff --git a/versioned_docs/version-0.52/build/building-apps/00-app-go.md b/versioned_docs/version-0.52/build/building-apps/00-app-go.md new file mode 100644 index 000000000..5a0524f3b --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/00-app-go.md @@ -0,0 +1,14 @@ +--- +sidebar_position: 1 +--- + +# Overview of `app.go` + +This section is intended to provide an overview of the `SimApp` `app.go` file and is still a work in progress. +For now please instead read the [tutorials](https://tutorials.cosmos.network) for a deep dive on how to build a chain. + +## Complete `app.go` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go +``` diff --git a/versioned_docs/version-0.52/build/building-apps/01-app-go-v2.md b/versioned_docs/version-0.52/build/building-apps/01-app-go-v2.md new file mode 100644 index 000000000..d77899d26 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/01-app-go-v2.md @@ -0,0 +1,154 @@ +--- +sidebar_position: 1 +--- + +# Overview of `app_di.go` + +:::note Synopsis + +The Cosmos SDK allows much easier wiring of an `app.go` thanks to App Wiring and [`depinject`](../packages/01-depinject.md). +Learn more about the rationale of App Wiring in [ADR-057](../architecture/adr-057-app-wiring.md). + +::: + +:::note Pre-requisite Readings + +* [ADR 057: App Wiring](../architecture/adr-057-app-wiring.md) +* [Depinject Documentation](../packages/01-depinject.md) +* [Modules depinject-ready](../building-modules/15-depinject.md) + +::: + +This section is intended to provide an overview of the `SimApp` `app_di.go` file with App Wiring. + +## `app_config.go` + +The `app_config.go` file is the single place to configure all modules parameters. + +1. Create the `AppConfig` variable: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_config.go#L103 + ``` + +2. Configure the `runtime` module: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_config.go#L103-L167 + ``` + +3. Configure the modules defined in the `PreBlocker`, `BeginBlocker` and `EndBlocker` and the `tx` module: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_config.go#L112-L129 + ``` + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_config.go#L200-L203 + ``` + +### Complete `app_config.go` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_config.go +``` + +### Alternative formats + +:::tip +The example above shows how to create an `AppConfig` using Go. However, it is also possible to create an `AppConfig` using YAML, or JSON. +The configuration can then be embed with `go:embed` and read with [`appconfig.LoadYAML`](https://pkg.go.dev/cosmossdk.io/core/appconfig#LoadYAML), or [`appconfig.LoadJSON`](https://pkg.go.dev/cosmossdk.io/core/appconfig#LoadJSON), in `app_di.go`. + +```go +//go:embed app_config.yaml +var ( + appConfigYaml []byte + appConfig = appconfig.LoadYAML(appConfigYaml) +) +``` + +::: + +```yaml +modules: + - name: runtime + config: + "@type": cosmos.app.runtime.v1alpha1.Module + app_name: SimApp + begin_blockers: [staking, auth, bank] + end_blockers: [bank, auth, staking] + init_genesis: [bank, auth, staking] + - name: auth + config: + "@type": cosmos.auth.module.v1.Module + bech32_prefix: cosmos + - name: bank + config: + "@type": cosmos.bank.module.v1.Module + - name: staking + config: + "@type": cosmos.staking.module.v1.Module + - name: tx + config: + "@type": cosmos.tx.config.v1.Config +``` + +A more complete example of `app.yaml` can be found [here](https://github.com/cosmos/cosmos-sdk/blob/91b1d83f1339e235a1dfa929ecc00084101a19e3/simapp/app.yaml). + +## `app_di.go` + +`app_di.go` is the place where `SimApp` is constructed. `depinject.Inject` facilitates that by automatically wiring the app modules and keepers, provided an application configuration `AppConfig` is provided. `SimApp` is constructed, when calling the injected `*runtime.AppBuilder`, with `appBuilder.Build(...)`. +In short `depinject` and the [`runtime` package](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/runtime) abstract the wiring of the app, and the `AppBuilder` is the place where the app is constructed. [`runtime`](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/runtime) takes care of registering the codecs, KV store, subspaces and instantiating `baseapp`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_v2.go#L101-L245 +``` + +:::warning +When using `depinject.Inject`, the injected types must be pointers. +::: + +### Advanced Configuration + +In advanced cases, it is possible to inject extra (module) configuration in a way that is not (yet) supported by `AppConfig`. +In this case, use `depinject.Configs` for combining the extra configuration and `AppConfig`, and `depinject.Supply` to providing that extra configuration. +More information on how work `depinject.Configs` and `depinject.Supply` can be found in the [`depinject` documentation](https://pkg.go.dev/cosmossdk.io/depinject). + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_v2.go#L114-L146 +``` + +### Registering non app wiring modules + +It is possible to combine app wiring / depinject enabled modules with non app wiring modules. +To do so, use the `app.RegisterModules` method to register the modules on your app, as well as `app.RegisterStores` for registering the extra stores needed. + +```go +// .... +app.App = appBuilder.Build(db, traceStore, baseAppOptions...) + +// register module manually +app.RegisterStores(storetypes.NewKVStoreKey(example.ModuleName)) +app.ExampleKeeper = examplekeeper.NewKeeper(app.appCodec, app.AccountKeeper.AddressCodec(), runtime.NewKVStoreService(app.GetKey(example.ModuleName)), authtypes.NewModuleAddress(govtypes.ModuleName).String()) +exampleAppModule := examplemodule.NewAppModule(app.ExampleKeeper) +if err := app.RegisterModules(&exampleAppModule); err != nil { + panic(err) +} + +// .... +``` + +:::warning +When using AutoCLI and combining app wiring and non app wiring modules. The AutoCLI options should be manually constructed instead of injected. +Otherwise it will miss the non depinject modules and not register their CLI. +::: + +### Complete `app_di.go` + +:::tip +Note that in the complete `SimApp` `app_di.go` file, testing utilities are also defined, but they could as well be defined in a separate file. +::: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_v2.go +``` diff --git a/versioned_docs/version-0.52/build/building-apps/02-app-mempool.md b/versioned_docs/version-0.52/build/building-apps/02-app-mempool.md new file mode 100644 index 000000000..9ef17bec6 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/02-app-mempool.md @@ -0,0 +1,96 @@ +--- +sidebar_position: 1 +--- + +# Application Mempool + +:::note Synopsis +This sections describes how the app-side mempool can be used and replaced. +::: + +Since `v0.47` the application has its own mempool to allow much more granular +block building than previous versions. This change was enabled by +[ABCI 1.0](https://docs.cometbft.com/v0.37/spec/abci/). +Notably it introduces the `PrepareProposal` and `ProcessProposal` steps of ABCI++. For more information please see [here](../abci/00-introduction.md) + +:::note Pre-requisite Readings + +* [BaseApp](../../learn/advanced/00-baseapp.md) +* [Abci](../abci/00-introduction.md) + +::: + +## Mempool + +* Before we delve into `PrepareProposal` and `ProcessProposal`, let's first walk through the mempool concepts. + +There are countless designs that an application developer can write for a mempool, the SDK opted to provide only simple mempool implementations. +Namely, the SDK provides the following mempools: + +* [No-op Mempool](#no-op-mempool) +* [Sender Nonce Mempool](#sender-nonce-mempool) +* [Priority Nonce Mempool](#priority-nonce-mempool) + +The default SDK is a [No-op Mempool](#no-op-mempool), but it can be replaced by the application developer in [`app.go`](./01-app-go-v2.md): + +```go +nonceMempool := mempool.NewSenderNonceMempool() +mempoolOpt := baseapp.SetMempool(nonceMempool) +baseAppOptions = append(baseAppOptions, mempoolOpt) +``` + +### No-op Mempool + +A no-op mempool is a mempool where transactions are completely discarded and ignored when BaseApp interacts with the mempool. +When this mempool is used, it is assumed that an application will rely on CometBFT's transaction ordering defined in `RequestPrepareProposal`, +which is FIFO-ordered by default. + +> Note: If a NoOp mempool is used, PrepareProposal and ProcessProposal both should be aware of this as +> PrepareProposal could include transactions that could fail verification in ProcessProposal. + +### Sender Nonce Mempool + +The nonce mempool is a mempool that keeps transactions from an sorted by nonce in order to avoid the issues with nonces. +It works by storing the transaction in a list sorted by the transaction nonce. When the proposer asks for transactions to be included in a block it randomly selects a sender and gets the first transaction in the list. It repeats this until the mempool is empty or the block is full. + +It is configurable with the following parameters: + +#### MaxTxs + +It is an integer value that sets the mempool in one of three modes, *bounded*, *unbounded*, or *disabled*. + +* **negative**: Disabled, mempool does not insert new transaction and return early. +* **zero**: Unbounded mempool has no transaction limit and will never fail with `ErrMempoolTxMaxCapacity`. +* **positive**: Bounded, it fails with `ErrMempoolTxMaxCapacity` when `maxTx` value is the same as `CountTx()` + +#### Seed + +Set the seed for the random number generator used to select transactions from the mempool. + +### Priority Nonce Mempool + +The [priority nonce mempool](https://github.com/cosmos/cosmos-sdk/blob/main/types/mempool/priority_nonce_spec.md) is a mempool implementation that stores txs in a partially ordered set by 2 dimensions: + +* priority +* sender-nonce (sequence number) + +Internally it uses one priority ordered [skip list](https://pkg.go.dev/github.com/huandu/skiplist) and one skip list per sender ordered by sender-nonce (sequence number). When there are multiple txs from the same sender, they are not always comparable by priority to other sender txs and must be partially ordered by both sender-nonce and priority. + +It is configurable with the following parameters: + +#### MaxTxs + +It is an integer value that sets the mempool in one of three modes, *bounded*, *unbounded*, or *disabled*. + +* **negative**: Disabled, mempool does not insert new transaction and return early. +* **zero**: Unbounded mempool has no transaction limit and will never fail with `ErrMempoolTxMaxCapacity`. +* **positive**: Bounded, it fails with `ErrMempoolTxMaxCapacity` when `maxTx` value is the same as `CountTx()` + +#### Callback + +The priority nonce mempool provides mempool options allowing the application sets callback(s). + +* **OnRead**: Set a callback to be called when a transaction is read from the mempool. +* **TxReplacement**: Sets a callback to be called when duplicated transaction nonce detected during mempool insert. Application can define a transaction replacement rule based on tx priority or certain transaction fields. + +More information on the SDK mempool implementation can be found in the [godocs](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/types/mempool). diff --git a/versioned_docs/version-0.52/build/building-apps/03-app-upgrade.md b/versioned_docs/version-0.52/build/building-apps/03-app-upgrade.md new file mode 100644 index 000000000..b9c487534 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/03-app-upgrade.md @@ -0,0 +1,218 @@ +--- +sidebar_position: 1 +--- + +# Application Upgrade + +:::note +This document describes how to upgrade your application. If you are looking specifically for the changes to perform between SDK versions, see the [SDK migrations documentation](https://docs.cosmos.network/main/migrations/intro). +::: + +:::warning +This section is currently incomplete. Track the progress of this document [here](https://github.com/cosmos/cosmos-sdk/issues/11504). +::: + +:::note Pre-requisite Readings + +* [`x/upgrade` Documentation](https://docs.cosmos.network/main/modules/upgrade) + +::: + +## General Workflow + +Let's assume we are running v0.38.0 of our software in our testnet and want to upgrade to v0.40.0. +How would this look in practice? First of all, we want to finalize the v0.40.0 release candidate +and then install a specially named upgrade handler (eg. "testnet-v2" or even "v0.40.0"). An upgrade +handler should be defined in a new version of the software to define what migrations +to run to migrate from the older version of the software. Naturally, this is app-specific rather +than module specific, and must be defined in `app.go`, even if it imports logic from various +modules to perform the actions. You can register them with `upgradeKeeper.SetUpgradeHandler` +during the app initialization (before starting the abci server), and they serve not only to +perform a migration, but also to identify if this is the old or new version (eg. presence of +a handler registered for the named upgrade). + +Once the release candidate along with an appropriate upgrade handler is frozen, +we can have a governance vote to approve this upgrade at some future block height (e.g. 200000). +This is known as an upgrade.Plan. The v0.38.0 code will not know of this handler, but will +continue to run until block 200000, when the plan kicks in at `BeginBlock`. It will check +for existence of the handler, and finding it missing, know that it is running the obsolete software, +and gracefully exit. + +Generally the application binary will restart on exit, but then will execute this BeginBlocker +again and exit, causing a restart loop. Either the operator can manually install the new software, +or you can make use of an external watcher daemon to possibly download and then switch binaries, +also potentially doing a backup. The SDK tool for doing such, is called [Cosmovisor](https://docs.cosmos.network/main/build/tooling/cosmovisor). + +When the binary restarts with the upgraded version (here v0.40.0), it will detect we have registered the +"testnet-v2" upgrade handler in the code, and realize it is the new version. It then will run the upgrade handler +and *migrate the database in-place*. Once finished, it marks the upgrade as done, and continues processing +the rest of the block as normal. Once 2/3 of the voting power has upgraded, the blockchain will immediately +resume the consensus mechanism. If the majority of operators add a custom `do-upgrade` script, this should +be a matter of minutes and not even require them to be awake at that time. + +## Integrating With An App + +:::tip +The following is not required for users using `depinject`, this is abstracted for them. +::: + +In addition to basic module wiring, setup the upgrade Keeper for the app and then define a `PreBlocker` that calls the upgrade +keeper's PreBlocker method: + +```go +func (app *myApp) PreBlocker(ctx sdk.Context, req req.RequestFinalizeBlock) (*sdk.ResponsePreBlock, error) { + // For demonstration sake, the app PreBlocker only returns the upgrade module pre-blocker. + // In a real app, the module manager should call all pre-blockers + // return app.ModuleManager.PreBlock(ctx, req) + return app.upgradeKeeper.PreBlocker(ctx, req) +} +``` + +The app must then integrate the upgrade keeper with its governance module as appropriate. The governance module +should call ScheduleUpgrade to schedule an upgrade and ClearUpgradePlan to cancel a pending upgrade. + +## Performing Upgrades + +Upgrades can be scheduled at a predefined block height. Once this block height is reached, the +existing software will cease to process ABCI messages and a new version with code that handles the upgrade must be deployed. +All upgrades are coordinated by a unique upgrade name that cannot be reused on the same blockchain. In order for the upgrade +module to know that the upgrade has been safely applied, a handler with the name of the upgrade must be installed. +Here is an example handler for an upgrade named "my-fancy-upgrade": + +```go +app.upgradeKeeper.SetUpgradeHandler("my-fancy-upgrade", func(ctx context.Context, plan upgrade.Plan) { + // Perform any migrations of the state store needed for this upgrade +}) +``` + +This upgrade handler performs the dual function of alerting the upgrade module that the named upgrade has been applied, +as well as providing the opportunity for the upgraded software to perform any necessary state migrations. Both the halt +(with the old binary) and applying the migration (with the new binary) are enforced in the state machine. Actually +switching the binaries is an ops task and not handled inside the sdk / abci app. + +Here is a sample code to set store migrations with an upgrade: + +```go +// this configures a no-op upgrade handler for the "my-fancy-upgrade" upgrade +app.UpgradeKeeper.SetUpgradeHandler("my-fancy-upgrade", func(ctx context.Context, plan upgrade.Plan) { + // upgrade changes here +}) +upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() +if err != nil { + // handle error +} +if upgradeInfo.Name == "my-fancy-upgrade" && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) { + storeUpgrades := store.StoreUpgrades{ + Renamed: []store.StoreRename{{ + OldKey: "foo", + NewKey: "bar", + }}, + Deleted: []string{}, + } + // configure store loader that checks if version == upgradeHeight and applies store upgrades + app.SetStoreLoader(upgrade.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades)) +} +``` + +## Halt Behavior + +Before halting the ABCI state machine in the BeginBlocker method, the upgrade module will log an error +that looks like: + +```text + UPGRADE "" NEEDED at height : +``` + +where `Name` and `Info` are the values of the respective fields on the upgrade Plan. + +To perform the actual halt of the blockchain, the upgrade keeper simply panics which prevents the ABCI state machine +from proceeding but doesn't actually exit the process. Exiting the process can cause issues for other nodes that start +to lose connectivity with the exiting nodes, thus this module prefers to just halt but not exit. + +## Automation + +Read more about [Cosmovisor](https://docs.cosmos.network/main/build/tooling/cosmovisor), the tool for automating upgrades. + +## Canceling Upgrades + +There are two ways to cancel a planned upgrade - with on-chain governance or off-chain social consensus. +For the first one, there is a `CancelSoftwareUpgrade` governance proposal, which can be voted on and will +remove the scheduled upgrade plan. Of course this requires that the upgrade was known to be a bad idea +well before the upgrade itself, to allow time for a vote. If you want to allow such a possibility, you +should set the upgrade height to be `2 * (votingperiod + depositperiod) + (safety delta)` from the beginning of +the first upgrade proposal. Safety delta is the time available from the success of an upgrade proposal +and the realization it was a bad idea (due to external testing). You can also start a `CancelSoftwareUpgrade` +proposal while the original `SoftwareUpgrade` proposal is still being voted upon, as long as the voting +period ends after the `SoftwareUpgrade` proposal. + +However, let's assume that we don't realize the upgrade has a bug until shortly before it will occur +(or while we try it out - hitting some panic in the migration). It would seem the blockchain is stuck, +but we need to allow an escape for social consensus to overrule the planned upgrade. To do so, there's +a `--unsafe-skip-upgrades` flag to the start command, which will cause the node to mark the upgrade +as done upon hitting the planned upgrade height(s), without halting and without actually performing a migration. +If over two-thirds run their nodes with this flag on the old binary, it will allow the chain to continue through +the upgrade with a manual override. (This must be well-documented for anyone syncing from genesis later on). + +Example: + +```shell + start --unsafe-skip-upgrades ... +``` + +## Pre-Upgrade Handling + +Cosmovisor supports custom pre-upgrade handling. Use pre-upgrade handling when you need to implement application config changes that are required in the newer version before you perform the upgrade. + +Using Cosmovisor pre-upgrade handling is optional. If pre-upgrade handling is not implemented, the upgrade continues. + +For example, make the required new-version changes to `app.toml` settings during the pre-upgrade handling. The pre-upgrade handling process means that the file does not have to be manually updated after the upgrade. + +Before the application binary is upgraded, Cosmovisor calls a `pre-upgrade` command that can be implemented by the application. + +The `pre-upgrade` command does not take in any command-line arguments and is expected to terminate with the following exit codes: + +| Exit status code | How it is handled in Cosmosvisor | +|------------------|---------------------------------------------------------------------------------------------------------------------| +| `0` | Assumes `pre-upgrade` command executed successfully and continues the upgrade. | +| `1` | Default exit code when `pre-upgrade` command has not been implemented. | +| `30` | `pre-upgrade` command was executed but failed. This fails the entire upgrade. | +| `31` | `pre-upgrade` command was executed but failed. But the command is retried until exit code `1` or `30` are returned. | + +## Sample + +Here is a sample structure of the `pre-upgrade` command: + +```go +func preUpgradeCommand() *cobra.Command { + cmd := &cobra.Command{ + Use: "pre-upgrade", + Short: "Pre-upgrade command", + Long: "Pre-upgrade command to implement custom pre-upgrade handling", + Run: func(cmd *cobra.Command, args []string) { + + err := HandlePreUpgrade() + + if err != nil { + os.Exit(30) + } + + os.Exit(0) + + }, + } + + return cmd +} +``` + +Ensure that the pre-upgrade command has been registered in the application: + +```go +rootCmd.AddCommand( + // .. + preUpgradeCommand(), + // .. + ) +``` + +When not using Cosmovisor, ensure to run ` pre-upgrade` before starting the application binary. diff --git a/versioned_docs/version-0.52/build/building-apps/04-security-part-1.md b/versioned_docs/version-0.52/build/building-apps/04-security-part-1.md new file mode 100644 index 000000000..bdda5c1e2 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/04-security-part-1.md @@ -0,0 +1,333 @@ +# The Cosmos Security Handbook: Part 1 - Core Chain + +> Thank you to **[Roman Akhtariev](https://twitter.com/akhtariev) and [Alpin Yukseloglu](https://twitter.com/0xalpo)** for authoring this post. The original post can be found [here](https://www.faulttolerant.xyz/2024-01-16-cosmos-security-1/). + +> [Trail of bits](https://www.trailofbits.com/) hosts another set of guidelines [here](https://github.com/crytic/building-secure-contracts/tree/master/not-so-smart-contracts/cosmos) + +The defining property of the Cosmos stack is that it is unconstrained. The layers of the stack are porous, and, to a sufficiently motivated developer, nothing is off-limits. From a security standpoint, this freedom can be terrifying. + +In this post, we aim to shed some light on the security landscape for the Cosmos stack. We will emphasize areas that are particularly unintuitive, either because they are unique to Cosmos or because they are areas that developers who have not built appchains before are unlikely to have encountered. + +Since the surface of new risks that come with developing appchains is vast, we cannot possibly fit everything into a single post. Thus, this article will be focused only on the security surface of the core chain. We are reserving CosmWasm and IBC-related risks for a future post. + +## Overview + +Application logic in Cosmos-based appchains can affect all parts of the stack. This level of expressivity necessitates important guardrails to be removed, which introduces certain risks that would otherwise be protected against. To a developer who is accustomed to building on general-purpose chains, the protections in place are often invisible to the point of going unnoticed. Thus, when faced with ultimate control, it can be difficult to differentiate between what is a new tool and what is an unmarked danger zone. + +In the sections that follow, we break down the common ways developers can shoot themselves in the foot when building appchains. Some of these risks are more severe than others, but almost all are relatively unique to building appchains with the Cosmos SDK. + +Specifically, we will cover the following areas with multiple concrete examples for each: + +* Non-determinism +* In-protocol panics +* Unmetered/unbounded computation +* Prefix iteration & key malleability +* Fee market & gas Issues + +## Non-Determinism + +One of the consequences of opening up the consensus layer to app developers is that the code they write must not break critical properties required to reach consensus. Determinism is one such property that is particularly easy to compromise. + +At a high level, determinism means that for the same input, all nodes in the network always produce the same output. It is an inherent requirement of blockchains. Without it, it is unclear what the nodes in the network are trying to agree on. + +Simply put, non-determinism in the executed code can trigger the chain to fork or for honest validators to be unfairly slashed. + +> As a brief side-note: while non-determinism should generally be avoided, we have provided a list in the appendix covering exactly which parts of the Cosmos SDK where the code needs to be deterministic. In general, anything that touches the state machine must be. +> + +### Randomness + +Trivially, any use of randomness should be prohibited in the state machine. Keep an eye on the use of the Go `rand` package. It should not be used within the state-machine scope, including the imported dependencies. + +In general, if randomness is used, it should be accessed in a deterministic way (much like [Chainlink's VRF](https://chain.link/vrf)). + +### Go map internals + +Under the hood, Go maps are implemented as a hash map of buckets where each bucket contains up to 8 key-value pairs. Since some key-value pairs within the bucket can be empty, Go uses randomness to select the starting element within the bucket. See [this article](https://medium.com/i0exception/map-iteration-in-go-275abb76f721) for the breakdown. + +**When building on the Cosmos SDK, you should never iterate over a Go map**. Doing so results in non-determinism. Instead, if `map` usage is inevitable, it is necessary to convert it to a `slice` and sort it. See an example [here](https://github.com/osmosis-labs/osmosis/blob/b0aee0006ce55d0851773084bd7880db7e32ad70/osmoutils/partialord/internal/dag/dag.go#L290-L302). + +### Invalid Time Handling + +Avoid using `time.Now()` since nodes are unlikely to process messages at the same point in time even if they are in the same timezone. Instead, always rely on `ctx.BlockTime()` which should be the canonical definition of what "now" is. + +### API calls + +Network requests are generally non-deterministic. As a result, they should be avoided in the state machine. + +### Concurrency And Multithreading + +Thread or goroutine pre-emption is likely to lack determinism. As a result, one should generally avoid using goroutines to be used anywhere within the state-machine scope. There are, of course, exceptions where we may process data concurrently for aggregation/counting which would be deterministic. However, such use cases are rare enough to consider the general use of goroutines in the app chain code as a red flag. + +### Cross-Platform Floats + +For reasons that could easily take up a [separate article](https://randomascii.wordpress.com/2013/07/16/floating-point-determinism/), it is safe to claim that float arithmetic is non-deterministic across platforms. Therefore, they must never be used in the app chain state-machine scope. + +## In-Protocol Panics + +One of the most unintuitive differences between developing a general-purpose chain and building one's own appchain is that code can be run in-protocol without being triggered by a specific transaction. + +While this feature unlocks an incredible amount of expressivity for developers (such as custom precompiles and in-protocol arbitrage/liquidations), it also exposes various ways for the chain to be halted. One of the common ways this happens is through panics. + +There are of course times when panics are appropriate to use instead of errors, but it is important to keep in mind that **panics in module-executed code (`Begin/EndBlock`) will cause the chain to halt**. + +While these halts are generally not difficult to recover when isolated, they still pose a valid attack vector, especially if the panics can be triggered repeatedly. They also result in expensive social coordination and reputation costs stemming from downtime. + +Thus, **we should be cognizant of when we use panics and ensure that we avoid them with behavior that could be handled well with an error.** Of course, it is still okay to guardrail unexpected flows with panics when needed, especially if the behavior is such that a chain halt *would* be appropriate. + +Cosmos SDK takes care of catching and recovering from panics in all of `PrepareProposal` , `ProcessProposal`, `DeliverTx` , leaving only `Begin/EndBlock` for this class of vulnerabilities. + +For reference, the Osmosis codebase catches and silently logs most panics stemming from `Begin/EndBlock` with [this](https://github.com/osmosis-labs/osmosis/blob/b0aee0006ce55d0851773084bd7880db7e32ad70/osmoutils/cache_ctx.go#L13-L44) helper. In almost all cases, it is most productive to understand the reason behind panic and reconcile it without halting the chain entirely. + +### Math Overflow + +By default, all SDK math operations panic on overflows. This means that any math that is done in functions that get called in `Begin/EndBlock` should make sure to catch overflow panics using a helper similar to the one linked above. + +For example, let's say a chain adds a feature that involves checking the spot price of arbitrary assets in `BeginBlock`. If the overflow panic is not caught, an attacker could create a market for a new asset and manipulate the price such that the spot price calculation overflows, triggering a panic at the top of each block. Since this is an easily repeatable attack, the attacker could presumably halt the chain in perpetuity until a hard fork patches the issue by catching overflow panics. + +**The solution to this problem is to catch panics whenever there is SDK math run in `Begin/EndBlock`.** + +### Bulk Coin Sends + +If a chain supports custom token transfer logic (e.g. blacklists for USDC), it needs to make sure all token transfers in `Begin/EndBlock` properly catch panics. While this is generally quite straightforward to do, it is commonly missed in one context: bulk coin sends. + +Specifically, the Cosmos SDK allows for multiple coins to be transferred in one function call through its `[SendCoins](https://github.com/cosmos/cosmos-sdk/blob/d55985637e1484309b09e76d29f04f2c7258c3de/x/bank/keeper/send.go#L202)` function. This is a black-box function that does not allow for individual validation of each token transfer, which often leads to it being overlooked. A single panic trigger in a call to `SendCoins` in `Begin/EndBlock` can trigger a chain halt. + +While one can catch the panic on the entire `SendCoins` call, this would mean that an attacker can DoS all transfers in the batch. Thus, **the solution for these situations is to transfer coins one by one with `SendCoin` and verify each transfer so that problematic ones can be skipped.** + +## Unmetered Computation + +In the standard Cosmos stack, only stateful operations are gas-metered. This implies that out-of-block compute that is not triggered by messages has no notion of a gas limit. Thus, any form of unbounded execution in such a context can be used to halt the chain. + +### Unmetered Execution in Hooks + +Whenever one implements functionality involving hooks to arbitrary CosmWasm contracts, it is crucial to check whether this logic can be triggered by module-executed code. If it is, then an attacker can simply upload a contract that runs an infinite loop to halt the chain. + +For instance, if a chain allows for arbitrary token transfer hooks and triggers them in `Begin/EndBlock`, then an attacker can create a token that executes an infinitely looping CosmWasm contract. Once this token is transferred in the next block's `BeginBlock`, the chain will halt. + +**The solution to this problem is to [wrap risky function calls](https://github.com/osmosis-labs/osmosis/blob/2a64b0b6171478b81b017a001f5179b199a38628/x/tokenfactory/keeper/before_send.go#L121-L128) in a separate Context that has a gas limit.** This assigns a gas budget to such calls that prevent them from running unboundedly and halting the chain. + +### Poorly Chosen Loop/Recursion Exit Condition + +This is a consideration that seems trivial but comes up much more frequently than one might expect. If a loop in unmetered code is never exited or a recursion base case is never hit, it might lead to an expensive chain halt. + +### Slow Convergence in Math Operations + +A few months ago, a security researcher [reported a vulnerability](https://blog.trailofbits.com/2023/10/23/numbers-turned-weapons-dos-in-osmosis-math-library/) in the Osmosis codebase stemming from [PowApprox function](https://github.com/osmosis-labs/osmosis/blob/44a6a100a92f2984a760b41b7486fb9000ac670e/osmomath/math.go#L86). The crux of the issue was centered around long-lasting convergence for certain input values. A determined attacker could in theory use such edge cases to temporarily halt the chain. **The solution in these cases is simple - [introduce a constant loop bound](https://github.com/osmosis-labs/osmosis/pull/6627).** + +As a side note, from our experience, rational approximation is a more accurate and performant substitute to Taylor expansion which is used in `PowApprox` of the above example. See [this article](https://xn--2-umb.com/22/approximation/) for details. + +## Key Malleability and Prefix Iteration + +When onboarding onto the Cosmos stack, developers must familiarize themselves with its [key/value stores](https://docs.cosmos.network/v0.46/core/store.html). One particularly insidious class of bugs is related to how one sets keys when writing to these stores. Even slight mistakes in this process can lead to critical vulnerabilities that are usually simple to detect and exploit. + +### Store Prefix Ending In Serialized ID + +To guarantee uniqueness, it is common to serialize IDs in a store key. However, since these IDs are often in the control of potential attackers (e.g. triggering higher pool IDs by creating more pools), some portion of the keys in these cases can be malleable. + +One way that this frequently surfaces is when developers run a prefix iterator over keys that end in a malleable component (e.g. the pool's ID, which an attacker can increase by creating empty pools). In these cases, the iterator might include objects that were not supposed to be in the loop in the first place, meaning that an attacker can trigger unintended behavior. For instance, the prefix iteration on a key that ends with `42` would also loop over ID `421`, etc. A more involved example covering a concrete attack vector can be found in the appendix. + +**This bug can be resolved in one of two ways:** + +a) Add a key separator suffix to the prefix as done [here](https://github.com/osmosis-labs/osmosis/blob/450f7570a34876b14c61e883f2bf2ea81944f9c7/x/concentrated-liquidity/types/keys.go#L191-L195). + +b) Convert malleable numbers in keys to big-endian as done [here](https://github.com/osmosis-labs/osmosis/blob/450f7570a34876b14c61e883f2bf2ea81944f9c7/x/gamm/types/key.go#L60-L62). + +### Key Uniqueness + +In many instances, it might be tempting to identify a data structure by a collection of their fields. For instance, one might want to key liquidity pools in the following way: + +```go +// KeyPool returns the key for the given pool +func KeyPool(pool Pool) []byte { + return []byte(fmt.Sprintf("%s%s%s%s", PoolPrefix, pool.GetToken0(), pool.GetToken1(), pool.GetSpreadFactor())) +} + +``` + +However, note that there can be multiple pools with the same tokens and spread factor. As a result, an existing entry could be completely overwritten. While the example above is somewhat trivial and would be easily caught by unit tests, more complex instances of this issue come up frequently enough to justify mentioning it. **The solution here is to ensure that keys are unique, usually through the addition of an ID component or some equivalent.** + +### Iterator Bounds + +This is another simple example that sometimes catches even the most seasoned Cosmos developers. Prefix iteration is inclusive of the start byte but exclusive of the end byte. + +As a result, iterating forwards requires initializing the iterator with the given start value. See this example from the Osmosis concentrated liquidity module: + +```go +// +startKey := types.TickIndexToBytes(currentTickIndex) +iter := prefixStore.Iterator(startKey, nil) + +``` + +On the other hand, iterating in reverse requires adding one more byte to the end byte of the reverse iterator: + +```go +// +startKey := types.TickIndexToBytes(currentTickIndex + 1) +iter := prefixStore.ReverseIterator(nil, startKey) + +``` + +Being off by one on such iterators is a frequent cause of critical vulnerabilities that are difficult to catch through testing. + +## Fee Market and Gas Issues + +For developers on general-purpose chains, fees are usually treated as a black box. Thus, fee-related issues can be particularly unintuitive for those who are not used to thinking about fees as an abstraction. Regardless of whether one is implementing [novel fee mechanisms](https://www.faulttolerant.xyz/2023-11-17-fee-credits/) or simply running something out-of-the-box, appchain developers generally have to grapple with risks that arise from having more control over gas. + +### Mispriced State Writes + +If a data structure is written to the state during a user-executed message flow, the creation of this data structure must be bounded by a high enough fee to deter spam. If this is not done properly (i.e. either there are insufficient or no fees), then an attacker can DoS the chain similar to how they would be able to through large/unbounded compute in unmetered flows. + +While this might seem trivial in simple cases, there are many scenarios where pricing state writes is nontrivial. This complexity generally surfaces in actions that create externalities for other users. + +For instance, if the protocol includes logic where an arbitrary number of tokens are iterated through linearly, then each new token can potentially push an increasing cost onto the system. In such cases, additional (and scaling) gas [must be charged](https://github.com/osmosis-labs/osmosis/blob/b0aee0006ce55d0851773084bd7880db7e32ad70/x/tokenfactory/keeper/createdenom.go#L19-L22) to ensure the protocol is not vulnerable to DoS attacks. + +### Fees on Contract-called Functions + +Charging fixed fees distinct from gas is a relatively common design pattern (for instance, Osmosis charges a governance-set fee denominated in OSMO for creating pools). However, introducing such fees often results in risks in scenarios where contracts act on behalf of users. Specifically, this design pattern can cause the fees to be charged incorrectly or, in some cases, even prevent the contract from being used at all. + +In such cases, it often makes sense to simply [charge the fee as additional gas](https://github.com/osmosis-labs/osmosis/blob/b0aee0006ce55d0851773084bd7880db7e32ad70/x/tokenfactory/keeper/createdenom.go#L95-L98) so that it gets floated up to the caller. If this is not possible due to the fee being too high, then the fee charge needs to be factored into the design of the contract(s) triggering it. + +### Broken or Missing Fee Market + +During periods of high network usage, it is critical to ensure that high-value transactions have a priority for getting on-chain. For example, liquidations can continue to happen to ensure the health of the market as long as a higher fee is provided. To achieve this, there has to be a proxy for demand. [EIP-1559](https://www.youtube.com/watch?v=62UI3Js30Io) is a protocol that incorporates a variable base fee that increases or decreases based on historic block sizes. [Osmosis recently implemented](https://github.com/osmosis-labs/osmosis/blob/b0aee0006ce55d0851773084bd7880db7e32ad70/x/txfees/keeper/mempool-1559) this directly in the mempool. + +However, the superior long-term solution is to integrate the fee market directly into consensus by leveraging `ABCI 2.0`. The Skip team has been spearheading [the implementation](https://github.com/skip-mev/feemarket) of this initiative which will be released as a pluggable component that chains can integrate. + +### Transaction Simulation and Execution Gas Consistency + +Before submitting a transaction on-chain, clients attempt to simulate its execution to determine how much gas to provide. There is a separate execution mode for simulation that does not commit state but attempts to estimate gas. + +Due to [challenges with how Cosmos SDK gas estimation works](https://github.com/cosmos/cosmos-sdk/issues/18834), the gas estimate often ends up being inconsistent with the actual execution. Many clients get around this today by multiplying their gas estimates by a constant multiplier. + +If the chain increases gas usage in ways that are not included in simulation logic, this could break many clients at chain upgrade time until they increase their gas multipliers. + +The specific area to pay attention to on this front is the `simulate` parameter in the `AnteHandler` API. An example that could cause issues might look like the following: + +```go +func (mfd MyMemPoolDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (newCtx sdk.Context, err error) { + if !simulate { + // Do some gas intensive logic such as many store reads/writes + // This will lead to inconsistencies between transaction simulation and execution + } +} + +``` + +While this does not affect chain security, it can cause severe client instability until clients update their gas estimation logic. If your appchain supports high-value and time-sensitive transactions such as ones related to collateral and leverage, such issues could be quite problematic. + +## Closing Remarks + +Over the past few months, Cosmos has seen an influx of liquidity and users. If we as an ecosystem want to thrive and compete with centralized entities, there has to be a shared focus on security and the desire to share insights on best practices around our stack. + +Our hope with this post was to lay the groundwork from a security standpoint for engineers onboarding onto the Cosmos SDK and appchains more broadly. While this is far from being exhaustive, it is a distillation of years of experience and scar tissue accumulated over many multi-million dollar war rooms. + +As an ecosystem, we cannot afford recklessness or lack of security awareness. **If you have any feedback or contributions you would like to see added to this post (or one of our future posts on CosmWasm and IBC), please reach out to us on Twitter/X @0xalpo and @akhtariev!** + +## Appendix + +### State-machine Scope + +The state-machine scope includes the following areas: + +* All messages supported by the chain, including: + * Every msg's `ValidateBasic` method + * Every msg's `MsgServer` method + * Net gas usage, in all execution paths + * Errors (assuming use of Cosmos-SDK errors) + * State changes (namely every store write) + * `AnteHandler`s in `execModeFinalize` + * `PostHandler`s in `execModeFinalize` + * Cosmwasm-whitelisted queries +* All BeginBlock/EndBlock logic +* ABCI 2.0 `ProcessProposal` +* ABCI 2.0 `VerifyVoteExtensions` + +The following are NOT in the state-machine scope: + +* Events +* Queries that are not Cosmwasm-whitelisted +* CLI interfaces +* Errors (assuming use of Go-native errors) +* ABCI 2.0 `PrepareProposal` +* ABCI 2.0 `ExtendVote` +* `AnteHandler`s in any mode other than `execModeFinalize` +* `PostHandler`s in any mode other than `execModeFinalize` + +### Key Malleability and Prefix Iteration Attack Example + +Consider the code below that checks whether the given address is the owner of a given position ID via an`IsPositionOwner` function. The `KeyAddressPoolIdPositionId` formats the key ending with a pool ID as a string. `HasAnyAtPrefix` function checks if there exists an entry at a given prefix. + +```go +// KeyAddressPoolIdPositionId returns the full key needed to store the position id for given addr + pool id + position id combination. +func KeyAddressPoolIdPositionId(addr sdk.AccAddress, poolId uint64, positionId uint64) []byte { + return []byte(fmt.Sprintf("%s%s%x%s%d%s%d", PositionPrefix, KeySeparator, addr.Bytes(), KeySeparator, poolId, KeySeparator, positionId)) +} + +``` + +`IsPositionOwner(address sdk.AccAddress, positionID uint64) bool` function checks if there exists an entry in the key-value store with the given prefix formatted per `KeyAddressPoolIdPositionId` structure. + +To achieve that, it might be tempting to use a store helper such as `HasAnyAtPrefix`. Unfortunately, this would be fatal. + +```go +// HasAnyAtPrefix returns true if there is at least one value in the given prefix. +func HasAnyAtPrefix[T any](storeObj store.KVStore, prefix []byte, parseValue func([]byte) (T, error)) (bool, error) { + _, err := GetFirstValueInRange(storeObj, prefix, sdk.PrefixEndBytes(prefix), false, parseValue) + if err != nil { + if err == ErrNoValuesInRange { + return false, nil + } + return false, err + } + + return true, nil +} + +// GetFirstValueInRange returns the first value between [keyStart, keyEnd) +func GetFirstValueInRange[T any](storeObj store.KVStore, keyStart []byte, keyEnd []byte, reverseIterate bool, parseValue func([]byte) (T, error)) (T, error) { + iterator := makeIterator(storeObj, keyStart, keyEnd, reverseIterate) + defer iterator.Close() + + if !iterator.Valid() { + var blankValue T + return blankValue, ErrNoValuesInRange + } + + return parseValue(iterator.Value()) +} + +``` + +To fully grasp the root of the issue, consider the following snapshot: + +```go +// +func test(poolId uint64) { + formattedString := fmt.Sprintf("%d", poolId) + byteSlice := []byte(formattedString) + fmt.Printf("Original String: %s\\n", formattedString) + fmt.Printf("Byte Slice: %v\\n", byteSlice) +} + +func main() { + poolIDOne := uint64(42) + poolIDTwo := uint64(421) + + // Prints: + // Original String: 42 + // Byte Slice: [52 50] + test(poolIDOne) + + // Prints: + // Original String: 421 + // Byte Slice: [52 50 49] + test(poolIDTwo) +} + +``` + +Both `poolIDOne` and `poolIDTwo` have the same prefix. + +Now, our original position existence check with `HasAnyAtPrefix` would pass if it was run on `poolID` of `42` when the user only owned a position ID `421`. This can result in malicious users getting access to positions that they do not own. diff --git a/versioned_docs/version-0.52/build/building-apps/05-app-testnet.md b/versioned_docs/version-0.52/build/building-apps/05-app-testnet.md new file mode 100644 index 000000000..c96e229e2 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/05-app-testnet.md @@ -0,0 +1,235 @@ +--- +sidebar_position: 1 +--- + +# Application Testnets + +Building an application is complicated and requires a lot of testing. The Cosmos SDK provides a way to test your application in a real-world environment: a testnet. + +We allow developers to take the state from their mainnet and run tests against the state. This is useful for testing upgrade migrations, or for testing the application in a real-world environment. + +## Testnet Setup + +We will be breaking down the steps to create a testnet from mainnet state. + +```go + // InitSimAppForTestnet is broken down into two sections: + // Required Changes: Changes that, if not made, will cause the testnet to halt or panic + // Optional Changes: Changes to customize the testnet to one's liking (lower vote times, fund accounts, etc) + func InitSimAppForTestnet(app *SimApp, newValAddr bytes.HexBytes, newValPubKey crypto.PubKey, newOperatorAddress, upgradeToTrigger string) *SimApp { + ... + } +``` + +### Required Changes + +#### Staking + +When creating a testnet the important part is migrate the validator set from many validators to one or a few. This allows developers to spin up the chain without needing to replace validator keys. + +```go + ctx := app.BaseApp.NewUncachedContext(true, tmproto.Header{}) + pubkey := &ed25519.PubKey{Key: newValPubKey.Bytes()} + pubkeyAny, err := types.NewAnyWithValue(pubkey) + if err != nil { + tmos.Exit(err.Error()) + } + + // STAKING + // + + // Create Validator struct for our new validator. + _, bz, err := bech32.DecodeAndConvert(newOperatorAddress) + if err != nil { + tmos.Exit(err.Error()) + } + bech32Addr, err := bech32.ConvertAndEncode("simvaloper", bz) + if err != nil { + tmos.Exit(err.Error()) + } + newVal := stakingtypes.Validator{ + OperatorAddress: bech32Addr, + ConsensusPubkey: pubkeyAny, + Jailed: false, + Status: stakingtypes.Bonded, + Tokens: sdk.NewInt(900000000000000), + DelegatorShares: sdk.MustNewDecFromStr("10000000"), + Description: stakingtypes.Description{ + Moniker: "Testnet Validator", + }, + Commission: stakingtypes.Commission{ + CommissionRates: stakingtypes.CommissionRates{ + Rate: sdk.MustNewDecFromStr("0.05"), + MaxRate: sdk.MustNewDecFromStr("0.1"), + MaxChangeRate: sdk.MustNewDecFromStr("0.05"), + }, + }, + MinSelfDelegation: sdk.OneInt(), + } + + // Remove all validators from power store + stakingKey := app.GetKey(stakingtypes.ModuleName) + stakingStore := ctx.KVStore(stakingKey) + iterator := app.StakingKeeper.ValidatorsPowerStoreIterator(ctx) + for ; iterator.Valid(); iterator.Next() { + stakingStore.Delete(iterator.Key()) + } + iterator.Close() + + // Remove all validators from last validators store + iterator = app.StakingKeeper.LastValidatorsIterator(ctx) + for ; iterator.Valid(); iterator.Next() { + app.StakingKeeper.LastValidatorPower.Delete(iterator.Key()) + } + iterator.Close() + + // Add our validator to power and last validators store + app.StakingKeeper.SetValidator(ctx, newVal) + err = app.StakingKeeper.SetValidatorByConsAddr(ctx, newVal) + if err != nil { + panic(err) + } + app.StakingKeeper.SetValidatorByPowerIndex(ctx, newVal) + app.StakingKeeper.SetLastValidatorPower(ctx, newVal.GetOperator(), 0) + if err := app.StakingKeeper.Hooks().AfterValidatorCreated(ctx, newVal.GetOperator()); err != nil { + panic(err) + } +``` + +#### Distribution + +Since the validator set has changed, we need to update the distribution records for the new validator. + + +```go + // Initialize records for this validator across all distribution stores + app.DistrKeeper.ValidatorHistoricalRewards.Set(ctx, newVal.GetOperator(), 0, distrtypes.NewValidatorHistoricalRewards(sdk.DecCoins{}, 1)) + app.DistrKeeper.ValidatorCurrentRewards.Set(ctx, newVal.GetOperator(), distrtypes.NewValidatorCurrentRewards(sdk.DecCoins{}, 1)) + app.DistrKeeper.ValidatorAccumulatedCommission.Set(ctx, newVal.GetOperator(), distrtypes.InitialValidatorAccumulatedCommission()) + app.DistrKeeper.ValidatorOutstandingRewards.Set(ctx, newVal.GetOperator(), distrtypes.ValidatorOutstandingRewards{Rewards: sdk.DecCoins{}}) +``` + +#### Slashing + +We also need to set the validator signing info for the new validator. + +```go + // SLASHING + // + + // Set validator signing info for our new validator. + newConsAddr := sdk.ConsAddress(newValAddr.Bytes()) + newValidatorSigningInfo := slashingtypes.ValidatorSigningInfo{ + Address: newConsAddr.String(), + StartHeight: app.LastBlockHeight() - 1, + Tombstoned: false, + } + app.SlashingKeeper.ValidatorSigningInfo.Set(ctx, newConsAddr, newValidatorSigningInfo) +``` + +#### Bank + +It is useful to create new accounts for your testing purposes. This avoids the need to have the same key as you may have on mainnet. + +```go + // BANK + // + + defaultCoins := sdk.NewCoins(sdk.NewInt64Coin("ustake", 1000000000000)) + + localSimAppAccounts := []sdk.AccAddress{ + sdk.MustAccAddressFromBech32("cosmos12smx2wdlyttvyzvzg54y2vnqwq2qjateuf7thj"), + sdk.MustAccAddressFromBech32("cosmos1cyyzpxplxdzkeea7kwsydadg87357qnahakaks"), + sdk.MustAccAddressFromBech32("cosmos18s5lynnmx37hq4wlrw9gdn68sg2uxp5rgk26vv"), + sdk.MustAccAddressFromBech32("cosmos1qwexv7c6sm95lwhzn9027vyu2ccneaqad4w8ka"), + sdk.MustAccAddressFromBech32("cosmos14hcxlnwlqtq75ttaxf674vk6mafspg8xwgnn53"), + sdk.MustAccAddressFromBech32("cosmos12rr534cer5c0vj53eq4y32lcwguyy7nndt0u2t"), + sdk.MustAccAddressFromBech32("cosmos1nt33cjd5auzh36syym6azgc8tve0jlvklnq7jq"), + sdk.MustAccAddressFromBech32("cosmos10qfrpash5g2vk3hppvu45x0g860czur8ff5yx0"), + sdk.MustAccAddressFromBech32("cosmos1f4tvsdukfwh6s9swrc24gkuz23tp8pd3e9r5fa"), + sdk.MustAccAddressFromBech32("cosmos1myv43sqgnj5sm4zl98ftl45af9cfzk7nhjxjqh"), + sdk.MustAccAddressFromBech32("cosmos14gs9zqh8m49yy9kscjqu9h72exyf295afg6kgk"), + sdk.MustAccAddressFromBech32("cosmos1jllfytsz4dryxhz5tl7u73v29exsf80vz52ucc")} + + // Fund localSimApp accounts + for _, account := range localSimAppAccounts { + err := app.BankKeeper.MintCoins(ctx, minttypes.ModuleName, defaultCoins) + if err != nil { + tmos.Exit(err.Error()) + } + err = app.BankKeeper.SendCoinsFromModuleToAccount(ctx, minttypes.ModuleName, account, defaultCoins) + if err != nil { + tmos.Exit(err.Error()) + } + } +``` + +#### Upgrade + +If you would like to schedule an upgrade the below can be used. + +```go + // UPGRADE + // + + if upgradeToTrigger != "" { + upgradePlan := upgradetypes.Plan{ + Name: upgradeToTrigger, + Height: app.LastBlockHeight(), + } + err = app.UpgradeKeeper.ScheduleUpgrade(ctx, upgradePlan) + if err != nil { + panic(err) + } + } +``` + +### Optional Changes + +If you have custom modules that rely on specific state from the above modules and/or you would like to test your custom module, you will need to update the state of your custom module to reflect your needs + +## Running the Testnet + +Before we can run the testnet we must plug everything together. + +in `root.go`, in the `initRootCmd` function we add: + +```diff +server.AddCommands(rootCmd, simapp.DefaultNodeHome, newApp, createSimAppAndExport) ++server.AddTestnetCreatorCommand(rootCmd, simapp.DefaultNodeHome, newTestnetApp) +``` + +Next we will add a newTestnetApp helper function: + +```diff +// newTestnetApp starts by running the normal newApp method. From there, the app interface returned is modified in order +// for a testnet to be created from the provided app. +func newTestnetApp(logger log.Logger, db cometbftdb.DB, traceStore io.Writer, appOpts servertypes.AppOptions) servertypes.Application { + // Create an app and type cast to an SimApp + app := newApp(logger, db, traceStore, appOpts) + simApp, ok := app.(*simapp.SimApp) + if !ok { + panic("app created from newApp is not of type simApp") + } + + newValAddr, ok := appOpts.Get(server.KeyNewValAddr).(bytes.HexBytes) + if !ok { + panic("newValAddr is not of type bytes.HexBytes") + } + newValPubKey, ok := appOpts.Get(server.KeyUserPubKey).(crypto.PubKey) + if !ok { + panic("newValPubKey is not of type crypto.PubKey") + } + newOperatorAddress, ok := appOpts.Get(server.KeyNewOpAddr).(string) + if !ok { + panic("newOperatorAddress is not of type string") + } + upgradeToTrigger, ok := appOpts.Get(server.KeyTriggerTestnetUpgrade).(string) + if !ok { + panic("upgradeToTrigger is not of type string") + } + + // Make modifications to the normal SimApp required to run the network locally + return simapp.InitSimAppForTestnet(simApp, newValAddr, newValPubKey, newOperatorAddress, upgradeToTrigger) +} +``` diff --git a/versioned_docs/version-0.52/build/building-apps/06-app-go-genesis.md b/versioned_docs/version-0.52/build/building-apps/06-app-go-genesis.md new file mode 100644 index 000000000..b9902858d --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/06-app-go-genesis.md @@ -0,0 +1,47 @@ +--- +sidebar_position: 1 +--- + +### Modifying the `DefaultGenesis` + +It is possible to modify the DefaultGenesis parameters for modules by wrapping the module, providing it to the `*module.Manager` and injecting it with `depinject`. + +Example ( staking ) : + +```go +type CustomStakingModule struct { + staking.AppModule + cdc codec.Codec +} + +// DefaultGenesis will override the Staking module DefaultGenesis AppModuleBasic method. +func (cm CustomStakingModule) DefaultGenesis() json.RawMessage { + params := stakingtypes.DefaultParams() + params.BondDenom = "mydenom" + + return cm.cdc.MustMarshalJSON(&stakingtypes.GenesisState{ + Params: params, + }) +} + +// option 1 ( for depinject users ): override previous module manager +depinject.Inject( +// ... provider/invoker/supplier +&moduleManager, +) + +oldStakingModule,_ := moduleManager.Modules()[stakingtypes.ModuleName].(staking.AppModule) +moduleManager.Modules()[stakingtypes.ModuleName] = CustomStakingModule{ + AppModule: oldStakingModule, + cdc: appCodec, +} + +// option 2 ( for non depinject users ): use new module manager +moduleManager := module.NewManagerFromMap(map[string]appmodule.AppModule{ +stakingtypes.ModuleName: CustomStakingModule{cdc: appCodec, AppModule: staking.NewAppModule(...)}, +// other modules ... +}) + +// set the module manager +app.ModuleManager = moduleManager +``` diff --git a/versioned_docs/version-0.52/build/building-apps/06-system-tests.md b/versioned_docs/version-0.52/build/building-apps/06-system-tests.md new file mode 100644 index 000000000..eb6d61ffe --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/06-system-tests.md @@ -0,0 +1,58 @@ +--- +sidebar_position: 1 +--- + +# System Tests + +System tests provide a framework to write and execute black box tests against a running chain. This adds another level +of confidence on top of unit, integration, and simulations tests, ensuring that business-critical scenarios +(like double signing prevention) or scenarios that can't be tested otherwise (like a chain upgrade) are covered. + +## Vanilla Go for Flow Control + +System tests are vanilla Go tests that interact with the compiled chain binary. The `test runner` component starts a +local testnet of 4 nodes (by default) and provides convenient helper methods for accessing the +`system under test (SUT)`. +A `CLI wrapper` makes it easy to access keys, submit transactions, or execute operations. Together, these components +enable the replication and validation of complex business scenarios. + +Here's an example of a double signing test, where a new node is added with the same key as the first validator: +[double signing test example](https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/tests/systemtests/fraud_test.go) + +The [getting started tutorial](https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/tests/systemtests/getting_started.md) +contains a step-by-step guide to building and running your first system test. It covers setting chain state via genesis +or +transactions and validation via transaction response or queries. + +## Design Principles and Guidelines + +System tests are slower compared to unit or integration tests as they interact with a running chain. Therefore, certain +principles can guide their usage: + +- **Perspective:** Tests should mimic a human interacting with the chain from the outside. Initial states can be set via + genesis or transactions to support a test scenario. +- **Roles:** The user can have multiple roles such as validator, delegator, granter, or group admin. +- **Focus:** Tests should concentrate on happy paths or business-critical workflows. Unit and integration tests are + better suited for more fine-grained testing. +- **Workflows:** Test workflows and scenarios, not individual units. Given the high setup costs, it is reasonable to + combine multiple steps and assertions in a single test method. +- **Genesis Mods:** Genesis modifications can incur additional time costs for resetting dirty states. Reuse existing + accounts (node0..n) whenever possible. +- **Framework:** Continuously improve the framework for better readability and reusability. + +## Errors and Debugging + +All output is logged to `systemtests/testnet/node{0..n}.out`. Usually, `node0.out` is very noisy as it receives the CLI +connections. Prefer any other node's log to find stack traces or error messages. + +Using system tests for state setup during debugging has become very handy: + +- Start the test with one node only and verbose output: + + ```sh + go test -v -tags=system_test ./ --run TestAccountCreation --verbose --nodes-count=1 + ``` + +- Copy the CLI command for the transaction and modify the test to stop before the command +- Start the node with `--home=/tests/systemtests/testnet/node0//` in debug mode +- Execute CLI command from shell and enter breakpoints diff --git a/versioned_docs/version-0.52/build/building-apps/_category_.json b/versioned_docs/version-0.52/build/building-apps/_category_.json new file mode 100644 index 000000000..342732cce --- /dev/null +++ b/versioned_docs/version-0.52/build/building-apps/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Building Apps", + "position": 0, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/building-modules/00-intro.md b/versioned_docs/version-0.52/build/building-modules/00-intro.md new file mode 100644 index 000000000..4eda49125 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/00-intro.md @@ -0,0 +1,73 @@ +--- +sidebar_position: 1 +--- + +# Introduction to Cosmos SDK Modules + +:::note Synopsis +Modules define most of the logic of Cosmos SDK applications. Developers compose modules together using the Cosmos SDK to build their custom application-specific blockchains. This document outlines the basic concepts behind SDK modules and how to approach module management. +::: + +:::note Pre-requisite Readings + +* [Anatomy of a Cosmos SDK application](../../learn/beginner/00-app-anatomy.md) +* [Lifecycle of a Cosmos SDK transaction](../../learn/beginner/01-tx-lifecycle.md) + +::: + +## Role of Modules in a Cosmos SDK Application + +The Cosmos SDK can be thought of as the Ruby-on-Rails of blockchain development. It comes with a core that provides the basic functionalities every blockchain application needs, like a [boilerplate implementation of the ABCI](../../learn/advanced/00-baseapp.md) to communicate with the underlying consensus engine, a [`multistore`](../../learn/advanced/04-store.md#multistore) to persist state, a [server](../../learn/advanced/03-node.md) to form a full-node and [interfaces](./09-module-interfaces.md) to handle queries. + +On top of this core, the Cosmos SDK enables developers to build modules that implement the business logic of their application. In other words, SDK modules implement the bulk of the logic of applications, while the core does the wiring and enables modules to be composed together. The end goal is to build a robust ecosystem of open-source Cosmos SDK modules, making it increasingly easier to build complex blockchain applications. + +Cosmos SDK modules can be seen as little state-machines within the state-machine. They generally define a subset of the state using one or more `KVStore`s in the [main multistore](../../learn/advanced/04-store.md), as well as a subset of [message types](./02-messages-and-queries.md#messages). These messages are routed by one of the main components of Cosmos SDK core, [`BaseApp`](../../learn/advanced/00-baseapp.md), to a module Protobuf [`Msg` service](./03-msg-services.md) that defines them. + +```mermaid +flowchart TD + A[Transaction relayed from the full-node's consensus engine to the node's application via DeliverTx] + A --> B[APPLICATION] + B --> C["Using baseapp's methods: Decode the Tx, extract and route the message(s)"] + C --> D[Message routed to the correct module to be processed] + D --> E[AUTH MODULE] + D --> F[BANK MODULE] + D --> G[STAKING MODULE] + D --> H[GOV MODULE] + H --> I[Handles message, Updates state] + E --> I + F --> I + G --> I + I --> J["Return result to the underlying consensus engine (e.g. CometBFT) (0=Ok, 1=Err)"] +``` + +As a result of this architecture, building a Cosmos SDK application usually revolves around writing modules to implement the specialized logic of the application and composing them with existing modules to complete the application. Developers will generally work on modules that implement logic needed for their specific use case that do not exist yet, and will use existing modules for more generic functionalities like staking, accounts, or token management. + + +### Modules as Sudo + +Modules have the ability to perform actions that are not available to regular users. This is because modules are given sudo permissions by the state machine. Modules can reject another modules desire to execute a function but this logic must be explicit. Examples of this can be seen when modules create functions to modify parameters: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/61da5d1c29c16a1eb5bb5488719fde604ec07b10/x/bank/keeper/msg_server.go#L147-L149 +``` + +## How to Approach Building Modules as a Developer + +While there are no definitive guidelines for writing modules, here are some important design principles developers should keep in mind when building them: + +* **Composability**: Cosmos SDK applications are almost always composed of multiple modules. This means developers need to carefully consider the integration of their module not only with the core of the Cosmos SDK, but also with other modules. The former is achieved by following standard design patterns outlined [here](#main-components-of-cosmos-sdk-modules), while the latter is achieved by properly exposing the store(s) of the module via the [`keeper`](./06-keeper.md). +* **Specialization**: A direct consequence of the **composability** feature is that modules should be **specialized**. Developers should carefully establish the scope of their module and not batch multiple functionalities into the same module. This separation of concerns enables modules to be re-used in other projects and improves the upgradability of the application. **Specialization** also plays an important role in the [object-capabilities model](https://docs.cosmos.network/main/learn/advanced/ocap#ocaps-in-practice) of the Cosmos SDK. +* **Capabilities**: Most modules need to read and/or write to the store(s) of other modules. However, in an open-source environment, it is possible for some modules to be malicious. That is why module developers need to carefully think not only about how their module interacts with other modules, but also about how to give access to the module's store(s). The Cosmos SDK takes a capabilities-oriented approach to inter-module security. This means that each store defined by a module is accessed by a `key`, which is held by the module's [`keeper`](./06-keeper.md). This `keeper` defines how to access the store(s) and under what conditions. Access to the module's store(s) is done by passing a reference to the module's `keeper`. + +## Main Components of Cosmos SDK Modules + +Modules are by convention defined in the `./x/` subfolder (e.g. the `bank` module will be defined in the `./x/bank` folder). They generally share the same core components: + +* A [`keeper`](./06-keeper.md), used to access the module's store(s) and update the state. +* A [`Msg` service](./02-messages-and-queries.md#messages), used to process messages when they are routed to the module by [`BaseApp`](../../learn/advanced/00-baseapp.md#message-routing) and trigger state-transitions. +* A [query service](./04-query-services.md), used to process user queries when they are routed to the module by [`BaseApp`](../../learn/advanced/00-baseapp.md#query-routing). +* Interfaces, for end users to query the subset of the state defined by the module and create `message`s of the custom types defined in the module. + +In addition to these components, modules implement the `AppModule` interface in order to be managed by the [`module manager`](./01-module-manager.md). + +Please refer to the [structure document](./11-structure.md) to learn about the recommended structure of a module's directory. diff --git a/versioned_docs/version-0.52/build/building-modules/01-module-manager.md b/versioned_docs/version-0.52/build/building-modules/01-module-manager.md new file mode 100644 index 000000000..974245432 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/01-module-manager.md @@ -0,0 +1,259 @@ +--- +sidebar_position: 1 +--- + +# Module Manager + +:::note Synopsis +Cosmos SDK modules need to implement the [`AppModule` interfaces](#application-module-interfaces), in order to be managed by the application's [module manager](#module-manager). The module manager plays an important role in [`message` and `query` routing](../../learn/advanced/00-baseapp.md#routing), and allows application developers to set the order of execution of a variety of functions like [`PreBlocker`](https://docs.cosmos.network/main/learn/beginner/app-anatomy) and [`BeginBlocker` and `EndBlocker`](https://docs.cosmos.network/main/learn/beginner/app-anatomy). +::: + +:::note Pre-requisite Readings + +* [Introduction to Cosmos SDK Modules](./00-intro.md) + +::: + +## Application Module Interfaces + +Application module interfaces exist to facilitate the composition of modules together to form a functional Cosmos SDK application. + +:::note + +It is recommended to implement interfaces from the [Core API](https://docs.cosmos.network/main/architecture/adr-063-core-module-api) `appmodule` package. This makes modules less dependent on the SDK. +For legacy reason modules can still implement interfaces from the SDK `module` package. +::: + +There are 2 main application module interfaces: + +* [`appmodule.AppModule` / `module.AppModule`](#appmodule) for inter-dependent module functionalities (except genesis-related functionalities). + +The above interfaces are mostly embedding smaller interfaces (extension interfaces), that defines specific functionalities: + + + +* (legacy) [`module.HasGenesisBasics`](#modulehasgenesisbasics): The legacy interface for stateless genesis methods. +* (legacy) [`module.HasGenesis`](#modulehasgenesis) for inter-dependent genesis-related module functionalities. +* (legacy) [`module.HasABCIGenesis`](#modulehasabcigenesis) for inter-dependent genesis-related module functionalities. +* [`appmodule.HasPreBlocker`](#haspreblocker): The extension interface that contains information about the `AppModule` and `PreBlock`. +* [`appmodule.HasBeginBlocker`](#hasbeginblocker): The extension interface that contains information about the `AppModule` and `BeginBlock`. +* [`appmodule.HasEndBlocker`](#hasendblocker): The extension interface that contains information about the `AppModule` and `EndBlock`. +* [`appmodule.HasService` / `module.HasServices`](#hasservices): The extension interface for modules to register services. +* [`module.HasABCIEndBlock`](#hasabciendblock): The extension interface that contains information about the `AppModule`, `EndBlock` and returns an updated validator set. +* (legacy) [`module.HasConsensusVersion`](#hasconsensusversion): The extension interface for declaring a module consensus version. + +The `AppModule` interface exists to define inter-dependent module methods. Many modules need to interact with other modules, typically through [`keeper`s](./06-keeper.md), which means there is a need for an interface where modules list their `keeper`s and other methods that require a reference to another module's object. `AppModule` interface extension, such as `HasBeginBlocker` and `HasEndBlocker`, also enables the module manager to set the order of execution between module's methods like `BeginBlock` and `EndBlock`, which is important in cases where the order of execution between modules matters in the context of the application. + +The usage of extension interfaces allows modules to define only the functionalities they need. For example, a module that does not need an `EndBlock` does not need to define the `HasEndBlocker` interface and thus the `EndBlock` method. `AppModule` and `AppModuleGenesis` are voluntarily small interfaces, that can take advantage of the `Module` patterns without having to define many placeholder functions. + +### `HasAminoCodec` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/eee5e21e1c8d0995b6d4f83b7f55ec0b58d27ba7/core/appmodule/module.go#L74-L78 +``` + +* `RegisterLegacyAminoCodec(registry.AminoRegistrar)`: Registers the `amino` codec for the module, which is used to marshal and unmarshal structs to/from `[]byte` in order to persist them in the module's `KVStore`. + +### `HasRegisterInterfaces` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/eee5e21e1c8d0995b6d4f83b7f55ec0b58d27ba7/core/appmodule/v2/module.go#L103-L106 +``` + +* `RegisterInterfaces(codectypes.InterfaceRegistry)`: Registers a module's interface types and their concrete implementations as `proto.Message`. + +### `HasGRPCGateway` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/eee5e21e1c8d0995b6d4f83b7f55ec0b58d27ba7/types/module/module.go#L84-L87 +``` + +* `RegisterGRPCGatewayRoutes(client.Context, *runtime.ServeMux)`: Registers gRPC routes for the module. + +### Genesis + +:::tip +For easily creating an `AppModule` that only has genesis functionalities, implement `module.HasGenesis/HasABCIGenesis`. +::: + +#### `module.HasGenesisBasics` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go#L76-L79 +``` + +Let us go through the methods: + +* `DefaultGenesis(codec.JSONCodec)`: Returns a default [`GenesisState`](./08-genesis.md#genesisstate) for the module, marshalled to `json.RawMessage`. The default `GenesisState` need to be defined by the module developer and is primarily used for testing. +* `ValidateGenesis(codec.JSONCodec, client.TxEncodingConfig, json.RawMessage)`: Used to validate the `GenesisState` defined by a module, given in its `json.RawMessage` form. It will usually unmarshall the `json` before running a custom [`ValidateGenesis`](./08-genesis.md#validategenesis) function defined by the module developer. + +#### `module.HasGenesis` + +`HasGenesis` is an extension interface for allowing modules to implement genesis functionalities. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/types/module/module.go#L184-L189 +``` + +#### `module.HasABCIGenesis` + +`HasABCIGenesis` is an extension interface for allowing modules to implement genesis functionalities and returns validator set updates. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/types/module/module.go#L94-L98 +``` + +### `AppModule` + +The `AppModule` interface defines a module. Modules can declare their functionalities by implementing extensions interfaces. +`AppModule`s are managed by the [module manager](#manager), which checks which extension interfaces are implemented by the module. + +#### `appmodule.AppModule` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/core/appmodule/module.go#L11-L20 +``` + +#### `module.AppModule` + +:::note +Previously the `module.AppModule` interface was containing all the methods that are defined in the extensions interfaces. This was leading to much boilerplate for modules that did not need all the functionalities. +::: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/core/appmodule/v2/module.go#L14-L20 +``` + +### `HasServices` + +This interface defines one method. It allows to checks if a module can register invariants. + +#### `appmodule.HasService` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/core/appmodule/module.go#L22-L40 +``` + +#### `module.HasServices` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go#L208-L211 +``` + +* `RegisterServices(Configurator)`: Allows a module to register services. + +### `HasConsensusVersion` + +This interface defines one method for checking a module consensus version. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/core/appmodule/v2/migrations.go#L6-L12 +``` + +* `ConsensusVersion() uint64`: Returns the consensus version of the module. + +### `HasPreBlocker` + +The `HasPreBlocker` is an extension interface from `appmodule.AppModule`. All modules that have an `PreBlock` method implement this interface. + +### `HasBeginBlocker` + +The `HasBeginBlocker` is an extension interface from `appmodule.AppModule`. All modules that have an `BeginBlock` method implement this interface. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8dfcb3208d3b1cfbae08eda519e4cc1560/core/appmodule/v2/module.go#L30-L38 +``` + +* `BeginBlock(context.Context) error`: This method gives module developers the option to implement logic that is automatically triggered at the beginning of each block. + +### `HasEndBlocker` + +The `HasEndBlocker` is an extension interface from `appmodule.AppModule`. All modules that have an `EndBlock` method implement this interface. If a module needs to return validator set updates (staking), they can use `HasABCIEndBlock` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8dfcb3208d3b1cfbae08eda519e4cc1560/core/appmodule/v2/module.go#L40-L48 +``` + +* `EndBlock(context.Context) error`: This method gives module developers the option to implement logic that is automatically triggered at the end of each block. + +### `HasABCIEndBlock` + +The `HasUpdateValidators` is an extension interface from `module.AppModule`. All modules that have an `EndBlock` which return validator set updates implement this interface. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8dfcb3208d3b1cfbae08eda519e4cc1560/core/appmodule/v2/module.go#L87-L94 +``` + +* `UpdateValidators(context.Context) ([]abci.ValidatorUpdate, error)`: This method gives module developers the option to inform the underlying consensus engine of validator set changes (e.g. the `staking` module). + + +### Implementing the Application Module Interfaces + + +Typically, the various application module interfaces are implemented in a file called `module.go`, located in the module's folder (e.g. `./x/module/module.go`). + +Almost every module needs to implement the `AppModule` interfaces. If the module is only used for genesis, it will implement `AppModuleGenesis` instead of `AppModule`. The concrete type that implements the interface can add parameters that are required for the implementation of the various methods of the interface. + +```go +// example +type AppModule struct { + keeper Keeper +} +``` + +In the example above, you can see that the `AppModule` concrete type references an `AppModuleBasic`, and not an `AppModuleGenesis`. That is because `AppModuleGenesis` only needs to be implemented in modules that focus on genesis-related functionalities. In most modules, the concrete `AppModule` type will have a reference to an `AppModuleBasic` and implement the two added methods of `AppModuleGenesis` directly in the `AppModule` type. + +## Module Manager + +The module manager is used to manage collections of `appmodule.AppModule` and `AppModule` and all the extensions interfaces. + +### `Manager` + +The `Manager` is a structure that holds all the `AppModule` of an application, and defines the order of execution between several key components of these modules: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/module/module.go#L267-L276 +``` + +The module manager is used throughout the application whenever an action on a collection of modules is required. It implements the following methods: + +* `NewManager(modules ...AppModule)`: Constructor function. It takes a list of the application's `AppModule`s and builds a new `Manager`. It is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderInitGenesis(moduleNames ...string)`: Sets the order in which the [`InitGenesis`](./08-genesis.md#initgenesis) function of each module will be called when the application is first started. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). + To initialize modules successfully, module dependencies should be considered. For example, the `genutil` module must occur after `staking` module so that the pools are properly initialized with tokens from genesis accounts, the `genutils` module must also occur after `auth` so that it can access the params from auth, IBC's `capability` module should be initialized before all other modules so that it can initialize any capabilities. +* `SetOrderExportGenesis(moduleNames ...string)`: Sets the order in which the [`ExportGenesis`](./08-genesis.md#exportgenesis) function of each module will be called in case of an export. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPreBlockers(moduleNames ...string)`: Sets the order in which the `PreBlock()` function of each module will be called before `BeginBlock()` of all modules. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderBeginBlockers(moduleNames ...string)`: Sets the order in which the `BeginBlock()` function of each module will be called at the beginning of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderEndBlockers(moduleNames ...string)`: Sets the order in which the `EndBlock()` function of each module will be called at the end of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPrecommiters(moduleNames ...string)`: Sets the order in which the `Precommit()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderPrepareCheckStaters(moduleNames ...string)`: Sets the order in which the `PrepareCheckState()` function of each module will be called during commit of each block. This function is generally called from the application's main [constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). +* `SetOrderMigrations(moduleNames ...string)`: Sets the order of migrations to be run. If not set then migrations will be run with an order defined in `DefaultMigrationsOrder`. +* `RegisterServices(cfg Configurator)`: Registers the services of modules implementing the `HasServices` interface. +* `InitGenesis(ctx context.Context, genesisData map[string]json.RawMessage)`: Calls the [`InitGenesis`](./08-genesis.md#initgenesis) function of each module when the application is first started, in the order defined in `OrderInitGenesis`. Returns an `abci.InitChainResponse` to the underlying consensus engine, which can contain validator updates. +* `ExportGenesis(ctx context.Context)`: Calls the [`ExportGenesis`](./08-genesis.md#exportgenesis) function of each module, in the order defined in `OrderExportGenesis`. The export constructs a genesis file from a previously existing state, and is mainly used when a hard-fork upgrade of the chain is required. +* `ExportGenesisForModules(ctx context.Context, modulesToExport []string)`: Behaves the same as `ExportGenesis`, except takes a list of modules to export. +* `BeginBlock(ctx context.Context) error`: At the beginning of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#beginblock) and, in turn, calls the [`BeginBlock`](./06-preblock-beginblock-endblock.md) function of each modules implementing the `appmodule.HasBeginBlocker` interface, in the order defined in `OrderBeginBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from each modules. +* `EndBlock(ctx context.Context) error`: At the end of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-preblock-beginblock-endblock.md) function of each modules implementing the `appmodule.HasEndBlocker` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). +* `EndBlock(context.Context) ([]abci.ValidatorUpdate, error)`: At the end of each block, this function is called from [`BaseApp`](../../learn/advanced/00-baseapp.md#endblock) and, in turn, calls the [`EndBlock`](./06-preblock-beginblock-endblock.md) function of each modules implementing the `module.HasABCIEndBlock` interface, in the order defined in `OrderEndBlockers`. It creates a child [context](../../learn/advanced/02-context.md) with an event manager to aggregate [events](../../learn/advanced/08-events.md) emitted from all modules. The function returns an `abci` which contains the aforementioned events, as well as validator set updates (if any). +* `Precommit(ctx context.Context)`: During [`Commit`](../../learn/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately before the [`deliverState`](../../learn/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../learn/advanced/04-store.md#commitmultistore) and, in turn calls the `Precommit` function of each modules implementing the `HasPrecommit` interface, in the order defined in `OrderPrecommiters`. It creates a child [context](../../learn/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the newly committed block's [`finalizeblockstate`](../../learn/advanced/00-baseapp.md#state-updates). +* `PrepareCheckState(ctx context.Context)`: During [`Commit`](../../learn/advanced/00-baseapp.md#commit), this function is called from `BaseApp` immediately after the [`deliverState`](../../learn/advanced/00-baseapp.md#state-updates) is written to the underlying [`rootMultiStore`](../../learn/advanced/04-store.md#commitmultistore) and, in turn calls the `PrepareCheckState` function of each module implementing the `HasPrepareCheckState` interface, in the order defined in `OrderPrepareCheckStaters`. It creates a child [context](../../learn/advanced/02-context.md) where the underlying `CacheMultiStore` is that of the next block's [`checkState`](../../learn/advanced/00-baseapp.md#state-updates). Writes to this state will be present in the [`checkState`](../../learn/advanced/00-baseapp.md#state-updates) of the next block, and therefore this method can be used to prepare the `checkState` for the next block. +* (Optional) `RegisterLegacyAminoCodec(cdc *codec.LegacyAmino)`: Registers the [`codec.LegacyAmino`s](../../learn/advanced/05-encoding.md#amino) of each of the application module. This function is usually called early on in the [application's construction](../../learn/beginner/00-app-anatomy.md#constructor). +* `RegisterInterfaces(registry codectypes.InterfaceRegistry)`: Registers interface types and implementations of each of the application's `AppModule`. +* (Optional) `RegisterGRPCGatewayRoutes(clientCtx client.Context, rtr *runtime.ServeMux)`: Registers gRPC routes for modules. +* `DefaultGenesis(cdc codec.JSONCodec)`: Provides default genesis information for modules in the application by calling the [`DefaultGenesis(cdc codec.JSONCodec)`](./08-genesis.md#defaultgenesis) function of each module. It only calls the modules that implements the `HasGenesisBasics` interfaces. +* `ValidateGenesis(cdc codec.JSONCodec, txEncCfg client.TxEncodingConfig, genesis map[string]json.RawMessage)`: Validates the genesis information modules by calling the [`ValidateGenesis(codec.JSONCodec, client.TxEncodingConfig, json.RawMessage)`](./08-genesis.md#validategenesis) function of modules implementing the `HasGenesisBasics` interface. + +Here's an example of a concrete integration within an `simapp`: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app.go#L411-L434 +``` + +This is the same example from `runtime` (the package that powers app di): + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/runtime/module.go#L61 +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/runtime/module.go#L82 +``` diff --git a/versioned_docs/version-0.52/build/building-modules/02-messages-and-queries.md b/versioned_docs/version-0.52/build/building-modules/02-messages-and-queries.md new file mode 100644 index 000000000..5ec1c7a70 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/02-messages-and-queries.md @@ -0,0 +1,121 @@ +--- +sidebar_position: 1 +--- + +# Messages and Queries + +:::note Synopsis +`Msg`s and `Queries` are the two primary objects handled by modules. Most of the core components defined in a module, like `Msg` services, `keeper`s and `Query` services, exist to process `message`s and `queries`. +::: + +:::note Pre-requisite Readings + +* [Introduction to Cosmos SDK Modules](./00-intro.md) + +::: + +## Messages + +`Msg`s are objects whose end-goal is to trigger state-transitions. They are wrapped in [transactions](../../learn/advanced/01-transactions.md), which may contain one or more of them. + +When a transaction is relayed from the underlying consensus engine to the Cosmos SDK application, it is first decoded by [`BaseApp`](../../learn/advanced/00-baseapp.md). Then, each message contained in the transaction is extracted and routed to the appropriate module via `BaseApp`'s `MsgServiceRouter` so that it can be processed by the module's [`Msg` service](./03-msg-services.md). For a more detailed explanation of the lifecycle of a transaction, click [here](../../learn/beginner/01-tx-lifecycle.md). + +### `Msg` Services + +Defining Protobuf `Msg` services is the recommended way to handle messages. A Protobuf `Msg` service should be created for each module, typically in `tx.proto` (see more info about [conventions and naming](../../learn/advanced/05-encoding.md#faq)). It must have an RPC service method defined for each message in the module. + + +Each `Msg` service method must have exactly one argument, which must implement the `transaction.Msg` interface, and a Protobuf response. The naming convention is to call the RPC argument `Msg` and the RPC response `MsgResponse`. For example: + +```protobuf + rpc Send(MsgSend) returns (MsgSendResponse); +``` + +See an example of a `Msg` service definition from `x/bank` module: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/x/bank/proto/cosmos/bank/v1beta1/tx.proto#L13-L41 +``` + +### `transaction.Msg` Interface + +`transaction.Msg` is an alias of `proto.Message`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/core/transaction/transaction.go#L8 +``` + +To attach a `ValidateBasic()` method to a message, then you must add methods to the type adhereing to the `HasValidateBasic`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/9c1e8b247cd47b5d3decda6e86fbc3bc996ee5d7/types/tx_msg.go#L84-L88 +``` + +In 0.50+ signers from the `GetSigners()` call is automated via a protobuf annotation. + +Read more about the signer field [here](./05-protobuf-annotations.md). + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/e6848d99b55a65d014375b295bdd7f9641aac95e/proto/cosmos/bank/v1beta1/tx.proto#L40 +``` + +If there is a need for custom signers then there is an alternative path which can be taken. A function which returns `signing.CustomGetSigner` for a specific message can be defined. + +```go +func ProvideBankSendTransactionGetSigners() signing.CustomGetSigner { + + // Extract the signer from the signature. + signer, err := coretypes.LatestSigner(Tx).Sender(ethTx) + if err != nil { + return nil, err + } + + // Return the signer in the required format. + return [][]byte{signer.Bytes()}, nil +} +``` + +When using dependency injection (depinject) this can be provided to the application via the provide method. + +```go +depinject.Provide(banktypes.ProvideBankSendTransactionGetSigners) +``` + +The Cosmos SDK uses Protobuf definitions to generate client and server code: + +* `MsgServer` interface defines the server API for the `Msg` service and its implementation is described as part of the [`Msg` services](./03-msg-services.md) documentation. +* Structures are generated for all RPC request and response types. + +A `RegisterMsgServer` method is also generated and should be used to register the module's `MsgServer` implementation in `RegisterServices` method from the [`AppModule` interface](./01-module-manager.md#appmodule). + +In order for clients (CLI and grpc-gateway) to have these URLs registered, the Cosmos SDK provides the function `RegisterMsgServiceDesc(registry codectypes.InterfaceRegistry, sd *grpc.ServiceDesc)` that should be called inside module's [`RegisterInterfaces`](01-module-manager.md#hasregisterinterfaces) method, using the proto-generated `&_Msg_serviceDesc` as `*grpc.ServiceDesc` argument. + + +## Queries + +A `query` is a request for information made by end-users of applications through an interface and processed by a full-node. A `query` is received by a full-node through its consensus engine and relayed to the application via the ABCI. It is then routed to the appropriate module via `BaseApp`'s `QueryRouter` so that it can be processed by the module's query service (./04-query-services.md). For a deeper look at the lifecycle of a `query`, click [here](../../learn/beginner/02-query-lifecycle.md). + +### gRPC Queries + +Queries should be defined using [Protobuf services](https://protobuf.dev/programming-guides/proto2/). A `Query` service should be created per module in `query.proto`. This service lists endpoints starting with `rpc`. + +Here's an example of such a `Query` service definition: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/auth/v1beta1/query.proto#L14-L89 +``` + +As `proto.Message`s, generated `Response` types implement by default `String()` method of [`fmt.Stringer`](https://pkg.go.dev/fmt#Stringer). + +A `RegisterQueryServer` method is also generated and should be used to register the module's query server in the `RegisterServices` method from the [`AppModule` interface](./01-module-manager.md#appmodule). + + +### Store Queries + +Store queries query directly for store keys. They use `clientCtx.QueryABCI(req abci.QueryRequest)` to return the full `abci.QueryResponse` with inclusion Merkle proofs. + +See following examples: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/abci.go#L864-L894 +``` diff --git a/versioned_docs/version-0.52/build/building-modules/03-msg-services.md b/versioned_docs/version-0.52/build/building-modules/03-msg-services.md new file mode 100644 index 000000000..14f906119 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/03-msg-services.md @@ -0,0 +1,165 @@ +--- +sidebar_position: 1 +--- + +# `Msg` Services + +:::note Synopsis +A Protobuf `Msg` service processes [messages](./02-messages-and-queries.md#messages). Protobuf `Msg` services are specific to the module in which they are defined, and only process messages defined within the said module. They are called from `BaseApp` during [`FinalizeBlock`](../../learn/advanced/00-baseapp.md#finalizeblock). +::: + +:::note Pre-requisite Readings + +* [Module Manager](./01-module-manager.md) +* [Messages and Queries](./02-messages-and-queries.md) + +::: + +## Implementation of a module `Msg` service + +Each module should define a Protobuf `Msg` service, which will be responsible for processing requests (implementing `sdk.Msg`) and returning responses. + +As further described in [ADR 031](../architecture/adr-031-msg-service.md), this approach has the advantage of clearly specifying return types and generating server and client code. + +Protobuf generates a `MsgServer` interface based on the definition of `Msg` service. It is the role of the module developer to implement this interface, by implementing the state transition logic that should happen upon receival of each `transaction.Msg`. As an example, here is the generated `MsgServer` interface for `x/bank`, which exposes two `transaction.Msg`s: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/x/bank/types/tx.pb.go#L564-L579 +``` + +When possible, the existing module's [`Keeper`](./06-keeper.md) should implement `MsgServer`, otherwise a `msgServer` struct that embeds the `Keeper` can be created, typically in `./keeper/msg_server.go`: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/x/bank/keeper/msg_server.go#L16-L19 +``` + +`msgServer` methods can retrieve the auxiliary information or services using the environment variable, it is always located in the keeper: + +Environment: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/07151304e2ec6a185243d083f59a2d543253cb15/core/appmodule/v2/environment.go#L14-L29 +``` + +Keeper Example: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/07151304e2ec6a185243d083f59a2d543253cb15/x/bank/keeper/keeper.go#L56-L58 +``` + +`transaction.Msg` processing usually follows these 3 steps: + +### Validation + +The message server must perform all validation required (both *stateful* and *stateless*) to make sure the `message` is valid. +The `signer` is charged for the gas cost of this validation. + +For example, a `msgServer` method for a `transfer` message should check that the sending account has enough funds to actually perform the transfer. + +It is recommended to implement all validation checks in a separate function that passes state values as arguments. This implementation simplifies testing. As expected, expensive validation functions charge additional gas. Example: + +```go +ValidateMsgA(msg MsgA, now Time, gm GasMeter) error { + if now.Before(msg.Expire) { + return sdkerrors.ErrInvalidRequest.Wrap("msg expired") + } + gm.ConsumeGas(1000, "signature verification") + return signatureVerificaton(msg.Prover, msg.Data) +} +``` + +:::warning +Previously, the `ValidateBasic` method was used to perform simple and stateless validation checks. +This way of validating is deprecated, this means the `msgServer` must perform all validation checks. +::: + +### State Transition + +After the validation is successful, the `msgServer` method uses the [`keeper`](./06-keeper.md) functions to access the state and perform a state transition. + +### Events + +Before returning, `msgServer` methods generally emit one or more [events](../../learn/advanced/08-events.md) by using the `EventManager` held in `environment`. + +There are two ways to emit events, typed events using protobuf or arbitrary key & values. + +Typed Events: + +```go +ctx.EventManager().EmitTypedEvent( + &group.EventABC{Key1: Value1, Key2, Value2}) +``` + +Arbitrary Events: + +```go +ctx.EventManager().EmitEvent( + sdk.NewEvent( + eventType, // e.g. sdk.EventTypeMessage for a message, types.CustomEventType for a custom event defined in the module + sdk.NewAttribute(key1, value1), + sdk.NewAttribute(key2, value2), + ), +) +``` + +These events are relayed back to the underlying consensus engine and can be used by service providers to implement services around the application. Click [here](../../learn/advanced/08-events.md) to learn more about events. + +The invoked `msgServer` method returns a `proto.Message` response and an `error`. These return values are then wrapped into an `*sdk.Result` or an `error`: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/msg_service_router.go#L160 +``` + +This method takes care of marshaling the `res` parameter to protobuf and attaching any events on the `EventManager()` to the `sdk.Result`. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/base/abci/v1beta1/abci.proto#L93-L113 +``` + +This diagram shows a typical structure of a Protobuf `Msg` service, and how the message propagates through the module. + +```mermaid +sequenceDiagram + participant User + participant baseApp + participant router + participant handler + participant msgServer + participant keeper + participant EventManager + + User->>baseApp: Transaction Type + baseApp->>router: Route(ctx, msgRoute) + router->>handler: handler + handler->>msgServer: Msg(Context, Msg(..)) + + alt addresses invalid, denominations wrong, etc. + msgServer->>handler: error + handler->>router: error + router->>baseApp: result, error code + else + msgServer->>keeper: perform action, update context + keeper->>msgServer: results, error code + msgServer->>EventManager: Emit relevant events + msgServer->>msgServer: maybe wrap results in more structure + msgServer->>handler: result, error code + handler->>router: result, error code + router->>baseApp: result, error code + end + + baseApp->>User: result, error code +``` + +## Telemetry + +New [telemetry metrics](../../learn/advanced/09-telemetry.md) can be created from `msgServer` methods when handling messages. + +This is an example from the `x/auth/vesting` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/vesting/msg_server.go#L76-L88 +``` + +:::Warning +Telemetry adds a performance overhead to the chain. It is recommended to only use this in critical paths +::: diff --git a/versioned_docs/version-0.52/build/building-modules/04-query-services.md b/versioned_docs/version-0.52/build/building-modules/04-query-services.md new file mode 100644 index 000000000..a787a0c22 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/04-query-services.md @@ -0,0 +1,57 @@ +--- +sidebar_position: 1 +--- + +# Query Services + +:::note Synopsis +A Protobuf Query service processes [`queries`](./02-messages-and-queries.md#queries). Query services are specific to the module in which they are defined, and only process `queries` defined within said module. They are called from `BaseApp`'s [`Query` method](../../learn/advanced/00-baseapp.md#query). +::: + +:::note Pre-requisite Readings + +* [Module Manager](./01-module-manager.md) +* [Messages and Queries](./02-messages-and-queries.md) + +::: + +## Implementation of a module query service + +### gRPC Service + +When defining a Protobuf `Query` service, a `QueryServer` interface is generated for each module with all the service methods: + +```go +type QueryServer interface { + QueryBalance(context.Context, *QueryBalanceParams) (*types.Coin, error) + QueryAllBalances(context.Context, *QueryAllBalancesParams) (*QueryAllBalancesResponse, error) +} +``` + +These custom queries methods should be implemented by a module's keeper, typically in `./keeper/grpc_query.go`. The first parameter of these methods is a generic `context.Context`. Therefore, the Cosmos SDK provides a function `sdk.UnwrapSDKContext` to retrieve the `context.Context` from the provided +`context.Context`. + +Here's an example implementation for the bank module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/keeper/grpc_query.go +``` + +### Calling queries from the State Machine + +The Cosmos SDK v0.47 introduces a new `cosmos.query.v1.module_query_safe` Protobuf annotation which is used to state that a query that is safe to be called from within the state machine, for example: + +* a Keeper's query function can be called from another module's Keeper, +* ADR-033 intermodule query calls, +* CosmWasm contracts can also directly interact with these queries. + +If the `module_query_safe` annotation set to `true`, it means: + +* The query is deterministic: given a block height it will return the same response upon multiple calls, and doesn't introduce any state-machine breaking changes across SDK patch versions. +* Gas consumption never fluctuates across calls and across patch versions. + +If you are a module developer and want to use `module_query_safe` annotation for your own query, you have to ensure the following things: + +* the query is deterministic and won't introduce state-machine-breaking changes without coordinated upgrades +* it has its gas tracked, to avoid the attack vector where no gas is accounted for + on potentially high-computation queries. diff --git a/versioned_docs/version-0.52/build/building-modules/05-protobuf-annotations.md b/versioned_docs/version-0.52/build/building-modules/05-protobuf-annotations.md new file mode 100644 index 000000000..9843a6339 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/05-protobuf-annotations.md @@ -0,0 +1,158 @@ +--- +sidebar_position: 1 +--- + +# ProtocolBuffer Annotations + +This document explains the various protobuf scalars that have been added to make working with protobuf easier for Cosmos SDK application developers + +## Signer + +Signer specifies which field should be used to determine the signer of a message for the Cosmos SDK. This field can be used for clients as well to infer which field should be used to determine the signer of a message. + +Read more about the signer field [here](./02-messages-and-queries.md). + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/e6848d99b55a65d014375b295bdd7f9641aac95e/proto/cosmos/bank/v1beta1/tx.proto#L40 +``` + +```proto +option (cosmos.msg.v1.signer) = "from_address"; +``` + +## Scalar + +The scalar type defines a way for clients to understand how to construct protobuf messages according to what is expected by the module and sdk. + +```proto +(cosmos_proto.scalar) = "cosmos.AddressString" +``` + +Example of account address string scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e6848d99b55a65d014375b295bdd7f9641aac95e/proto/cosmos/bank/v1beta1/tx.proto#L46 +``` + +Example of validator address string scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/query.proto#L87 +``` + +Example of pubkey scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/11068bfbcd44a7db8af63b6a8aa079b1718f6040/proto/cosmos/staking/v1beta1/tx.proto#L94 +``` + +Example of Decimals scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/distribution.proto#L26 +``` + +Example of Int scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/gov/v1/gov.proto#L137 +``` + +There are a few options for what can be provided as a scalar: `cosmos.AddressString`, `cosmos.ValidatorAddressString`, `cosmos.ConsensusAddressString`, `cosmos.Int`, `cosmos.Dec`. + +## Implements_Interface + +Implement interface is used to provide information to client tooling like [telescope](https://github.com/cosmology-tech/telescope) on how to encode and decode protobuf messages. + +```proto +option (cosmos_proto.implements_interface) = "cosmos.auth.v1beta1.AccountI"; +``` + +## Method,Field,Message Added In + +`method_added_in`, `field_added_in` and `message_added_in` are annotations to denotate to clients that a field has been supported in a later version. This is useful when new methods or fields are added in later versions and that the client needs to be aware of what it can call. + +The annotation should be worded as follow: + +```proto +option (cosmos_proto.method_added_in) = "cosmos-sdk v0.50.1"; +option (cosmos_proto.method_added_in) = "x/epochs v1.0.0"; +option (cosmos_proto.method_added_in) = "simapp v24.0.0"; +``` + +## Amino + +The amino codec was removed in `v0.50+`, this means there is not a need register `legacyAminoCodec`. To replace the amino codec, Amino protobuf annotations are used to provide information to the amino codec on how to encode and decode protobuf messages. + +:::note +Amino annotations are only used for backwards compatibility with amino. New modules are not required use amino annotations. +::: + +The below annotations are used to provide information to the amino codec on how to encode and decode protobuf messages in a backwards compatible manner. + +### Name + +Name specifies the amino name that would show up for the user in order for them see which message they are signing. + +```proto +option (amino.name) = "cosmos-sdk/BaseAccount"; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/tx.proto#L41 +``` + +### Field_Name + +Field name specifies the amino name that would show up for the user in order for them see which field they are signing. + +```proto +uint64 height = 1 [(amino.field_name) = "public_key"]; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/distribution.proto#L166 +``` + +### Dont_OmitEmpty + +Dont omitempty specifies that the field should not be omitted when encoding to amino. + +```proto +repeated cosmos.base.v1beta1.Coin amount = 3 [(amino.dont_omitempty) = true]; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/bank.proto#L56 +``` + +### Encoding + +Encoding instructs the amino json marshaler how to encode certain fields that may differ from the standard encoding behaviour. The most common example of this is how `repeated cosmos.base.v1beta1.Coin` is encoded when using the amino json encoding format. The `legacy_coins` option tells the json marshaler [how to encode a null slice](https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/x/tx/signing/aminojson/json_marshal.go#L65) of `cosmos.base.v1beta1.Coin`. + +```proto +(amino.encoding) = "legacy_coins", +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/genesis.proto#L23 +``` + +Another example is a protobuf `bytes` that contains a valid JSON document. +The `inline_json` option tells the json marshaler to embed the JSON bytes into the wrapping document without escaping. + +```proto +(amino.encoding) = "inline_json", +``` + +E.g. the bytes containing `{"foo":123}` in the `envelope` field would lead to the following JSON: + +```json +{ + "envelope": { + "foo": 123 + } +} +``` + +If the bytes are not valid JSON, this leads to JSON broken documents. Thus a JSON validity check needs to be in place at some point of the process. diff --git a/versioned_docs/version-0.52/build/building-modules/06-keeper.md b/versioned_docs/version-0.52/build/building-modules/06-keeper.md new file mode 100644 index 000000000..0bd776ff9 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/06-keeper.md @@ -0,0 +1,69 @@ +--- +sidebar_position: 1 +--- + +# Keepers + +:::note Synopsis +`Keeper`s refer to a Cosmos SDK abstraction whose role is to manage access to the subset of the state defined by various modules. `Keeper`s are module-specific, i.e. the subset of state defined by a module can only be accessed by a `keeper` defined in said module. If a module needs to access the subset of state defined by another module, a reference to the second module's internal `keeper` needs to be passed to the first one. This is done in `app.go` during the instantiation of module keepers. +::: + +:::note Pre-requisite Readings + +* [Introduction to Cosmos SDK Modules](./00-intro.md) + +::: + +## Motivation + +The Cosmos SDK is a framework that makes it easy for developers to build complex decentralized applications from scratch, mainly by composing modules together. As the ecosystem of open-source modules for the Cosmos SDK expands, it will become increasingly likely that some of these modules contain vulnerabilities, as a result of the negligence or malice of their developer. + +The Cosmos SDK adopts an [object-capabilities-based approach](https://docs.cosmos.network/main/learn/advanced/ocap#ocaps-in-practice) to help developers better protect their application from unwanted inter-module interactions, and `keeper`s are at the core of this approach. A `keeper` can be considered quite literally to be the gatekeeper of a module's store(s). Each store (typically an [`IAVL` Store](../../learn/advanced/04-store.md#iavl-store)) defined within a module comes with a `storeKey`, which grants unlimited access to it. The module's `keeper` holds this `storeKey` (which should otherwise remain unexposed), and defines [methods](#implementing-methods) for reading and writing to the store(s). + +The core idea behind the object-capabilities approach is to only reveal what is necessary to get the work done. In practice, this means that instead of handling permissions of modules through access-control lists, module `keeper`s are passed a reference to the specific instance of the other modules' `keeper`s that they need to access (this is done in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function)). As a consequence, a module can only interact with the subset of state defined in another module via the methods exposed by the instance of the other module's `keeper`. This is a great way for developers to control the interactions that their own module can have with modules developed by external developers. + +## Type Definition + +`keeper`s are generally implemented in a `/keeper/keeper.go` file located in the module's folder. By convention, the type `keeper` of a module is simply named `Keeper` and usually follows the following structure: + +```go +type Keeper struct { + // External keepers, if any + + // Store key(s) + + // codec + + // authority +} +``` + +For example, here is the type definition of the `keeper` from the `staking` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/staking/keeper/keeper.go#L54-L115 +``` + +Let us go through the different parameters: + +* An expected `keeper` is a `keeper` external to a module that is required by the internal `keeper` of said module. External `keeper`s are listed in the internal `keeper`'s type definition as interfaces. These interfaces are themselves defined in an `expected_keepers.go` file in the root of the module's folder. In this context, interfaces are used to reduce the number of dependencies, as well as to facilitate the maintenance of the module itself. +* `KVStoreService`s grant access to the store(s) of the [multistore](../../learn/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules. +* `cdc` is the [codec](../../learn/advanced/05-encoding.md) used to marshal and unmarshal structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces. +* The authority listed is a module account or user account that has the right to change module level parameters. Previously this was handled by the param module, which has been deprecated. + +Of course, it is possible to define different types of internal `keeper`s for the same module (e.g. a read-only `keeper`). Each type of `keeper` comes with its own constructor function, which is called from the [application's constructor function](../../learn/beginner/00-app-anatomy.md). This is where `keeper`s are instantiated, and where developers make sure to pass correct instances of modules' `keeper`s to other modules that require them. + +## Implementing Methods + +`Keeper`s primarily expose methods for business logic, as validity checks should have already been performed by the [`Msg` server](./03-msg-services.md) when `keeper`s' methods are called. + + +State management is recommended to be done via [Collections](../packages/collections) + + +## State Management + +In the Cosmos SDK, it is crucial to be methodical and selective when managing state within a module, as improper state management can lead to inefficiency, security risks, and scalability issues. Not all data belongs in the on-chain state; it's important to store only essential blockchain data that needs to be verified by consensus. Storing unnecessary information, especially client-side data, can bloat the state and slow down performance. Instead, developers should focus on using an off-chain database to handle supplementary data, extending the API as needed. This approach minimizes on-chain complexity, optimizes resource usage, and keeps the blockchain state lean and efficient, ensuring scalability and smooth operations. + + +The Cosmos SDK leverages Protocol Buffers (protobuf) for efficient state management, providing a well-structured, binary encoding format that ensures compatibility and performance across different modules. The SDK’s recommended approach for managing state is through the [collections package](../pacakges/02-collections.md), which simplifies state handling by offering predefined data structures like maps and indexed sets, reducing the complexity of managing raw state data. While users can opt for custom encoding schemes if they need more flexibility or have specialized requirements, they should be aware that such custom implementations may not integrate seamlessly with indexers that decode state data on the fly. This could lead to challenges in data retrieval, querying, and interoperability, making protobuf a safer and more future-proof choice for most use cases. diff --git a/versioned_docs/version-0.52/build/building-modules/06-preblock-beginblock-endblock.md b/versioned_docs/version-0.52/build/building-modules/06-preblock-beginblock-endblock.md new file mode 100644 index 000000000..a7a890eff --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/06-preblock-beginblock-endblock.md @@ -0,0 +1,71 @@ +--- +sidebar_position: 1 +--- + +# PreBlocker, BeginBlocker and EndBlocker + +:::note Synopsis +`PreBlocker`, `BeginBlocker` and `EndBlocker` are optional methods module developers can implement in their module. +They will be triggered at the beginning and at the end of each block respectively, when the [`BeginBlock`](../../learn/advanced/00-baseapp.md#beginblock) and [`EndBlock`](../../learn/advanced/00-baseapp.md#endblock) inside within ABCI `FinalizeBlock` +::: + +:::note Pre-requisite Readings + +* [Module Manager](./01-module-manager.md) + +::: + +## PreBlocker + +There are two semantics around the new lifecycle method: + +* It runs before the `BeginBlocker` of all modules +* It can modify consensus parameters in storage, and signal the caller through the return value. + +:::warning +Modules are required to get the consensus params from the consensus module. Consensus params located in `sdk.Context` were deprecated and should be treated as unsafe. `sdk.Context` is deprecated due to it being a global state within the entire state machine, it has been replaced with `appmodule.Environment`. +::: + +## BeginBlocker and EndBlocker + +`BeginBlocker` and `EndBlocker` are a way for module developers to add automatic execution of logic to their module. This is a powerful tool that should be used carefully, as complex automatic functions can slow down or even halt the chain. + +In 0.47.0, `PrepareProposal` and `ProcessProposal` were added that allow app developers to do arbitrary work at those phases, but they do not influence the work that will be done in `BeginBlock`, nor are they accessible from modules. + +When needed, `BeginBlocker` and `EndBlocker` are implemented as part of the [`HasBeginBlocker`, `HasABCIEndBlocker` and `EndBlocker` interfaces](./01-module-manager.md#appmodule). This means either can be left-out if not required. The `BeginBlock` and `EndBlock` methods of the interface implemented in `module.go` generally defer to `BeginBlocker` and `EndBlocker` methods respectively, which are usually implemented in `abci.go`. + +The actual implementation of `BeginBlocker` and `EndBlocker` in `abci.go` are very similar to that of a [`Msg` service](./03-msg-services.md): + +* They generally use the [`keeper`](./06-keeper.md) and [`ctx`](https://pkg.go.dev/context) to retrieve information about the latest state. +* If needed, they use the `keeper` and `ctx` to trigger state-transitions. +* If needed, they can emit [`events`](../../learn/advanced/08-events.md) via the `environments`'s `EventManager`. + +A specific method (`UpdateValidators`) is available to return validator updates to the underlying consensus engine in the form of an [`[]appmodule.ValidatorUpdates`](https://github.com/cosmos/cosmos-sdk/blob/07151304e2ec6a185243d083f59a2d543253cb15/core/appmodule/v2/module.go#L87-L101). This is the preferred way to implement custom validator changes (in v1). + +It is possible for developers to define the order of execution between the `BeginBlocker`/`EndBlocker` functions of each of their application's modules via the module's manager `SetOrderBeginBlocker`/`SetOrderEndBlocker` methods. For more on the module manager, click [here](./01-module-manager.md#manager). + +### Implementation + +A module must implement those core interface to make use of the `PreBlocker`, `BeginBlocker` or `EndBlocker` capabilities: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/core/v1.0.0-alpha.4/core/appmodule/v2/module.go#L22-L48 +``` + +See an example implementation of `BeginBlocker` from the `distribution` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/distribution/keeper/abci.go#L13-L40 +``` + +and an example of `EndBlocker` from the `gov` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/keeper/abci.go#L22 +``` + +and an example implementation of `EndBlocker` with validator updates from the `staking` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/staking/keeper/abci.go#L12-L17 +``` diff --git a/versioned_docs/version-0.52/build/building-modules/08-genesis.md b/versioned_docs/version-0.52/build/building-modules/08-genesis.md new file mode 100644 index 000000000..b7b35478b --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/08-genesis.md @@ -0,0 +1,78 @@ +--- +sidebar_position: 1 +--- + +# Module Genesis + +:::note Synopsis +Modules generally handle a subset of the state and, as such, they need to define the related subset of the genesis file as well as methods to initialize, verify and export it. +::: + +:::note Pre-requisite Readings + +* [Module Manager](./01-module-manager.md) +* [Keepers](./06-keeper.md) + +::: + +## Type Definition + +The subset of the genesis state defined from a given module is generally defined in a `genesis.proto` file ([more info](../../learn/advanced/05-encoding.md#gogoproto) on how to define protobuf messages). The struct defining the module's subset of the genesis state is usually called `GenesisState` and contains all the module-related values that need to be initialized during the genesis process. + +See an example of `GenesisState` protobuf message definition from the `auth` module: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/auth/v1beta1/genesis.proto +``` + +Next we present the main genesis-related methods that need to be implemented by module developers in order for their module to be used in Cosmos SDK applications. + +### `DefaultGenesis` + +The `DefaultGenesis()` method is a simple method that calls the constructor function for `GenesisState` with the default value for each parameter. See an example from the `auth` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/module.go#L63-L67 +``` + +### `ValidateGenesis` + +The `ValidateGenesis(data GenesisState)` method is called to verify that the provided `genesisState` is correct. It should perform validity checks on each of the parameters listed in `GenesisState`. See an example from the `auth` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/types/genesis.go#L62-L75 +``` + +## Other Genesis Methods + +Other than the methods related directly to `GenesisState`, module developers are expected to implement two other methods as part of the [`AppModuleGenesis` interface](./01-module-manager.md#appmodulegenesis) (only if the module needs to initialize a subset of state in genesis). These methods are [`InitGenesis`](#initgenesis) and [`ExportGenesis`](#exportgenesis). + +### `InitGenesis` + +The `InitGenesis` method is executed during [`InitChain`](../../learn/advanced/00-baseapp.md#initchain) when the application is first started. Given a `GenesisState`, it initializes the subset of the state managed by the module by using the module's [`keeper`](./06-keeper.md) setter function on each parameter within the `GenesisState`. + +The [module manager](./01-module-manager.md#manager) of the application is responsible for calling the `InitGenesis` method of each of the application's modules in order. This order is set by the application developer via the manager's `SetOrderGenesisMethod`, which is called in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function). + +See an example of `InitGenesis` from the `auth` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/452129d6aa45134f598f05be13f3fd961ff9734e/x/auth/keeper/genesis.go#L12-L43 +``` + +### `ExportGenesis` + +The `ExportGenesis` method is executed whenever an export of the state is made. It takes the latest known version of the subset of the state managed by the module and creates a new `GenesisState` out of it. This is mainly used when the chain needs to be upgraded via a hard fork. + +See an example of `ExportGenesis` from the `auth` module. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/452129d6aa45134f598f05be13f3fd961ff9734e/x/auth/keeper/genesis.go#L45-L60 +``` + +### GenesisTxHandler + +`GenesisTxHandler` is a way for modules to submit state transitions prior to the first block. This is used by `x/genutil` to submit the genesis transactions for the validators to be added to staking. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/core/genesis/txhandler.go#L3-L6 +``` diff --git a/versioned_docs/version-0.52/build/building-modules/09-module-interfaces.md b/versioned_docs/version-0.52/build/building-modules/09-module-interfaces.md new file mode 100644 index 000000000..c579aa82d --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/09-module-interfaces.md @@ -0,0 +1,94 @@ +--- +sidebar_position: 1 +--- + +# Module Interfaces + +:::note Synopsis +This document details how to build CLI and REST interfaces for a module. Examples from various Cosmos SDK modules are included. +::: + +:::note Pre-requisite Readings + +* [Building Modules Intro](./00-intro.md) + +::: + +## CLI + +One of the main interfaces for an application is the [command-line interface](../../learn/advanced/07-cli.md). This entrypoint adds commands from the application's modules enabling end-users to create [**messages**](./02-messages-and-queries.md#messages) wrapped in transactions and [**queries**](./02-messages-and-queries.md#queries). The CLI files are typically found in the module's `./client/cli` folder. + +### Transaction Commands + +In order to create messages that trigger state changes, end-users must create [transactions](../../learn/advanced/01-transactions.md) that wrap and deliver the messages. A transaction command creates a transaction that includes one or more messages. + +Transaction commands typically have their own `tx.go` file that lives within the module's `./client/cli` folder. The commands are specified in getter functions and the name of the function should include the name of the command. + +Here is an example from the `x/bank` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/client/cli/tx.go#L37-L76 +``` + +In the example, `NewSendTxCmd()` creates and returns the transaction command for a transaction that wraps and delivers `MsgSend`. `MsgSend` is the message used to send tokens from one account to another. + +In general, the getter function does the following: + +* **Constructs the command:** Read the [Cobra Documentation](https://pkg.go.dev/github.com/spf13/cobra) for more detailed information on how to create commands. + * **Use:** Specifies the format of the user input required to invoke the command. In the example above, `send` is the name of the transaction command and `[from_key_or_address]`, `[to_address]`, and `[amount]` are the arguments. + * **Args:** The number of arguments the user provides. In this case, there are exactly three: `[from_key_or_address]`, `[to_address]`, and `[amount]`. + * **Short and Long:** Descriptions for the command. A `Short` description is expected. A `Long` description can be used to provide additional information that is displayed when a user adds the `--help` flag. + * **RunE:** Defines a function that can return an error. This is the function that is called when the command is executed. This function encapsulates all of the logic to create a new transaction. + * The function typically starts by getting the `clientCtx`, which can be done with `client.GetClientTxContext(cmd)`. The `clientCtx` contains information relevant to transaction handling, including information about the user. In this example, the `clientCtx` is used to retrieve the address of the sender by calling `clientCtx.GetFromAddress()`. + * If applicable, the command's arguments are parsed. In this example, the arguments `[to_address]` and `[amount]` are both parsed. + * A [message](./02-messages-and-queries.md) is created using the parsed arguments and information from the `clientCtx`. The constructor function of the message type is called directly. In this case, `types.NewMsgSend(fromAddr, toAddr, amount)`. Its good practice to call, if possible, the necessary [message validation methods](../building-modules/03-msg-services.md#Validation) before broadcasting the message. + * Depending on what the user wants, the transaction is either generated offline or signed and broadcasted to the preconfigured node using `tx.GenerateOrBroadcastTxCLI(clientCtx, flags, msg)`. +* **Adds transaction flags:** All transaction commands must add a set of transaction flags. The transaction flags are used to collect additional information from the user (e.g. the amount of fees the user is willing to pay). The transaction flags are added to the constructed command using `AddTxFlagsToCmd(cmd)`. +* **Returns the command:** Finally, the transaction command is returned. + +Each module can implement `NewTxCmd()`, which aggregates all of the transaction commands of the module. Here is an example from the `x/bank` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/client/cli/tx.go#L20-L35 +``` + +Each module then can also implement a `GetTxCmd()` method that simply returns `NewTxCmd()`. This allows the root command to easily aggregate all of the transaction commands for each module. Here is an example: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/module.go#L84-L86 +``` + +### Query Commands + +:::warning +This section is being rewritten. Refer to [AutoCLI](https://docs.cosmos.network/main/core/autocli) while this section is being updated. +::: + +## gRPC + +[gRPC](https://grpc.io/) is a Remote Procedure Call (RPC) framework. RPC is the preferred way for external clients like wallets and exchanges to interact with a blockchain. + +In addition to providing an ABCI query pathway, the Cosmos SDK provides a gRPC proxy server that routes gRPC query requests to ABCI query requests. + +In order to do that, modules must implement the `module.HasGRPCGateway` interface on `AppModule` to wire the client gRPC requests to the correct handler inside the module. + +Here's an example from the `x/auth` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/module.go#L71-L76 +``` + +## gRPC-gateway REST + +Applications need to support web services that use HTTP requests (e.g. a web wallet like [Keplr](https://keplr.app)). [grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) translates REST calls into gRPC calls, which might be useful for clients that do not use gRPC. + +Modules that want to expose REST queries should add `google.api.http` annotations to their `rpc` methods, such as in the example below from the `x/auth` module: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/auth/v1beta1/query.proto#L14-L89 +``` + + +gRPC gateway is started in-process along with the application and CometBFT. It can be enabled or disabled by setting gRPC Configuration `enable` in [`app.toml`](../run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml). + +The Cosmos SDK provides a command for generating [Swagger](https://swagger.io/) documentation (`protoc-gen-swagger`). Setting `swagger` in [`app.toml`](../run-node/01-run-node.md#configuring-the-node-using-apptoml-and-configtoml) defines if swagger documentation should be automatically registered. diff --git a/versioned_docs/version-0.52/build/building-modules/11-structure.md b/versioned_docs/version-0.52/build/building-modules/11-structure.md new file mode 100644 index 000000000..95750c4e4 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/11-structure.md @@ -0,0 +1,99 @@ +--- +sidebar_position: 1 +--- + +# Folder Structure + +:::note Synopsis +This document outlines the structure of Cosmos SDK modules. These ideas are meant to be applied as suggestions. Application developers are encouraged to improve upon and contribute to module structure and development design. + +The required interface for a module is located in the module.go. Everything beyond this is suggestive. +::: + +## Structure + +A typical Cosmos SDK module can be structured as follows: + +```shell +proto +└── {project_name} +    └── {module_name} +    └── {proto_version} +       ├── {module_name}.proto +       ├── genesis.proto +       ├── query.proto +       └── tx.proto +``` + +* `{module_name}.proto`: The module's common message type definitions. +* `genesis.proto`: The module's message type definitions related to genesis state. +* `query.proto`: The module's Query service and related message type definitions. +* `tx.proto`: The module's Msg service and related message type definitions. + +```shell +x/{module_name} +├── client +│   ├── cli +│   │ ├── query.go +│   │   └── tx.go +│   └── testutil +│   ├── cli_test.go +│   └── suite.go +├── exported +│   └── exported.go +├── keeper +│   ├── genesis.go +│   ├── grpc_query.go +│   ├── hooks.go +│   ├── invariants.go +│   ├── keeper.go +│   ├── keys.go +│   ├── msg_server.go +│   └── querier.go +├── simulation +│   ├── decoder.go +│   ├── genesis.go +│   ├── operations.go +│   └── params.go +├── types +│   ├── {module_name}.pb.go +│ ├── codec.go +│ ├── errors.go +│ ├── events.go +│ ├── events.pb.go +│ ├── expected_keepers.go +│ ├── genesis.go +│ ├── genesis.pb.go +│ ├── keys.go +│ ├── msgs.go +│ ├── params.go +│ ├── query.pb.go +│ └── tx.pb.go +├── module.go +├── abci.go +├── autocli.go +├── depinject.go +└── README.md +``` + +* `client/`: The module's CLI client functionality implementation and the module's CLI testing suite. +* `exported/`: The module's exported types - typically interface types. If a module relies on keepers from another module, it is expected to receive the keepers as interface contracts through the `expected_keepers.go` file (see below) in order to avoid a direct dependency on the module implementing the keepers. However, these interface contracts can define methods that operate on and/or return types that are specific to the module that is implementing the keepers and this is where `exported/` comes into play. The interface types that are defined in `exported/` use canonical types, allowing for the module to receive the keepers as interface contracts through the `expected_keepers.go` file. This pattern allows for code to remain DRY and also alleviates import cycle chaos. +* `keeper/`: The module's `Keeper` and `MsgServer` implementation. + * `abci.go`: The module's `BeginBlocker` and `EndBlocker` implementations (this file is only required if `BeginBlocker` and/or `EndBlocker` need to be defined). +* `simulation/`: The module's [simulation](./14-simulator.md) package defines functions used by the blockchain simulator application (`simapp`). +* `README.md`: The module's specification documents outlining important concepts, state storage structure, and message and event type definitions. Learn more how to write module specs in the [spec guidelines](../spec/SPEC_MODULE.md). +* `types/`: includes type definitions for messages, events, and genesis state, including the type definitions generated by Protocol Buffers. + * `codec.go`: The module's registry methods for interface types. + * `errors.go`: The module's sentinel errors. + * `events.go`: The module's event types and constructors. + * `expected_keepers.go`: The module's [expected keeper](./06-keeper.md#type-definition) interfaces. + * `genesis.go`: The module's genesis state methods and helper functions. + * `keys.go`: The module's store keys and associated helper functions. + * `msgs.go`: The module's message type definitions and associated methods. + * `params.go`: The module's parameter type definitions and associated methods. + * `*.pb.go`: The module's type definitions generated by Protocol Buffers (as defined in the respective `*.proto` files above). +* The root directory includes the module's `AppModule` implementation. + * `autocli.go`: The module [autocli](https://docs.cosmos.network/main/core/autocli) options. + * `depinject.go`: The module [depinject](./15-depinject.md#type-definition) options. + +> Note: although the above pattern is followed by most of the Cosmos SDK modules, there are some modules that don't follow this pattern. E.g `x/group` and `x/nft` dont have a `types` folder, instead all of the type definitions for messages, events, and genesis state are live in the root directory and the module's `AppModule` implementation lives in the `module` folder. diff --git a/versioned_docs/version-0.52/build/building-modules/12-errors.md b/versioned_docs/version-0.52/build/building-modules/12-errors.md new file mode 100644 index 000000000..20c7c5121 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/12-errors.md @@ -0,0 +1,59 @@ +--- +sidebar_position: 1 +--- + +# Errors + +:::note Synopsis +This document outlines the recommended usage and APIs for error handling in Cosmos SDK modules. +::: + +Modules are encouraged to define and register their own errors to provide better +context on failed message or handler execution. Typically, these errors should be +common or general errors which can be further wrapped to provide additional specific execution context. + +There are two ways to return errors. You can register custom errors with a codespace that is meant to provide more information to clients and normal go errors. The Cosmos SDK uses a mixture of both. + +:::Note +Errors v2 has been created as a zero dependency errors package. GRPC errors and tracing support is removed natively from the errors package. Users are required to wrap stack traces and add tracing information to their errors. +::: + +:::Warning +If errors are registered they are part of consensus and cannot be changed in a minor release +::: + +## Registration + +Modules should define and register their custom errors in `x/{module}/errors.go`. +Registration of errors is handled via the [`errors` package](https://github.com/cosmos/cosmos-sdk/blob/main/errors/errors.go). + +Example: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/distribution/types/errors.go +``` + +Each custom module error must provide the codespace, which is typically the module name +(e.g. "distribution") and is unique per module, and a uint32 code. Together, the codespace and code +provide a globally unique Cosmos SDK error. Typically, the code is monotonically increasing but does not +necessarily have to be. The only restrictions on error codes are the following: + +* Must be greater than one, as a code value of one is reserved for internal errors. +* Must be unique within the module. + +## Wrapping + +The custom module errors can be returned as their concrete type as they already fulfill the `error` +interface. However, module errors can be wrapped to provide further context and meaning to failed execution. + +Example: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/keeper/keeper.go#L141-L182 +``` + +## ABCI + +If a module error is registered, the Cosmos SDK `errors` package allows ABCI information to be extracted +through the `ABCIInfo` function. The package also provides `ResponseCheckTx` and `ResponseDeliverTx` as +auxiliary functions to automatically get `CheckTx` and `DeliverTx` responses from an error. diff --git a/versioned_docs/version-0.52/build/building-modules/13-upgrade.md b/versioned_docs/version-0.52/build/building-modules/13-upgrade.md new file mode 100644 index 000000000..fc3dc384b --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/13-upgrade.md @@ -0,0 +1,65 @@ +--- +sidebar_position: 1 +--- + +# Upgrading Modules + +:::note Synopsis +[In-Place Store Migrations](../../learn/advanced/15-upgrade.md) allow your modules to upgrade to new versions that include breaking changes. This document outlines how to build modules to take advantage of this functionality. +::: + +:::note Pre-requisite Readings + +* [In-Place Store Migration](../../learn/advanced/15-upgrade.md) + +::: + +## Consensus Version + +Successful upgrades of existing modules require each `AppModule` to implement the function `ConsensusVersion() uint64`. + +* The versions must be hard-coded by the module developer. +* The initial version **must** be set to 1. + +Consensus versions serve as state-breaking versions of app modules and must be incremented when the module introduces breaking changes. + +## Registering Migrations + +To register the functionality that takes place during a module upgrade, you must register which migrations you want to take place. + +Migration registration takes place in the `Configurator` using the `RegisterMigration` method. The `AppModule` reference to the configurator is in the `RegisterServices` method. + +You can register one or more migrations. If you register more than one migration script, list the migrations in increasing order and ensure there are enough migrations that lead to the desired consensus version. For example, to migrate to version 3 of a module, register separate migrations for version 1 and version 2 as shown in the following example: + +```go +func (am AppModule) RegisterServices(cfg module.Configurator) { + // --snip-- + cfg.RegisterMigration(types.ModuleName, 1, func(ctx sdk.Context) error { + // Perform in-place store migrations from ConsensusVersion 1 to 2. + }) + cfg.RegisterMigration(types.ModuleName, 2, func(ctx sdk.Context) error { + // Perform in-place store migrations from ConsensusVersion 2 to 3. + }) +} +``` + +Since these migrations are functions that need access to a Keeper's store, use a wrapper around the keepers called `Migrator` as shown in this example: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/keeper/migrations.go +``` + + + +## Writing Migration Scripts + +To define the functionality that takes place during an upgrade, write a migration script and place the functions in a `migrations/` directory. For example, to write migration scripts for the bank module, place the functions in `x/bank/migrations/`. Use the recommended naming convention for these functions. For example, `v2bank` is the script that migrates the package `x/bank/migrations/v2`: + +```go +// Migrating bank module from version 1 to 2 +func (m Migrator) Migrate1to2(ctx sdk.Context) error { + return v2bank.MigrateStore(ctx, m.keeper.storeKey) // v2bank is package `x/bank/migrations/v2`. +} +``` + +To see example code of changes that were implemented in a migration of balance keys, check out [migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/migrations/v2/store.go#L55-L76). For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](../architecture/adr-028-public-key-addresses.md). diff --git a/versioned_docs/version-0.52/build/building-modules/14-simulator.md b/versioned_docs/version-0.52/build/building-modules/14-simulator.md new file mode 100644 index 000000000..966df4cb8 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/14-simulator.md @@ -0,0 +1,139 @@ +--- +sidebar_position: 1 +--- + +# Module Simulation + +:::note Pre-requisite Readings + +* [Cosmos Blockchain Simulator](../../learn/advanced/12-simulation.md) +::: + +## Synopsis + +This document details how to define each module simulation functions to be +integrated with the application `SimulationManager`. + +* [Simulation package](#simulation-package) + * [Store decoders](#store-decoders) + * [Randomized genesis](#randomized-genesis) + * [Random weighted operations](#random-weighted-operations) + * [Random proposal contents](#random-proposal-contents) +* [Registering simulation functions](#registering-simulation-functions) +* [App Simulator manager](#app-simulator-manager) + +## Simulation package + +Every module that implements the Cosmos SDK simulator needs to have a `x//simulation` +package which contains the primary functions required by the fuzz tests: store +decoders, randomized genesis state and parameters, weighted operations and proposal +contents. + +### Store decoders + +Registering the store decoders is required for the `AppImportExport`. This allows +for the key-value pairs from the stores to be decoded (_i.e_ unmarshalled) +to their corresponding types. In particular, it matches the key to a concrete type +and then unmarshals the value from the `KVPair` to the type provided. + +You can use the example [here](https://github.com/cosmos/cosmos-sdk/blob/main/x/distribution/simulation/decoder.go) from the distribution module to implement your store decoders. + +If the module uses the `collections` package, you can use the example [here](https://github.com/cosmos/cosmos-sdk/blob/23cf89cce1882ba9c8280e64735ae200504bfdce/x/bank/module.go#L166) from the Bank module to implement your store decoders. + +### Randomized genesis + +The simulator tests different scenarios and values for genesis parameters +in order to fully test the edge cases of specific modules. The `simulator` package from each module must expose a `RandomizedGenState` function to generate the initial random `GenesisState` from a given seed. + +Once the module genesis parameter are generated randomly (or with the key and +values defined in a `params` file), they are marshaled to JSON format and added +to the app genesis JSON to use it on the simulations. + +You can check an example on how to create the randomized genesis [here](https://github.com/cosmos/cosmos-sdk/blob/main/x/staking/simulation/genesis.go). + +### Random weighted operations + +Operations are one of the crucial parts of the Cosmos SDK simulation. They are the transactions +(`Msg`) that are simulated with random field values. The sender of the operation +is also assigned randomly. + +Operations on the simulation are simulated using the full [transaction cycle](../../learn/advanced/01-transactions.md) of a +`ABCI` application that exposes the `BaseApp`. + +Shown below is how weights are set: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/23cf89cce1882ba9c8280e64735ae200504bfdce/x/staking/depinject.go#L144-L154 +``` + +As you can see, the weights are predefined in this case. Options exist to override this behavior with different weights. One option is to use `*rand.Rand` to define a random weight for the operation, or you can inject your own predefined weights. + +The SDK simulations can be executed like normal tests in Go from the shell or within an IDE. +Make sure that you pass the `-tags='sims` parameter to enable them and other params that make sense for your scenario. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/23cf89cce1882ba9c8280e64735ae200504bfdce/scripts/build/simulations.mk#L19 +``` + +### Random proposal contents + +Randomized governance proposals are also supported on the Cosmos SDK simulator. Each +module must register the message to be used for governance proposals. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/23cf89cce1882ba9c8280e64735ae200504bfdce/x/staking/depinject.go#L139-L142 +``` + +## Registering simulation functions + +Now that all the required functions are defined, we need to integrate them into the module pattern within the `module.go`: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/23cf89cce1882ba9c8280e64735ae200504bfdce/x/staking/depinject.go#L127-L154 +``` + +## App Simulator manager + +The following step is setting up the `SimulatorManager` at the app level. This +is required for the simulation test files on the next step. + +```go +type CustomApp struct { + ... + sm *module.SimulationManager +} +``` + +Then at the instantiation of the application, we create the `SimulationManager` +instance in the same way we create the `ModuleManager` but this time we only pass +the modules that implement the simulation functions from the `AppModuleSimulation` +interface described above. + +```go +func NewCustomApp(...) { + // create the simulation manager and define the order of the modules for deterministic simulations + app.sm = module.NewSimulationManager( + auth.NewAppModule(app.accountKeeper), + bank.NewAppModule(app.bankKeeper, app.accountKeeper), + supply.NewAppModule(app.supplyKeeper, app.accountKeeper), + gov.NewAppModule(app.govKeeper, app.accountKeeper, app.supplyKeeper), + mint.NewAppModule(app.mintKeeper), + distr.NewAppModule(app.distrKeeper, app.accountKeeper, app.supplyKeeper, app.stakingKeeper), + staking.NewAppModule(cdc, app.stakingKeeper), + slashing.NewAppModule(app.slashingKeeper, app.accountKeeper, app.stakingKeeper), + ) + + // register the store decoders for simulation tests + app.sm.RegisterStoreDecoders() + ... +} +``` + +## Integration with the Go fuzzer framework + +The simulations provide deterministic behaviour already. The integration with the [Go fuzzer](https://go.dev/doc/security/fuzz/) +can be done at a high level with the deterministic pseudo random number generator where the fuzzer provides varying numbers. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/23cf89cce1882ba9c8280e64735ae200504bfdce/scripts/build/simulations.mk#L80-L84 +``` diff --git a/versioned_docs/version-0.52/build/building-modules/15-depinject.md b/versioned_docs/version-0.52/build/building-modules/15-depinject.md new file mode 100644 index 000000000..c07c9aa96 --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/15-depinject.md @@ -0,0 +1,130 @@ +--- +sidebar_position: 1 +--- + +# Modules depinject-ready + +:::note Pre-requisite Readings + +* [Depinject Documentation](../packages/01-depinject.md) + +::: + +[`depinject`](../packages/01-depinject.md) is used to wire any module in `app.go`. +All core modules are already configured to support dependency injection. + +To work with `depinject` a module must define its configuration and requirements so that `depinject` can provide the right dependencies. + +In brief, as a module developer, the following steps are required: + +1. Define the module configuration using Protobuf +2. Define the module dependencies in `x/{moduleName}/module.go` + +A chain developer can then use the module by following these two steps: + +1. Configure the module in `app_config.go` or `app.yaml` +2. Inject the module in `app.go` + +## Module Configuration + +The module available configuration is defined in a Protobuf file, located at `{moduleName}/module/v1/module.proto`. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/group/module/v1/module.proto +``` + +* `go_import` must point to the Go package of the custom module. +* Message fields define the module configuration. + That configuration can be set in the `app_config.go` / `app.yaml` file for a chain developer to configure the module. + Taking `group` as example, a chain developer is able to decide, thanks to `uint64 max_metadata_len`, what the maximum metadata length allowed for a group proposal is. + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/simapp/app_config.go#L228-L234 + ``` + +That message is generated using [`pulsar`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/scripts/protocgen-pulsar.sh) (by running `make proto-gen`). +In the case of the `group` module, this file is generated here: https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/api/cosmos/group/module/v1/module.pulsar.go. + +The part that is relevant for the module configuration is: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/api/cosmos/group/module/v1/module.pulsar.go#L515-L527 +``` + +:::note +Pulsar is optional. The official [`protoc-gen-go`](https://protobuf.dev/reference/go/go-generated/) can be used as well. +::: + +## Dependency Definition + +Once the configuration proto is defined, the module's `module.go` must define what dependencies are required by the module. +The boilerplate is similar for all modules. + +:::warning +All methods, structs and their fields must be public for `depinject`. +::: + +1. Import the module configuration generated package: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/group/module/module.go#L12-L14 + ``` + + Define an `init()` function for defining the `providers` of the module configuration: + This registers the module configuration message and the wiring of the module. + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/group/module/module.go#L194-L199 + ``` + +2. Ensure that the module implements the `appmodule.AppModule` interface: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.47.0/x/group/module/module.go#L58-L64 + ``` + +3. Define a struct that inherits `depinject.In` and define the module inputs (i.e. module dependencies): + * `depinject` provides the right dependencies to the module. + * `depinject` also checks that all dependencies are provided. + + :::tip + For making a dependency optional, add the `optional:"true"` struct tag. + ::: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/group/module/module.go#L201-L211 + ``` + +4. Define the module outputs with a public struct that inherits `depinject.Out`: + The module outputs are the dependencies that the module provides to other modules. It is usually the module itself and its keeper. + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/group/module/module.go#L213-L218 + ``` + +5. Create a function named `ProvideModule` (as called in 1.) and use the inputs for instantiating the module outputs. + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/group/module/module.go#L220-L235 + ``` + + The `ProvideModule` function should return an instance of `cosmossdk.io/core/appmodule.AppModule` which implements + one or more app module extension interfaces for initializing the module. + + Following is the complete app wiring configuration for `group`: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/group/module/module.go#L194-L235 + ``` + +6. All modules must implement `depinject.OnePerModuleType` interface. This is used in order to tell the dependency injection framework that the module can only be instantiated once. + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/f4bdec3433373cc4950f4680743e969495763fbb/x/group/module/module.go#L64-L65 + ``` + +The module is now ready to be used with `depinject` by a chain developer. + +## Integrate in an application + +The App Wiring is done in `app_config.go` / `app.yaml` and `app_di.go` and is explained in detail in the [overview of `app_di.go`](https://docs.cosmos.network/main/build/building-apps/app-go-di). diff --git a/versioned_docs/version-0.52/build/building-modules/16-testing.md b/versioned_docs/version-0.52/build/building-modules/16-testing.md new file mode 100644 index 000000000..6b9441bbe --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/16-testing.md @@ -0,0 +1,112 @@ +--- +sidebar_position: 1 +--- + +# Testing + +The Cosmos SDK contains different types of [tests](https://martinfowler.com/articles/practical-test-pyramid.html). +These tests have different goals and are used at different stages of the development cycle. +We advice, as a general rule, to use tests at all stages of the development cycle. +It is advised, as a chain developer, to test your application and modules in a similar way than the SDK. + +The rationale behind testing can be found in [ADR-59](https://docs.cosmos.network/main/build/architecture/adr-059-test-scopes.html). + +## Unit Tests + +Unit tests are the lowest test category of the [test pyramid](https://martinfowler.com/articles/practical-test-pyramid.html). +All packages and modules should have unit test coverage. Modules should have their dependencies mocked: this means mocking keepers. + +The SDK uses `mockgen` to generate mocks for keepers: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/scripts/mockgen.sh#L3-L6 +``` + +You can read more about mockgen [here](https://github.com/golang/mock). + +### Example + +As an example, we will walkthrough the [keeper tests](https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/gov/keeper/keeper_test.go) of the `x/gov` module. + +The `x/gov` module has a `Keeper` type, which requires a few external dependencies (ie. imports outside `x/gov` to work properly). + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/gov/keeper/keeper.go#L22-L24 +``` + +In order to only test `x/gov`, we mock the [expected keepers](https://docs.cosmos.network/v0.46/building-modules/keeper.html#type-definition) and instantiate the `Keeper` with the mocked dependencies. Note that we may need to configure the mocked dependencies to return the expected values: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/gov/keeper/common_test.go#L67-L81 +``` + +This allows us to test the `x/gov` module without having to import other modules. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/gov/keeper/keeper_test.go#L3-L42 +``` + +We can test then create unit tests using the newly created `Keeper` instance. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/gov/keeper/keeper_test.go#L83-L107 +``` + +## Integration Tests + +Integration tests are at the second level of the [test pyramid](https://martinfowler.com/articles/practical-test-pyramid.html). +In the SDK, we locate our integration tests under [`/tests/integrations`](https://github.com/cosmos/cosmos-sdk/tree/main/tests/integration). + +The goal of these integration tests is to test how a component interacts with other dependencies. Compared to unit tests, integration tests do not mock dependencies. Instead, they use the direct dependencies of the component. This differs as well from end-to-end tests, which test the component with a full application. + +Integration tests interact with the tested module via the defined `Msg` and `Query` services. The result of the test can be verified by checking the state of the application, by checking the emitted events or the response. It is advised to combine two of these methods to verify the result of the test. + +The SDK provides small helpers for quickly setting up an integration tests. These helpers can be found at . + +### Example + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/a2f73a7dd37bea0ab303792c55fa1e4e1db3b898/testutil/integration/example_test.go#L30-L116 +``` + +## Deterministic and Regression tests + +Tests are written for queries in the Cosmos SDK which have `module_query_safe` Protobuf annotation. + +Each query is tested using 2 methods: + +* Use property-based testing with the [`rapid`](https://pkg.go.dev/pgregory.net/rapid@v0.5.3) library. The property that is tested is that the query response and gas consumption are the same upon 1000 query calls. +* Regression tests are written with hardcoded responses and gas, and verify they don't change upon 1000 calls and between SDK patch versions. + +Here's an example of regression tests: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/tests/integration/bank/keeper/deterministic_test.go#L134-L151 +``` + +## Simulations + +Simulations fuzz tests for deterministic message execution. They use a minimal application, built with [`depinject`](../packages/01-depinject.md): + +:::note +Simulations have been refactored to message factories +::: + +An example for `x/bank/` simulations: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/bank/simulation/msg_factory.go#L13-L20 +``` + +## System Tests + +System tests are at the top of the [test pyramid](https://martinfowler.com/articles/practical-test-pyramid.html). +They test the whole application flow as black box, from the user perspective. They are located under [`/tests/systemtests`](https://github.com/cosmos/cosmos-sdk/tree/main/tests/systemtests). + +For that, the SDK is using the `simapp` binary, but you should use your own binary. +More details about system test can be found in [building-apps](https://docs.cosmos.network/main/build/building-apps/system-tests) + + +## Learn More + +Learn more about testing scope in [ADR-59](https://docs.cosmos.network/main/build/architecture/adr-059-test-scopes.html). diff --git a/versioned_docs/version-0.52/build/building-modules/18-define-hooks.md b/versioned_docs/version-0.52/build/building-modules/18-define-hooks.md new file mode 100644 index 000000000..8b19f238b --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/18-define-hooks.md @@ -0,0 +1,57 @@ +--- +sidebar_position: 1 +--- + +# Hooks + +Hooks are functions that are called before and/or after certain events in the module's lifecycle. + +## Defining Hooks + +1. Define the hook interface and a wrapper implementing `depinject.OnePerModuleType`: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/71c603a2a5a103df00f216d78ec8b108ed64ae28/testutil/x/counter/types/expected_keepers.go#L5-L12 + ``` + +2. Add a `CounterHooks` field to the keeper: + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/71c603a2a5a103df00f216d78ec8b108ed64ae28/testutil/x/counter/keeper/keeper.go#L25 + + ``` + +3. Create a `depinject` invoker function + + ```go reference + https://github.com/cosmos/cosmos-sdk/blob/71c603a2a5a103df00f216d78ec8b108ed64ae28/testutil/x/counter/depinject.go#L53-L75 + ``` + +4. Inject the hooks during app initialization: + + ```go + appConfig = appconfig.Compose(&appv1alpha1.Config{ + Modules: []*appv1alpha1.ModuleConfig{ + // .... + { + Name: types.ModuleName, + Config: appconfig.WrapAny(&types.Module{}), + }, + } + }) + appConfig = depinject.Configs( + AppConfig(), + runtime.DefaultServiceBindings(), + depinject.Supply( + logger, + viper, + map[string]types.CounterHooksWrapper{ + "counter": types.CounterHooksWrapper{&types.Hooks{}}, + }, + )) + ``` + +## Examples in the SDK + +For examples of hooks implementation in the Cosmos SDK, refer to the [Epochs Hooks documentation](https://docs.cosmos.network/main/build/modules/epochs#hooks) and [Distribution Hooks Documentation](https://docs.cosmos.network/main/build/modules/distribution#hooks). + diff --git a/versioned_docs/version-0.52/build/building-modules/_category_.json b/versioned_docs/version-0.52/build/building-modules/_category_.json new file mode 100644 index 000000000..2d50f8b3e --- /dev/null +++ b/versioned_docs/version-0.52/build/building-modules/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Building Modules", + "position": 1, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/migrations/01-intro.md b/versioned_docs/version-0.52/build/migrations/01-intro.md new file mode 100644 index 000000000..47c5c245a --- /dev/null +++ b/versioned_docs/version-0.52/build/migrations/01-intro.md @@ -0,0 +1,15 @@ +--- +sidebar_position: 1 +--- + +# SDK Migrations + +To smoothen the update to the latest stable release, the SDK includes a CLI command for hard-fork migrations (under the ` genesis migrate` subcommand). +Additionally, the SDK includes in-place migrations for its core modules. These in-place migrations are useful to migrate between major releases. + +* Hard-fork migrations are supported from the last major release to the current one. +* [In-place module migrations](https://docs.cosmos.network/main/core/upgrade#overwriting-genesis-functions) are supported from the last two major releases to the current one. + +Migration from a version older than the last two major releases is not supported. + +When migrating from a previous version, refer to the [`UPGRADING.md`](./02-upgrading.md) and the `CHANGELOG.md` of the version you are migrating to. diff --git a/versioned_docs/version-0.52/build/migrations/02-upgrading.md b/versioned_docs/version-0.52/build/migrations/02-upgrading.md new file mode 100644 index 000000000..f6f2a5289 --- /dev/null +++ b/versioned_docs/version-0.52/build/migrations/02-upgrading.md @@ -0,0 +1,471 @@ +# Upgrading Cosmos SDK + +This guide provides instructions for upgrading to specific versions of Cosmos SDK. +Note, always read the **SimApp** section for more information on application wiring updates. + +## [v0.52.x](https://github.com/cosmos/cosmos-sdk/releases/tag/v0.52.0-alpha.0) + +Documentation to migrate an application from v0.50.x to server/v2 is available elsewhere. +It is additional to the changes described here. + +### SimApp + +In this section we describe the changes made in Cosmos SDK' SimApp. +**These changes are directly applicable to your application wiring.** +Please read this section first, but for an exhaustive list of changes, refer to the [CHANGELOG](./simapp/CHANGELOG.md). + +#### Client (`root.go`) + +The `client` package has been refactored to make use of the address codecs (address, validator address, consensus address, etc.) +and address bech32 prefixes (address and validator address). +This is part of the work of abstracting the SDK from the global bech32 config. + +This means the address codecs and prefixes must be provided in the `client.Context` in the application client (usually `root.go`). + +```diff +clientCtx = clientCtx. ++ WithAddressCodec(addressCodec). ++ WithValidatorAddressCodec(validatorAddressCodec). ++ WithConsensusAddressCodec(consensusAddressCodec). ++ WithAddressPrefix("cosmos"). ++ WithValidatorPrefix("cosmosvaloper") +``` + +**When using `depinject` / `app_di`, the client codecs can be provided directly from application config.** + +Refer to SimApp `root_v2.go` and `root.go` for an example with an app di and a legacy app. + +Additionally, a simplification of the start command leads to the following change: + +```diff +- server.AddCommands(rootCmd, newApp, func(startCmd *cobra.Command) {}) ++ server.AddCommands(rootCmd, newApp, server.StartCmdOptions[servertypes.Application]{}) +``` + +#### Server (`app.go`) + +##### Module Manager + +The basic module manager has been deleted. It was not necessary anymore and was simplified to use the `module.Manager` directly. +It can be removed from your `app.go`. + +For depinject users, it isn't necessary anymore to supply a `map[string]module.AppModuleBasic` for customizing the app module basic instantiation. +The custom parameters (such as genutil message validator or gov proposal handler, or evidence handler) can directly be supplied. +When requiring a module manager in `root.go`, inject `*module.Manager` using `depinject.Inject`. + +For non depinject users, simply call `RegisterLegacyAminoCodec` and `RegisterInterfaces` on the module manager: + +```diff +-app.BasicModuleManager = module.NewBasicManagerFromManager(...) +-app.BasicModuleManager.RegisterLegacyAminoCodec(legacyAmino) +-app.BasicModuleManager.RegisterInterfaces(interfaceRegistry) ++app.ModuleManager.RegisterLegacyAminoCodec(legacyAmino) ++app.ModuleManager.RegisterInterfaces(interfaceRegistry) +``` + +Additionally, thanks to the genesis simplification, as explained in [the genesis interface update](#genesis-interface), the module manager `InitGenesis` and `ExportGenesis` methods do not require the codec anymore. + +##### gRPC Web + +Grpc-web embedded client has been removed from the server. If you would like to use grpc-web, you can use the [envoy proxy](https://www.envoyproxy.io/docs/envoy/latest/start/start). Here's how to set it up: + +
+Step by step guide + +1. Install Envoy following the [official installation guide](https://www.envoyproxy.io/docs/envoy/latest/start/install). + +2. Create an Envoy configuration file named `envoy.yaml` with the following content: + + ```yaml + static_resources: + listeners: + - name: listener_0 + address: + socket_address: { address: 0.0.0.0, port_value: 8080 } + filter_chains: + - filters: + - name: envoy.filters.network.http_connection_manager + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager + codec_type: auto + stat_prefix: ingress_http + route_config: + name: local_route + virtual_hosts: + - name: local_service + domains: ["*"] + routes: + - match: { prefix: "/" } + route: + cluster: grpc_service + timeout: 0s + max_stream_duration: + grpc_timeout_header_max: 0s + cors: + allow_origin_string_match: + - prefix: "*" + allow_methods: GET, PUT, DELETE, POST, OPTIONS + allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout + max_age: "1728000" + expose_headers: custom-header-1,grpc-status,grpc-message + http_filters: + - name: envoy.filters.http.grpc_web + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_web.v3.GrpcWeb + - name: envoy.filters.http.cors + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors + - name: envoy.filters.http.router + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router + clusters: + - name: grpc_service + connect_timeout: 0.25s + type: logical_dns + http2_protocol_options: {} + lb_policy: round_robin + load_assignment: + cluster_name: cluster_0 + endpoints: + - lb_endpoints: + - endpoint: + address: + socket_address: + address: 0.0.0.0 + port_value: 9090 + ``` + + This configuration tells Envoy to listen on port 8080 and forward requests to your gRPC service on port 9090. Note that this configuration is a starting point and can be modified according to your specific needs and preferences. You may need to adjust ports, addresses, or add additional settings based on your particular setup and requirements. + +3. Start your Cosmos SDK application, ensuring it's configured to serve gRPC on port 9090. + +4. Start Envoy with the configuration file: + + ```bash + envoy -c envoy.yaml + ``` + +5. If Envoy starts successfully, you should see output similar to this: + + ```bash + [2024-08-29 10:47:08.753][6281320][info][config] [source/common/listener_manager/listener_manager_impl.cc:930] all dependencies initialized. starting workers + [2024-08-29 10:47:08.754][6281320][info][main] [source/server/server.cc:978] starting main dispatch loop + ``` + + This indicates that Envoy has started and is ready to proxy requests. + +6. Update your client applications to connect to Envoy (http://localhost:8080 by default). + +
+ +By following these steps, Envoy will handle the translation between gRPC-Web and gRPC, allowing your existing gRPC-Web clients to continue functioning without modifications to your Cosmos SDK application. + +To test the setup, you can use a tool like [grpcurl](https://github.com/fullstorydev/grpcurl). For example: + +```bash +grpcurl -plaintext localhost:8080 cosmos.base.tendermint.v1beta1.Service/GetLatestBlock +``` + +##### AnteHandlers + +The `GasConsumptionDecorator` and `IncreaseSequenceDecorator` have been merged with the SigVerificationDecorator, so you'll +need to remove them both from your app.go code, they will yield to unresolvable symbols when compiling. + +#### Unordered Transactions + +The Cosmos SDK now supports unordered transactions. This means that transactions +can be executed in any order and doesn't require the client to deal with or manage +nonces. This also means the order of execution is not guaranteed. + +Unordered transactions are automatically enabled when using `depinject` / app di, simply supply the `servertypes.AppOptions` in `app.go`: + +```diff + depinject.Supply( ++ // supply the application options ++ appOpts, + // supply the logger + logger, + ) +``` + +
+Step-by-step Wiring +If you are still using the legacy wiring, you must enable unordered transactions manually: + +* Update the `App` constructor to create, load, and save the unordered transaction + manager. + + ```go + func NewApp(...) *App { + // ... + + // create, start, and load the unordered tx manager + utxDataDir := filepath.Join(cast.ToString(appOpts.Get(flags.FlagHome)), "data") + app.UnorderedTxManager = unorderedtx.NewManager(utxDataDir) + app.UnorderedTxManager.Start() + + if err := app.UnorderedTxManager.OnInit(); err != nil { + panic(fmt.Errorf("failed to initialize unordered tx manager: %w", err)) + } + } + ``` + +* Add the decorator to the existing AnteHandler chain, which should be as early + as possible. + + ```go + anteDecorators := []sdk.AnteDecorator{ + ante.NewSetUpContextDecorator(), + // ... + ante.NewUnorderedTxDecorator(unorderedtx.DefaultMaxTimeoutDuration, options.TxManager, options.Environment), + // ... + } + + return sdk.ChainAnteDecorators(anteDecorators...), nil + ``` + +* If the App has a SnapshotManager defined, you must also register the extension + for the TxManager. + + ```go + if manager := app.SnapshotManager(); manager != nil { + err := manager.RegisterExtensions(unorderedtx.NewSnapshotter(app.UnorderedTxManager)) + if err != nil { + panic(fmt.Errorf("failed to register snapshot extension: %s", err)) + } + } + ``` + +* Create or update the App's `Preblocker()` method to call the unordered tx + manager's `OnNewBlock()` method. + + ```go + ... + app.SetPreblocker(app.PreBlocker) + ... + + func (app *SimApp) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) (*sdk.ResponsePreBlock, error) { + app.UnorderedTxManager.OnNewBlock(ctx.BlockTime()) + return app.ModuleManager.PreBlock(ctx, req) + } + ``` + +* Create or update the App's `Close()` method to close the unordered tx manager. + Note, this is critical as it ensures the manager's state is written to file + such that when the node restarts, it can recover the state to provide replay + protection. + + ```go + func (app *App) Close() error { + // ... + + // close the unordered tx manager + if e := app.UnorderedTxManager.Close(); e != nil { + err = errors.Join(err, e) + } + + return err + } + ``` + +
+ +To submit an unordered transaction, the client must set the `unordered` flag to +`true` and ensure a reasonable `timeout_height` is set. The `timeout_height` is +used as a TTL for the transaction and is used to provide replay protection. See +[ADR-070](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-070-unordered-transactions.md) +for more details. + +#### Sign Mode Textual + +With the split of `x/auth/tx/config` in two (x/auth/tx/config as depinject module for txconfig and tx options) and `x/validate`, sign mode textual is no more automatically configured when using runtime (it was previously the case). +For the same instructions than for legacy app wiring to enable sign mode textual (see in v0.50 UPGRADING documentation). + +### Depinject `app_config.go` / `app.yml` + +With the introduction of [environment in modules](#core-api), depinject automatically creates the environment for all modules. +Learn more about environment [here](https://example.com) . Given the fields of environment, this means runtime creates a kv store service for all modules by default. It can happen that some modules do not have a store necessary (such as `x/auth/tx` for instance). In this case, the store creation should be skipped in `app_config.go`: + +```diff +InitGenesis: []string{ + "..." +}, ++ // SkipStoreKeys is an optional list of store keys to skip when constructing the ++ // module's keeper. This is useful when a module does not have a store key. ++ SkipStoreKeys: []string{ ++ "tx", ++ }, +``` + +### Protobuf + +The `cosmossdk.io/api/tendermint` package has been removed as CometBFT now publishes its protos to `buf.build/tendermint` and `buf.build/cometbft`. +There is no longer a need for the Cosmos SDK to host these protos for itself and its dependencies. +That package containing proto v2 generated code, but the SDK now uses [buf generated go SDK instead](https://buf.build/docs/bsr/generated-sdks/go). +If you were depending on `cosmossdk.io/api/tendermint`, please use the buf generated go SDK instead, or ask CometBFT host the generated proto v2 code. + +The `codectypes.Any` has moved to `github.com/cosmos/gogoproto/types/any`. Module developers need to update the `buf.gen.gogo.yaml` configuration files by adjusting the corresponding `opt` option to `Mgoogle/protobuf/any.proto=github.com/cosmos/gogoproto/types/any` for directly mapping the`Any` type to its new location: + +```diff +version: v1 +plugins: + - name: gocosmos + out: .. +- opt: plugins=grpc,Mgoogle/protobuf/any.proto=github.com/cosmos/cosmos-sdk/codec/types,Mcosmos/orm/v1/orm.proto=cosmossdk.io/orm ++ opt: plugins=grpc,Mgoogle/protobuf/any.proto=github.com/cosmos/gogoproto/types/any,Mcosmos/orm/v1/orm.proto=cosmossdk.io/orm + - name: grpc-gateway + out: .. + opt: logtostderr=true,allow_colon_final_segments=true + +``` + +Also, any usages of the interfaces `AnyUnpacker` and `UnpackInterfacesMessage` must be replaced with the interfaces of the same name in the `github.com/cosmos/gogoproto/types/any` package. + +### Modules + +#### `**all**` + +All modules (expect `auth`) were spun out into their own `go.mod`. Replace their imports by `cosmossdk.io/x/{moduleName}`. + +##### Core API + +Core API has been introduced for modules since v0.47. With the deprecation of `sdk.Context`, we strongly recommend to use the `cosmossdk.io/core/appmodule` interfaces for the modules. This will allow the modules to work out of the box with server/v2 and baseapp, as well as limit their dependencies on the SDK. + +Additionally, the `appmodule.Environment` struct is introduced to fetch different services from the application. +This should be used as an alternative to using `sdk.UnwrapContext(ctx)` to fetch the services. +It needs to be passed into a module at instantiation (or depinject will inject the correct environment). + +`x/circuit` is used as an example: + +```go +app.CircuitKeeper = circuitkeeper.NewKeeper(runtime.NewEnvironment(runtime.NewKVStoreService(keys[circuittypes.StoreKey]), logger.With(log.ModuleKey, "x/circuit")), appCodec, authtypes.NewModuleAddress(govtypes.ModuleName).String(), app.AuthKeeper.AddressCodec()) +``` + +If your module requires a message server or query server, it should be passed in the environment as well. + +```diff +-govKeeper := govkeeper.NewKeeper(appCodec, runtime.NewKVStoreService(keys[govtypes.StoreKey]), app.AuthKeeper, app.BankKeeper,app.StakingKeeper, app.PoolKeeper, app.MsgServiceRouter(), govConfig, authtypes.NewModuleAddress(govtypes.ModuleName).String()) ++govKeeper := govkeeper.NewKeeper(appCodec, runtime.NewEnvironment(runtime.NewKVStoreService(keys[govtypes.StoreKey]), logger.With(log.ModuleKey, "x/circuit"), runtime.EnvWithMsgRouterService(app.MsgServiceRouter()), runtime.EnvWithQueryRouterService(app.GRPCQueryRouter())), app.AuthKeeper, app.BankKeeper, app.StakingKeeper, app.PoolKeeper, govConfig, authtypes.NewModuleAddress(govtypes.ModuleName).String()) +``` + +The signature of the extension interface `HasRegisterInterfaces` has been changed to accept a `cosmossdk.io/core/registry.InterfaceRegistrar` instead of a `codec.InterfaceRegistry`. `HasRegisterInterfaces` is now a part of `cosmossdk.io/core/appmodule`. Modules should update their `HasRegisterInterfaces` implementation to accept a `cosmossdk.io/core/registry.InterfaceRegistrar` interface. + +```diff +-func (AppModule) RegisterInterfaces(registry codectypes.InterfaceRegistry) { ++func (AppModule) RegisterInterfaces(registry registry.InterfaceRegistrar) { +``` + +The signature of the extension interface `HasAminoCodec` has been changed to accept a `cosmossdk.io/core/registry.AminoRegistrar` instead of a `codec.LegacyAmino`. Modules should update their `HasAminoCodec` implementation to accept a `cosmossdk.io/core/registry.AminoRegistrar` interface. + +```diff +-func (AppModule) RegisterLegacyAminoCodec(cdc *codec.LegacyAmino) { ++func (AppModule) RegisterLegacyAminoCodec(registrar registry.AminoRegistrar) { +``` + +##### Simulation + +`MsgSimulatorFn` has been updated to return an error. Its context argument has been removed, and an address.Codec has +been added to avoid the use of the Accounts.String() method. + +```diff +-type MsgSimulatorFn func(r *rand.Rand, ctx sdk.Context, accs []Account) sdk.Msg ++type MsgSimulatorFn func(r *rand.Rand, accs []Account, cdc address.Codec) (sdk.Msg, error) +``` + +##### Depinject + +Previously `cosmossdk.io/core` held functions `Invoke`, `Provide` and `Register` were moved to `cosmossdk.io/depinject/appconfig`. +All modules using dependency injection must update their imports. + +##### Params + +Previous module migrations have been removed. It is required to migrate to v0.50 prior to upgrading to v0.52 for not missing any module migrations. + +##### Genesis Interface + +All genesis interfaces have been migrated to take `context.Context` instead of `sdk.Context`. +Secondly, the codec is no longer passed in by the framework. The codec is now passed in by the module. +Lastly, all InitGenesis and ExportGenesis functions now return an error. + +```go +// InitGenesis performs genesis initialization for the module. +func (am AppModule) InitGenesis(ctx context.Context, data json.RawMessage) error { +} + +// ExportGenesis returns the exported genesis state as raw bytes for the module. +func (am AppModule) ExportGenesis(ctx context.Context) (json.RawMessage, error) { +} +``` + +##### Migration to Collections + +Most of Cosmos SDK modules have migrated to [collections](https://docs.cosmos.network/main/build/packages/collections). +Many functions have been removed due to this changes as the API can be smaller thanks to collections. +For modules that have migrated, verify you are checking against `collections.ErrNotFound` when applicable. + +#### `x/auth` + +Vesting accounts messages (and CLIs) have been removed. Existing vesting accounts will keep working but no new vesting accounts can be created. +Use `x/accounts` lockup accounts or implement an `x/accounts` vesting account instead. + +#### `x/accounts` + +Accounts's AccountNumber will be used as a global account number tracking replacing Auth legacy AccountNumber. Must set accounts's AccountNumber with auth's AccountNumber value in upgrade handler. This is done through auth keeper MigrateAccountNumber function. + +```go +import authkeeper "github.com/cosmos/cosmos-sdk/x/auth/keeper" +... +err := authkeeper.MigrateAccountNumberUnsafe(ctx, &app.AuthKeeper) +if err != nil { + return nil, err +} +``` + +#### `x/crisis` + +The `x/crisis` module was removed due to it not being supported or functional any longer. + +#### `x/distribution` + +Existing chains using `x/distribution` module must add the new `x/protocolpool` module. + +#### `x/gov` + +Gov v1beta1 proposal handler has been changed to take in a `context.Context` instead of `sdk.Context`. +This change was made to allow legacy proposals to be compatible with server/v2. +If you wish to migrate to server/v2, you should update your proposal handler to take in a `context.Context` and use services. +On the other hand, if you wish to keep using baseapp, simply unwrap the sdk context in your proposal handler. + +#### `x/protocolpool` + +Introducing a new `x/protocolpool` module to handle community pool funds. Its store must be added while upgrading to v0.52.x. + +Example: + +```go +func (app SimApp) RegisterUpgradeHandlers() { + app.UpgradeKeeper.SetUpgradeHandler( + UpgradeName, + func(ctx sdk.Context, _ upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { + return app.ModuleManager.RunMigrations(ctx, app.Configurator(), fromVM) + }, + ) + + // ... +} +``` + +Add `x/protocolpool` store while upgrading to v0.52.x: + +```go +storetypes.StoreUpgrades{ + Added: []string{ + protocolpooltypes.ModuleName, + }, +} +``` + +#### `x/validate` + +Introducing `x/validate` a module that is solely used for registering default ante/post handlers and global tx validators when using runtime and runtime/v2. If you wish to set your custom ante/post handlers, no need to use this module. +You can however always extend them by adding extra tx validators (see `x/validate` documentation). diff --git a/versioned_docs/version-0.52/build/migrations/_category_.json b/versioned_docs/version-0.52/build/migrations/_category_.json new file mode 100644 index 000000000..5a06c3ebb --- /dev/null +++ b/versioned_docs/version-0.52/build/migrations/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Migrations", + "position": 3, + "link": null +} diff --git a/versioned_docs/version-0.52/build/modules/README.md b/versioned_docs/version-0.52/build/modules/README.md new file mode 100644 index 000000000..721157391 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/README.md @@ -0,0 +1,44 @@ +--- +sidebar_position: 0 +--- + +# List of Modules + +Here are some production-grade modules that can be used in Cosmos SDK applications, along with their respective documentation: + +* [Auth](./auth/README.md) - Authentication of accounts and transactions for Cosmos SDK applications. +* [Authz](./authz/README.md) - Authorization for accounts to perform actions on behalf of other accounts. +* [Bank](./bank/README.md) - Token transfer functionalities. +* [Circuit](./circuit/README.md) - Circuit breaker module for pausing messages. +* [Consensus](./consensus/README.md) - Consensus module for modifying CometBFT's ABCI consensus params. +* [Distribution](./distribution/README.md) - Fee distribution, and staking token provision distribution. +* [Epochs](./epochs/README.md) - Allow other modules to set that they would like to be signaled once every period +* [Evidence](./evidence/README.md) - Evidence handling for double signing, misbehaviour, etc. +* [Feegrant](./feegrant/README.md) - Grant fee allowances for executing transactions. +* [Genutil](./genutil/README.md) - Genesis utilities for the Cosmos SDK. +* [Governance](./gov/README.md) - On-chain proposals and voting. +* [Mint](./mint/README.md) - Creation of new units of staking token. +* [NFT](./nft/README.md) - NFT module implemented based on [ADR43](https://docs.cosmos.network/main/build/architecture/adr-043-nft-module). +* [Params](./params/README.md) - Globally available parameter store. +* [Protocolpool](./protocolpool/README.md) - Functionalities handling community pool funds. +* [Slashing](./slashing/README.md) - Validator punishment mechanisms. +* [Staking](./staking/README.md) - Proof-of-Stake layer for public blockchains. +* [tx](./tx/README.md) - Tx utilities for the Cosmos SDK. +* [Upgrade](./upgrade/README.md) - Software upgrades handling and coordination. +* [Validate](./validate/README.md) - Global ante/post handlers and tx validator setup. + +To learn more about the process of building modules, visit the [building modules reference documentation](https://docs.cosmos.network/main/building-modules/intro). + +## IBC + +The IBC module for the SDK is maintained by the IBC Go team in its [own repository](https://github.com/cosmos/ibc-go). + +Additionally, the [capability module](https://github.com/cosmos/ibc-go/tree/fdd664698d79864f1e00e147f9879e58497b5ef1/modules/capability) is from v0.50+ maintained by the IBC Go team in its [own repository](https://github.com/cosmos/ibc-go/tree/fdd664698d79864f1e00e147f9879e58497b5ef1/modules/capability). + +## CosmWasm + +The CosmWasm module enables smart contracts, learn more by going to their [documentation site](https://book.cosmwasm.com/), or visit [the repository](https://github.com/CosmWasm/cosmwasm). + +## EVM + +Read more about writing smart contracts with solidity at the official [`evm` documentation page](https://docs.evmos.org/). diff --git a/versioned_docs/version-0.52/build/modules/_category_.json b/versioned_docs/version-0.52/build/modules/_category_.json new file mode 100644 index 000000000..72d229c0b --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Modules", + "position": 2, + "link": null +} diff --git a/versioned_docs/version-0.52/build/modules/accounts/README.md b/versioned_docs/version-0.52/build/modules/accounts/README.md new file mode 100644 index 000000000..213a003fd --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/accounts/README.md @@ -0,0 +1,540 @@ +# x/accounts + +The x/accounts module enhances the Cosmos SDK by providing tools and infrastructure for creating advanced smart accounts. + +## Basics + +An account can be thought of as a simplified cosmos-sdk module that supports multiple deployments. This means: + +1. A single account implementation can be deployed to multiple addresses, similar to how CosmWasm allows multiple contract instances from one WASM upload. + +2. Each account address is mapped to its corresponding account code. + +3. Accounts maintain their own state partition, similar to modules. + +4. Accounts can define both message and query handlers. + +This design allows for flexible and reusable account structures within the ecosystem. + +### Example account creation + +#### Basic + +Defining an account begins with creating a struct that encapsulates the account's state. If the account has no state, the +struct is empty `type Account struct{}`. + +By default, accounts utilize collections to manage their state. + +##### State Isolation + +It's crucial to understand that an account's state is isolated. This means: + +1. States are not shared between accounts of different types. +2. States are not shared even between accounts of the same type. + +For example, consider two accounts of type Counter: + +- One located at address "cosmos123" +- Another at address "cosmos456" + +These accounts do not share the same collections.Item instance. Instead, each maintains its own separate state. + +```go +type Account struct { + // We will define that the account contains in its state a counter, it's an item. + // It could also be a map or whatever! + Counter collections.Item[uint64] +} +``` + +#### Init + +Creating an account begins with defining its init message. This message is processed when an account is created, similar to: + +- The `instantiate` method in a CosmWasm contract +- The `constructor` in an EVM contract + +For an account to be a valid `x/account` implementer, it must define both: + +1. An `Init` method +2. An init message + +We start by defining the `MsgInit` and its corresponding `MsgInitResponse` as protobuf messages: + +```protobuf +message MsgInit { + uint64 counter = 1; +} + +message MsgInitResponse {} +``` + +Next, we implement the Init method, which sets the initial counter. We also implement a method of the `Account` interface. This method: + +Signals to the x/accounts runtime what the Init entrypoint is +Performs some generic operations to maintain type safety in the system + +Here's the Go implementation: + +```go +package counter + +import ( + "context" + "cosmossdk.io/x/accounts/accountstd" +) + +type Account struct { + Counter collections.Item[uint64] +} + +func (a Account) Init(ctx context.Context, msg *MsgInit) (*MsgInitResponse, error) { + err := a.Counter.Set(ctx, msg.Counter) + if err != nil { + return nil, err + } + + return &MsgInitResponse{}, nil +} + +func (a Account) RegisterInitHandler(builder *accountstd.InitBuilder) { + accountstd.RegisterInitHandler(builder, a.Init) +} +``` + +#### Execute Handlers + +Execute handlers are methods that an account can execute, defined as messages. These executions can be triggered: + +- During block execution (not queries) through transactions +- During begin or end block + +To define an execute handler, we start by creating its proto message: + +```protobuf +message MsgIncreaseCounter { + uint64 amount = 1; +} + +message MsgIncreaseCounterResponse { + uint64 new_value = 1; +} +``` + +Next, we implement the handling code for this message and register it using the `RegisterExecuteHandlers` method: + +```go +package counter + +import ( + "context" + "cosmossdk.io/x/accounts/accountstd" +) + +type Account struct { + Counter collections.Item[uint64] +} + +func (a Account) Init(ctx context.Context, msg *MsgInit) (*MsgInitResponse, error) { + err := a.Counter.Set(ctx, msg.Counter) + if err != nil { + return nil, err + } + return &MsgInitResponse{}, nil +} + +// Handler for MsgIncreaseCounter +func (a Account) IncreaseCounter(ctx context.Context, msg *MsgIncreaseCounter) (*MsgIncreaseCounterResponse, error) { + counter, err := a.Counter.Get(ctx) + if err != nil { + return nil, err + } + + newValue := counter + msg.Amount + err = a.Counter.Set(ctx, newValue) + if err != nil { + return nil, err + } + + return &MsgIncreaseCounterResponse{NewValue: newValue}, nil +} + +// Registration of the handler in the runtime +func (a Account) RegisterExecuteHandlers(builder *accountstd.ExecuteBuilder) { + accountstd.RegisterExecuteHandler(builder, a.IncreaseCounter) +} + +func (a Account) RegisterInitHandler(builder *accountstd.InitBuilder) { + accountstd.RegisterInitHandler(builder, a.Init) +} +``` + +This implementation defines an IncreaseCounter method that handles the MsgIncreaseCounter message, updating the counter +value and returning the new value in the response. + +#### Query Handlers + +Query Handlers are read-only methods implemented by an account to expose information about itself. This information can be accessed by: + +- External clients (e.g., CLI, wallets) +- Other modules and accounts within the system + +Query handlers can be invoked: + +1. By external clients +2. During block execution + +To define a query handler, we follow a similar process to execute handlers: + +1. Define the request and response proto messages: + +```protobuf +message QueryCounter {} + +message QueryCounterResponse { + uint64 value = 1; +} +``` + +2. Implement and register the query handler: + +```go +package counter + +import ( + "context" + "cosmossdk.io/x/accounts/accountstd" +) + +func (a Account) QueryCounter(ctx context.Context, _ *QueryCounterRequest) (*QueryCounterResponse, error) { + counter, err := a.Counter.Get(ctx) + if err != nil { + return nil, err + } + return &QueryCounterResponse{ + Value: counter, + }, nil +} + +func (a Account) RegisterQueryHandlers(builder *accountstd.QueryBuilder) { + accountstd.RegisterQueryHandler(builder, a.QueryCounter) +} +``` + +This implementation defines a `QueryCounter` method that retrieves the current counter value and returns it in the response. +The `RegisterQueryHandlers` method registers this query handler with the system. + +#### The Account Constructor + +After creating our basic counter account, we implement the account constructor function: + +```go +package counter + +import ( + "cosmossdk.io/collections" + "cosmossdk.io/x/accounts/accountstd" +) + +func NewAccount(deps accountstd.Dependencies) (Account, error) { + return Account{ + Counter: collections.NewItem(deps.SchemaBuilder, CounterPrefix, "counter", collections.Uint64Value), + }, nil +} + +type Account struct { + Counter collections.Item[uint64] +} + +// Rest of the Account implementation... +``` + +The `accountstd.Dependencies` type provides an environment with essential components: + +1. `AddressCodec`: For encoding and decoding addresses +2. `SchemaBuilder`: For schema construction (handled by the accounts module) +3. `HeaderService`: For accessing block header information +4. Other useful services and utilities + +These dependencies allow the account to interact with the blockchain system. + +## App Wiring + +Note: This assumes you've already wired the `x/accounts` module in your application. If not, refer to the Simapp example. + +After creating our basic account, we wire it to the `x/accounts` module. + +### Depinject Method + +Define the depinject constructor: + +```go +package counterdepinject + +func ProvideAccount() accountstd.DepinjectAccount { + return accountstd.DIAccount("counter", counter.NewAccount) +} +``` + +Add this to the application: + +```go +package app + +func NewApp() *App { + // ... + appConfig = depinject.Configs( + AppConfig(), + depinject.Supply( + appOpts, + logger, + ), + depinject.Provide( + counterdepinject.ProvideAccount, + ), + ) + // ... +} +``` + +### Manual Method + +Add the account to the x/accounts Keeper: + +```go +accountsKeeper, err := accounts.NewKeeper( + appCodec, + runtime.NewEnvironment(/* ... */), + signingCtx.AddressCodec(), + appCodec.InterfaceRegistry(), + + accountstd.AddAccount("counter", counter.NewAccount), // Add account here + // Add more accounts if needed +) +``` + +Choose the method that best fits your application structure. + +### The accountsstd Package + +The `accountsstd` package provides utility functions for use within account init, execution, or query handlers. Key functions include: + +1. `Whoami()`: Retrieves the address of the current account. +2. `Sender()`: Gets the address of the transaction sender (not available in queries). +3. `Funds()`: Retrieves funds provided by the sender during Init or Execution. +4. `ExecModule()`: Allows the account to execute a module message. + Note: Impersonation is prevented. An account can't send messages on behalf of others. +5. `QueryModule()`: Enables querying a module. + +These functions, along with others, facilitate account operations and interactions within the system. +For a comprehensive list of available utilities, refer to the Go documentation. + +### Interfaces via Messages and Queries + +Accounts can handle various messages and queries, allowing for flexible interface definitions: + +1. Multiple account types can handle the same message or query. +2. Different accounts (even with the same type but different addresses) can process identical messages or queries. + +This flexibility enables defining interfaces as common sets of messages and/or queries that accounts can handle. + +Example: Transaction Authentication + +- We define a `MsgAuthenticate` message. +- Any account capable of handling `MsgAuthenticate` is considered to implement the `Authentication` interface. +- This approach allows for standardized interaction patterns across different account types. + +(Note: More details on the `Authentication` interface will be provided later.) + +### Full Examples + +Some examples can be found in the [defaults](./defaults) package. + +## The Authentication Interface + +x/accounts introduces the `Authentication` interface, allowing for flexible transaction (TX) authentication beyond traditional public key cryptography. + +Chain developers can implement tailored authentication methods for their accounts. Any account that implements the `Authentication` interface can be authenticated within a transaction. + +To implement the `Authentication` interface in x/accounts, an account must expose an execution handler capable of processing a specific message type. + +The key message type for authentication is `MsgAuthenticate`, which is defined in the module's protocol buffer files: + +[interfaces/account_abstraction/v1/interface.proto](./proto/cosmos/accounts/interfaces/account_abstraction/v1/interface.proto) + +### Authentication Mechanism + +#### AnteHandler in the SDK + +The Cosmos SDK utilizes an `AnteHandler` to verify transaction (TX) integrity. Its primary function is to ensure that the messages within a transaction are correctly signed by the purported sender. + +#### Authentication Flow for x/accounts Module + +When the `AnteHandler` identifies that a message sender (and transaction signer) belongs to the x/accounts module, it delegates the authentication process to that module. + +##### Authentication Interface Requirement + +For successful authentication, the account must implement the `Authentication` interface. If an account fails to implement this interface, it's considered non-externally owned, resulting in transaction rejection. + +###### Sequence Diagram + +```mermaid +graph TD + A[Tx Is Received] --> B[Execute Signature Verification Ante Handler] + B --> D{Is signer an x/accounts account?} + D -->|No| E[Continue with signature verification ante handler] + D -->|Yes| F{Does account handle MsgAuthenticate?} + F -->|No| G[Fail TX: Non-externally owned account] + F -->|Yes| H[Invoke signer account MsgAuthenticate] + E --> I[End] + G --> I + H --> I +``` + +### Implementing the Authentication Interface + +To implement the Authentication interface, an account must handle the execution of `MsgAuthenticate`. Here's an example of how to do this: + +```go +package base + +import ( + "context" + "errors" + aa_interface_v1 "github.com/cosmos/cosmos-sdk/x/accounts/interfaces/account_abstraction/v1" + "github.com/cosmos/cosmos-sdk/x/accounts/std" +) + +// Account represents a base account structure +type Account struct { + // Account fields... +} + +// Authenticate implements the authentication flow for an abstracted base account. +func (a Account) Authenticate(ctx context.Context, msg *aa_interface_v1.MsgAuthenticate) (*aa_interface_v1.MsgAuthenticateResponse, error) { + if !accountstd.SenderIsAccountsModule(ctx) { + return nil, errors.New("unauthorized: only accounts module is allowed to call this") + } + // Implement your authentication logic here + // ... + return &aa_interface_v1.MsgAuthenticateResponse{}, nil +} + +// RegisterExecuteHandlers registers the execution handlers for the account. +func (a Account) RegisterExecuteHandlers(builder *accountstd.ExecuteBuilder) { + accountstd.RegisterExecuteHandler(builder, a.SwapPubKey) // Other handlers + accountstd.RegisterExecuteHandler(builder, a.Authenticate) // Implements the Authentication interface +} +``` + +#### Key Implementation Points + +1. **Sender Verification**: Always verify that the sender is the x/accounts module. This prevents unauthorized accounts from triggering authentication. +2. **Authentication Safety**: Ensure your authentication mechanism is secure: + - Prevent replay attacks by making it impossible to reuse the same action with the same signature. + +##### Implementation example + +Please find an example [here](./defaults/base/account.go). + +## Supporting Custom Accounts in the x/auth gRPC Server + +### Overview + +The x/auth module provides a mechanism for custom account types to be exposed via its `Account` and `AccountInfo` gRPC +queries. This feature is particularly useful for ensuring compatibility with existing wallets that have not yet integrated +with x/accounts but still need to parse account information post-migration. + +### Implementation + +To support this feature, your custom account type needs to implement the `auth.QueryLegacyAccount` handler. Here are some important points to consider: + +1. **Selective Implementation**: This implementation is not required for every account type. It's only necessary for accounts you want to expose through the x/auth gRPC `Account` and `AccountInfo` methods. +2. **Flexible Response**: The `info` field in the `QueryLegacyAccountResponse` is optional. If your custom account cannot be represented as a `BaseAccount`, you can leave this field empty. + +### Example Implementation + +A concrete example of implementation can be found in `defaults/base/account.go`. Here's a simplified version: + +```go +func (a Account) AuthRetroCompatibility(ctx context.Context, _ *authtypes.QueryLegacyAccount) (*authtypes.QueryLegacyAccountResponse, error) { + seq := a.GetSequence() + num := a.GetNumber() + address := a.GetAddress() + pubKey := a.GetPubKey() + + baseAccount := &authtypes.BaseAccount{ + AccountNumber: num, + Sequence: seq, + Address: address, + } + + // Convert pubKey to Any type + pubKeyAny, err := gogotypes.NewAnyWithValue(pubKey) + if err != nil { + return nil, err + } + baseAccount.PubKey = pubKeyAny + + // Convert the entire baseAccount to Any type + accountAny, err := gogotypes.NewAnyWithValue(baseAccount) + if err != nil { + return nil, err + } + + return &authtypes.QueryLegacyAccountResponse{ + Account: accountAny, + Info: baseAccount, + }, nil +} +``` + +### Usage Notes + +- Implement this handler only for account types you want to expose via x/auth gRPC methods. +- The `info` field in the response can be nil if your account doesn't fit the `BaseAccount` structure. + +## Genesis + +### Creating accounts on genesis + +In order to create accounts at genesis, the `x/accounts` module allows developers to provide +a list of genesis `MsgInit` messages that will be executed in the `x/accounts` genesis flow. + +The init messages are generated offline. You can also use the following CLI command to generate the +json messages: `simd accounts tx init [account type] [msg] --from me --genesis`. This will generate +a jsonified init message wrapped in an x/accounts `MsgInit`. + +This follows the same initialization flow and rules that would happen if the chain is running. +The only concrete difference is that this is happening at the genesis block. + +For example, given the following `genesis.json` file: + +```json +{ + "app_state": { + "accounts": { + "init_account_msgs": [ + { + "sender": "account_creator_address", + "account_type": "lockup", + "message": { + "@type": "cosmos.accounts.defaults.lockup.MsgInitLockupAccount", + "owner": "some_owner", + "end_time": "..", + "start_time": ".." + }, + "funds": [ + { + "denom": "stake", + "amount": "1000" + } + ] + } + ] + } + } +} +``` + +The accounts module will run the lockup account initialization message. \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/modules/auth/1-vesting.md b/versioned_docs/version-0.52/build/modules/auth/1-vesting.md new file mode 100644 index 000000000..5fd33462a --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/auth/1-vesting.md @@ -0,0 +1,584 @@ +--- +sidebar_position: 1 +--- + +# `x/auth/vesting` + +:::warning +Vesting accounts are deprecated in favor of `x/accounts`. +The creation of vesting account, using `x/auth/vesting`, is not possible since v0.52. +For existing chains, importing the `x/auth/vesting module` is still required for backward compatibility purposes. +::: + +* [Intro and Requirements](#intro-and-requirements) +* [Note](#note) +* [Vesting Account Types](#vesting-account-types) + * [BaseVestingAccount](#basevestingaccount) + * [ContinuousVestingAccount](#continuousvestingaccount) + * [DelayedVestingAccount](#delayedvestingaccount) + * [Period](#period) + * [PeriodicVestingAccount](#periodicvestingaccount) + * [PermanentLockedAccount](#permanentlockedaccount) +* [Vesting Account Specification](#vesting-account-specification) + * [Determining Vesting & Vested Amounts](#determining-vesting--vested-amounts) + * [Periodic Vesting Accounts](#periodic-vesting-accounts) + * [Transferring/Sending](#transferringsending) + * [Delegating](#delegating) + * [Undelegating](#undelegating) +* [Keepers & Handlers](#keepers--handlers) +* [Genesis Initialization](#genesis-initialization) +* [Examples](#examples) + * [Simple](#simple) + * [Slashing](#slashing) + * [Periodic Vesting](#periodic-vesting) +* [Glossary](#glossary) + +## Intro and Requirements + +This specification defines the vesting account implementation that is used by the Cosmos Hub. The requirements for this vesting account is that it should be initialized during genesis with a starting balance `X` and a vesting end time `ET`. A vesting account may be initialized with a vesting start time `ST` and a number of vesting periods `P`. If a vesting start time is included, the vesting period does not begin until start time is reached. If vesting periods are included, the vesting occurs over the specified number of periods. + +For all vesting accounts, the owner of the vesting account is able to delegate and undelegate from validators, however they cannot transfer coins to another account until those coins are vested. This specification allows for four different kinds of vesting: + +* Delayed vesting, where all coins are vested once `ET` is reached. +* Continuous vesting, where coins begin to vest at `ST` and vest linearly with respect to time until `ET` is reached +* Periodic vesting, where coins begin to vest at `ST` and vest periodically according to number of periods and the vesting amount per period. The number of periods, length per period, and amount per period are configurable. A periodic vesting account is distinguished from a continuous vesting account in that coins can be released in staggered tranches. For example, a periodic vesting account could be used for vesting arrangements where coins are released quarterly, yearly, or over any other function of tokens over time. +* Permanent locked vesting, where coins are locked forever. Coins in this account can still be used for delegating and for governance votes even while locked. + +## Note + +Vesting accounts can be initialized with some vesting and non-vesting coins. The non-vesting coins would be immediately transferable. DelayedVesting ContinuousVesting, PeriodicVesting and PermenantVesting accounts can be created with normal messages after genesis. Other types of vesting accounts must be created at genesis, or as part of a manual network upgrade. The current specification only allows for _unconditional_ vesting (ie. there is no possibility of reaching `ET` and +having coins fail to vest). + +## Vesting Account Types + +```go +// VestingAccount defines an interface that any vesting account type must +// implement. +type VestingAccount interface { + Account + + GetVestedCoins(Time) Coins + GetVestingCoins(Time) Coins + + // TrackDelegation performs internal vesting accounting necessary when + // delegating from a vesting account. It accepts the current block time, the + // delegation amount and balance of all coins whose denomination exists in + // the account's original vesting balance. + TrackDelegation(Time, Coins, Coins) + + // TrackUndelegation performs internal vesting accounting necessary when a + // vesting account performs an undelegation. + TrackUndelegation(Coins) + + GetStartTime() int64 + GetEndTime() int64 +} +``` + +### BaseVestingAccount + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/vesting/v1beta1/vesting.proto#L11-L35 +``` + +### ContinuousVestingAccount + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/vesting/v1beta1/vesting.proto#L37-L46 +``` + +### DelayedVestingAccount + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/vesting/v1beta1/vesting.proto#L48-L57 +``` + +### Period + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/vesting/v1beta1/vesting.proto#L59-L69 +``` + +```go +// Stores all vesting periods passed as part of a PeriodicVestingAccount +type Periods []Period + +``` + +### PeriodicVestingAccount + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/vesting/v1beta1/vesting.proto#L71-L81 +``` + +In order to facilitate less ad-hoc type checking and assertions and to support flexibility in account balance usage, the existing `x/bank` `ViewKeeper` interface is updated to contain the following: + +```go +type ViewKeeper interface { + // ... + + // Calculates the total locked account balance. + LockedCoins(ctx sdk.Context, addr sdk.AccAddress) sdk.Coins + + // Calculates the total spendable balance that can be sent to other accounts. + SpendableCoins(ctx sdk.Context, addr sdk.AccAddress) sdk.Coins +} +``` + +### PermanentLockedAccount + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/vesting/v1beta1/vesting.proto#L83-L94 +``` + +## Vesting Account Specification + +Given a vesting account, we define the following in the proceeding operations: + +* `OV`: The original vesting coin amount. It is a constant value. +* `V`: The number of `OV` coins that are still _vesting_. It is derived by +`OV`, `StartTime` and `EndTime`. This value is computed on demand and not on a per-block basis. +* `V'`: The number of `OV` coins that are _vested_ (unlocked). This value is computed on demand and not a per-block basis. +* `DV`: The number of delegated _vesting_ coins. It is a variable value. It is stored and modified directly in the vesting account. +* `DF`: The number of delegated _vested_ (unlocked) coins. It is a variable value. It is stored and modified directly in the vesting account. +* `BC`: The number of `OV` coins less any coins that are transferred +(which can be negative or delegated). It is considered to be balance of the embedded base account. It is stored and modified directly in the vesting account. + +### Determining Vesting & Vested Amounts + +It is important to note that these values are computed on demand and not on a mandatory per-block basis (e.g. `BeginBlocker` or `EndBlocker`). + +#### Continuously Vesting Accounts + +To determine the amount of coins that are vested for a given block time `T`, the +following is performed: + +1. Compute `X := T - StartTime` +2. Compute `Y := EndTime - StartTime` +3. Compute `V' := OV * (X / Y)` +4. Compute `V := OV - V'` + +Thus, the total amount of _vested_ coins is `V'` and the remaining amount, `V`, +is _vesting_. + +```go +func (cva ContinuousVestingAccount) GetVestedCoins(t Time) Coins { + if t <= cva.StartTime { + // We must handle the case where the start time for a vesting account has + // been set into the future or when the start of the chain is not exactly + // known. + return ZeroCoins + } else if t >= cva.EndTime { + return cva.OriginalVesting + } + + x := t - cva.StartTime + y := cva.EndTime - cva.StartTime + + return cva.OriginalVesting * (x / y) +} + +func (cva ContinuousVestingAccount) GetVestingCoins(t Time) Coins { + return cva.OriginalVesting - cva.GetVestedCoins(t) +} +``` + +### Periodic Vesting Accounts + +Periodic vesting accounts require calculating the coins released during each period for a given block time `T`. Note that multiple periods could have passed when calling `GetVestedCoins`, so we must iterate over each period until the end of that period is after `T`. + +1. Set `CT := StartTime` +2. Set `V' := 0` + +For each Period P: + + 1. Compute `X := T - CT` + 2. IF `X >= P.Length` + 1. Compute `V' += P.Amount` + 2. Compute `CT += P.Length` + 3. ELSE break + 3. Compute `V := OV - V'` + +```go +func (pva PeriodicVestingAccount) GetVestedCoins(t Time) Coins { + if t < pva.StartTime { + return ZeroCoins + } + ct := pva.StartTime // The start of the vesting schedule + vested := 0 + periods = pva.GetPeriods() + for _, period := range periods { + if t - ct < period.Length { + break + } + vested += period.Amount + ct += period.Length // increment ct to the start of the next vesting period + } + return vested +} + +func (pva PeriodicVestingAccount) GetVestingCoins(t Time) Coins { + return pva.OriginalVesting - cva.GetVestedCoins(t) +} +``` + +#### Delayed/Discrete Vesting Accounts + +Delayed vesting accounts are easier to reason about as they only have the full amount vesting up until a certain time, then all the coins become vested (unlocked). This does not include any unlocked coins the account may have initially. + +```go +func (dva DelayedVestingAccount) GetVestedCoins(t Time) Coins { + if t >= dva.EndTime { + return dva.OriginalVesting + } + + return ZeroCoins +} + +func (dva DelayedVestingAccount) GetVestingCoins(t Time) Coins { + return dva.OriginalVesting - dva.GetVestedCoins(t) +} +``` + +### Transferring/Sending + +At any given time, a vesting account may transfer: `min((BC + DV) - V, BC)`. + +In other words, a vesting account may transfer the minimum of the base account balance and the base account balance plus the number of currently delegated vesting coins less the number of coins vested so far. + +However, given that account balances are tracked via the `x/bank` module and that we want to avoid loading the entire account balance, we can instead determine the locked balance, which can be defined as `max(V - DV, 0)`, and infer the spendable balance from that. + +```go +func (va VestingAccount) LockedCoins(t Time) Coins { + return max(va.GetVestingCoins(t) - va.DelegatedVesting, 0) +} +``` + +The `x/bank` `ViewKeeper` can then provide APIs to determine locked and spendable coins for any account: + +```go +func (k Keeper) LockedCoins(ctx Context, addr AccAddress) Coins { + acc := k.GetAccount(ctx, addr) + if acc != nil { + if acc.IsVesting() { + return acc.LockedCoins(ctx.HeaderInfo().Time) + } + } + + // non-vesting accounts do not have any locked coins + return NewCoins() +} +``` + +#### Keepers/Handlers + +The corresponding `x/bank` keeper should appropriately handle sending coins based on if the account is a vesting account or not. + +```go +func (k Keeper) SendCoins(ctx Context, from Account, to Account, amount Coins) { + bc := k.GetBalances(ctx, from) + v := k.LockedCoins(ctx, from) + + spendable := bc - v + newCoins := spendable - amount + assert(newCoins >= 0) + + from.SetBalance(newCoins) + to.AddBalance(amount) + + // save balances... +} +``` + +### Delegating + +For a vesting account attempting to delegate `D` coins, the following is performed: + +1. Verify `BC >= D > 0` +2. Compute `X := min(max(V - DV, 0), D)` (portion of `D` that is vesting) +3. Compute `Y := D - X` (portion of `D` that is free) +4. Set `DV += X` +5. Set `DF += Y` + +```go +func (va VestingAccount) TrackDelegation(t Time, balance Coins, amount Coins) { + assert(balance <= amount) + x := min(max(va.GetVestingCoins(t) - va.DelegatedVesting, 0), amount) + y := amount - x + + va.DelegatedVesting += x + va.DelegatedFree += y +} +``` + +**Note** `TrackDelegation` only modifies the `DelegatedVesting` and `DelegatedFree` fields, so upstream callers MUST modify the `Coins` field by subtracting `amount`. + +#### Keepers/Handlers + +```go +func DelegateCoins(t Time, from Account, amount Coins) { + if isVesting(from) { + from.TrackDelegation(t, amount) + } else { + from.SetBalance(sc - amount) + } + + // save account... +} +``` + +### Undelegating + +For a vesting account attempting to undelegate `D` coins, the following is performed: + +> NOTE: `DV < D` and `(DV + DF) < D` may be possible due to quirks in the rounding of delegation/undelegation logic. + +1. Verify `D > 0` +2. Compute `X := min(DF, D)` (portion of `D` that should become free, prioritizing free coins) +3. Compute `Y := min(DV, D - X)` (portion of `D` that should remain vesting) +4. Set `DF -= X` +5. Set `DV -= Y` + +```go +func (cva ContinuousVestingAccount) TrackUndelegation(amount Coins) { + x := min(cva.DelegatedFree, amount) + y := amount - x + + cva.DelegatedFree -= x + cva.DelegatedVesting -= y +} +``` + +**Note** `TrackUnDelegation` only modifies the `DelegatedVesting` and `DelegatedFree` fields, so upstream callers MUST modify the `Coins` field by adding `amount`. + +**Note**: If a delegation is slashed, the continuous vesting account ends up with an excess `DV` amount, even after all its coins have vested. This is because undelegating free coins are prioritized. + +**Note**: The undelegation (bond refund) amount may exceed the delegated vesting (bond) amount due to the way undelegation truncates the bond refund, which can increase the validator's exchange rate (tokens/shares) slightly if the undelegated tokens are non-integral. + +#### Keepers/Handlers + +```go +func UndelegateCoins(to Account, amount Coins) { + if isVesting(to) { + if to.DelegatedFree + to.DelegatedVesting >= amount { + to.TrackUndelegation(amount) + // save account ... + } + } else { + AddBalance(to, amount) + // save account... + } +} +``` + +## Keepers & Handlers + +The `VestingAccount` implementations reside in `x/auth`. However, any keeper in a module (e.g. staking in `x/staking`) wishing to potentially utilize any vesting coins, must call explicit methods on the `x/bank` keeper (e.g. `DelegateCoins`) opposed to `SendCoins` and `SubtractCoins`. + +In addition, the vesting account should also be able to spend any coins it receives from other users. Thus, the bank module's `MsgSend` handler should error if a vesting account is trying to send an amount that exceeds their unlocked coin amount. + +See the above specification for full implementation details. + +## Genesis Initialization + +To initialize both vesting and non-vesting accounts, the `GenesisAccount` struct includes new fields: `Vesting`, `StartTime`, and `EndTime`. Accounts meant to be of type `BaseAccount` or any non-vesting type have `Vesting = false`. The genesis initialization logic (e.g. `initFromGenesisState`) must parse and return the correct accounts accordingly based off of these fields. + +```go +type GenesisAccount struct { + // ... + + // vesting account fields + OriginalVesting sdk.Coins `json:"original_vesting"` + DelegatedFree sdk.Coins `json:"delegated_free"` + DelegatedVesting sdk.Coins `json:"delegated_vesting"` + StartTime int64 `json:"start_time"` + EndTime int64 `json:"end_time"` +} + +func ToAccount(gacc GenesisAccount) Account { + bacc := NewBaseAccount(gacc) + + if gacc.OriginalVesting > 0 { + if ga.StartTime != 0 && ga.EndTime != 0 { + // return a continuous vesting account + } else if ga.EndTime != 0 { + // return a delayed vesting account + } else { + // invalid genesis vesting account provided + panic() + } + } + + return bacc +} +``` + +## Examples + +### Simple + +Given a continuous vesting account with 10 vesting coins. + +```text +OV = 10 +DF = 0 +DV = 0 +BC = 10 +V = 10 +V' = 0 +``` + +1. Immediately receives 1 coin + + ```text + BC = 11 + ``` + +2. Time passes, 2 coins vest + + ```text + V = 8 + V' = 2 + ``` + +3. Delegates 4 coins to validator A + + ```text + DV = 4 + BC = 7 + ``` + +4. Sends 3 coins + + ```text + BC = 4 + ``` + +5. More time passes, 2 more coins vest + + ```text + V = 6 + V' = 4 + ``` + +6. Sends 2 coins. At this point the account cannot send anymore until further +coins vest or it receives additional coins. It can still however, delegate. + + ```text + BC = 2 + ``` + +### Slashing + +Same initial starting conditions as the simple example. + +1. Time passes, 5 coins vest + + ```text + V = 5 + V' = 5 + ``` + +2. Delegate 5 coins to validator A + + ```text + DV = 5 + BC = 5 + ``` + +3. Delegate 5 coins to validator B + + ```text + DF = 5 + BC = 0 + ``` + +4. Validator A gets slashed by 50%, making the delegation to A now worth 2.5 coins +5. Undelegate from validator A (2.5 coins) + + ```text + DF = 5 - 2.5 = 2.5 + BC = 0 + 2.5 = 2.5 + ``` + +6. Undelegate from validator B (5 coins). The account at this point can only +send 2.5 coins unless it receives more coins or until more coins vest. +It can still however, delegate. + + ```text + DV = 5 - 2.5 = 2.5 + DF = 2.5 - 2.5 = 0 + BC = 2.5 + 5 = 7.5 + ``` + + Notice how we have an excess amount of `DV`. + +### Periodic Vesting + +A vesting account is created where 100 tokens will be released over 1 year, with +1/4 of tokens vesting each quarter. The vesting schedule would be as follows: + +```yaml +Periods: +- amount: 25stake, length: 7884000 +- amount: 25stake, length: 7884000 +- amount: 25stake, length: 7884000 +- amount: 25stake, length: 7884000 +``` + +```text +OV = 100 +DF = 0 +DV = 0 +BC = 100 +V = 100 +V' = 0 +``` + +1. Immediately receives 1 coin + + ```text + BC = 101 + ``` + +2. Vesting period 1 passes, 25 coins vest + + ```text + V = 75 + V' = 25 + ``` + +3. During vesting period 2, 5 coins are transferred and 5 coins are delegated + + ```text + DV = 5 + BC = 91 + ``` + +4. Vesting period 2 passes, 25 coins vest + + ```text + V = 50 + V' = 50 + ``` + +## Glossary + +* OriginalVesting: The amount of coins (per denomination) that are initially +part of a vesting account. These coins are set at genesis. +* StartTime: The BFT time at which a vesting account starts to vest. +* EndTime: The BFT time at which a vesting account is fully vested. +* DelegatedFree: The tracked amount of coins (per denomination) that are +delegated from a vesting account that have been fully vested at time of delegation. +* DelegatedVesting: The tracked amount of coins (per denomination) that are +delegated from a vesting account that were vesting at time of delegation. +* ContinuousVestingAccount: A vesting account implementation that vests coins +linearly over time. +* DelayedVestingAccount: A vesting account implementation that only fully vests +all coins at a given time. +* PeriodicVestingAccount: A vesting account implementation that vests coins +according to a custom vesting schedule. +* PermanentLockedAccount: It does not ever release coins, locking them indefinitely. +Coins in this account can still be used for delegating and for governance votes even while locked. + + diff --git a/versioned_docs/version-0.52/build/modules/auth/2-tx.md b/versioned_docs/version-0.52/build/modules/auth/2-tx.md new file mode 100644 index 000000000..07941952c --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/auth/2-tx.md @@ -0,0 +1,293 @@ +--- +sidebar_position: 1 +--- + +# `x/auth/tx` + +:::note Pre-requisite Readings + +* [Transactions](https://docs.cosmos.network/main/core/transactions#transaction-generation) +* [Encoding](https://docs.cosmos.network/main/core/encoding#transaction-encoding) + +::: + +## Abstract + +This document specifies the `x/auth/tx` package of the Cosmos SDK. + +This package represents the Cosmos SDK implementation of the `client.TxConfig`, `client.TxBuilder`, `client.TxEncoder` and `client.TxDecoder` interfaces. + +## Contents + +* [`x/auth/tx`](#xauthtx) + * [Abstract](#abstract) + * [Contents](#contents) + * [Transactions](#transactions) + * [`TxConfig`](#txconfig) + * [`TxBuilder`](#txbuilder) + * [`TxEncoder`/ `TxDecoder`](#txencoder-txdecoder) + * [`x/auth/tx/config`](#xauthtxconfig) + * [Storage](#storage) + * [Client](#client) + * [CLI](#cli) + * [Query](#query) + * [Transactions](#transactions-1) + * [`encode`](#encode) + * [`decode`](#decode) + * [gRPC](#grpc) + * [`TxDecode`](#txdecode) + * [`TxEncode`](#txencode) + * [`TxDecodeAmino`](#txdecodeamino) + * [`TxEncodeAmino`](#txencodeamino) + +## Transactions + +### `TxConfig` + +`client.TxConfig` defines an interface a client can utilize to generate an application-defined concrete transaction type. +The interface defines a set of methods for creating a `client.TxBuilder`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/client/tx_config.go#L25-L31 +``` + +The default implementation of `client.TxConfig` is instantiated by `NewTxConfig` in `x/auth/tx` module. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/auth/tx/config.go#L22-L28 +``` + +### `TxBuilder` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/client/tx_config.go#L33-L50 +``` + +The [`client.TxBuilder`](https://docs.cosmos.network/main/core/transactions#transaction-generation) interface is as well implemented by `x/auth/tx`. +A `client.TxBuilder` can be accessed with `TxConfig.NewTxBuilder()`. + +### `TxEncoder`/ `TxDecoder` + +More information about `TxEncoder` and `TxDecoder` can be found [here](https://docs.cosmos.network/main/core/encoding#transaction-encoding). + +## `x/auth/tx/config` + +The `x/auth/tx/config` contains a depinject module. +The depinject module is to outputs the `TxConfig` and `TxConfigOptions` for the app. + +### Storage + +This module has no store key. Do not forget to add the module name in the `SkipStoreKeys` runtime config present in the app config. + +```go +SkipStoreKeys: []string{ + authtxconfig.DepinjectModuleName, + validate.ModuleName, +}, +``` + +## Client + +### CLI + +#### Query + +The `x/auth/tx` module provides a CLI command to query any transaction, given its hash, transaction sequence or signature. + +Without any argument, the command will query the transaction using the transaction hash. + +```shell +simd query tx DFE87B78A630C0EFDF76C80CD24C997E252792E0317502AE1A02B9809F0D8685 +``` + +When querying a transaction from an account given its sequence, use the `--type=acc_seq` flag: + +```shell +simd query tx --type=acc_seq cosmos1u69uyr6v9qwe6zaaeaqly2h6wnedac0xpxq325/1 +``` + +When querying a transaction given its signature, use the `--type=signature` flag: + +```shell +simd query tx --type=signature Ofjvgrqi8twZfqVDmYIhqwRLQjZZ40XbxEamk/veH3gQpRF0hL2PH4ejRaDzAX+2WChnaWNQJQ41ekToIi5Wqw== +``` + +When querying a transaction given its events, use the `--type=events` flag: + +```shell +simd query txs --events 'message.sender=cosmos...' --page 1 --limit 30 +``` + +The `x/auth/block` module provides a CLI command to query any block, given its hash, height, or events. + +When querying a block by its hash, use the `--type=hash` flag: + +```shell +simd query block --type=hash DFE87B78A630C0EFDF76C80CD24C997E252792E0317502AE1A02B9809F0D8685 +``` + +When querying a block by its height, use the `--type=height` flag: + +```shell +simd query block --type=height 1357 +``` + +When querying a block by its events, use the `--query` flag: + +```shell +simd query blocks --query 'message.sender=cosmos...' --page 1 --limit 30 +``` + +#### Transactions + +The `x/auth/tx` module provides a convenient CLI command for decoding and encoding transactions. + +#### `encode` + +The `encode` command encodes a transaction created with the `--generate-only` flag or signed with the sign command. +The transaction is serialized it to Protobuf and returned as base64. + +```bash +$ simd tx encode tx.json +Co8BCowBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmwKLWNvc21vczFsNnZzcWhoN3Jud3N5cjJreXozampnM3FkdWF6OGd3Z3lsODI3NRItY29zbW9zMTU4c2FsZHlnOHBteHU3Znd2dDBkNng3amVzd3A0Z3d5a2xrNnkzGgwKBXN0YWtlEgMxMDASBhIEEMCaDA== +$ simd tx encode tx.signed.json +``` + +More information about the `encode` command can be found running `simd tx encode --help`. + +#### `decode` + +The `decode` commands decodes a transaction encoded with the `encode` command. + + +```bash +simd tx decode Co8BCowBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmwKLWNvc21vczFsNnZzcWhoN3Jud3N5cjJreXozampnM3FkdWF6OGd3Z3lsODI3NRItY29zbW9zMTU4c2FsZHlnOHBteHU3Znd2dDBkNng3amVzd3A0Z3d5a2xrNnkzGgwKBXN0YWtlEgMxMDASBhIEEMCaDA== +``` + +More information about the `decode` command can be found running `simd tx decode --help`. + +### gRPC + +A user can query the `x/auth/tx` module using gRPC endpoints. + +#### `TxDecode` + +The `TxDecode` endpoint allows to decode a transaction. + +```shell +cosmos.tx.v1beta1.Service/TxDecode +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"tx_bytes":"Co8BCowBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmwKLWNvc21vczFsNnZzcWhoN3Jud3N5cjJreXozampnM3FkdWF6OGd3Z3lsODI3NRItY29zbW9zMTU4c2FsZHlnOHBteHU3Znd2dDBkNng3amVzd3A0Z3d5a2xrNnkzGgwKBXN0YWtlEgMxMDASBhIEEMCaDA=="}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/TxDecode +``` + +Example Output: + +```json +{ + "tx": { + "body": { + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"100"}],"fromAddress":"cosmos1l6vsqhh7rnwsyr2kyz3jjg3qduaz8gwgyl8275","toAddress":"cosmos158saldyg8pmxu7fwvt0d6x7jeswp4gwyklk6y3"} + ] + }, + "authInfo": { + "fee": { + "gasLimit": "200000" + } + } + } +} +``` + +#### `TxEncode` + +The `TxEncode` endpoint allows to encode a transaction. + +```shell +cosmos.tx.v1beta1.Service/TxEncode +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"tx": { + "body": { + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"100"}],"fromAddress":"cosmos1l6vsqhh7rnwsyr2kyz3jjg3qduaz8gwgyl8275","toAddress":"cosmos158saldyg8pmxu7fwvt0d6x7jeswp4gwyklk6y3"} + ] + }, + "authInfo": { + "fee": { + "gasLimit": "200000" + } + } + }}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/TxEncode +``` + +Example Output: + +```json +{ + "txBytes": "Co8BCowBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmwKLWNvc21vczFsNnZzcWhoN3Jud3N5cjJreXozampnM3FkdWF6OGd3Z3lsODI3NRItY29zbW9zMTU4c2FsZHlnOHBteHU3Znd2dDBkNng3amVzd3A0Z3d5a2xrNnkzGgwKBXN0YWtlEgMxMDASBhIEEMCaDA==" +} +``` + +#### `TxDecodeAmino` + +The `TxDecode` endpoint allows to decode an amino transaction. + +```shell +cosmos.tx.v1beta1.Service/TxDecodeAmino +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"amino_binary": "KCgWqQpvqKNhmgotY29zbW9zMXRzeno3cDJ6Z2Q3dnZrYWh5ZnJlNHduNXh5dTgwcnB0ZzZ2OWg1Ei1jb3Ntb3MxdHN6ejdwMnpnZDd2dmthaHlmcmU0d241eHl1ODBycHRnNnY5aDUaCwoFc3Rha2USAjEwEhEKCwoFc3Rha2USAjEwEMCaDCIGZm9vYmFy"}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/TxDecodeAmino +``` + +Example Output: + +```json +{ + "aminoJson": "{\"type\":\"cosmos-sdk/StdTx\",\"value\":{\"msg\":[{\"type\":\"cosmos-sdk/MsgSend\",\"value\":{\"from_address\":\"cosmos1tszz7p2zgd7vvkahyfre4wn5xyu80rptg6v9h5\",\"to_address\":\"cosmos1tszz7p2zgd7vvkahyfre4wn5xyu80rptg6v9h5\",\"amount\":[{\"denom\":\"stake\",\"amount\":\"10\"}]}}],\"fee\":{\"amount\":[{\"denom\":\"stake\",\"amount\":\"10\"}],\"gas\":\"200000\"},\"signatures\":null,\"memo\":\"foobar\",\"timeout_height\":\"0\"}}" +} +``` + +#### `TxEncodeAmino` + +The `TxEncodeAmino` endpoint allows to encode an amino transaction. + +```shell +cosmos.tx.v1beta1.Service/TxEncodeAmino +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"amino_json":"{\"type\":\"cosmos-sdk/StdTx\",\"value\":{\"msg\":[{\"type\":\"cosmos-sdk/MsgSend\",\"value\":{\"from_address\":\"cosmos1tszz7p2zgd7vvkahyfre4wn5xyu80rptg6v9h5\",\"to_address\":\"cosmos1tszz7p2zgd7vvkahyfre4wn5xyu80rptg6v9h5\",\"amount\":[{\"denom\":\"stake\",\"amount\":\"10\"}]}}],\"fee\":{\"amount\":[{\"denom\":\"stake\",\"amount\":\"10\"}],\"gas\":\"200000\"},\"signatures\":null,\"memo\":\"foobar\",\"timeout_height\":\"0\"}}"}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/TxEncodeAmino +``` + +Example Output: + +```json +{ + "amino_binary": "KCgWqQpvqKNhmgotY29zbW9zMXRzeno3cDJ6Z2Q3dnZrYWh5ZnJlNHduNXh5dTgwcnB0ZzZ2OWg1Ei1jb3Ntb3MxdHN6ejdwMnpnZDd2dmthaHlmcmU0d241eHl1ODBycHRnNnY5aDUaCwoFc3Rha2USAjEwEhEKCwoFc3Rha2USAjEwEMCaDCIGZm9vYmFy" +} +``` diff --git a/versioned_docs/version-0.52/build/modules/auth/README.md b/versioned_docs/version-0.52/build/modules/auth/README.md new file mode 100644 index 000000000..4adf48bdb --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/auth/README.md @@ -0,0 +1,716 @@ +--- +sidebar_position: 1 +--- + +# `x/auth` + +## Abstract + +This document specifies the auth module of the Cosmos SDK. + +The auth module is responsible for specifying the base transaction and account types +for an application, since the SDK itself is agnostic to these particulars. It contains +the middlewares, where all basic transaction validity checks (signatures, nonces, auxiliary fields) +are performed, and exposes the account keeper, which allows other modules to read, write, and modify accounts. + +This module is used in the Cosmos Hub. + +## Contents + +* [Concepts](#concepts) + * [Gas & Fees](#gas--fees) +* [State](#state) + * [Accounts](#accounts) +* [AnteHandlers](#antehandlers) +* [Keepers](#keepers) + * [Account Keeper](#account-keeper) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) + +## Concepts + +**Note:** The auth module is different from the [authz module](../modules/authz/). + +The differences are: + +* `auth` - authentication of accounts and transactions for Cosmos SDK applications and is responsible for specifying the base transaction and account types. +* `authz` - authorization for accounts to perform actions on behalf of other accounts and enables a granter to grant authorizations to a grantee that allows the grantee to execute messages on behalf of the granter. + +### Gas & Fees + +Fees serve two purposes for an operator of the network. + +Fees limit the growth of the state stored by every full node and allow for +general purpose censorship of transactions of little economic value. Fees +are best suited as an anti-spam mechanism where validators are disinterested in +the use of the network and identities of users. + +Fees are determined by the gas limits and gas prices transactions provide, where +`fees = ceil(gasLimit * gasPrices)`. Txs incur gas costs for all state reads/writes, +signature verification, as well as costs proportional to the tx size. Operators +should set minimum gas prices when starting their nodes. They must set the unit +costs of gas in each token denomination they wish to support: + +`simd start ... --minimum-gas-prices=0.00001stake;0.05photinos` + +When adding transactions to mempool or gossipping transactions, validators check +if the transaction's gas prices, which are determined by the provided fees, meet +any of the validator's minimum gas prices. In other words, a transaction must +provide a fee of at least one denomination that matches a validator's minimum +gas price. + +CometBFT does not currently provide fee based mempool prioritization, and fee +based mempool filtering is local to node and not part of consensus. But with +minimum gas prices set, such a mechanism could be implemented by node operators. + +Because the market value for tokens will fluctuate, validators are expected to +dynamically adjust their minimum gas prices to a level that would encourage the +use of the network. + +## State + +### Accounts + +Accounts contain authentication information for a uniquely identified external user of an SDK blockchain, +including public key, address, and account number / sequence number for replay protection. For efficiency, +since account balances must also be fetched to pay fees, account structs also store the balance of a user +as `sdk.Coins`. + +Accounts are exposed externally as an interface, and stored internally as +either a base account or vesting account. Module clients wishing to add more +account types may do so. + +* `0x01 | Address -> ProtocolBuffer(account)` + +#### Account Interface + +The account interface exposes methods to read and write standard account information. +Note that all of these methods operate on an account struct conforming to the +interface - in order to write the account to the store, the account keeper will +need to be used. + +```go +// AccountI is an interface used to store coins at a given address within state. +// It presumes a notion of sequence numbers for replay protection, +// a notion of account numbers for replay protection for previously pruned accounts, +// and a pubkey for authentication purposes. +// +// Many complex conditions can be used in the concrete struct which implements AccountI. +type AccountI interface { + proto.Message + + GetAddress() sdk.AccAddress + SetAddress(sdk.AccAddress) error // errors if already set. + + GetPubKey() crypto.PubKey // can return nil. + SetPubKey(crypto.PubKey) error + + GetAccountNumber() uint64 + SetAccountNumber(uint64) error + + GetSequence() uint64 + SetSequence(uint64) error + + // Ensure that account implements stringer + String() string +} +``` + +##### Base Account + +A base account is the simplest and most common account type, which just stores all requisite +fields directly in a struct. + +```protobuf +// BaseAccount defines a base account type. It contains all the necessary fields +// for basic account functionality. Any custom account type should extend this +// type for additional functionality (e.g. vesting). +message BaseAccount { + string address = 1; + google.protobuf.Any pub_key = 2; + uint64 account_number = 3; + uint64 sequence = 4; +} +``` + +### Vesting Account + +:::warning +Vesting accounts are deprecated in favor of `x/accounts`. +The creation of vesting account, using `x/auth/vesting`, is not possible since v0.52. +For existing chains, importing the `x/auth/vesting module` is still required for backward compatibility purposes. +::: + +See [Vesting](https://docs.cosmos.network/main/modules/auth/vesting/). + +## AnteHandlers + +The `x/auth` module presently has no transaction handlers of its own, but does expose the special `AnteHandler`, used for performing basic validity checks on a transaction, such that it could be thrown out of the mempool. +The `AnteHandler` can be seen as a set of decorators that check transactions within the current context, per [ADR 010](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-010-modular-antehandler.md). + +Note that the `AnteHandler` is called on both `CheckTx` and `DeliverTx`, as CometBFT proposers presently have the ability to include in their proposed block transactions which fail `CheckTx`. + +### Decorators + +The auth module provides `AnteDecorator`s that are recursively chained together into a single `AnteHandler` in the following order: + +* `SetUpContextDecorator`: Sets the `GasMeter` in the `Context` and wraps the next `AnteHandler` with a defer clause to recover from any downstream `OutOfGas` panics in the `AnteHandler` chain to return an error with information on gas provided and gas used. + +* `RejectExtensionOptionsDecorator`: Rejects all extension options which can optionally be included in protobuf transactions. + +* `MempoolFeeDecorator`: Checks if the `tx` fee is above local mempool `minFee` parameter during `CheckTx`. + +* `ValidateBasicDecorator`: Calls `tx.ValidateBasic` and returns any non-nil error. + +* `TxTimeoutHeightDecorator`: Check for a `tx` height timeout. + +* `ValidateMemoDecorator`: Validates `tx` memo with application parameters and returns any non-nil error. + +* `ConsumeGasTxSizeDecorator`: Consumes gas proportional to the `tx` size based on application parameters. + +* `DeductFeeDecorator`: Deducts the `FeeAmount` from first signer of the `tx`. If the `x/feegrant` module is enabled and a fee granter is set, it deducts fees from the fee granter account. + +* `SetPubKeyDecorator`: Sets the pubkey from a `tx`'s signers that does not already have its corresponding pubkey saved in the state machine and in the current context. + +* `ValidateSigCountDecorator`: Validates the number of signatures in `tx` based on app-parameters. + +* `SigGasConsumeDecorator`: Consumes parameter-defined amount of gas for each signature. This requires pubkeys to be set in context for all signers as part of `SetPubKeyDecorator`. + +* `SigVerificationDecorator`: Verifies all signatures are valid. This requires pubkeys to be set in context for all signers as part of `SetPubKeyDecorator`. + +* `IncrementSequenceDecorator`: Increments the account sequence for each signer to prevent replay attacks. + +## Keepers + +The auth module only exposes one keeper, the account keeper, which can be used to read and write accounts. + +### Account Keeper + +Presently only one fully-permissioned account keeper is exposed, which has the ability to both read and write +all fields of all accounts, and to iterate over all stored accounts. + +```go +// AccountKeeperI is the interface contract that x/auth's keeper implements. +type AccountKeeperI interface { + // Return a new account with the next account number and the specified address. Does not save the new account to the store. + NewAccountWithAddress(sdk.Context, sdk.AccAddress) types.AccountI + + // Return a new account with the next account number. Does not save the new account to the store. + NewAccount(sdk.Context, types.AccountI) types.AccountI + + // Check if an account exists in the store. + HasAccount(sdk.Context, sdk.AccAddress) bool + + // Retrieve an account from the store. + GetAccount(sdk.Context, sdk.AccAddress) types.AccountI + + // Set an account in the store. + SetAccount(sdk.Context, types.AccountI) + + // Remove an account from the store. + RemoveAccount(sdk.Context, types.AccountI) + + // Iterate over all accounts, calling the provided function. Stop iteration when it returns true. + IterateAccounts(sdk.Context, func(types.AccountI) bool) + + // Fetch the public key of an account at a specified address + GetPubKey(sdk.Context, sdk.AccAddress) (crypto.PubKey, error) + + // Fetch the sequence of an account at a specified address. + GetSequence(sdk.Context, sdk.AccAddress) (uint64, error) + + // Fetch the next account number, and increment the internal counter. + NextAccountNumber(sdk.Context) uint64 +} +``` + +## Parameters + +The auth module contains the following parameters: + +| Key | Type | Example | +| ---------------------- | --------------- | ------- | +| MaxMemoCharacters | uint64 | 256 | +| TxSigLimit | uint64 | 7 | +| TxSizeCostPerByte | uint64 | 10 | +| SigVerifyCostED25519 | uint64 | 590 | +| SigVerifyCostSecp256k1 | uint64 | 1000 | + +## Client + +### CLI + +A user can query and interact with the `auth` module using the CLI. + +### Query + +The `query` commands allow users to query `auth` state. + +```bash +simd query auth --help +``` + +#### account + +The `account` command allow users to query for an account by it's address. + +```bash +simd query auth account [address] [flags] +``` + +Example: + +```bash +simd query auth account cosmos1... +``` + +Example Output: + +```bash +'@type': /cosmos.auth.v1beta1.BaseAccount +account_number: "0" +address: cosmos1zwg6tpl8aw4rawv8sgag9086lpw5hv33u5ctr2 +pub_key: + '@type': /cosmos.crypto.secp256k1.PubKey + key: ApDrE38zZdd7wLmFS9YmqO684y5DG6fjZ4rVeihF/AQD +sequence: "1" +``` + +#### accounts + +The `accounts` command allow users to query all the available accounts. + +```bash +simd query auth accounts [flags] +``` + +Example: + +```bash +simd query auth accounts +``` + +Example Output: + +```bash +accounts: +- '@type': /cosmos.auth.v1beta1.BaseAccount + account_number: "0" + address: cosmos1zwg6tpl8aw4rawv8sgag9086lpw5hv33u5ctr2 + pub_key: + '@type': /cosmos.crypto.secp256k1.PubKey + key: ApDrE38zZdd7wLmFS9YmqO684y5DG6fjZ4rVeihF/AQD + sequence: "1" +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "8" + address: cosmos1yl6hdjhmkf37639730gffanpzndzdpmhwlkfhr + pub_key: null + sequence: "0" + name: transfer + permissions: + - minter + - burner +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "4" + address: cosmos1fl48vsnmsdzcv85q5d2q4z5ajdha8yu34mf0eh + pub_key: null + sequence: "0" + name: bonded_tokens_pool + permissions: + - burner + - staking +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "5" + address: cosmos1tygms3xhhs3yv487phx3dw4a95jn7t7lpm470r + pub_key: null + sequence: "0" + name: not_bonded_tokens_pool + permissions: + - burner + - staking +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "6" + address: cosmos10d07y265gmmuvt4z0w9aw880jnsr700j6zn9kn + pub_key: null + sequence: "0" + name: gov + permissions: + - burner +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "3" + address: cosmos1jv65s3grqf6v6jl3dp4t6c9t9rk99cd88lyufl + pub_key: null + sequence: "0" + name: distribution + permissions: [] +- '@type': /cosmos.auth.v1beta1.BaseAccount + account_number: "1" + address: cosmos147k3r7v2tvwqhcmaxcfql7j8rmkrlsemxshd3j + pub_key: null + sequence: "0" +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "7" + address: cosmos1m3h30wlvsf8llruxtpukdvsy0km2kum8g38c8q + pub_key: null + sequence: "0" + name: mint + permissions: + - minter +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "2" + address: cosmos17xpfvakm2amg962yls6f84z3kell8c5lserqta + pub_key: null + sequence: "0" + name: fee_collector + permissions: [] +pagination: + next_key: null + total: "0" +``` + +#### params + +The `params` command allow users to query the current auth parameters. + +```bash +simd query auth params [flags] +``` + +Example: + +```bash +simd query auth params +``` + +Example Output: + +```bash +max_memo_characters: "256" +sig_verify_cost_ed25519: "590" +sig_verify_cost_secp256k1: "1000" +tx_sig_limit: "7" +tx_size_cost_per_byte: "10" +``` + +### Transactions + +The `auth` module supports transactions commands to help you with signing and more. Compared to other modules you can access directly the `auth` module transactions commands using the only `tx` command. + +Use directly the `--help` flag to get more information about the `tx` command. + +```bash +simd tx --help +``` + +#### `sign` + +The `sign` command allows users to sign transactions that was generated offline. + +```bash +simd tx sign tx.json --from $ALICE > tx.signed.json +``` + +The result is a signed transaction that can be broadcasted to the network thanks to the broadcast command. + +More information about the `sign` command can be found running `simd tx sign --help`. + +#### `sign-batch` + +The `sign-batch` command allows users to sign multiples offline generated transactions. +The transactions can be in one file, with one tx per line, or in multiple files. + +```bash +simd tx sign txs.json --from $ALICE > tx.signed.json +``` + +or + +```bash +simd tx sign tx1.json tx2.json tx3.json --from $ALICE > tx.signed.json +``` + +The result is multiples signed transactions. For combining the signed transactions into one transactions, use the `--append` flag. + +More information about the `sign-batch` command can be found running `simd tx sign-batch --help`. + +#### `multi-sign` + +The `multi-sign` command allows users to sign transactions that was generated offline by a multisig account. + +```bash +simd tx multi-sign transaction.json k1k2k3 k1sig.json k2sig.json k3sig.json +``` + +Where `k1k2k3` is the multisig account address, `k1sig.json` is the signature of the first signer, `k2sig.json` is the signature of the second signer, and `k3sig.json` is the signature of the third signer. + +##### Nested multisig transactions + +To allow transactions to be signed by nested multisigs, meaning that a participant of a multisig account can be another multisig account, the `--skip-signature-verification` flag must be used. + +```bash +# First aggregate signatures of the multisig participant +simd tx multi-sign transaction.json ms1 ms1p1sig.json ms1p2sig.json --signature-only --skip-signature-verification > ms1sig.json + +# Then use the aggregated signatures and the other signatures to sign the final transaction +simd tx multi-sign transaction.json k1ms1 k1sig.json ms1sig.json --skip-signature-verification +``` + +Where `ms1` is the nested multisig account address, `ms1p1sig.json` is the signature of the first participant of the nested multisig account, `ms1p2sig.json` is the signature of the second participant of the nested multisig account, and `ms1sig.json` is the aggregated signature of the nested multisig account. + +`k1ms1` is a multisig account comprised of an individual signer and another nested multisig account (`ms1`). `k1sig.json` is the signature of the first signer of the individual member. + +More information about the `multi-sign` command can be found running `simd tx multi-sign --help`. + +#### `multisign-batch` + +The `multisign-batch` works the same way as `sign-batch`, but for multisig accounts. +With the difference that the `multisign-batch` command requires all transactions to be in one file, and the `--append` flag does not exist. + +More information about the `multisign-batch` command can be found running `simd tx multisign-batch --help`. + +#### `validate-signatures` + +The `validate-signatures` command allows users to validate the signatures of a signed transaction. + +```bash +$ simd tx validate-signatures tx.signed.json +Signers: + 0: cosmos1l6vsqhh7rnwsyr2kyz3jjg3qduaz8gwgyl8275 + +Signatures: + 0: cosmos1l6vsqhh7rnwsyr2kyz3jjg3qduaz8gwgyl8275 [OK] +``` + +More information about the `validate-signatures` command can be found running `simd tx validate-signatures --help`. + +#### `broadcast` + +The `broadcast` command allows users to broadcast a signed transaction to the network. + +```bash +simd tx broadcast tx.signed.json +``` + +More information about the `broadcast` command can be found running `simd tx broadcast --help`. + + +### gRPC + +A user can query the `auth` module using gRPC endpoints. + +#### Account + +The `account` endpoint allow users to query for an account by it's address. + +```bash +cosmos.auth.v1beta1.Query/Account +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' \ + localhost:9090 \ + cosmos.auth.v1beta1.Query/Account +``` + +Example Output: + +```bash +{ + "account":{ + "@type":"/cosmos.auth.v1beta1.BaseAccount", + "address":"cosmos1zwg6tpl8aw4rawv8sgag9086lpw5hv33u5ctr2", + "pubKey":{ + "@type":"/cosmos.crypto.secp256k1.PubKey", + "key":"ApDrE38zZdd7wLmFS9YmqO684y5DG6fjZ4rVeihF/AQD" + }, + "sequence":"1" + } +} +``` + +#### Accounts + +The `accounts` endpoint allow users to query all the available accounts. + +```bash +cosmos.auth.v1beta1.Query/Accounts +``` + +Example: + +```bash +grpcurl -plaintext \ + localhost:9090 \ + cosmos.auth.v1beta1.Query/Accounts +``` + +Example Output: + +```bash +{ + "accounts":[ + { + "@type":"/cosmos.auth.v1beta1.BaseAccount", + "address":"cosmos1zwg6tpl8aw4rawv8sgag9086lpw5hv33u5ctr2", + "pubKey":{ + "@type":"/cosmos.crypto.secp256k1.PubKey", + "key":"ApDrE38zZdd7wLmFS9YmqO684y5DG6fjZ4rVeihF/AQD" + }, + "sequence":"1" + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1yl6hdjhmkf37639730gffanpzndzdpmhwlkfhr", + "accountNumber":"8" + }, + "name":"transfer", + "permissions":[ + "minter", + "burner" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1fl48vsnmsdzcv85q5d2q4z5ajdha8yu34mf0eh", + "accountNumber":"4" + }, + "name":"bonded_tokens_pool", + "permissions":[ + "burner", + "staking" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1tygms3xhhs3yv487phx3dw4a95jn7t7lpm470r", + "accountNumber":"5" + }, + "name":"not_bonded_tokens_pool", + "permissions":[ + "burner", + "staking" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos10d07y265gmmuvt4z0w9aw880jnsr700j6zn9kn", + "accountNumber":"6" + }, + "name":"gov", + "permissions":[ + "burner" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1jv65s3grqf6v6jl3dp4t6c9t9rk99cd88lyufl", + "accountNumber":"3" + }, + "name":"distribution" + }, + { + "@type":"/cosmos.auth.v1beta1.BaseAccount", + "accountNumber":"1", + "address":"cosmos147k3r7v2tvwqhcmaxcfql7j8rmkrlsemxshd3j" + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1m3h30wlvsf8llruxtpukdvsy0km2kum8g38c8q", + "accountNumber":"7" + }, + "name":"mint", + "permissions":[ + "minter" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos17xpfvakm2amg962yls6f84z3kell8c5lserqta", + "accountNumber":"2" + }, + "name":"fee_collector" + } + ], + "pagination":{ + "total":"9" + } +} +``` + +#### Params + +The `params` endpoint allow users to query the current auth parameters. + +```bash +cosmos.auth.v1beta1.Query/Params +``` + +Example: + +```bash +grpcurl -plaintext \ + localhost:9090 \ + cosmos.auth.v1beta1.Query/Params +``` + +Example Output: + +```bash +{ + "params": { + "maxMemoCharacters": "256", + "txSigLimit": "7", + "txSizeCostPerByte": "10", + "sigVerifyCostEd25519": "590", + "sigVerifyCostSecp256k1": "1000" + } +} +``` + +### REST + +A user can query the `auth` module using REST endpoints. + +#### Account + +The `account` endpoint allow users to query for an account by it's address. + +```bash +/cosmos/auth/v1beta1/account?address={address} +``` + +#### Accounts + +The `accounts` endpoint allow users to query all the available accounts. + +```bash +/cosmos/auth/v1beta1/accounts +``` + +#### Params + +The `params` endpoint allow users to query the current auth parameters. + +```bash +/cosmos/auth/v1beta1/params +``` diff --git a/versioned_docs/version-0.52/build/modules/authz/README.md b/versioned_docs/version-0.52/build/modules/authz/README.md new file mode 100644 index 000000000..405bd5427 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/authz/README.md @@ -0,0 +1,364 @@ +--- +sidebar_position: 1 +--- + +# `x/authz` + +## Abstract + +`x/authz` is an implementation of a Cosmos SDK module, per [ADR 30](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md), that allows +granting arbitrary privileges from one account (the granter) to another account (the grantee). Authorizations must be granted for a particular Msg service method one by one using an implementation of the `Authorization` interface. + +## Contents + +* [Concepts](#concepts) + * [Authorization and Grant](#authorization-and-grant) + * [Built-in Authorizations](#built-in-authorizations) + * [Gas](#gas) +* [State](#state) + * [Grant](#grant) + * [GrantQueue](#grantqueue) +* [Messages](#messages) + * [MsgGrant](#msggrant) + * [MsgRevoke](#msgrevoke) + * [MsgRevokeAll](#msgrevokeall) + * [MsgExec](#msgexec) + * [MsgPruneExpiredGrants](#msgpruneexpiredgrants) +* [Events](#events) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) + +## Concepts + +### Authorization and Grant + +The `x/authz` module defines interfaces and messages grant authorizations to perform actions +on behalf of one account to other accounts. The design is defined in the [ADR 030](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md). + +A *grant* is an allowance to execute a Msg by the grantee on behalf of the granter. +Authorization is an interface that must be implemented by a concrete authorization logic to validate and execute grants. Authorizations are extensible and can be defined for any Msg service method, even if the Msg method is defined outside of the module. See the `SendAuthorization` example in the next section for more details. + +**Note:** The authz module is different from the [auth (authentication)](../modules/auth/) module, which is responsible for specifying the base transaction and account types. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/authorizations.go#L14-L28 +``` + +### Built-in Authorizations + +The Cosmos SDK `x/authz` module comes with following authorization types: + +#### GenericAuthorization + +`GenericAuthorization` implements the `Authorization` interface that gives unrestricted permission to execute the provided Msg on behalf of granter's account. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/proto/cosmos/authz/v1beta1/authz.proto#L14-L22 +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/generic_authorization.go#L18-L34 +``` + +* `msg` stores Msg type URL. + +#### SendAuthorization + +`SendAuthorization` implements the `Authorization` interface for the `cosmos.bank.v1beta1.MsgSend` Msg. + +* It takes a (positive) `SpendLimit` that specifies the maximum amount of tokens the grantee can spend. The `SpendLimit` is updated as the tokens are spent. +* It takes an (optional) `AllowList` that specifies to which addresses a grantee can send token. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/bank/proto/cosmos/bank/v1beta1/authz.proto#L11-L29 +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/bank/types/send_authorization.go#L33-L73 +``` + +* `spend_limit` keeps track of how many coins are left in the authorization. +* `allow_list` specifies an optional list of addresses to whom the grantee can send tokens on behalf of the granter. + +#### StakeAuthorization + +`StakeAuthorization` implements the `Authorization` interface for messages in the [staking module](https://docs.cosmos.network/main/build/modules/staking). It takes an `AuthorizationType` to specify whether you want to authorize delegation, undelegation, redelegation or cancel unbonding delegation, each of which must be authorized separately. It also takes an optional `MaxTokens` that keeps track of a limit to the amount of tokens that can be delegated/undelegated/redelegated. If left empty, the amount is unlimited. Additionally, this Msg takes an `AllowList` or a `DenyList`, enabling you to specify which validators the grantee can or cannot stake with. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/staking/proto/cosmos/staking/v1beta1/authz.proto#L11-L34 +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/staking/types/authz.go#L78-L166 +``` + +### Gas + +To prevent DoS attacks, granting `StakeAuthorization`s with `x/authz` incurs gas. `StakeAuthorization` allows you to authorize another account to delegate, undelegate, or redelegate tokens to validators. The granter can define a list of validators for which they allow or deny delegations. The Cosmos SDK then iterates over these lists and charge 10 gas for each validator included in both lists. + +Since the state maintains a list of granter-grantee pairs with same expiration, we iterate over this list to remove the grant from the list (in case of any revoke of particular `msgType`), charging 20 gas for each iteration. + +## State + +### Grant + +Grants are identified by combining granter address (the address bytes of the granter), grantee address (the address bytes of the grantee) and Authorization type (its type URL). Hence we only allow one grant for the (granter, grantee, Authorization) triple. + +* Grant: `0x01 | granter_address_len (1 byte) | granter_address_bytes | grantee_address_len (1 byte) | grantee_address_bytes | msgType_bytes -> ProtocolBuffer(AuthorizationGrant)` + +The grant object encapsulates an `Authorization` type and an expiration timestamp: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/proto/cosmos/authz/v1beta1/authz.proto#L24-L32 +``` + +### GrantQueue + +We are maintaining a queue for authz pruning. Whenever a grant is created, an item will be added to `GrantQueue` with a key of expiration, granter, grantee. + +In `EndBlock` (which runs for every block) we continuously check and prune the expired grants by forming a prefix key with current blocktime that passed the stored expiration in `GrantQueue`, we iterate through all the matched records from `GrantQueue` and delete maximum of 200 grants from the `GrantQueue` & `Grant`s store for each run. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/keeper/keeper.go#L479-L520 +``` + +* GrantQueue: `0x02 | expiration_bytes | granter_address_len (1 byte) | granter_address_bytes | grantee_address_len (1 byte) | grantee_address_bytes -> ProtocolBuffer(GrantQueueItem)` + +The `expiration_bytes` are the expiration date in UTC with the format `"2006-01-02T15:04:05.000000000"`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/keeper/keys.go#L84-L100 +``` + +The `GrantQueueItem` object contains the list of type urls between granter and grantee that expire at the time indicated in the key. + +## Messages + +In this section we describe the processing of messages for the authz module. + +### MsgGrant + +An authorization grant is created using the `MsgGrant` message. +If there is already a grant for the `(granter, grantee, Authorization)` triple, then the new grant overwrites the previous one. To update or extend an existing grant, a new grant with the same `(granter, grantee, Authorization)` triple should be created. + +An authorization grant for authz `MsgGrant` is not allowed and will return an error. This is for preventing user from accidentally authorizing their entire account to a different account. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/proto/cosmos/authz/v1beta1/tx.proto#L45-L55 +``` + +The message handling should fail if: + +* both granter and grantee have the same address. +* provided `Expiration` time is less than current unix timestamp (but a grant will be created if no `expiration` time is provided since `expiration` is optional). +* provided `Grant.Authorization` is not implemented. +* `Authorization.MsgTypeURL()` is not defined in the router (there is no defined handler in the app router to handle that Msg types). + +### MsgRevoke + +A grant can be removed with the `MsgRevoke` message. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/proto/cosmos/authz/v1beta1/tx.proto#L79-L88 +``` + +The message handling should fail if: + +* both granter and grantee have the same address. +* provided `MsgTypeUrl` is empty. + +NOTE: The `MsgExec` message removes a grant if the grant has expired. + +### MsgRevokeAll + +The `MsgRevokeAll` message revokes all grants issued by the specified granter. This is useful for quickly removing all authorizations granted by a single granter without specifying individual message types or grantees. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/proto/cosmos/authz/v1beta1/tx.proto#L93-L100 +``` + +The message handling should fail if: + +* the `granter` address is not provided or invalid. +* the `granter` does not have any active grants. + +### MsgExec + +When a grantee wants to execute a transaction on behalf of a granter, they must send `MsgExec`. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/authz/proto/cosmos/authz/v1beta1/tx.proto#L60-L72 +``` + +The message handling should fail if: + +* provided `Authorization` is not implemented. +* grantee doesn't have permission to run the transaction. +* if granted authorization is expired. + +### MsgPruneExpiredGrants + +Message that clean up 75 expired grants. A user has no benefit sending this transaction, it is only used by the chain to clean up expired grants. + +## Events + +The authz module emits proto events defined in [the Protobuf reference](https://buf.build/cosmos/cosmos-sdk/docs/main/cosmos.authz.v1beta1#cosmos.authz.v1beta1.EventGrant). + +## Client + +### CLI + +A user can query and interact with the `authz` module using the CLI. + +#### Query + +The `query` commands allow users to query `authz` state. + +```bash +simd query authz --help +``` + +##### grants + +The `grants` command allows users to query grants for a granter-grantee pair. If the message type URL is set, it selects grants only for that message type. + +```bash +simd query authz grants [granter-addr] [grantee-addr] [msg-type-url]? [flags] +``` + +Example: + +```bash +simd query authz grants cosmos1.. cosmos1.. /cosmos.bank.v1beta1.MsgSend +``` + +Example Output: + +```bash +grants: +- authorization: + '@type': /cosmos.bank.v1beta1.SendAuthorization + spend_limit: + - amount: "100" + denom: stake + expiration: "2022-01-01T00:00:00Z" +pagination: null +``` + +#### Transactions + +The `tx` commands allow users to interact with the `authz` module. + +```bash +simd tx authz --help +``` + +##### exec + +The `exec` command allows a grantee to execute a transaction on behalf of granter. + +```bash + simd tx authz exec [tx-json-file] --from [grantee] [flags] +``` + +Example: + +```bash +simd tx authz exec tx.json --from=cosmos1.. +``` + +##### grant + +The `grant` command allows a granter to grant an authorization to a grantee. + +```bash +simd tx authz grant --from [flags] +``` +- The `send` authorization_type refers to the built-in `SendAuthorization` type. The custom flags available are `spend-limit` (required) and `allow-list` (optional) , documented [here](#SendAuthorization) + +Example: + +```bash + simd tx authz grant cosmos1.. send --spend-limit=100stake --allow-list=cosmos1...,cosmos2... --from=cosmos1.. +``` +- The `generic` authorization_type refers to the built-in `GenericAuthorization` type. The custom flag available is `msg-type` ( required) documented [here](#GenericAuthorization). + +> Note: `msg-type` is any valid Cosmos SDK `Msg` type url. + +Example: +```bash + simd tx authz grant cosmos1.. generic --msg-type=/cosmos.bank.v1beta1.MsgSend --from=cosmos1.. +``` +- The `delegate`,`unbond`,`redelegate` authorization_types refer to the built-in `StakeAuthorization` type. The custom flags available are `spend-limit` (optional), `allowed-validators` (optional) and `deny-validators` (optional) documented [here](#StakeAuthorization). +> Note: `allowed-validators` and `deny-validators` cannot both be empty. `spend-limit` represents the `MaxTokens` + +Example: + +```bash +simd tx authz grant cosmos1.. delegate --spend-limit=100stake --allowed-validators=cosmos...,cosmos... --deny-validators=cosmos... --from=cosmos1.. +``` + +##### revoke + +The `revoke` command allows a granter to revoke an authorization from a grantee. + +```bash +simd tx authz revoke [grantee] [msg-type-url] --from=[granter] [flags] +``` + +Example: + +```bash +simd tx authz revoke cosmos1.. /cosmos.bank.v1beta1.MsgSend --from=cosmos1.. +``` + +##### revoke-all + +The `revoke-all` command allows a granter to revoke all authorizations created by the granter. + +```bash +simd tx authz revoke-all --from=[granter] [flags] +``` + +Example: + +```bash +simd tx authz revoke-all --from=cosmos1.. +``` + +### gRPC + +A user can query the `authz` module using gRPC endpoints. + +#### Grants + +The `Grants` endpoint allows users to query grants for a granter-grantee pair. If the message type URL is set, it selects grants only for that message type. + +```bash +cosmos.authz.v1beta1.Query/Grants +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"granter":"cosmos1..","grantee":"cosmos1..","msg_type_url":"/cosmos.bank.v1beta1.MsgSend"}' \ + localhost:9090 \ + cosmos.authz.v1beta1.Query/Grants +``` + +### REST + +A user can query the `authz` module using REST endpoints. + +```bash +/cosmos/authz/v1beta1/grants +``` + +Example: + +```bash +curl "localhost:1317/cosmos/authz/v1beta1/grants?granter=cosmos1..&grantee=cosmos1..&msg_type_url=/cosmos.bank.v1beta1.MsgSend" +``` diff --git a/versioned_docs/version-0.52/build/modules/bank/README.md b/versioned_docs/version-0.52/build/modules/bank/README.md new file mode 100644 index 000000000..f6f77f1f9 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/bank/README.md @@ -0,0 +1,938 @@ +--- +sidebar_position: 1 +--- + +# `x/bank` + +## Abstract + +This document specifies the bank module of the Cosmos SDK. + +The bank module is responsible for handling multi-asset coin transfers between +accounts and tracking special-case pseudo-transfers which must work differently +with particular kinds of accounts (notably delegating/undelegating for legacy vesting +accounts). It exposes several interfaces with varying capabilities for secure +interaction with other modules which must alter user balances. + +In addition, the bank module tracks and provides query support for the total +supply of all assets used in the application. + +## Contents + +* [Supply](#supply) + * [Total Supply](#total-supply) +* [Module Accounts](#module-accounts) + * [Permissions](#permissions) +* [State](#state) +* [Params](#params) +* [Keepers](#keepers) +* [Messages](#messages) +* [Events](#events) + * [Message Events](#message-events) + * [Keeper Events](#keeper-events) +* [Parameters](#parameters) + * [SendEnabled](#sendenabled) + * [DefaultSendEnabled](#defaultsendenabled) +* [Client](#client) + * [CLI](#cli) + * [Query](#query) + * [Transactions](#transactions) +* [gRPC](#grpc) + +## Supply + +The `supply` functionality: + +* passively tracks the total supply of coins within a chain, +* provides a pattern for modules to hold/interact with `Coins`, and +* introduces the invariant check to verify a chain's total supply. + +### Total Supply + +The total `Supply` of the network is equal to the sum of all coins from all +accounts within a chain. The total supply is updated every time a `Coin` is minted +(eg: as part of the inflation mechanism) or burned (eg: due to slashing or if a governance +proposal is vetoed). + +## Module Accounts + +The supply functionality introduces a new type of `auth.Account` which can be used by +modules to allocate tokens and in special cases mint or burn tokens. At a base +level these module accounts are capable of sending/receiving tokens to and from +`auth.Account`s and other module accounts. This design replaces previous +alternative designs where, to hold tokens, modules would burn the incoming +tokens from the sender account, and then track those tokens internally. Later, +in order to send tokens, the module would need to effectively mint tokens +within a destination account. The new design removes duplicate logic between +modules to perform this accounting. + +The `ModuleAccount` interface is defined as follows: + +```go +type ModuleAccount interface { + auth.Account // same methods as the Account interface + + GetName() string // name of the module; used to obtain the address + GetPermissions() []string // permissions of module account + HasPermission(string) bool +} +``` + +> **WARNING!** +> Any module or message handler that allows either direct or indirect sending of funds must explicitly guarantee those funds cannot be sent to module accounts (unless allowed). + +The supply `Keeper` interface also introduces new wrapper functions for the auth `Keeper` +and the bank `SendKeeper` in order to be able to: + +* Get `ModuleAccount`s by providing its `Name`. +* Send coins from and to other `ModuleAccount`s by passing only the `Name` or standard `Account`s + (`BaseAccount` or legacy `VestingAccount`). +* `Mint` or `Burn` coins for a `ModuleAccount` (restricted to its permissions). + +### Permissions + +Each `ModuleAccount` has a different set of permissions that provide different +object capabilities to perform certain actions. Permissions need to be +registered upon the creation of the supply `Keeper` so that every time a +`ModuleAccount` calls the allowed functions, the `Keeper` can lookup the +permissions to that specific account and perform or not perform the action. + +The available permissions are: + +* `Minter`: allows for a module to mint a specific amount of coins. +* `Burner`: allows for a module to burn a specific amount of coins. +* `Staking`: allows for a module to delegate and undelegate a specific amount of coins. + +## State + +The `x/bank` module keeps state of the following primary objects: + +1. Account balances +2. Denomination metadata +3. The total supply of all balances +4. Information on which denominations are allowed to be sent. + +In addition, the `x/bank` module keeps the following indexes to manage the +aforementioned state: + +* Supply Index: `0x0 | byte(denom) -> byte(amount)` +* Denom Metadata Index: `0x1 | byte(denom) -> ProtocolBuffer(Metadata)` +* Balances Index: `0x2 | byte(address length) | []byte(address) | []byte(balance.Denom) -> ProtocolBuffer(balance)` +* Reverse Denomination to Address Index: `0x03 | byte(denom) | 0x00 | []byte(address) -> 0` +* Send enabled Denoms: `0x4 | string -> bool` + +## Params + +The bank module stores its params in state with the prefix of `0x05`, +it can be updated with governance or the address with authority. + +* Params: `0x05 | ProtocolBuffer(Params)` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/bank/proto/cosmos/bank/v1beta1/bank.proto#L12-L23 +``` + +## Keepers + +The bank module provides these exported keeper interfaces that can be +passed to other modules that read or update account balances. Modules +should use the least-permissive interface that provides the functionality they +require. + +Best practices dictate careful review of `bank` module code to ensure that +permissions are limited in the way that you expected. + +### Denied Addresses + +The `x/bank` module accepts a map of addresses (`blockedAddrs`) that are considered blocklisted +from directly and explicitly receiving funds through means such as `MsgSend` and +`MsgMultiSend` and direct API calls like `SendCoinsFromModuleToAccount`. + +Typically, these addresses are module accounts. If these addresses receive funds +outside the expected rules of the state machine, invariants are likely to be +broken and could result in a halted network. + +By providing the `x/bank` module with a blocklisted set of addresses, an error occurs for the operation if a user or client attempts to directly or indirectly send funds to a blocklisted account, for example, by using [IBC](https://ibc.cosmos.network). + +### Common Types + +#### Input + +An input of a multi-send transaction + +```protobuf +// Input models transaction input. +message Input { + string address = 1; + repeated cosmos.base.v1beta1.Coin coins = 2; +} +``` + +#### Output + +An output of a multi-send transaction. + +```protobuf +// Output models transaction outputs. +message Output { + string address = 1; + repeated cosmos.base.v1beta1.Coin coins = 2; +} +``` + +### BaseKeeper + +The base keeper provides full-permission access: the ability to arbitrary modify any account's balance and mint or burn coins. + +Restricted permission to mint per module could be achieved by using baseKeeper with `WithMintCoinsRestriction` to give specific restrictions to mint (e.g. only minting certain denom). + +```go +// Keeper defines a module interface that facilitates the transfer of coins +// between accounts. +type Keeper interface { + SendKeeper + WithMintCoinsRestriction(MintingRestrictionFn) BaseKeeper + + InitGenesis(context.Context, *types.GenesisState) error + ExportGenesis(context.Context) (*types.GenesisState, error) + + GetSupply(ctx context.Context, denom string) sdk.Coin + HasSupply(ctx context.Context, denom string) bool + GetPaginatedTotalSupply(ctx context.Context, pagination *query.PageRequest) (sdk.Coins, *query.PageResponse, error) + IterateTotalSupply(ctx context.Context, cb func(sdk.Coin) bool) + GetDenomMetaData(ctx context.Context, denom string) (types.Metadata, bool) + HasDenomMetaData(ctx context.Context, denom string) bool + SetDenomMetaData(ctx context.Context, denomMetaData types.Metadata) + IterateAllDenomMetaData(ctx context.Context, cb func(types.Metadata) bool) + + SendCoinsFromModuleToAccount(ctx context.Context, senderModule string, recipientAddr sdk.AccAddress, amt sdk.Coins) error + SendCoinsFromModuleToModule(ctx context.Context, senderModule, recipientModule string, amt sdk.Coins) error + SendCoinsFromAccountToModule(ctx context.Context, senderAddr sdk.AccAddress, recipientModule string, amt sdk.Coins) error + DelegateCoinsFromAccountToModule(ctx context.Context, senderAddr sdk.AccAddress, recipientModule string, amt sdk.Coins) error + UndelegateCoinsFromModuleToAccount(ctx context.Context, senderModule string, recipientAddr sdk.AccAddress, amt sdk.Coins) error + MintCoins(ctx context.Context, moduleName string, amt sdk.Coins) error + BurnCoins(ctx context.Context, address []byte, amt sdk.Coins) error + + DelegateCoins(ctx context.Context, delegatorAddr, moduleAccAddr sdk.AccAddress, amt sdk.Coins) error + UndelegateCoins(ctx context.Context, moduleAccAddr, delegatorAddr sdk.AccAddress, amt sdk.Coins) error + + types.QueryServer +} +``` + +### SendKeeper + +The send keeper provides access to account balances and the ability to transfer coins between +accounts. The send keeper does not alter the total supply (mint or burn coins). + +```go +// SendKeeper defines a module interface that facilitates the transfer of coins +// between accounts without the possibility of creating coins. +type SendKeeper interface { + ViewKeeper + + AppendSendRestriction(restriction SendRestrictionFn) + PrependSendRestriction(restriction SendRestrictionFn) + ClearSendRestriction() + + InputOutputCoins(ctx context.Context, input types.Input, outputs []types.Output) error + SendCoins(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) error + + GetParams(ctx context.Context) types.Params + SetParams(ctx context.Context, params types.Params) error + + IsSendEnabledDenom(ctx context.Context, denom string) bool + SetSendEnabled(ctx context.Context, denom string, value bool) + SetAllSendEnabled(ctx context.Context, sendEnableds []*types.SendEnabled) + DeleteSendEnabled(ctx context.Context, denom string) + IterateSendEnabledEntries(ctx context.Context, cb func(denom string, sendEnabled bool) (stop bool)) + GetAllSendEnabledEntries(ctx context.Context) []types.SendEnabled + + IsSendEnabledCoin(ctx context.Context, coin sdk.Coin) bool + IsSendEnabledCoins(ctx context.Context, coins ...sdk.Coin) error + + BlockedAddr(addr sdk.AccAddress) bool +} +``` + +#### Send Restrictions + +The `SendKeeper` applies a `SendRestrictionFn` before each transfer of funds. + +```golang +// A SendRestrictionFn can restrict sends and/or provide a new receiver address. +type SendRestrictionFn func(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) (newToAddr sdk.AccAddress, err error) +``` + +After the `SendKeeper` (or `BaseKeeper`) has been created, send restrictions can be added to it using the `AppendSendRestriction` or `PrependSendRestriction` functions. +Both functions compose the provided restriction with any previously provided restrictions. +`AppendSendRestriction` adds the provided restriction to be run after any previously provided send restrictions. +`PrependSendRestriction` adds the restriction to be run before any previously provided send restrictions. +The composition will short-circuit when an error is encountered. I.e. if the first one returns an error, the second is not run. +Send restrictions can also be cleared by using `ClearSendRestriction`. + +During `SendCoins`, the send restriction is applied before coins are removed from the `from_address` and adding them to the `to_address`. +During `InputOutputCoins`, the send restriction is applied before the input coins are removed, once for each output before the funds are added. + +Send Restrictions are not placed on `ModuleToAccount` or `ModuleToModule` transfers. This is done due to modules needing to move funds to user accounts and other module accounts. This is a design decision to allow for more flexibility in the state machine. The state machine should be able to move funds between module accounts and user accounts without restrictions. + +Secondly this limitation would limit the usage of the state machine even for itself. users would not be able to receive rewards, not be able to move funds between module accounts. In the case that a user sends funds from a user account to the community pool and then a governance proposal is used to get those tokens into the users account this would fall under the discretion of the app chain developer to what they would like to do here. We can not make strong assumptions here. + +Thirdly, this issue could lead into a chain halt if a token is disabled and the token is moved in the begin/endblock. This is the last reason we see the current change as they are more damaging then beneficial for users. + +A send restriction function should make use of a custom value in the context to allow bypassing that specific restriction. +For example, in your module's keeper package, you'd define the send restriction function: + +```golang +var _ banktypes.SendRestrictionFn = Keeper{}.SendRestrictionFn + +func (k Keeper) SendRestrictionFn(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) (sdk.AccAddress, error) { + // Bypass if the context says to. + if mymodule.HasBypass(ctx) { + return toAddr, nil + } + + // Your custom send restriction logic goes here. + return nil, errors.New("not implemented") +} +``` + +The bank keeper should be provided to your keeper's constructor so the send restriction can be added to it: + +```golang +func NewKeeper(cdc codec.BinaryCodec, storeKey storetypes.StoreKey, bankKeeper mymodule.BankKeeper) Keeper { + rv := Keeper{/*...*/} + bankKeeper.AppendSendRestriction(rv.SendRestrictionFn) + return rv +} +``` + +Then, in the `mymodule` package, define the context helpers: + +```golang +const bypassKey = "bypass-mymodule-restriction" + +// WithBypass returns a new context that will cause the mymodule bank send restriction to be skipped. +func WithBypass(ctx context.Context) context.Context { + return sdk.UnwrapSDKContext(ctx).WithValue(bypassKey, true) +} + +// WithoutBypass returns a new context that will cause the mymodule bank send restriction to not be skipped. +func WithoutBypass(ctx context.Context) context.Context { + return sdk.UnwrapSDKContext(ctx).WithValue(bypassKey, false) +} + +// HasBypass checks the context to see if the mymodule bank send restriction should be skipped. +func HasBypass(ctx context.Context) bool { + bypassValue := ctx.Value(bypassKey) + if bypassValue == nil { + return false + } + bypass, isBool := bypassValue.(bool) + return isBool && bypass +} +``` + +Now, anywhere where you want to use `SendCoins` or `InputOutputCoins` but you don't want your send restriction applied +you just need to apply custom value in the context: + +```golang +func (k Keeper) DoThing(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) error { + return k.bankKeeper.SendCoins(mymodule.WithBypass(ctx), fromAddr, toAddr, amt) +} +``` + +### ViewKeeper + +The view keeper provides read-only access to account balances. The view keeper does not have balance alteration functionality. All balance lookups are `O(1)`. + +```go +// ViewKeeper defines a module interface that facilitates read only access to +// account balances. +type ViewKeeper interface { + ValidateBalance(ctx context.Context, addr sdk.AccAddress) error + HasBalance(ctx context.Context, addr sdk.AccAddress, amt sdk.Coin) bool + + GetAllBalances(ctx context.Context, addr sdk.AccAddress) sdk.Coins + GetAccountsBalances(ctx context.Context) []types.Balance + GetBalance(ctx context.Context, addr sdk.AccAddress, denom string) sdk.Coin + LockedCoins(ctx context.Context, addr sdk.AccAddress) sdk.Coins + SpendableCoins(ctx context.Context, addr sdk.AccAddress) sdk.Coins + SpendableCoin(ctx context.Context, addr sdk.AccAddress, denom string) sdk.Coin + + IterateAccountBalances(ctx context.Context, addr sdk.AccAddress, cb func(coin sdk.Coin) (stop bool)) + IterateAllBalances(ctx context.Context, cb func(address sdk.AccAddress, coin sdk.Coin) (stop bool)) +} +``` + +## Messages + +### MsgSend + +Send coins from one address to another. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/bank/proto/cosmos/bank/v1beta1/tx.proto#L44-L59 +``` + +The message will fail under the following conditions: + +* The coins do not have sending enabled +* The `to_address` is restricted + +### MsgMultiSend + +Send coins from one sender and to a series of different address. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/bank/proto/cosmos/bank/v1beta1/tx.proto#L65-L75 +``` + +The message will fail under the following conditions: + +* Any of the coins do not have sending enabled +* Any of the `to_address` are restricted +* Any of the coins are locked +* The inputs and outputs do not correctly correspond to one another (eg: total_in not equal to total_out) + +### MsgUpdateParams + +The `bank` module params can be updated through `MsgUpdateParams`, which can be done using governance proposal. The signer will always be the `gov` module account address. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/bank/proto/cosmos/bank/v1beta1/tx.proto#L81-L93 +``` + +The message handling can fail if: + +* signer is not the gov module account address. + +### MsgSetSendEnabled + +Used with the x/gov module to create or edit SendEnabled entries. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/bank/proto/cosmos/bank/v1beta1/tx.proto#L106-L122 +``` + +The message will fail under the following conditions: + +* The authority is not a decodable address +* The authority is not x/gov module's address +* There are multiple SendEnabled entries with the same Denom +* One or more SendEnabled entries has an invalid Denom + +### MsgBurn + +Used to burn coins from an account. The coins are removed from the account and the total supply is reduced. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/bank/proto/cosmos/bank/v1beta1/tx.proto#L130-L139 +``` + +This message will fail under the following conditions: + +* The `from_address` is not a decodable address +* The coins are not spendable +* The coins are not positive +* The coins are not valid + +## Events + +The bank module emits the following events: + +### Message Events + +#### MsgSend + +| Type | Attribute Key | Attribute Value | +| -------- | ------------- | ------------------ | +| transfer | recipient | {recipientAddress} | +| transfer | sender | {senderAddress} | +| transfer | amount | {amount} | +| message | module | bank | +| message | action | send | + +#### MsgMultiSend + +| Type | Attribute Key | Attribute Value | +| -------- | ------------- | ------------------ | +| transfer | recipient | {recipientAddress} | +| transfer | sender | {senderAddress} | +| transfer | amount | {amount} | +| message | module | bank | +| message | action | multisend | + +### Keeper Events + +In addition to message events, the bank keeper will produce events when the following methods are called (or any method which ends up calling them) + +#### MintCoins + +```json +{ + "type": "coinbase", + "attributes": [ + { + "key": "minter", + "value": "{{sdk.AccAddress of the module minting coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being minted}}", + "index": true + } + ] +} +``` + +```json +{ + "type": "coin_received", + "attributes": [ + { + "key": "receiver", + "value": "{{sdk.AccAddress of the module minting coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being received}}", + "index": true + } + ] +} +``` + +#### BurnCoins + +```json +{ + "type": "burn", + "attributes": [ + { + "key": "burner", + "value": "{{sdk.AccAddress of the module burning coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being burned}}", + "index": true + } + ] +} +``` + +```json +{ + "type": "coin_spent", + "attributes": [ + { + "key": "spender", + "value": "{{sdk.AccAddress of the module burning coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being burned}}", + "index": true + } + ] +} +``` + +#### addCoins + +```json +{ + "type": "coin_received", + "attributes": [ + { + "key": "receiver", + "value": "{{sdk.AccAddress of the address beneficiary of the coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being received}}", + "index": true + } + ] +} +``` + +#### subUnlockedCoins/DelegateCoins + +```json +{ + "type": "coin_spent", + "attributes": [ + { + "key": "spender", + "value": "{{sdk.AccAddress of the address which is spending coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being spent}}", + "index": true + } + ] +} +``` + +## Parameters + +The bank module contains the following parameters + +### SendEnabled + +SendEnabled is depreacted and only kept for backward compatibility. For genesis, use the newly added send_enabled field in the genesis object. Storage, lookup, and manipulation of this information is now in the keeper. + +### DefaultSendEnabled + +The default send enabled value controls send transfer capability for all +coin denominations unless specifically included in the array of `SendEnabled` +parameters. + +## Client + +### CLI + +A user can query and interact with the `bank` module using the CLI. + +#### Query + +The `query` commands allow users to query `bank` state. + +```shell +simd query bank --help +``` + +##### balance + +The `balance` command allows users to query account balance by specific denom. + +```shell +simd query bank balance [address] [denom] [flags] +``` + +Example: + +```shell +simd query bank balance cosmos1.. stake +``` + +##### balances + +The `balances` command allows users to query account balances by address. + +```shell +simd query bank balances [address] [flags] +``` + +Example: + +```shell +simd query bank balances cosmos1.. +``` + +##### spendable balances + +The `spendable-balances` command allows users to query account spendable balances by address. + +```shell +simd query spendable-balances [address] [flags] +``` + +Example: + +```shell +simd query bank spendable-balances cosmos1.. +``` + +##### spendable balance by denom + +The `spendable-balance` command allows users to query account spendable balance by address for a specific denom. + +```shell +simd query spendable-balance [address] [denom] [flags] +``` + +Example: + +```shell +simd query bank spendable-balance cosmos1.. stake +``` + +##### denom-metadata + +The `denom-metadata` command allows users to query metadata for coin denominations. + +```shell +simd query bank denom-metadata [denom] +``` + +Example: + +```shell +simd query bank denom-metadata stake +``` + +##### denoms-metadata + +The `denoms-metadata` command allows users to query metadata for all coin denominations. + +```shell +simd query bank denoms-metadata [flags] +``` + +Example: + +```shell +simd query bank denoms-metadata +``` + +##### total supply + +The `total-supply` (or `total` for short) command allows users to query the total supply of coins. + +```shell +simd query bank total [flags] +``` + +Example: + +```shell +simd query bank total --denom stake +``` + +##### total supply of + +The `total-supply-of` command allows users to query the total supply for a specific coin denominations. + +```shell +simd query bank total-supply-of [denom] +``` + +Example: + +```shell +simd query bank total-supply-of stake +``` + +##### send-enabled + +The `send-enabled` command allows users to query for all or some SendEnabled entries. + +```shell +simd query bank send-enabled [denom1 ...] [flags] +``` + +Example: + +```shell +simd query bank send-enabled +``` + +##### params + +The `params` command allows users to query for the current bank parameters. + +```shell +simd query bank params [flags] +``` + +##### denom owners + +The `denom-owners` command allows users to query for all account addresses that own a particular token denomination. + +```shell +simd query bank denom-owners [denom] [flags] +``` + +Example: + +```shell +simd query bank denom-owners stake +``` + +#### Transactions + +The `tx` commands allow users to interact with the `bank` module. + +```shell +simd tx bank --help +``` + +##### send + +The `send` command allows users to send funds from one account to another. + +```shell +simd tx bank send [from_key_or_address] [to_address] [amount] [flags] +``` + +Example: + +```shell +simd tx bank send cosmos1.. cosmos1.. 100stake +``` + +## gRPC + +A user can query the `bank` module using gRPC endpoints. + +### Balance + +The `Balance` endpoint allows users to query account balance by address for a given denomination. + +```shell +cosmos.bank.v1beta1.Query/Balance +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"address":"cosmos1..","denom":"stake"}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/Balance +``` + +### AllBalances + +The `AllBalances` endpoint allows users to query account balance by address for all denominations. + +```shell +cosmos.bank.v1beta1.Query/AllBalances +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/AllBalances +``` + +### DenomMetadata + +The `DenomMetadata` endpoint allows users to query metadata for a single coin denomination. + +```shell +cosmos.bank.v1beta1.Query/DenomMetadata +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"denom":"stake"}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/DenomMetadata +``` + +### DenomsMetadata + +The `DenomsMetadata` endpoint allows users to query metadata for all coin denominations. + +```shell +cosmos.bank.v1beta1.Query/DenomsMetadata +``` + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/DenomsMetadata +``` + +### DenomOwners + +The `DenomOwners` endpoint allows users to query metadata for a single coin denomination. + +```shell +cosmos.bank.v1beta1.Query/DenomOwners +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"denom":"stake"}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/DenomOwners +``` + +### TotalSupply + +The `TotalSupply` endpoint allows users to query the total supply of all coins. + +```shell +cosmos.bank.v1beta1.Query/TotalSupply +``` + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/TotalSupply +``` + +### SupplyOf + +The `SupplyOf` endpoint allows users to query the total supply of a single coin. + +```shell +cosmos.bank.v1beta1.Query/SupplyOf +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"denom":"stake"}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/SupplyOf +``` + +### Params + +The `Params` endpoint allows users to query the parameters of the `bank` module. + +```shell +cosmos.bank.v1beta1.Query/Params +``` + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/Params +``` + +### SendEnabled + +The `SendEnabled` endpoints allows users to query the SendEnabled entries of the `bank` module. + +Any denominations NOT returned, use the `Params.DefaultSendEnabled` value. + +```shell +cosmos.bank.v1beta1.Query/SendEnabled +``` + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/SendEnabled +``` + diff --git a/versioned_docs/version-0.52/build/modules/circuit/README.md b/versioned_docs/version-0.52/build/modules/circuit/README.md new file mode 100644 index 000000000..0a8d3ee95 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/circuit/README.md @@ -0,0 +1,203 @@ +# `x/circuit` + +## Concepts + +Circuit Breaker is a module that is meant to avoid a chain needing to halt/shut down in the presence of a vulnerability, instead the module will allow specific messages or all messages to be disabled. When operating a chain, if it is app specific then a halt of the chain is less detrimental, but if there are applications built on top of the chain then halting is expensive due to the disturbance to applications. + +## How it works + +Circuit Breaker works with the idea that an address or set of addresses have the right to block messages from being executed and/or included in the mempool. Any address with a permission is able to reset the circuit breaker for the message. + +The transactions are checked and can be rejected at two points: + +* In `CircuitBreakerDecorator` [ante handler](https://docs.cosmos.network/main/learn/advanced/baseapp#antehandler): + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/x/circuit/v0.1.0/x/circuit/ante/circuit.go#L27-L41 +``` + +* With a [message router check](https://docs.cosmos.network/main/learn/advanced/baseapp#msg-service-router): + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/baseapp/msg_service_router.go#L123-L133 +``` + +:::note +The `CircuitBreakerDecorator` works for most use cases, but [does not check the inner messages of a transaction](https://docs.cosmos.network/main/learn/beginner/tx-lifecycle#antehandler). This some transactions (such as `x/authz` transactions or some `x/gov` transactions) may pass the ante handler. **This does not affect the circuit breaker** as the message router check will still fail the transaction. +This tradeoff is to avoid introducing more dependencies in the `x/circuit` module. Chains can re-define the `CircuitBreakerDecorator` to check for inner messages if they wish to do so. +::: + +## State + +### Accounts + +* AccountPermissions `0x1 | account_address -> ProtocolBuffer(CircuitBreakerPermissions)` + +```go +type level int32 + +const ( + // LEVEL_NONE_UNSPECIFIED indicates that the account will have no circuit + // breaker permissions. + LEVEL_NONE_UNSPECIFIED = iota + // LEVEL_SOME_MSGS indicates that the account will have permission to + // trip or reset the circuit breaker for some Msg type URLs. If this level + // is chosen, a non-empty list of Msg type URLs must be provided in + // limit_type_urls. + LEVEL_SOME_MSGS + // LEVEL_ALL_MSGS indicates that the account can trip or reset the circuit + // breaker for Msg's of all type URLs. + LEVEL_ALL_MSGS + // LEVEL_SUPER_ADMIN indicates that the account can take all circuit breaker + // actions and can grant permissions to other accounts. + LEVEL_SUPER_ADMIN +) + +type Access struct { + level int32 + msgs []string // if full permission, msgs can be empty +} +``` + +### Disable List + +List of type urls that are disabled. + +* DisableList `0x2 | msg_type_url -> []byte{}` + +## State Transitions + +### Authorize + +Authorize, is called by the module authority (default governance module account) or any account with `LEVEL_SUPER_ADMIN` to give permission to disable/enable messages to another account. There are three levels of permissions that can be granted. `LEVEL_SOME_MSGS` limits the number of messages that can be disabled. `LEVEL_ALL_MSGS` permits all messages to be disabled. `LEVEL_SUPER_ADMIN` allows an account to take all circuit breaker actions including authorizing and deauthorizing other accounts. + +```protobuf + // AuthorizeCircuitBreaker allows a super-admin to grant (or revoke) another + // account's circuit breaker permissions. + rpc AuthorizeCircuitBreaker(MsgAuthorizeCircuitBreaker) returns (MsgAuthorizeCircuitBreakerResponse); +``` + +### Trip + +Trip, is called by an authorized account to disable message execution for a specific msgURL. If empty, all the msgs will be disabled. + +```protobuf + // TripCircuitBreaker pauses processing of Msg's in the state machine. + rpc TripCircuitBreaker(MsgTripCircuitBreaker) returns (MsgTripCircuitBreakerResponse); +``` + +### Reset + +Reset is called by an authorized account to enable execution for a specific msgURL of previously disabled message. If empty, all the disabled messages will be enabled. + +```protobuf + // ResetCircuitBreaker resumes processing of Msg's in the state machine that + // have been paused using TripCircuitBreaker. + rpc ResetCircuitBreaker(MsgResetCircuitBreaker) returns (MsgResetCircuitBreakerResponse); +``` + +## Messages + +### MsgAuthorizeCircuitBreaker + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/circuit/proto/cosmos/circuit/v1/tx.proto#L25-L40 +``` + +This message is expected to fail if: + +* the granter is not an account with permission level `LEVEL_SUPER_ADMIN` or the module authority + +### MsgTripCircuitBreaker + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/circuit/proto/cosmos/circuit/v1/tx.proto#L47-L60 +``` + +This message is expected to fail if: + +* if the signer does not have a permission level with the ability to disable the specified type url message + +### MsgResetCircuitBreaker + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/circuit/proto/cosmos/circuit/v1/tx.proto#L67-L78 +``` + +This message is expected to fail if: + +* if the type url is not disabled + +## Events + +The circuit module emits the following events: + +### Message Events + +#### MsgAuthorizeCircuitBreaker + +| Type | Attribute Key | Attribute Value | +|---------|---------------|---------------------------| +| string | granter | {granterAddress} | +| string | grantee | {granteeAddress} | +| string | permission | {granteePermissions} | +| message | module | circuit | +| message | action | authorize_circuit_breaker | + +#### MsgTripCircuitBreaker + +| Type | Attribute Key | Attribute Value | +|----------|---------------|--------------------| +| string | authority | {authorityAddress} | +| []string | msg_urls | []string{msg_urls} | +| message | module | circuit | +| message | action | trip_circuit_breaker | + +#### ResetCircuitBreaker + +| Type | Attribute Key | Attribute Value | +|----------|---------------|--------------------| +| string | authority | {authorityAddress} | +| []string | msg_urls | []string{msg_urls} | +| message | module | circuit | +| message | action | reset_circuit_breaker | + + +## Keys + +* `AccountPermissionPrefix` - `0x01` +* `DisableListPrefix` - `0x02` + +## Client + +### CLI + +`x/circuit` module client provides the following CLI commands: + +```shell +$ tx circuit +Transactions commands for the circuit module + +Usage: + simd tx circuit [flags] + simd tx circuit [command] + +Available Commands: + authorize Authorize an account to trip the circuit breaker. + disable Disable a message from being executed + reset Enable a message to be executed +``` + +```shell +$ query circuit +Querying commands for the circuit module + +Usage: + simd query circuit [flags] + simd query circuit [command] + +Available Commands: + account Query a specific account's permissions + accounts Query all account permissions + disabled-list Query a list of all disabled message types +``` diff --git a/versioned_docs/version-0.52/build/modules/consensus/README.md b/versioned_docs/version-0.52/build/modules/consensus/README.md new file mode 100644 index 000000000..83add3f0b --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/consensus/README.md @@ -0,0 +1,87 @@ +--- +sidebar_position: 1 +--- + +# `x/consensus` + +## Abstract + +Functionality to modify CometBFT's ABCI consensus params. + +## Contents + +* [Abstract](#abstract) +* [Contents](#contents) +* [State](#state) +* [Params](#params) +* [Keeper](#keeper) +* [Messages](#messages) + * [UpdateParams](#updateparams) +* [Events](#events) + +## State + +The `x/consensus` module keeps state of the consensus params from CometBFT. + +## Params + +The consensus module stores its params in state with the prefix of `0x05`, +it can be updated with governance or the address with authority. + +* Params: `0x05 | ProtocolBuffer(cometbft.ConsensusParams)` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/consensus/proto/cosmos/consensus/v1/query.proto#L21-L27 +``` + +```protobuf reference +https://github.com/cometbft/cometbft/blob/v0.34.35/proto/tendermint/types/params.proto#L11-L18 +``` + +## Keeper + +The Keeper of the `x/consensus` module provides the following functions: + +* `Params`: Retrieves the current consensus parameters. + +* `UpdateParams`: Updates the consensus parameters. Only the authority can perform this operation. + +* `BlockParams`: Returns the maximum gas and bytes allowed in a block. + +* `ValidatorPubKeyTypes`: Provides the list of public key types allowed for validators. + +* `EvidenceParams`: Returns the evidence parameters, including maximum age and bytes. + +* `AppVersion`: Returns the current application version. + + +Note: It is recommended to use the `x/consensus` module keeper to get consensus params instead of accessing them through the context. + + +## Messages + +### UpdateParams + +Update consensus params. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/consensus/proto/cosmos/consensus/v1/tx.proto#L24-L44 +``` + +The message will fail under the following conditions: + +* The signer is not the set authority +* Not all values are set + +## Events + +The consensus module emits the following events: + +### Message Events + +#### MsgUpdateParams + +| Type | Attribute Key | Attribute Value | +|--------|---------------|---------------------| +| string | authority | msg.Signer | +| string | parameters | consensus Parameters | diff --git a/versioned_docs/version-0.52/build/modules/distribution/README.md b/versioned_docs/version-0.52/build/modules/distribution/README.md new file mode 100644 index 000000000..8cd03dc5e --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/distribution/README.md @@ -0,0 +1,948 @@ +--- +sidebar_position: 1 +--- + +# `x/distribution` + +## Overview + +This _simple_ distribution mechanism describes a functional way to passively +distribute rewards between validators and delegators. Note that this mechanism does +not distribute funds in as precisely as active reward distribution mechanisms and +will therefore be upgraded in the future. + +The mechanism operates as follows. Collected rewards are pooled globally and +divided out passively to validators and delegators. Each validator has the +opportunity to charge commission to the delegators on the rewards collected on +behalf of the delegators. Fees are collected directly into a global reward pool +and validator proposer-reward pool. Due to the nature of passive accounting, +whenever changes to parameters which affect the rate of reward distribution +occurs, withdrawal of rewards must also occur. + +* Whenever withdrawing, one must withdraw the maximum amount they are entitled + to, leaving nothing in the pool. +* Whenever bonding, unbonding, or re-delegating tokens to an existing account, a + full withdrawal of the rewards must occur (as the rules for lazy accounting + change). +* Whenever a validator chooses to change the commission on rewards, all accumulated + commission rewards must be simultaneously withdrawn. + +The above scenarios are covered in `hooks.md`. + +The distribution mechanism outlined herein is used to lazily distribute the +following rewards between validators and associated delegators: + +* multi-token fees to be socially distributed +* inflated staked asset provisions +* validator commission on all rewards earned by their delegators stake + +Fees are pooled within a global pool. The mechanisms used allow for validators +and delegators to independently and lazily withdraw their rewards. + +## Shortcomings + +As a part of the lazy computations, each delegator holds an accumulation term +specific to each validator which is used to estimate what their approximate +fair portion of tokens held in the fee pool is owed to them. + +```text +entitlement = delegator-accumulation / all-delegators-accumulation +``` + +Under the circumstance that there was constant and equal flow of incoming +reward tokens every block, this distribution mechanism would be equal to the +active distribution (distribute individually to all delegators each block). +However, this is unrealistic so deviations from the active distribution will +occur based on fluctuations of incoming reward tokens as well as timing of +reward withdrawal by other delegators. + +If you happen to know that incoming rewards are about to significantly increase, +you are incentivized to not withdraw until after this event, increasing the +worth of your existing _accum_. See [#2764](https://github.com/cosmos/cosmos-sdk/issues/2764) +for further details. + +## Effect on Staking + +Charging commission on Atom provisions while also allowing for Atom-provisions +to be auto-bonded (distributed directly to the validators bonded stake) is +problematic within BPoS. Fundamentally, these two mechanisms are mutually +exclusive. If both commission and auto-bonding mechanisms are simultaneously +applied to the staking-token then the distribution of staking-tokens between +any validator and its delegators will change with each block. This then +necessitates a calculation for each delegation records for each block - +which is considered computationally expensive. + +In conclusion, we can only have Atom commission and unbonded atoms +provisions or bonded atom provisions with no Atom commission, and we elect to +implement the former. Stakeholders wishing to rebond their provisions, may elect +to set up a script to periodically withdraw and rebond rewards. + +## Contents + +* [Concepts](#concepts) +* [State](#state) + * [Validator Distribution](#validator-distribution) + * [Delegation Distribution](#delegation-distribution) + * [Params](#params) +* [Begin Block](#begin-block) +* [Messages](#messages) +* [Hooks](#hooks) +* [Events](#events) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + +## Concepts + +In Proof of Stake (PoS) blockchains, rewards gained from transaction fees are paid to validators. The fee distribution module fairly distributes the rewards to the validators' constituent delegators. + +Rewards are calculated per period. The period is updated each time a validator's delegation changes, for example, when the validator receives a new delegation. +The rewards for a single validator can then be calculated by taking the total rewards for the period before the delegation started, minus the current total rewards. +To learn more, see the [F1 Fee Distribution paper](https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/docs/spec/fee_distribution/f1_fee_distr.pdf). + +The commission to the validator is paid when the validator is removed or when the validator requests a withdrawal. +The commission is calculated and incremented at every `BeginBlock` operation to update accumulated fee amounts. + +The rewards to a delegator are distributed when the delegation is changed or removed, or a withdrawal is requested. +Before rewards are distributed, all slashes to the validator that occurred during the current delegation are applied. + +### Reference Counting in F1 Fee Distribution + +In F1 fee distribution, the rewards a delegator receives are calculated when their delegation is withdrawn. This calculation must read the terms of the summation of rewards divided by the share of tokens from the period which they ended when they delegated, and the final period that was created for the withdrawal. + +Additionally, as slashes change the amount of tokens a delegation will have (but we calculate this lazily, +only when a delegator un-delegates), we must calculate rewards in separate periods before / after any slashes +which occurred in between when a delegator delegated and when they withdrew their rewards. Thus slashes, like +delegations, reference the period which was ended by the slash event. + +All stored historical rewards records for periods which are no longer referenced by any delegations +or any slashes can thus be safely removed, as they will never be read (future delegations and future +slashes will always reference future periods). This is implemented by tracking a `ReferenceCount` +along with each historical reward storage entry. Each time a new object (delegation or slash) +is created which might need to reference the historical record, the reference count is incremented. +Each time one object which previously needed to reference the historical record is deleted, the reference +count is decremented. If the reference count hits zero, the historical record is deleted. + +## State + +### FeePool + +The `FeePool` is used to store decimal rewards to allow for fractions of coins to be received from operations like inflation. + +Once those rewards are big enough, they are sent as `sdk.Coins` to the community pool. + +* FeePool: `0x00 -> ProtocolBuffer(FeePool)` + +```go +// coins with decimal +type DecCoins []DecCoin + +type DecCoin struct { + Amount math.LegacyDec + Denom string +} +``` + +### Validator Distribution + +Validator distribution information for the relevant validator is updated each time: + +1. delegation amount to a validator is updated, +2. any delegator withdraws from a validator, or +3. the validator withdraws its commission. + +* ValidatorDistInfo: `0x02 | ValOperatorAddrLen (1 byte) | ValOperatorAddr -> ProtocolBuffer(validatorDistribution)` + +```go +type ValidatorDistInfo struct { + OperatorAddress sdk.AccAddress + SelfBondRewards sdkmath.DecCoins + ValidatorCommission types.ValidatorAccumulatedCommission +} +``` + +### Delegation Distribution + +Each delegation distribution only needs to record the height at which it last +withdrew fees. Because a delegation must withdraw fees each time it's +properties change (aka bonded tokens etc.) its properties will remain constant +and the delegator's _accumulation_ factor can be calculated passively knowing +only the height of the last withdrawal and its current properties. + +* DelegationDistInfo: `0x02 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValOperatorAddrLen (1 byte) | ValOperatorAddr -> ProtocolBuffer(delegatorDist)` + +```go +type DelegationDistInfo struct { + WithdrawalHeight int64 // last time this delegation withdrew rewards +} +``` + +### Params + +The distribution module stores it's params in state with the prefix of `0x09`, +it can be updated with governance or the address with authority. + +* Params: `0x09 | ProtocolBuffer(Params)` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/distribution/proto/cosmos/distribution/v1beta1/distribution.proto#L12-L44 +``` + +## Begin Block + +At each `BeginBlock`, all fees received in the previous block are transferred to +the distribution `ModuleAccount` account. When a delegator or validator +withdraws their rewards, they are taken out of the `ModuleAccount`. During begin +block, the different claims on the fees collected are updated as follows: + +* The reserve community tax is charged. +* The remainder is distributed proportionally by voting power to all bonded validators + +### The Distribution Scheme + +See [params](#params) for description of parameters. + +Let `fees` be the total fees collected in the previous block, including +inflationary rewards to the stake. All fees are collected in a specific module +account during the block. During `BeginBlock`, they are sent to the +`"distribution"` `ModuleAccount`. No other sending of tokens occurs. Instead, the +rewards each account is entitled to are stored, and withdrawals can be triggered +through the messages `WithdrawValidatorCommission` and +`WithdrawDelegatorReward`. + +#### Reward to the Community Pool + +The community pool (x/protocolpool) gets `community_tax * fees`, plus any remaining dust after +validators get their rewards that are always rounded down to the nearest +integer value. + +#### Reward To the Validators + +The proposer receives no extra rewards. All fees are distributed among all the +bonded validators, including the proposer, in proportion to their consensus power. + +```text +powFrac = validator power / total bonded validator power +voteMul = 1 - community_tax +``` + +All validators receive `fees * voteMul * powFrac`. + +#### Rewards to Delegators + +Each validator's rewards are distributed to its delegators. The validator also +has a self-delegation that is treated like a regular delegation in +distribution calculations. + +The validator sets a commission rate. The commission rate is flexible, but each +validator sets a maximum rate and a maximum daily increase. These maximums cannot be exceeded and protect delegators from sudden increases of validator commission rates to prevent validators from taking all of the rewards. + +The outstanding rewards that the operator is entitled to are stored in +`ValidatorAccumulatedCommission`, while the rewards the delegators are entitled +to are stored in `ValidatorCurrentRewards`. The [F1 fee distribution scheme](#concepts) is used to calculate the rewards per delegator as they +withdraw or update their delegation, and is thus not handled in `BeginBlock`. + +#### Example Distribution + +For this example distribution, the underlying consensus engine selects block proposers in +proportion to their power relative to the entire bonded power. + +All validators are equally performant at including pre-commits in their proposed +blocks. Then hold `(pre_commits included) / (total bonded validator power)` +constant so that the amortized block reward for the validator is `( validator power / total bonded power) * (1 - community tax rate)` of +the total rewards. Consequently, the reward for a single delegator is: + +```text +(delegator proportion of the validator power / validator power) * (validator power / total bonded power) + * (1 - community tax rate) * (1 - validator commission rate) += (delegator proportion of the validator power / total bonded power) * (1 - +community tax rate) * (1 - validator commission rate) +``` + +## Messages + +### MsgSetWithdrawAddress + +By default, the withdraw address is the delegator address. To change its withdraw address, a delegator must send a `MsgSetWithdrawAddress` message. +Changing the withdraw address is possible only if the parameter `WithdrawAddrEnabled` is set to `true`. + +The withdraw address cannot be any of the module accounts. These accounts are blocked from being withdraw addresses by being added to the distribution keeper's `blockedAddrs` array at initialization. + +Response: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/distribution/proto/cosmos/distribution/v1beta1/tx.proto#L62-L73 +``` + +```go +func (k Keeper) SetWithdrawAddr(ctx context.Context, delegatorAddr sdk.AccAddress, withdrawAddr sdk.AccAddress) error + if k.blockedAddrs[withdrawAddr.String()] { + fail with "`{withdrawAddr}` is not allowed to receive external funds" + } + + if !k.GetWithdrawAddrEnabled(ctx) { + fail with `ErrSetWithdrawAddrDisabled` + } + + k.SetDelegatorWithdrawAddr(ctx, delegatorAddr, withdrawAddr) +``` + +### MsgWithdrawDelegatorReward + +A delegator can withdraw its rewards. +Internally in the distribution module, this transaction simultaneously removes the previous delegation with associated rewards, the same as if the delegator simply started a new delegation of the same value. +The rewards are sent immediately from the distribution `ModuleAccount` to the withdraw address. +Any remainder (truncated decimals) are sent to the community pool. +The starting height of the delegation is set to the current validator period, and the reference count for the previous period is decremented. +The amount withdrawn is deducted from the `ValidatorOutstandingRewards` variable for the validator. + +In the F1 distribution, the total rewards are calculated per validator period, and a delegator receives a piece of those rewards in proportion to their stake in the validator. +In basic F1, the total rewards that all the delegators are entitled to between to periods is calculated the following way. +Let `R(X)` be the total accumulated rewards up to period `X` divided by the tokens staked at that time. The delegator allocation is `R(X) * delegator_stake`. +Then the rewards for all the delegators for staking between periods `A` and `B` are `(R(B) - R(A)) * total stake`. +However, these calculated rewards don't account for slashing. + +Taking the slashes into account requires iteration. +Let `F(X)` be the fraction a validator is to be slashed for a slashing event that happened at period `X`. +If the validator was slashed at periods `P1, ..., PN`, where `A < P1`, `PN < B`, the distribution module calculates the individual delegator's rewards, `T(A, B)`, as follows: + +```go +stake := initial stake +rewards := 0 +previous := A +for P in P1, ..., PN`: + rewards = (R(P) - previous) * stake + stake = stake * F(P) + previous = P +rewards = rewards + (R(B) - R(PN)) * stake +``` + +The historical rewards are calculated retroactively by playing back all the slashes and then attenuating the delegator's stake at each step. +The final calculated stake is equivalent to the actual staked coins in the delegation with a margin of error due to rounding errors. + +Response: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/distribution/proto/cosmos/distribution/v1beta1/tx.proto#L79-L90 +``` + +### WithdrawValidatorCommission + +The validator can send the WithdrawValidatorCommission message to withdraw their accumulated commission. +The commission is calculated in every block during `BeginBlock`, so no iteration is required to withdraw. +The amount withdrawn is deducted from the `ValidatorOutstandingRewards` variable for the validator. +Only integer amounts can be sent. If the accumulated awards have decimals, the amount is truncated before the withdrawal is sent, and the remainder is left to be withdrawn later. + +### Common distribution operations + +These operations take place during many different messages. + +#### Initialize delegation + +Each time a delegation is changed, the rewards are withdrawn and the delegation is reinitialized. +Initializing a delegation increments the validator period and keeps track of the starting period of the delegation. + +```go +// initialize starting info for a new delegation +func (k Keeper) initializeDelegation(ctx context.Context, val sdk.ValAddress, del sdk.AccAddress) { + // period has already been incremented - we want to store the period ended by this delegation action + previousPeriod := k.GetValidatorCurrentRewards(ctx, val).Period - 1 + + // increment reference count for the period we're going to track + k.incrementReferenceCount(ctx, val, previousPeriod) + + validator := k.stakingKeeper.Validator(ctx, val) + delegation := k.stakingKeeper.Delegation(ctx, del, val) + + // calculate delegation stake in tokens + // we don't store directly, so multiply delegation shares * (tokens per share) + // note: necessary to truncate so we don't allow withdrawing more rewards than owed + stake := validator.TokensFromSharesTruncated(delegation.GetShares()) + k.SetDelegatorStartingInfo(ctx, val, del, types.NewDelegatorStartingInfo(previousPeriod, stake, uint64(ctx.BlockHeight()))) +} +``` + +### MsgUpdateParams + +Distribution module params can be updated through `MsgUpdateParams`, which can be done using governance proposal and the signer will always be gov module account address. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/distribution/proto/cosmos/distribution/v1beta1/tx.proto#L159:L172 +``` + +The message handling can fail if: + +* signer is not the gov module account address. + +## Hooks + +Available hooks that can be called by and from this module. + +### Create or modify delegation distribution + +* triggered-by: `staking.MsgDelegate`, `staking.MsgBeginRedelegate`, `staking.MsgUndelegate` + +#### Before + +* The delegation rewards are withdrawn to the withdraw address of the delegator. + The rewards include the current period and exclude the starting period. +* The validator period is incremented. + The validator period is incremented because the validator's power and share distribution might have changed. +* The reference count for the delegator's starting period is decremented. + +#### After + +The starting height of the delegation is set to the previous period. +Because of the `Before`-hook, this period is the last period for which the delegator was rewarded. + +### Validator created + +* triggered-by: `staking.MsgCreateValidator` + +When a validator is created, the following validator variables are initialized: + +* Historical rewards +* Current accumulated rewards +* Accumulated commission +* Total outstanding rewards +* Period + +By default, all values are set to a `0`, except period, which is set to `1`. + +### Validator removed + +* triggered-by: `staking.RemoveValidator` + +Outstanding commission is sent to the validator's self-delegation withdrawal address. +Remaining delegator rewards get sent to the community pool. + +Note: The validator gets removed only when it has no remaining delegations. +At that time, all outstanding delegator rewards will have been withdrawn. +Any remaining rewards are dust amounts. + +### Validator is slashed + +* triggered-by: `staking.Slash` +* The current validator period reference count is incremented. + The reference count is incremented because the slash event has created a reference to it. +* The validator period is incremented. +* The slash event is stored for later use. + The slash event will be referenced when calculating delegator rewards. + +## Events + +The distribution module emits the following events: + +### BeginBlocker + +| Type | Attribute Key | Attribute Value | +|-----------------|---------------|--------------------| +| commission | amount | {commissionAmount} | +| commission | validator | {validatorAddress} | +| rewards | amount | {rewardAmount} | +| rewards | validator | {validatorAddress} | + +### Handlers + +#### MsgSetWithdrawAddress + +| Type | Attribute Key | Attribute Value | +|----------------------|------------------|----------------------| +| set_withdraw_address | withdraw_address | {withdrawAddress} | +| message | module | distribution | +| message | action | set_withdraw_address | +| message | sender | {senderAddress} | + +#### MsgWithdrawDelegatorReward + +| Type | Attribute Key | Attribute Value | +|---------|---------------|---------------------------| +| withdraw_rewards | amount | {rewardAmount} | +| withdraw_rewards | validator | {validatorAddress} | +| message | module | distribution | +| message | action | withdraw_delegator_reward | +| message | sender | {senderAddress} | + +#### MsgWithdrawValidatorCommission + +| Type | Attribute Key | Attribute Value | +|------------|---------------|-------------------------------| +| withdraw_commission | amount | {commissionAmount} | +| message | module | distribution | +| message | action | withdraw_validator_commission | +| message | sender | {senderAddress} | + +## Parameters + +The distribution module contains the following parameters: + +| Key | Type | Example | +| ------------------- | ------------ | -------------------------- | +| communitytax | string (dec) | "0.020000000000000000" [0] | +| withdrawaddrenabled | bool | true | + +* [0] `communitytax` must be positive and cannot exceed 1.00. +* `baseproposerreward` and `bonusproposerreward` were parameters that are deprecated in v0.47 and are not used. + +:::note +The community tax is collected and sent to the community pool (x/protocolpool). +::: + +## Client + +### CLI + +A user can query and interact with the `distribution` module using the CLI. + +#### Query + +The `query` commands allow users to query `distribution` state. + +```shell +simd query distribution --help +``` + +##### commission + +The `commission` command allows users to query validator commission rewards by address. + +```shell +simd query distribution commission [address] [flags] +``` + +Example: + +```shell +simd query distribution commission cosmosvaloper1... +``` + +Example Output: + +```yml +commission: +- amount: "1000000.000000000000000000" + denom: stake +``` + +##### params + +The `params` command allows users to query the parameters of the `distribution` module. + +```shell +simd query distribution params [flags] +``` + +Example: + +```shell +simd query distribution params +``` + +Example Output: + +```yml +base_proposer_reward: "0.000000000000000000" +bonus_proposer_reward: "0.000000000000000000" +community_tax: "0.020000000000000000" +withdraw_addr_enabled: true +``` + +##### rewards + +The `rewards` command allows users to query delegator rewards. Users can optionally include the validator address to query rewards earned from a specific validator. + +```shell +simd query distribution rewards [delegator-addr] [validator-addr] [flags] +``` + +Example: + +```shell +simd query distribution rewards cosmos1... +``` + +Example Output: + +```yml +rewards: +- reward: + - amount: "1000000.000000000000000000" + denom: stake + validator_address: cosmosvaloper1.. +total: +- amount: "1000000.000000000000000000" + denom: stake +``` + +##### slashes + +The `slashes` command allows users to query all slashes for a given block range. + +```shell +simd query distribution slashes [validator] [start-height] [end-height] [flags] +``` + +Example: + +```shell +simd query distribution slashes cosmosvaloper1... 1 1000 +``` + +Example Output: + +```yml +pagination: + next_key: null + total: "0" +slashes: +- validator_period: 20, + fraction: "0.009999999999999999" +``` + +##### validator-outstanding-rewards + +The `validator-outstanding-rewards` command allows users to query all outstanding (un-withdrawn) rewards for a validator and all their delegations. + +```shell +simd query distribution validator-outstanding-rewards [validator] [flags] +``` + +Example: + +```shell +simd query distribution validator-outstanding-rewards cosmosvaloper1... +``` + +Example Output: + +```yml +rewards: +- amount: "1000000.000000000000000000" + denom: stake +``` + +##### validator-distribution-info + +The `validator-distribution-info` command allows users to query validator commission and self-delegation rewards for validator. + +```shell +simd query distribution validator-distribution-info cosmosvaloper1... +``` + +Example Output: + +```yml +commission: +- amount: "100000.000000000000000000" + denom: stake +operator_address: cosmosvaloper1... +self_bond_rewards: +- amount: "100000.000000000000000000" + denom: stake +``` + +#### Transactions + +The `tx` commands allow users to interact with the `distribution` module. + +```shell +simd tx distribution --help +``` + +##### set-withdraw-addr + +The `set-withdraw-addr` command allows users to set the withdraw address for rewards associated with a delegator address. + +```shell +simd tx distribution set-withdraw-addr [withdraw-addr] [flags] +``` + +Example: + +```shell +simd tx distribution set-withdraw-addr cosmos1... --from cosmos1... +``` + +##### withdraw-all-rewards + +The `withdraw-all-rewards` command allows users to withdraw all rewards for a delegator. + +```shell +simd tx distribution withdraw-all-rewards [flags] +``` + +Example: + +```shell +simd tx distribution withdraw-all-rewards --from cosmos1... +``` + +##### withdraw-rewards + +The `withdraw-rewards` command allows users to withdraw all rewards from a given delegation address, +and optionally withdraw validator commission if the delegation address given is a validator operator and the user proves the `--commission` flag. + +```shell +simd tx distribution withdraw-rewards [validator-addr] [flags] +``` + +Example: + +```shell +simd tx distribution withdraw-rewards cosmosvaloper1... --from cosmos1... --commission +``` + +### gRPC + +A user can query the `distribution` module using gRPC endpoints. + +#### Params + +The `Params` endpoint allows users to query parameters of the `distribution` module. + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/Params +``` + +Example Output: + +```json +{ + "params": { + "communityTax": "20000000000000000", + "baseProposerReward": "00000000000000000", + "bonusProposerReward": "00000000000000000", + "withdrawAddrEnabled": true + } +} +``` + +#### ValidatorDistributionInfo + +The `ValidatorDistributionInfo` queries validator commission and self-delegation rewards for validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvalop1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorDistributionInfo +``` + +Example Output: + +```json +{ + "commission": { + "commission": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] + }, + "self_bond_rewards": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ], + "validator_address": "cosmosvalop1..." +} +``` + +#### ValidatorOutstandingRewards + +The `ValidatorOutstandingRewards` endpoint allows users to query rewards of a validator address. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvalop1.."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorOutstandingRewards +``` + +Example Output: + +```json +{ + "rewards": { + "rewards": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] + } +} +``` + +#### ValidatorCommission + +The `ValidatorCommission` endpoint allows users to query accumulated commission for a validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvalop1.."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorCommission +``` + +Example Output: + +```json +{ + "commission": { + "commission": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] + } +} +``` + +#### ValidatorSlashes + +The `ValidatorSlashes` endpoint allows users to query slash events of a validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvalop1.."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorSlashes +``` + +Example Output: + +```json +{ + "slashes": [ + { + "validator_period": "20", + "fraction": "0.009999999999999999" + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### DelegationRewards + +The `DelegationRewards` endpoint allows users to query the total rewards accrued by a delegation. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1...","validator_address":"cosmosvalop1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegationRewards +``` + +Example Output: + +```json +{ + "rewards": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] +} +``` + +#### DelegationTotalRewards + +The `DelegationTotalRewards` endpoint allows users to query the total rewards accrued by each validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegationTotalRewards +``` + +Example Output: + +```json +{ + "rewards": [ + { + "validatorAddress": "cosmosvaloper1...", + "reward": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] + } + ], + "total": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] +} +``` + +#### DelegatorValidators + +The `DelegatorValidators` endpoint allows users to query all validators for given delegator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegatorValidators +``` + +Example Output: + +```json +{ + "validators": ["cosmosvaloper1..."] +} +``` + +#### DelegatorWithdrawAddress + +The `DelegatorWithdrawAddress` endpoint allows users to query the withdraw address of a delegator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegatorWithdrawAddress +``` + +Example Output: + +```json +{ + "withdrawAddress": "cosmos1..." +} +``` diff --git a/versioned_docs/version-0.52/build/modules/epochs/README.md b/versioned_docs/version-0.52/build/modules/epochs/README.md new file mode 100644 index 000000000..7b0b0b285 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/epochs/README.md @@ -0,0 +1,177 @@ +--- +sidebar_position: 1 +--- + +# `x/epochs` + +## Abstract + +Often in the SDK, we would like to run certain code every-so often. The +purpose of `epochs` module is to allow other modules to set that they +would like to be signaled once every period. So another module can +specify it wants to execute code once a week, starting at UTC-time = x. +`epochs` creates a generalized epoch interface to other modules so that +they can easily be signaled upon such events. + +## Contents + +1. **[Concept](#concepts)** +2. **[State](#state)** +3. **[Events](#events)** +4. **[Keeper](#keepers)** +5. **[Hooks](#hooks)** +6. **[Queries](#queries)** + +## Concepts + +The epochs module defines on-chain timers that execute at fixed time intervals. +Other SDK modules can then register logic to be executed at the timer ticks. +We refer to the period in between two timer ticks as an "epoch". + +Every timer has a unique identifier. +Every epoch will have a start time, and an end time, where `end time = start time + timer interval`. +On mainnet, we only utilize one identifier, with a time interval of `one day`. + +The timer will tick at the first block whose block time is greater than the timer end time, +and set the start as the prior timer end time. (Notably, it's not set to the block time!) +This means that if the chain has been down for a while, you will get one timer tick per block, +until the timer has caught up. + +## State + +The Epochs module keeps a single `EpochInfo` per identifier. +This contains the current state of the timer with the corresponding identifier. +Its fields are modified at every timer tick. +EpochInfos are initialized as part of genesis initialization or upgrade logic, +and are only modified on begin blockers. + +## Events + +The `epochs` module emits the following events: + +### BeginBlocker + +| Type | Attribute Key | Attribute Value | +| ----------- | ------------- | --------------- | +| epoch_start | epoch_number | {epoch_number} | +| epoch_start | start_time | {start_time} | + +### EndBlocker + +| Type | Attribute Key | Attribute Value | +| --------- | ------------- | --------------- | +| epoch_end | epoch_number | {epoch_number} | + +## Keepers + +### Keeper functions + +Epochs keeper module provides utility functions to manage epochs. + +## Hooks + +```go + // the first block whose timestamp is after the duration is counted as the end of the epoch + AfterEpochEnd(ctx sdk.Context, epochIdentifier string, epochNumber int64) + // new epoch is next block of epoch end block + BeforeEpochStart(ctx sdk.Context, epochIdentifier string, epochNumber int64) +``` + +### How modules receive hooks + +On hook receiver function of other modules, they need to filter +`epochIdentifier` and only do executions for only specific +epochIdentifier. Filtering epochIdentifier could be in `Params` of other +modules so that they can be modified by governance. + +This is the standard dev UX of this: + +```golang +func (k MyModuleKeeper) AfterEpochEnd(ctx sdk.Context, epochIdentifier string, epochNumber int64) { + params := k.GetParams(ctx) + if epochIdentifier == params.DistrEpochIdentifier { + // my logic + } +} +``` + +### Panic isolation + +If a given epoch hook panics, its state update is reverted, but we keep +proceeding through the remaining hooks. This allows more advanced epoch +logic to be used, without concern over state machine halting, or halting +subsequent modules. + +This does mean that if there is behavior you expect from a prior epoch +hook, and that epoch hook reverted, your hook may also have an issue. So +do keep in mind "what if a prior hook didn't get executed" in the safety +checks you consider for a new epoch hook. + +## Queries + +The Epochs module provides the following queries to check the module's state. + +```protobuf +service Query { + // EpochInfos provide running epochInfos + rpc EpochInfos(QueryEpochsInfoRequest) returns (QueryEpochsInfoResponse) {} + // CurrentEpoch provide current epoch of specified identifier + rpc CurrentEpoch(QueryCurrentEpochRequest) returns (QueryCurrentEpochResponse) {} +} +``` + +### Epoch Infos + +Query the currently running epochInfos + +```sh + query epochs epoch-infos +``` + +:::details Example + +An example output: + +```sh +epochs: +- current_epoch: "183" + current_epoch_start_height: "2438409" + current_epoch_start_time: "2021-12-18T17:16:09.898160996Z" + duration: 86400s + epoch_counting_started: true + identifier: day + start_time: "2021-06-18T17:00:00Z" +- current_epoch: "26" + current_epoch_start_height: "2424854" + current_epoch_start_time: "2021-12-17T17:02:07.229632445Z" + duration: 604800s + epoch_counting_started: true + identifier: week + start_time: "2021-06-18T17:00:00Z" +``` + +::: + +### Current Epoch + +Query the current epoch by the specified identifier + +```sh + query epochs current-epoch [identifier] +``` + +:::details Example + +Query the current `day` epoch: + +```sh + query epochs current-epoch day +``` + +Which in this example outputs: + +```sh +current_epoch: "183" +``` + +::: diff --git a/versioned_docs/version-0.52/build/modules/evidence/README.md b/versioned_docs/version-0.52/build/modules/evidence/README.md new file mode 100644 index 000000000..82cd03bac --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/evidence/README.md @@ -0,0 +1,440 @@ +--- +sidebar_position: 1 +--- + +# `x/evidence` + +* [Concepts](#concepts) +* [State](#state) +* [Messages](#messages) +* [Events](#events) +* [Parameters](#parameters) +* [BeginBlock](#beginblock) +* [Client](#client) + * [CLI](#cli) + * [REST](#rest) + * [gRPC](#grpc) + +## Abstract + +`x/evidence` is an implementation of a Cosmos SDK module, per [ADR 009](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-009-evidence-module.md), +that allows for the submission and handling of arbitrary evidence of misbehavior such +as equivocation and counterfactual signing. + +The evidence module differs from standard evidence handling which typically expects the +underlying consensus engine, e.g. CometBFT, to automatically submit evidence when +it is discovered by allowing clients and foreign chains to submit more complex evidence +directly. + +All concrete evidence types must implement the `Evidence` interface contract. Submitted +`Evidence` is first routed through the evidence module's `Router` in which it attempts +to find a corresponding registered `Handler` for that specific `Evidence` type. +Each `Evidence` type must have a `Handler` registered with the evidence module's +keeper in order for it to be successfully routed and executed. + +Each corresponding handler must also fulfill the `Handler` interface contract. The +`Handler` for a given `Evidence` type can perform any arbitrary state transitions +such as slashing, jailing, and tombstoning. + +## Concepts + +### Evidence + +Any concrete type of evidence submitted to the `x/evidence` module must fulfill the +`Evidence` contract outlined below. Not all concrete types of evidence will fulfill +this contract in the same way and some data may be entirely irrelevant to certain +types of evidence. An additional `ValidatorEvidence`, which extends `Evidence`, +has also been created to define a contract for evidence against malicious validators. + +```go +// Evidence defines the contract which concrete evidence types of misbehavior +// must implement. +type Evidence interface { + proto.Message + + Route() string + String() string + Hash() []byte + ValidateBasic() error + + // Height at which the infraction occurred + GetHeight() int64 +} + +// ValidatorEvidence extends Evidence interface to define contract +// for evidence against malicious validators +type ValidatorEvidence interface { + Evidence + + // The consensus address of the malicious validator at time of infraction + GetConsensusAddress() sdk.ConsAddress + + // The total power of the malicious validator at time of infraction + GetValidatorPower() int64 + + // The total validator set power at time of infraction + GetTotalPower() int64 +} +``` + +### Registration & Handling + +The `x/evidence` module must first know about all types of evidence it is expected +to handle. This is accomplished by registering the `Route` method in the `Evidence` +contract with what is known as a `Router` (defined below). The `Router` accepts +`Evidence` and attempts to find the corresponding `Handler` for the `Evidence` +via the `Route` method. + +```go +type Router interface { + AddRoute(r string, h Handler) Router + HasRoute(r string) bool + GetRoute(path string) Handler + Seal() + Sealed() bool +} +``` + +The `Handler` (defined below) is responsible for executing the entirety of the +business logic for handling `Evidence`. This typically includes validating the +evidence, both stateless checks via `ValidateBasic` and stateful checks via any +keepers provided to the `Handler`. In addition, the `Handler` may also perform +capabilities such as slashing and jailing a validator. All `Evidence` handled +by the `Handler` should be persisted. + +```go +// Handler defines an agnostic Evidence handler. The handler is responsible +// for executing all corresponding business logic necessary for verifying the +// evidence as valid. In addition, the Handler may execute any necessary +// slashing and potential jailing. +type Handler func(context.Context, Evidence) error +``` + + +## State + +Currently the `x/evidence` module only stores valid submitted `Evidence` in state. +The evidence state is also stored and exported in the `x/evidence` module's `GenesisState`. + +```protobuf +// GenesisState defines the evidence module's genesis state. +message GenesisState { + // evidence defines all the evidence at genesis. + repeated google.protobuf.Any evidence = 1; +} + +``` + +All `Evidence` is retrieved and stored via a prefix `KVStore` using prefix `0x00` (`KeyPrefixEvidence`). + + +## Messages + +### MsgSubmitEvidence + +Evidence is submitted through a `MsgSubmitEvidence` message: + +```protobuf +// MsgSubmitEvidence represents a message that supports submitting arbitrary +// Evidence of misbehavior such as equivocation or counterfactual signing. +message MsgSubmitEvidence { + string submitter = 1; + google.protobuf.Any evidence = 2; +} +``` + +Note, the `Evidence` of a `MsgSubmitEvidence` message must have a corresponding +`Handler` registered with the `x/evidence` module's `Router` in order to be processed +and routed correctly. + +Given the `Evidence` is registered with a corresponding `Handler`, it is processed +as follows: + +```go +func SubmitEvidence(ctx Context, evidence Evidence) error { + if _, err := GetEvidence(ctx, evidence.Hash()); err == nil { + return errorsmod.Wrap(types.ErrEvidenceExists, strings.ToUpper(hex.EncodeToString(evidence.Hash()))) + } + if !router.HasRoute(evidence.Route()) { + return errorsmod.Wrap(types.ErrNoEvidenceHandlerExists, evidence.Route()) + } + + handler := router.GetRoute(evidence.Route()) + if err := handler(ctx, evidence); err != nil { + return errorsmod.Wrap(types.ErrInvalidEvidence, err.Error()) + } + + ctx.EventManager().EmitEvent( + sdk.NewEvent( + types.EventTypeSubmitEvidence, + sdk.NewAttribute(types.AttributeKeyEvidenceHash, strings.ToUpper(hex.EncodeToString(evidence.Hash()))), + ), + ) + + SetEvidence(ctx, evidence) + return nil +} +``` + +First, there must not already exist valid submitted `Evidence` of the exact same +type. Secondly, the `Evidence` is routed to the `Handler` and executed. Finally, +if there is no error in handling the `Evidence`, an event is emitted and it is persisted to state. + + +## Events + +The `x/evidence` module emits the following events: + +### Handlers + +#### MsgSubmitEvidence + +| Type | Attribute Key | Attribute Value | +| --------------- | ------------- | --------------- | +| submit_evidence | evidence_hash | {evidenceHash} | +| message | module | evidence | +| message | sender | {senderAddress} | +| message | action | submit_evidence | + + +## Parameters + +The evidence module does not contain any parameters. + + +## BeginBlock + +### Evidence Handling + +CometBFT blocks can include +[Evidence](https://github.com/cometbft/cometbft/blob/main/spec/abci/abci%2B%2B_basic_concepts.md#evidence) that indicates if a validator committed malicious behavior. The relevant information is forwarded to the application as ABCI Evidence in `abci.RequestBeginBlock` so that the validator can be punished accordingly. + +#### Equivocation + +The Cosmos SDK handles two types of evidence inside the ABCI `BeginBlock`: + +* `DuplicateVoteEvidence`, +* `LightClientAttackEvidence`. + +The evidence module handles these two evidence types the same way. First, the Cosmos SDK converts the CometBFT concrete evidence type to an SDK `Evidence` interface using `Equivocation` as the concrete type. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/evidence/v1beta1/evidence.proto#L12-L32 +``` + +For some `Equivocation` submitted in `block` to be valid, it must satisfy: + +`Evidence.Timestamp >= block.Timestamp - MaxEvidenceAge` + +Where: + +* `Evidence.Timestamp` is the timestamp in the block at height `Evidence.Height` +* `block.Timestamp` is the current block timestamp. + +If valid `Equivocation` evidence is included in a block, the validator's stake is +reduced (slashed) by `SlashFractionDoubleSign` as defined by the `x/slashing` module +of what their stake was when the infraction occurred, rather than when the evidence was discovered. +We want to "follow the stake", i.e., the stake that contributed to the infraction +should be slashed, even if it has since been redelegated or started unbonding. + +In addition, the validator is permanently jailed and tombstoned to make it impossible for that +validator to ever re-enter the validator set. + +The `Equivocation` evidence is handled as follows: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/evidence/keeper/infraction.go#L26-L140 +``` + +**Note:** The slashing, jailing, and tombstoning calls are delegated through the `x/slashing` module +that emits informative events and finally delegates calls to the `x/staking` module. See documentation +on slashing and jailing in [State Transitions](../staking/README.md#state-transitions). + +## Client + +### CLI + +A user can query and interact with the `evidence` module using the CLI. + +#### Query + +The `query` commands allows users to query `evidence` state. + +```bash +simd query evidence --help +``` + +#### evidence + +The `evidence` command allows users to list all evidence or evidence by hash. + +Usage: + +```bash +simd query evidence evidence [flags] +``` + +To query evidence by hash + +Example: + +```bash +simd query evidence evidence "DF0C23E8634E480F84B9D5674A7CDC9816466DEC28A3358F73260F68D28D7660" +``` + +Example Output: + +```bash +evidence: + consensus_address: cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h + height: 11 + power: 100 + time: "2021-10-20T16:08:38.194017624Z" +``` + +To get all evidence + +Example: + +```bash +simd query evidence list +``` + +Example Output: + +```bash +evidence: + consensus_address: cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h + height: 11 + power: 100 + time: "2021-10-20T16:08:38.194017624Z" +pagination: + next_key: null + total: "1" +``` + +### REST + +A user can query the `evidence` module using REST endpoints. + +#### Evidence + +Get evidence by hash + +```bash +/cosmos/evidence/v1beta1/evidence/{hash} +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/evidence/v1beta1/evidence/DF0C23E8634E480F84B9D5674A7CDC9816466DEC28A3358F73260F68D28D7660" +``` + +Example Output: + +```bash +{ + "evidence": { + "consensus_address": "cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h", + "height": "11", + "power": "100", + "time": "2021-10-20T16:08:38.194017624Z" + } +} +``` + +#### All evidence + +Get all evidence + +```bash +/cosmos/evidence/v1beta1/evidence +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/evidence/v1beta1/evidence" +``` + +Example Output: + +```bash +{ + "evidence": [ + { + "consensus_address": "cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h", + "height": "11", + "power": "100", + "time": "2021-10-20T16:08:38.194017624Z" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### gRPC + +A user can query the `evidence` module using gRPC endpoints. + +#### Evidence + +Get evidence by hash + +```bash +cosmos.evidence.v1beta1.Query/Evidence +``` + +Example: + +```bash +grpcurl -plaintext -d '{"evidence_hash":"DF0C23E8634E480F84B9D5674A7CDC9816466DEC28A3358F73260F68D28D7660"}' localhost:9090 cosmos.evidence.v1beta1.Query/Evidence +``` + +Example Output: + +```bash +{ + "evidence": { + "consensus_address": "cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h", + "height": "11", + "power": "100", + "time": "2021-10-20T16:08:38.194017624Z" + } +} +``` + +#### All evidence + +Get all evidence + +```bash +cosmos.evidence.v1beta1.Query/AllEvidence +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.evidence.v1beta1.Query/AllEvidence +``` + +Example Output: + +```bash +{ + "evidence": [ + { + "consensus_address": "cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h", + "height": "11", + "power": "100", + "time": "2021-10-20T16:08:38.194017624Z" + } + ], + "pagination": { + "total": "1" + } +} +``` diff --git a/versioned_docs/version-0.52/build/modules/feegrant/README.md b/versioned_docs/version-0.52/build/modules/feegrant/README.md new file mode 100644 index 000000000..63e563d2e --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/feegrant/README.md @@ -0,0 +1,449 @@ +--- +sidebar_position: 1 +--- + +# `x/feegrant` + +## Abstract + +This document specifies the fee grant module. For the full ADR, please see [Fee Grant ADR-029](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-029-fee-grant-module.md). + +This module allows accounts to grant fee allowances and to use fees from their accounts. Grantees can execute any transaction without the need to maintain sufficient fees. + +## Contents + +* [Concepts](#concepts) +* [State](#state) + * [FeeAllowance](#feeallowance) + * [FeeAllowanceQueue](#feeallowancequeue) +* [Messages](#messages) + * [Msg/GrantAllowance](#msggrantallowance) + * [Msg/RevokeAllowance](#msgrevokeallowance) +* [Events](#events) +* [Msg Server](#msg-server) + * [MsgGrantAllowance](#msggrantallowance-1) + * [MsgRevokeAllowance](#msgrevokeallowance-1) + * [Exec fee allowance](#exec-fee-allowance) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + +## Concepts + +### Grant + +`Grant` is stored in the KVStore to record a grant with full context. Every grant will contain `granter`, `grantee` and what kind of `allowance` is granted. `granter` is an account address who is giving permission to `grantee` (the beneficiary account address) to pay for some or all of `grantee`'s transaction fees. `allowance` defines what kind of fee allowance (`BasicAllowance` or `PeriodicAllowance`, see below) is granted to `grantee`. `allowance` accepts an interface which implements `FeeAllowanceI`, encoded as `Any` type. There can be only one existing fee grant allowed for a `grantee` and `granter`, self grants are not allowed. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/feegrant/proto/cosmos/feegrant/v1beta1/feegrant.proto#L86-L96 +``` + +`FeeAllowanceI` looks like: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/feegrant/fees.go#L10-L34 +``` + +### Fee Allowance types + +There are two types of fee allowances present at the moment: + +* `BasicAllowance` +* `PeriodicAllowance` +* `AllowedMsgAllowance` + +### BasicAllowance + +`BasicAllowance` is permission for `grantee` to use fee from a `granter`'s account. If any of the `spend_limit` or `expiration` reaches its limit, the grant will be removed from the state. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/feegrant/proto/cosmos/feegrant/v1beta1/feegrant.proto#L15-L33 +``` + +* `spend_limit` is the limit of coins that are allowed to be used from the `granter` account. If it is empty, it assumes there's no spend limit, `grantee` can use any number of available coins from `granter` account address before the expiration. + +* `expiration` specifies an optional time when this allowance expires. If the value is left empty, there is no expiry for the grant. + +* When a grant is created with empty values for `spend_limit` and `expiration`, it is still a valid grant. It won't restrict the `grantee` to use any number of coins from `granter` and it won't have any expiration. The only way to restrict the `grantee` is by revoking the grant. + +### PeriodicAllowance + +`PeriodicAllowance` is a repeating fee allowance for the mentioned period, we can mention when the grant can expire as well as when a period can reset. We can also define the maximum number of coins that can be used in a mentioned period of time. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/feegrant/proto/cosmos/feegrant/v1beta1/feegrant.proto#L35-L71 +``` + +* `basic` is the instance of `BasicAllowance` which is optional for periodic fee allowance. If empty, the grant will have no `expiration` and no `spend_limit`. + +* `period` is the specific period of time, after each period passes, `period_can_spend` will be reset. + +* `period_spend_limit` specifies the maximum number of coins that can be spent in the period. + +* `period_can_spend` is the number of coins left to be spent before the period_reset time. + +* `period_reset` keeps track of when a next period reset should happen. + +### AllowedMsgAllowance + +`AllowedMsgAllowance` is a fee allowance, it can be any of `BasicFeeAllowance`, `PeriodicAllowance` but restricted only to the allowed messages mentioned by the granter. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/feegrant/proto/cosmos/feegrant/v1beta1/feegrant.proto#L73-L84 +``` + +* `allowance` is either `BasicAllowance` or `PeriodicAllowance`. + +* `allowed_messages` is array of messages allowed to execute the given allowance. + +### FeeGranter flag + +`feegrant` module introduces a `FeeGranter` flag for CLI for the sake of executing transactions with fee granter. When this flag is set, `clientCtx` will append the granter account address for transactions generated through CLI. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/client/cmd.go#L269-L280 +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/client/tx/tx.go#L129-L131 +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/auth/tx/builder.go#L208 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/proto/cosmos/tx/v1beta1/tx.proto#L216-L243 +``` + +Example cmd: + +```shell +simd tx gov submit-legacy-proposal --title="Test Proposal" --description="My awesome proposal" --type="Text" --from validator-key --fee-granter=cosmos1xh44hxt7spr67hqaa7nyx5gnutrz5fraw6grxn --chain-id=testnet --fees="10stake" +``` + +### Granted Fee Deductions + +Fees are deducted from grants in the `x/auth` ante handler. To learn more about how ante handlers work, read the [Auth Module AnteHandlers Guide](../auth/README.md#antehandlers). + +### Gas + +In order to prevent DoS attacks, using a filtered `x/feegrant` incurs gas. The SDK must assure that the `grantee`'s transactions all conform to the filter set by the `granter`. The SDK does this by iterating over the allowed messages in the filter and charging 10 gas per filtered message. The SDK will then iterate over the messages being sent by the `grantee` to ensure the messages adhere to the filter, also charging 10 gas per message. The SDK will stop iterating and fail the transaction if it finds a message that does not conform to the filter. + +**WARNING**: The gas is charged against the granted allowance. Ensure your messages conform to the filter, if any, before sending transactions using your allowance. + +### Pruning + +A queue in the state maintained with the prefix of expiration of the grants and checks them on EndBlock with the current block time for every block to prune. + +## State + +### FeeAllowance + +Fee Allowances are identified by combining `Grantee` (the account address of fee allowance grantee) with the `Granter` (the account address of fee allowance granter). + +Fee allowance grants are stored in the state as follows: + +* Grant: `0x00 | grantee_addr_len (1 byte) | grantee_addr_bytes | granter_addr_len (1 byte) | granter_addr_bytes -> ProtocolBuffer(Grant)` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/feegrant/feegrant.pb.go#L222-L230 +``` + +### FeeAllowanceQueue + +Fee Allowances queue items are identified by combining the `FeeAllowancePrefixQueue` (i.e., 0x01), `expiration`, `grantee` (the account address of fee allowance grantee), `granter` (the account address of fee allowance granter). Endblocker checks `FeeAllowanceQueue` state for the expired grants and prunes them from `FeeAllowance` if there are any found. + +Fee allowance queue keys are stored in the state as follows: + +* Grant: `0x01 | expiration_bytes | grantee_addr_len (1 byte) | grantee_addr_bytes | granter_addr_len (1 byte) | granter_addr_bytes -> EmptyBytes` + +## Messages + +### Msg/GrantAllowance + +A fee allowance grant will be created with the `MsgGrantAllowance` message. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/feegrant/proto/cosmos/feegrant/v1beta1/tx.proto#L30-L44 +``` + +### Msg/RevokeAllowance + +An allowed grant fee allowance can be removed with the `MsgRevokeAllowance` message. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/feegrant/proto/cosmos/feegrant/v1beta1/tx.proto#L49-L62 +``` + +## Events + +The feegrant module emits the following events: + +## Msg Server + +### MsgGrantAllowance + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ---------------- | +| message | action | set_feegrant | +| message | granter | {granterAddress} | +| message | grantee | {granteeAddress} | + +### MsgRevokeAllowance + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ---------------- | +| message | action | revoke_feegrant | +| message | granter | {granterAddress} | +| message | grantee | {granteeAddress} | + +### Exec fee allowance + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ---------------- | +| message | action | use_feegrant | +| message | granter | {granterAddress} | +| message | grantee | {granteeAddress} | + +### Prune fee allowances + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ---------------- | +| message | action | prune_feegrant | +| message | pruner | {prunerAddress} | + + +## Client + +### CLI + +A user can query and interact with the `feegrant` module using the CLI. + +#### Query + +The `query` commands allow users to query `feegrant` state. + +```shell +simd query feegrant --help +``` + +##### grant + +The `grant` command allows users to query a grant for a given granter-grantee pair. + +```shell +simd query feegrant grant [granter] [grantee] [flags] +``` + +Example: + +```shell +simd query feegrant grant cosmos1.. cosmos1.. +``` + +Example Output: + +```yml +allowance: + '@type': /cosmos.feegrant.v1beta1.BasicAllowance + expiration: null + spend_limit: + - amount: "100" + denom: stake +grantee: cosmos1.. +granter: cosmos1.. +``` + +##### grants-by-grantee + +The `grants-by-grantee ` command allows users to query all grants for a given grantee. + +```shell +simd query feegrant grants-by-grantee [grantee] [flags] +``` + +Example: + +```shell +simd query feegrant grants-by-grantee cosmos1.. +``` + +Example Output: + +```yml +allowances: +- allowance: + '@type': /cosmos.feegrant.v1beta1.BasicAllowance + expiration: null + spend_limit: + - amount: "100" + denom: stake + grantee: cosmos1.. + granter: cosmos1.. +pagination: + next_key: null + total: "0" +``` + +##### grants-by-granter + +The `grants-by-granter` command allows users to query all grants created by a given granter. + +```shell +simd query feegrant grants-by-granter [granter] [flags] +``` + +Example: + +```shell +simd query feegrant grants-by-granter cosmos1.. +``` + +Example Output: + +```yml +allowances: +- allowance: + '@type': /cosmos.feegrant.v1beta1.BasicAllowance + expiration: null + spend_limit: + - amount: "100" + denom: stake + grantee: cosmos1.. + granter: cosmos1.. +pagination: + next_key: null + total: "0" +``` + +#### Transactions + +The `tx` commands allow users to interact with the `feegrant` module. + +```shell +simd tx feegrant --help +``` + +##### grant + +The `grant` command allows users to grant fee allowances to another account. The fee allowance can have an expiration date, a total spend limit, a periodic spend limit, and/or allowed messages. + +```shell +simd tx feegrant grant [granter] [grantee] [flags] +``` + +Examples: + +###### One-time spend limit + +```shell +simd tx feegrant grant cosmos1.. cosmos1.. --spend-limit 100stake +``` + +###### Periodic spend limit + +```shell +simd tx feegrant grant cosmos1.. cosmos1.. --spend-limit 100stake --period 3600 --period-limit 10stake +``` + +###### With expiration + +```shell +simd tx feegrant grant cosmos1.. cosmos1.. --spend-limit 100stake --expiration 2024-10-31T15:04:05Z +``` + +###### With allowed messages + +```shell +simd tx feegrant grant cosmos1.. cosmos1.. --spend-limit 100stake --expiration 2024-10-31T15:04:05Z --allowed-messages "/cosmos.gov.v1beta1.MsgSubmitProposal,/cosmos.gov.v1beta1.MsgVote" +``` + +Available flags: + +- `--spend-limit`: The maximum amount of tokens the grantee can spend +- `--period`: The time duration in seconds for periodic allowance +- `--period-limit`: The maximum amount of tokens the grantee can spend within each period +- `--expiration`: The date and time when the grant expires (RFC3339 format) +- `--allowed-messages`: Comma-separated list of allowed message type URLs + +##### revoke + +The `revoke` command allows users to revoke a granted fee allowance. + +```shell +simd tx feegrant revoke [granter] [grantee] [flags] +``` + +Example: + +```shell +simd tx feegrant revoke cosmos1.. cosmos1.. +``` + +### gRPC + +A user can query the `feegrant` module using gRPC endpoints. + +#### Allowance + +The `Allowance` endpoint allows users to query a granted fee allowance. + +```shell +cosmos.feegrant.v1beta1.Query/Allowance +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"grantee":"cosmos1..","granter":"cosmos1.."}' \ + localhost:9090 \ + cosmos.feegrant.v1beta1.Query/Allowance +``` + +Example Output: + +```json +{ + "allowance": { + "granter": "cosmos1..", + "grantee": "cosmos1..", + "allowance": {"@type":"/cosmos.feegrant.v1beta1.BasicAllowance","spendLimit":[{"denom":"stake","amount":"100"}]} + } +} +``` + +#### Allowances + +The `Allowances` endpoint allows users to query all granted fee allowances for a given grantee. + +```shell +cosmos.feegrant.v1beta1.Query/Allowances +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' \ + localhost:9090 \ + cosmos.feegrant.v1beta1.Query/Allowances +``` + +Example Output: + +```json +{ + "allowances": [ + { + "granter": "cosmos1..", + "grantee": "cosmos1..", + "allowance": {"@type":"/cosmos.feegrant.v1beta1.BasicAllowance","spendLimit":[{"denom":"stake","amount":"100"}]} + } + ], + "pagination": { + "total": "1" + } +} +``` diff --git a/versioned_docs/version-0.52/build/modules/genutil/README.md b/versioned_docs/version-0.52/build/modules/genutil/README.md new file mode 100644 index 000000000..c9ceb87b0 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/genutil/README.md @@ -0,0 +1,105 @@ +# `x/genutil` + +## Concepts + +The `genutil` package contains a variety of genesis utility functionalities for usage within a blockchain application. Namely: + +* Genesis transactions related (gentx) +* Commands for collection and creation of gentxs +* `InitChain` processing of gentxs +* Genesis file creation +* Genesis file validation +* Genesis file migration +* CometBFT related initialization + * Translation of an app genesis to a CometBFT genesis +* Application state export into a genesis file + +## Genesis + +Genutil contains the data structure that defines an application genesis. +An application genesis consist of a consensus genesis (g.e. CometBFT genesis) and application related genesis data. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-rc.0/x/genutil/types/genesis.go#L24-L34 +``` + +The application genesis can then be translated to the consensus engine to the right format: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-rc.0/x/genutil/types/genesis.go#L126-L136 +``` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-rc.0/server/start.go#L397-L407 +``` + +## Client + +### CLI + +The genutil commands are available under the `genesis` subcommand. + +#### add-genesis-account + +Add a genesis account to `genesis.json`. Learn more [here](https://docs.cosmos.network/main/run-node/run-node#adding-genesis-accounts). + +#### collect-gentxs + +Collect genesis txs and output a `genesis.json` file. + +```shell +simd genesis collect-gentxs +``` + +This will create a new `genesis.json` file that includes data from all the validators (we sometimes call it the "super genesis file" to distinguish it from single-validator genesis files). + +#### gentx + +Generate a genesis tx carrying a self delegation. + +```shell +simd genesis gentx [key_name] [amount] --chain-id [chain-id] +``` + +This will create the genesis transaction for your new chain. Here `amount` should be at least `1000000000stake`. +If you provide too much or too little, you will encounter an error when starting a node. + +#### migrate + +Migrate genesis to a specified target (SDK) version. + +```shell +simd genesis migrate [target-version] +``` + +:::tip +The `migrate` command is extensible and takes a `MigrationMap`. This map is a mapping of target versions to genesis migrations functions. +When not using the default `MigrationMap`, it is recommended to still call the default `MigrationMap` corresponding the SDK version of the chain and prepend/append your own genesis migrations. +::: + +#### validate-genesis + +Validates the genesis file at the default location or at the location passed as an argument. + +```shell +simd genesis validate-genesis +``` + +:::warning +Validate genesis only validates if the genesis is valid at the **current application binary**. For validating a genesis from a previous version of the application, use the `migrate` command to migrate the genesis to the current version. +::: + +#### export + +Export state to genesis file. + +```shell +simd genesis export +``` + +Some flags are available to customize the export: + +* `--for-zero-height`: export the genesis file for a chain with zero height +* `--height [height]`: export the genesis file for a chain with a given height + +Read the help for more information. diff --git a/versioned_docs/version-0.52/build/modules/gov/README.md b/versioned_docs/version-0.52/build/modules/gov/README.md new file mode 100644 index 000000000..543639749 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/gov/README.md @@ -0,0 +1,1295 @@ +--- +sidebar_position: 1 +--- + +# `x/gov` + +## Abstract + +This paper specifies the Governance module of the Cosmos SDK, which was first +described in the [Cosmos Whitepaper](https://cosmos.network/about/whitepaper) in +June 2016. + +The module enables Cosmos SDK based blockchain to support an on-chain governance +system. In this system, holders of the native staking token of the chain can vote +on proposals on a 1 token 1 vote basis. Next is a list of features the module +currently supports: + +* **Proposal submission:** Users can submit proposals with a deposit. Once the +minimum deposit is reached, the proposal enters voting period. The minimum deposit can be reached by collecting deposits from different users (including proposer) within deposit period. +* **Vote:** Participants can vote on proposals that reached MinDeposit and entered voting period. +* **Inheritance and penalties:** Delegators, by default, inherit their validator's vote if they don't vote themselves. +* **Claiming deposit:** Users that deposited on proposals can recover their +deposits if the proposal was accepted or rejected. If the proposal was vetoed, or never entered voting period (minimum deposit not reached within deposit period), the deposit is burned (or refunded depending on the gov parameters). + +## Contents + +The following specification uses *ATOM* as the native staking token. The module +can be adapted to any Proof-Of-Stake blockchain by replacing *ATOM* with the native +staking token of the chain. + +* [Concepts](#concepts) + * [Proposal submission](#proposal-submission) + * [Deposit](#deposit) + * [Vote](#vote) +* [State](#state) + * [Proposals](#proposals) + * [Parameters and base types](#parameters-and-base-types) + * [Deposit](#deposit-1) + * [ValidatorGovInfo](#validatorgovinfo) + * [Legacy Proposal](#legacy-proposal) +* [Messages](#messages) + * [Proposal Submission](#proposal-submission-1) + * [Deposit](#deposit-2) + * [Vote](#vote-1) +* [Events](#events) + * [EndBlocker](#endblocker) + * [Handlers](#handlers) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) +* [Metadata](#metadata) + * [Proposal](#proposal-3) + * [Vote](#vote-5) + +## Concepts + +The governance process is divided into a few steps that are outlined below: + +* **Proposal submission:** Proposal is submitted to the blockchain with a + deposit. +* **Vote:** Once deposit reaches a certain value (`MinDeposit`), proposal is + confirmed and vote opens. Bonded Atom holders can then send `TxGovVote` + transactions to vote on the proposal. +* **Execution** After a period of time, the votes are tallied and depending + on the result, the messages in the proposal will be executed. + +### Proposal submission + +#### Right to submit a proposal + +Every account can submit proposals by sending a `MsgSubmitProposal` transaction. +Once a proposal is submitted, it is identified by its unique `proposalID`. + +#### Proposal Messages + +A proposal includes an array of `sdk.Msg`s which are executed automatically if the +proposal passes. The messages are executed by the governance `ModuleAccount` itself. Modules +such as `x/upgrade`, that want to allow certain messages to be executed by governance +only should add a whitelist within the respective msg server, granting the governance +module the right to execute the message once a quorum has been reached. The governance +module uses the core `router.Service` to check that these messages are correctly constructed +and have a respective path to execute on but do not perform a full validity check. + +:::warning +Ultimately, governance is able to execute any proposal, even if they weren't meant to be executed by governance (ie. no authority present). +Messages without authority are messages meant to be executed by users. Using the `MsgSudoExec` message in a proposal, let governance be able to execute any message, effectively acting as super user. +::: + +### Deposit + +To prevent spam, proposals must be submitted with a deposit in the coins defined by +the `MinDeposit` param. + +When a proposal is submitted, it has to be accompanied with a deposit that must be +strictly positive, but can be inferior to `MinDeposit`. The submitter doesn't need +to pay for the entire deposit on their own. The newly created proposal is stored in +an *inactive proposal queue* and stays there until its deposit passes the `MinDeposit`. +Other token holders can increase the proposal's deposit by sending a `Deposit` +transaction. If a proposal doesn't pass the `MinDeposit` before the deposit end time +(the time when deposits are no longer accepted), the proposal will be destroyed: the +proposal will be removed from state and the deposit will be burned (see x/gov `EndBlocker`). +When a proposal deposit passes the `MinDeposit` threshold (even during the proposal +submission) before the deposit end time, the proposal will be moved into the +*active proposal queue* and the voting period will begin. + +The deposit is kept in escrow and held by the governance `ModuleAccount` until the +proposal is finalized (passed or rejected). + +#### Deposit refund and burn + +When a proposal is finalized, the coins from the deposit are either refunded or burned +according to the final tally of the proposal and the governance module parameters: + +* All refunded or burned deposits are removed from the state. Events are issued when + burning or refunding a deposit. +* If the proposal is approved or rejected but *not* vetoed, each deposit will be + automatically refunded to its respective depositor (transferred from the governance + `ModuleAccount`). +* If the proposal is marked as Spam, the deposit will be burned. + +For other cases, they are three parameters that define if the deposit of a proposal should be burned or returned to the depositors. + +* `BurnVoteVeto` burns the proposal deposit if the proposal gets vetoed. +* `BurnVoteQuorum` burns the proposal deposit if the vote does not reach quorum. +* `BurnProposalDepositPrevote` burns the proposal deposit if it does not enter the voting phase. + +> Note: These parameters are modifiable via governance. + +### Vote + +#### Participants + +*Participants* are users that have the right to vote on proposals. On the +Cosmos Hub, participants are bonded Atom holders. Unbonded Atom holders and +other users do not get the right to participate in governance. However, they +can submit and deposit on proposals. + +Note that when *participants* have bonded and unbonded Atoms, their voting power is calculated from their bonded Atom holdings only. + +#### Voting period + +Once a proposal reaches `MinDeposit`, it immediately enters `Voting period`. We +define `Voting period` as the interval between the moment the vote opens and +the moment the vote closes. The default value of `Voting period` is 2 weeks but is modifiable at genesis or governance. + +#### Option set + +The option set of a proposal refers to the set of choices a participant can +choose from when casting its vote. + +The initial option set includes the following options: + +* `Yes` / `Option 1` +* `Abstain` / `Option 2` +* `No` / `Option 3` +* `NoWithVeto` / `Option 4` +* `Spam` / `Option Spam` + +`NoWithVeto` counts as `No` but also adds a `Veto` vote. `Abstain` option +allows voters to signal that they do not intend to vote in favor or against the +proposal but accept the result of the vote. + +#### Weighted Votes + +[ADR-037](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-037-gov-split-vote.md) introduces the weighted vote feature which allows a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No. + +Often times the entity owning that address might not be a single individual. For example, a company might have different stakeholders who want to vote differently, and so it makes sense to allow them to split their voting power. Currently, it is not possible for them to do "passthrough voting" and giving their users voting rights over their tokens. However, with this system, exchanges can poll their users for voting preferences, and then vote on-chain proportionally to the results of the poll. + +To represent weighted vote on chain, we use the following Protobuf message. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/gov.proto#L56-L63 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/gov.proto#L202-L219 +``` + +For a weighted vote to be valid, the `options` field must not contain duplicate vote options, and the sum of weights of all options must be equal to 1. + +The maximum number of weighted vote options can be limited by the developer via a config parameter, named `MaxVoteOptionsLen`, which gets passed into the gov keeper. + +### Quorum + +Quorum is defined as the minimum percentage of voting power that needs to be +cast on a proposal for the result to be valid. + +### Expedited Quorum + +Expedited Quorum is defined as the minimum percentage of voting power that needs to be +cast on an **expedited** proposal for the result to be valid. + +### Yes Quorum + +Yes quorum is a more restrictive quorum that is used to determine if a proposal passes. +It is defined as the minimum percentage of voting power that needs to have voted `Yes` for the proposal to pass. +It differs from `Threshold` as it takes the whole voting power into account, not only `Yes` and `No` votes. +By default, `YesQuorum` is set to 0, which means no minimum. + +### Proposal Types + +Proposal types have been introduced in [ADR-069](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-069-gov-improvements.md). + +#### Standard proposal + +A standard proposal is a proposal that can contain any messages. The proposal follows the standard governance flow and governance parameters. + +#### Expedited Proposal + +A proposal can be expedited, making the proposal use shorter voting duration and a higher tally threshold by its default. If an expedited proposal fails to meet the threshold within the scope of shorter voting duration, the expedited proposal is then converted to a regular proposal and restarts voting under regular voting conditions. + +#### Optimistic Proposal + +An optimistic proposal is a proposal that passes unless a threshold of NO votes is reached. +Voter can only vote NO on the proposal. If the NO threshold is reached, the optimistic proposal is converted to a standard proposal. + +That threshold is defined by the `optimistic_rejected_threshold` governance parameter. +A chain can optionally set a list of authorized addresses that can submit optimistic proposals using the `optimistic_authorized_addresses` governance parameter. + +#### Multiple Choice Proposals + +A multiple choice proposal is a proposal where the voting options can be defined by the proposer. +The number of voting options is limited to a maximum of **4**. +Multiple choice proposals, contrary to any other proposal type, cannot have messages to execute. They are only text proposals. + +### Threshold + +Threshold is defined as the minimum proportion of `Yes` votes (excluding `Abstain` votes) for the proposal to be accepted. + +Initially, the threshold is set at 50% of `Yes` votes, excluding `Abstain` votes. +A possibility to veto exists if more than 1/3rd of all votes are `NoWithVeto` votes. +Note, both of these values are derived from the `Params` +on-chain parameter, which is modifiable by governance. +This means that proposals are accepted iff: + +* There exist bonded tokens. +* Quorum has been achieved. +* The proportion of `Abstain` votes is inferior to 1/1. +* The proportion of `NoWithVeto` votes is inferior to 1/3, including + `Abstain` votes. +* The proportion of `Yes` votes, excluding `Abstain` votes, at the end of + the voting period is superior to 1/2. + +For expedited proposals, by default, the threshold is higher than with a *normal proposal*, namely, 66.7%. + +### Inheritance + +If a delegator does not vote, by default, it will inherit its validator vote. + +* If the delegator votes before its validator, it will not inherit from the validator's vote. +* If the delegator votes after its validator, it will override its validator vote with its own. + If the proposal is urgent, it is possible that the vote will close before delegators have a chance to react and + override their validator's vote. This is not a problem, as proposals require more than 2/3rd of the total voting power to pass, when tallied at the end of the voting period. Because as little as 1/3 + 1 validation power could collude to censor transactions, non-collusion is already assumed for ranges exceeding this threshold. + +This behavior can be changed by passing a custom tally calculation function to the governance module. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/keeper/config.go#L33-L35 +``` + +#### Validator’s punishment for non-voting + +At present, validators are not punished for failing to vote. + +#### Execution + +Execution is the process of executing the messages contained in a proposal. The execution phase will commence after the proposal has been accepted by the network. The messages contained in the proposal will be executed in the order they were submitted. All messages must be executed successfully for the proposal to be considered successful. I + +If a proposal passes but fails to execute, the proposal will be marked as `StatusFailed`. This status is different from `StatusRejected`, which is used when a proposal fails to pass. + +Execution has an upper limit on how much gas can be consumed in a single block. This limit is defined by the `ProposalExecutionGas` parameter. + +## State + +The governance module uses [collections](https://docs.cosmos.network/v0.50/build/packages/collections) for state management. + +### Constitution + +`Constitution` is found in the genesis state. It is a string field intended to be used to describe the purpose of a particular blockchain, and its expected norms. A few examples of how the constitution field can be used: + +* define the purpose of the chain, laying a foundation for its future development +* set expectations for delegators +* set expectations for validators +* define the chain's relationship to "meatspace" entities, like a foundation or corporation + +Since this is more of a social feature than a technical feature, we'll now get into some items that may have been useful to have in a genesis constitution: + +* What limitations on governance exist, if any? + * is it okay for the community to slash the wallet of a whale that they no longer feel that they want around? (viz: Juno Proposal 4 and 16) + * can governance "socially slash" a validator who is using unapproved MEV? (viz: commonwealth.im/osmosis) + * In the event of an economic emergency, what should validators do? + * Terra crash of May, 2022, saw validators choose to run a new binary with code that had not been approved by governance, because the governance token had been inflated to nothing. +* What is the purpose of the chain, specifically? + * best example of this is the Cosmos hub, where different founding groups, have different interpretations of the purpose of the network. + +This genesis entry, "constitution" hasn't been designed for existing chains, who should likely just ratify a constitution using their governance system. Instead, this is for new chains. It will allow for validators to have a much clearer idea of purpose and the expectations placed on them while operating their nodes. Likewise, for community members, the constitution will give them some idea of what to expect from both the "chain team" and the validators, respectively. + +This constitution is designed to be immutable, and placed only in genesis, though that could change over time by a pull request to the cosmos-sdk that allows for the constitution to be changed by governance. Communities wishing to make amendments to their original constitution should use the governance mechanism and a "signaling proposal" to do exactly that. + +**Ideal use scenario for a cosmos chain constitution** + +As a chain developer, you decide that you'd like to provide clarity to your key user groups: + +* validators +* token holders +* developers (yourself) + +You use the constitution to immutably store some Markdown in genesis, so that when difficult questions come up, the constitution can provide guidance to the community. + +### Proposals + +`Proposal` objects are used to tally votes and generally track the proposal's state. +They contain an array of arbitrary `sdk.Msg`'s which the governance module will attempt +to resolve and then execute if the proposal passes. `Proposal`'s are identified by a +unique id and contains a series of timestamps: `submit_time`, `deposit_end_time`, +`voting_start_time`, `voting_end_time` which track the lifecycle of a proposal + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/gov.proto#L78-L134 +``` + +A proposal will generally require more than just a set of messages to explain its +purpose but need some greater justification and allow a means for interested participants +to discuss and debate the proposal. +In most cases, **it is encouraged to have an off-chain system that supports the on-chain governance process**. +To accommodate for this, a proposal contains a special **`metadata`** field, a string, +which can be used to add context to the proposal. The `metadata` field allows custom use for networks, +however, it is expected that the field contains a URL or some form of CID using a system such as +[IPFS](https://docs.ipfs.io/concepts/content-addressing/). To support the case of +interoperability across networks, the SDK recommends that the `metadata` represents +the following `JSON` template: + +```json +{ + "title": "...", + "description": "...", + "forum": "...", // a link to the discussion platform (i.e. Discord) + "other": "..." // any extra data that doesn't correspond to the other fields +} +``` + +This makes it far easier for clients to support multiple networks. + +Fields metadata, title and summary have a maximum length that is chosen by the app developer, and +passed into the gov keeper as a config (`x/gov/keeper/config`). + +The default maximum length are: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/keeper/config.go#L38-L47 +``` + +#### Writing a module that uses governance + +There are many aspects of a chain, or of the individual modules that you may want to +use governance to perform such as changing various parameters. This is very simple +to do. First, write out your message types and `MsgServer` implementation. Add an +`authority` field to the keeper which will be populated in the constructor with the +governance module account: `govKeeper.GetGovernanceAccount().GetAddress()`. Then for +the methods in the `msg_server.go`, perform a check on the message that the signer +matches `authority`. This will prevent any user from executing that message. + +:::warning +Note, any message can be executed by governance if embedded in `MsgSudoExec`. +::: + +### Parameters and base types + +`Params` define the rules according to which votes are run. If governance wants to change a +parameter it can do so by submitting a gov `MsgUpdateParams` governance proposal. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/gov.proto#L259-L348 +``` + +Parameters are stored in the `gov` store under the key `ParamsKey`. + +Additionally, we introduce some basic types: + +```go +type ProposalStatus byte + +const ( + StatusNil ProposalStatus = 0x00 + StatusDepositPeriod ProposalStatus = 0x01 // Proposal is submitted. Participants can deposit on it but not vote + StatusVotingPeriod ProposalStatus = 0x02 // MinDeposit is reached, participants can vote + StatusPassed ProposalStatus = 0x03 // Proposal passed and successfully executed + StatusRejected ProposalStatus = 0x04 // Proposal has been rejected + StatusFailed ProposalStatus = 0x05 // Proposal passed but failed execution +) +``` + +### Deposit + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/gov.proto#L65-L76 +``` + +### ValidatorGovInfo + +This type is used in a temp map when tallying + +```go + type ValidatorGovInfo struct { + Minus sdk.Dec + Vote Vote + } +``` + +### Legacy Proposal + +:::warning +Legacy proposals (`gov/v1beta1`) are deprecated. Use the new proposal flow by granting the governance module the right to execute the message. +::: + +## Messages + +### Proposal Submission + +Proposals can be submitted by any account via a `MsgSubmitProposal` or a `MsgSubmitMultipleChoiceProposal` transaction. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/tx.proto#L64-L102 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/tx.proto#L229-L256 +``` + +:::tip +A multiple choice proposal is a proposal where the voting options can be defined by the proposer. +It cannot have messages to execute. It is only a text proposal. +This means submitting a multiple choice proposal using `MsgSubmitProposal` is invalid, as vote options cannot be defined. +::: + +All `sdk.Msgs` passed into the `messages` field of a `MsgSubmitProposal` message +must be registered in the app's message router. Each of these messages must +have one signer, namely the gov module account. And finally, the metadata length +must not be larger than the `maxMetadataLen` config passed into the gov keeper. +The `initialDeposit` must be strictly positive and conform to the accepted denom of the `MinDeposit` param. + +### Deposit + +Once a proposal is submitted, if `Proposal.TotalDeposit < GovParams.MinDeposit`, Atom holders can send +`MsgDeposit` transactions to increase the proposal's deposit. + +A deposit is accepted iff: + +* The proposal exists +* The proposal is not in the voting period +* The deposited coins are conform to the accepted denom from the `MinDeposit` param + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/tx.proto#L167-L180 +``` + +### Vote + +Once `GovParams.MinDeposit` is reached, voting period starts. From there, +bonded Atom holders are able to send `MsgVote` transactions to cast their +vote on the proposal. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/proto/cosmos/gov/v1/tx.proto#L125-L141 +``` + +## Events + +The governance module emits the following events: + +### EndBlocker + +| Type | Attribute Key | Attribute Value | +| ----------------- | --------------- | ---------------- | +| inactive_proposal | proposal_id | {proposalID} | +| inactive_proposal | proposal_result | {proposalResult} | +| active_proposal | proposal_id | {proposalID} | +| active_proposal | proposal_result | {proposalResult} | + +### Handlers + +#### MsgSubmitProposal, MsgSubmitMultipleChoiceProposal + +| Type | Attribute Key | Attribute Value | +| ------------------- | ------------------- | --------------- | +| submit_proposal | proposal_id | {proposalID} | +| submit_proposal [0] | voting_period_start | {proposalID} | +| proposal_deposit | amount | {depositAmount} | +| proposal_deposit | proposal_id | {proposalID} | +| message | module | governance | +| message | action | submit_proposal | +| message | sender | {senderAddress} | + +* [0] Event only emitted if the voting period starts during the submission. + +#### MsgVote + +| Type | Attribute Key | Attribute Value | +| ------------- | ------------- | --------------- | +| proposal_vote | option | {voteOption} | +| proposal_vote | proposal_id | {proposalID} | +| message | module | governance | +| message | action | vote | +| message | sender | {senderAddress} | + +#### MsgVoteWeighted + +| Type | Attribute Key | Attribute Value | +| ------------- | ------------- | --------------------- | +| proposal_vote | option | {weightedVoteOptions} | +| proposal_vote | proposal_id | {proposalID} | +| message | module | governance | +| message | action | vote | +| message | sender | {senderAddress} | + +#### MsgDeposit + +| Type | Attribute Key | Attribute Value | +| -------------------- | ------------------- | --------------- | +| proposal_deposit | amount | {depositAmount} | +| proposal_deposit | proposal_id | {proposalID} | +| proposal_deposit [0] | voting_period_start | {proposalID} | +| message | module | governance | +| message | action | deposit | +| message | sender | {senderAddress} | + +* [0] Event only emitted if the voting period starts during the submission. + +## Parameters + +The governance module contains the following parameters: + +| Key | Type | Example | +| ------------------------------- | ----------------- | --------------------------------------- | +| min_deposit | array (coins) | [{"denom":"uatom","amount":"10000000"}] | +| max_deposit_period | string (time ns) | "172800000000000" (17280s) | +| voting_period | string (time ns) | "172800000000000" (17280s) | +| quorum | string (dec) | "0.334000000000000000" | +| yes_quorum | string (dec) | "0.4" | +| threshold | string (dec) | "0.500000000000000000" | +| veto | string (dec) | "0.334000000000000000" | +| expedited_threshold | string (time ns) | "0.667000000000000000" | +| expedited_voting_period | string (time ns) | "86400000000000" (8600s) | +| expedited_min_deposit | array (coins) | [{"denom":"uatom","amount":"50000000"}] | +| expedited_quorum | string (dec) | "0.5" | +| burn_proposal_deposit_prevote | bool | false | +| burn_vote_quorum | bool | false | +| burn_vote_veto | bool | true | +| min_initial_deposit_ratio | string | "0.1" | +| proposal_cancel_ratio | string (dec) | "0.5" | +| proposal_cancel_dest | string (address) | "cosmos1.." or empty for burn | +| proposal_cancel_max_period | string (dec) | "0.5" | +| optimistic_rejected_threshold | string (dec) | "0.1" | +| optimistic_authorized_addresses | array (addresses) | [] | + +**NOTE**: The governance module contains parameters that are objects unlike other +modules. If only a subset of parameters are desired to be changed, only they need +to be included and not the entire parameter object structure. + +### Message Based Parameters + +In addition to the parameters above, the governance module can also be configured to have different parameters for a given proposal message. + +| Key | Type | Example | +| ------------- | ---------------- | -------------------------- | +| voting_period | string (time ns) | "172800000000000" (17280s) | +| yes_quorum | string (dec) | "0.4" | +| quorum | string (dec) | "0.334000000000000000" | +| threshold | string (dec) | "0.500000000000000000" | +| veto | string (dec) | "0.334000000000000000" | + +If configured, these params will take precedence over the global params for a specific proposal. + +:::warning +Currently, messaged based parameters limit the number of messages that can be included in a proposal. +Only messages that have the same message parameters can be included in the same proposal. +::: + +## Metadata + +The gov module has two locations for metadata where users can provide further context about the on-chain actions they are taking. By default all metadata fields have a 255 character length field where metadata can be stored in json format, either on-chain or off-chain depending on the amount of data required. Here we provide a recommendation for the json structure and where the data should be stored. There are two important factors in making these recommendations. First, that the gov and group modules are consistent with one another, note the number of proposals made by all groups may be quite large. Second, that client applications such as block explorers and governance interfaces have confidence in the consistency of metadata structure across chains. + +### Proposal + +Location: off-chain as json object stored on IPFS (mirrors [group proposal](../group/README.md#metadata)) + +```json +{ + "title": "", + "authors": [""], + "summary": "", + "details": "", + "proposal_forum_url": "", + "vote_option_context": "", +} +``` + +:::note +The `authors` field is an array of strings, this is to allow for multiple authors to be listed in the metadata. +In v0.46, the `authors` field is a comma-separated string. Frontends are encouraged to support both formats for backwards compatibility. +::: + +### Vote + +Location: on-chain as json within 255 character limit (mirrors [group vote](../group/README.md#metadata)) + +```json +{ + "justification": "", +} +``` + +## Client + +### CLI + +A user can query and interact with the `gov` module using the CLI. + +#### Query + +The `query` commands allow users to query `gov` state. + +```bash +simd query gov --help +``` + +##### deposit + +The `deposit` command allows users to query a deposit for a given proposal from a given depositor. + +```bash +simd query gov deposit [proposal-id] [depositor-addr] [flags] +``` + +Example: + +```bash +simd query gov deposit 1 cosmos1.. +``` + +Example Output: + +```bash +amount: +- amount: "100" + denom: stake +depositor: cosmos1.. +proposal_id: "1" +``` + +##### deposits + +The `deposits` command allows users to query all deposits for a given proposal. + +```bash +simd query gov deposits [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov deposits 1 +``` + +Example Output: + +```bash +deposits: +- amount: + - amount: "100" + denom: stake + depositor: cosmos1.. + proposal_id: "1" +pagination: + next_key: null + total: "0" +``` + +##### params + +The `params` command allows users to query all parameters for the `gov` module. + +```bash +simd query gov params [flags] +``` + +Example: + +```bash +simd query gov params +``` + +Example Output: + +```bash +params: + expedited_min_deposit: + - amount: "50000000" + denom: stake + expedited_threshold: "0.670000000000000000" + expedited_voting_period: 86400s + max_deposit_period: 172800s + min_deposit: + - amount: "10000000" + denom: stake + min_initial_deposit_ratio: "0.000000000000000000" + proposal_cancel_burn_rate: "0.500000000000000000" + quorum: "0.334000000000000000" + threshold: "0.500000000000000000" + veto_threshold: "0.334000000000000000" + voting_period: 172800s +``` + +##### proposal + +The `proposal` command allows users to query a given proposal. + +```bash +simd query gov proposal [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov proposal 1 +``` + +Example Output: + +```bash +deposit_end_time: "2022-03-30T11:50:20.819676256Z" +final_tally_result: + abstain_count: "0" + no_count: "0" + no_with_veto_count: "0" + yes_count: "0" +id: "1" +messages: +- '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "10" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. +metadata: AQ== +status: PROPOSAL_STATUS_DEPOSIT_PERIOD +submit_time: "2022-03-28T11:50:20.819676256Z" +total_deposit: +- amount: "10" + denom: stake +voting_end_time: null +voting_start_time: null +``` + +##### proposals + +The `proposals` command allows users to query all proposals with optional filters. + +```bash +simd query gov proposals [flags] +``` + +Example: + +```bash +simd query gov proposals +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "0" +proposals: +- deposit_end_time: "2022-03-30T11:50:20.819676256Z" + final_tally_result: + abstain_count: "0" + no_count: "0" + no_with_veto_count: "0" + yes_count: "0" + id: "1" + messages: + - '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "10" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. + metadata: AQ== + status: PROPOSAL_STATUS_DEPOSIT_PERIOD + submit_time: "2022-03-28T11:50:20.819676256Z" + total_deposit: + - amount: "10" + denom: stake + voting_end_time: null + voting_start_time: null +- deposit_end_time: "2022-03-30T14:02:41.165025015Z" + final_tally_result: + abstain_count: "0" + no_count: "0" + no_with_veto_count: "0" + yes_count: "0" + id: "2" + messages: + - '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "10" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. + metadata: AQ== + status: PROPOSAL_STATUS_DEPOSIT_PERIOD + submit_time: "2022-03-28T14:02:41.165025015Z" + total_deposit: + - amount: "10" + denom: stake + voting_end_time: null + voting_start_time: null +``` + +##### proposer + +The `proposer` command allows users to query the proposer for a given proposal. + +```bash +simd query gov proposer [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov proposer 1 +``` + +Example Output: + +```bash +proposal_id: "1" +proposer: cosmos1.. +``` + +##### tally + +The `tally` command allows users to query the tally of a given proposal vote. + +```bash +simd query gov tally [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov tally 1 +``` + +Example Output: + +```bash +abstain: "0" +"no": "0" +no_with_veto: "0" +"yes": "1" +``` + +##### vote + +The `vote` command allows users to query a vote for a given proposal. + +```bash +simd query gov vote [proposal-id] [voter-addr] [flags] +``` + +Example: + +```bash +simd query gov vote 1 cosmos1.. +``` + +Example Output: + +```bash +option: VOTE_OPTION_YES +options: +- option: VOTE_OPTION_YES + weight: "1.000000000000000000" +proposal_id: "1" +voter: cosmos1.. +``` + +##### votes + +The `votes` command allows users to query all votes for a given proposal. + +```bash +simd query gov votes [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov votes 1 +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "0" +votes: +- option: VOTE_OPTION_YES + options: + - option: VOTE_OPTION_YES + weight: "1.000000000000000000" + proposal_id: "1" + voter: cosmos1.. +``` + +#### Transactions + +The `tx` commands allow users to interact with the `gov` module. + +```bash +simd tx gov --help +``` + +##### deposit + +The `deposit` command allows users to deposit tokens for a given proposal. + +```bash +simd tx gov deposit [proposal-id] [deposit] [flags] +``` + +Example: + +```bash +simd tx gov deposit 1 10000000stake --from cosmos1.. +``` + +##### draft-proposal + +The `draft-proposal` command allows users to draft any type of proposal. +The command returns a `draft_proposal.json`, to be used by `submit-proposal` after being completed. +The `draft_metadata.json` is meant to be uploaded to [IPFS](#metadata). + +```bash +simd tx gov draft-proposal +``` + +##### submit-proposal + +The `submit-proposal` command allows users to submit a governance proposal along with some messages and metadata. +Messages, metadata and deposit are defined in a JSON file. + +```bash +simd tx gov submit-proposal [path-to-proposal-json] [flags] +``` + +Example: + +```bash +simd tx gov submit-proposal /path/to/proposal.json --from cosmos1.. +``` + +where `proposal.json` contains: + +```json +{ + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1...", // The gov module module address + "to_address": "cosmos1...", + "amount":[{"denom": "stake","amount": "10"}] + } + ], + "metadata": "AQ==", + "deposit": "10stake", + "title": "Proposal Title", + "summary": "Proposal Summary" +} +``` + +:::note +By default the metadata, summary and title are both limited by 255 characters, this can be overridden by the application developer. +::: + +:::tip +When metadata is not specified, the title is limited to 255 characters and the summary 40x the title length. +::: + +##### cancel-proposal + +Once proposal is canceled, from the deposits of proposal `deposits * proposal_cancel_ratio` will be burned or sent to `ProposalCancelDest` address , if `ProposalCancelDest` is empty then deposits will be burned. The `remaining deposits` will be sent to depositors. + +```bash +simd tx gov cancel-proposal [proposal-id] [flags] +``` + +Example: + +```bash +simd tx gov cancel-proposal 1 --from cosmos1... +``` + +##### vote + +The `vote` command allows users to submit a vote for a given governance proposal. + +```bash +simd tx gov vote [command] [flags] +``` + +Example: + +```bash +simd tx gov vote 1 yes --from cosmos1.. +``` + +##### weighted-vote + +The `weighted-vote` command allows users to submit a weighted vote for a given governance proposal. + +```bash +simd tx gov weighted-vote [proposal-id] [weighted-options] [flags] +``` + +Example: + +```bash +simd tx gov weighted-vote 1 yes=0.5,no=0.5 --from cosmos1.. +``` + +### gRPC + +A user can query the `gov` module using gRPC endpoints. + +#### Proposal + +The `Proposal` endpoint allows users to query a given proposal. + +```bash +cosmos.gov.v1.Query/Proposal +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Proposal +``` + +#### Proposals + +The `Proposals` endpoint allows users to query all proposals with optional filters. + +```bash +cosmos.gov.v1.Query/Proposals +``` + +Example: + +```bash +grpcurl -plaintext \ + localhost:9090 \ + cosmos.gov.v1.Query/Proposals +``` + +#### Vote + +The `Vote` endpoint allows users to query a vote for a given proposal. + +```bash +cosmos.gov.v1.Query/Vote +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1","voter":"cosmos1.."}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Vote +``` + +#### Votes + +The `Votes` endpoint allows users to query all votes for a given proposal. + +```bash +cosmos.gov.v1.Query/Votes +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Votes +``` + +#### Params + +The `Params` endpoint allows users to query all parameters for the `gov` module. + +```bash +cosmos.gov.v1.Query/Params +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"params_type":"voting"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Params +``` + +#### Deposit + +The `Deposit` endpoint allows users to query a deposit for a given proposal from a given depositor. + +```bash +cosmos.gov.v1.Query/Deposit +``` + +Example: + +```bash +grpcurl -plaintext \ + '{"proposal_id":"1","depositor":"cosmos1.."}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Deposit +``` + +#### deposits + +The `Deposits` endpoint allows users to query all deposits for a given proposal. + +```bash +cosmos.gov.v1.Query/Deposits +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Deposits +``` + +#### TallyResult + +The `TallyResult` endpoint allows users to query the tally of a given proposal. + +```bash +cosmos.gov.v1.Query/TallyResult +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/TallyResult +``` + +### REST + +A user can query the `gov` module using REST endpoints. + +#### proposal + +The `proposals` endpoint allows users to query a given proposal. + +```bash +/cosmos/gov/v1/proposals/{proposal_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1 +``` + +#### proposals + +The `proposals` endpoint also allows users to query all proposals with optional filters. + + +```bash +/cosmos/gov/v1/proposals +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals +``` + +#### voter vote + +The `votes` endpoint allows users to query a vote for a given proposal. + + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/votes/{voter} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/votes/cosmos1.. +``` + +#### votes + +The `votes` endpoint allows users to query all votes for a given proposal. + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/votes +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/votes +``` + +#### params + +The `params` endpoint allows users to query all parameters for the `gov` module. + +```bash +/cosmos/gov/v1/params/{params_type} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/params/voting +``` + +Note: `params_type` are deprecated in v1 since all params are stored in Params. + +#### deposits + +The `deposits` endpoint allows users to query a deposit for a given proposal from a given depositor. + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/deposits/{depositor} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/deposits/cosmos1.. +``` + +#### proposal deposits + +The `deposits` endpoint allows users to query all deposits for a given proposal. + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/deposits +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/deposits +``` + +#### tally + +The `tally` endpoint allows users to query the tally of a given proposal. + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/tally +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/tally +``` diff --git a/versioned_docs/version-0.52/build/modules/group/README.md b/versioned_docs/version-0.52/build/modules/group/README.md new file mode 100644 index 000000000..a6e703bda --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/group/README.md @@ -0,0 +1,2166 @@ +--- +sidebar_position: 1 +--- + +# `x/group` + +## Abstract + +The following documents specify the group module. + +This module allows the creation and management of on-chain multisig accounts and enables voting for message execution based on configurable decision policies. + +## Contents + +* [Concepts](#concepts) + * [Group](#group) + * [Group Policy](#group-policy) + * [Decision Policy](#decision-policy) + * [Proposal](#proposal) + * [Pruning](#pruning) +* [State](#state) + * [Group Table](#group-table) + * [Group Member Table](#group-member-table) + * [Group Policy Table](#group-policy-table) + * [Proposal Table](#proposal-table) + * [Vote Table](#vote-table) +* [Msg Service](#msg-service) + * [Msg/CreateGroup](#msgcreategroup) + * [Msg/UpdateGroupMembers](#msgupdategroupmembers) + * [Msg/UpdateGroupAdmin](#msgupdategroupadmin) + * [Msg/UpdateGroupMetadata](#msgupdategroupmetadata) + * [Msg/CreateGroupPolicy](#msgcreategrouppolicy) + * [Msg/CreateGroupWithPolicy](#msgcreategroupwithpolicy) + * [Msg/UpdateGroupPolicyAdmin](#msgupdategrouppolicyadmin) + * [Msg/UpdateGroupPolicyDecisionPolicy](#msgupdategrouppolicydecisionpolicy) + * [Msg/UpdateGroupPolicyMetadata](#msgupdategrouppolicymetadata) + * [Msg/SubmitProposal](#msgsubmitproposal) + * [Msg/WithdrawProposal](#msgwithdrawproposal) + * [Msg/Vote](#msgvote) + * [Msg/Exec](#msgexec) + * [Msg/LeaveGroup](#msgleavegroup) +* [Events](#events) + * [EventCreateGroup](#eventcreategroup) + * [EventUpdateGroup](#eventupdategroup) + * [EventCreateGroupPolicy](#eventcreategrouppolicy) + * [EventUpdateGroupPolicy](#eventupdategrouppolicy) + * [EventCreateProposal](#eventcreateproposal) + * [EventWithdrawProposal](#eventwithdrawproposal) + * [EventVote](#eventvote) + * [EventExec](#eventexec) + * [EventLeaveGroup](#eventleavegroup) + * [EventProposalPruned](#eventproposalpruned) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) +* [Metadata](#metadata) + +## Concepts + +### Group + +A group is simply an aggregation of accounts with associated weights. It is not +an account and doesn't have a balance. It doesn't in and of itself have any +sort of voting or decision weight. It does have an "administrator" which has +the ability to add, remove and update members in the group. Note that a +group policy account could be an administrator of a group, and that the +administrator doesn't necessarily have to be a member of the group. + +### Group Policy + +A group policy is an account associated with a group and a decision policy. +Group policies are abstracted from groups because a single group may have +multiple decision policies for different types of actions. Managing group +membership separately from decision policies results in the least overhead +and keeps membership consistent across different policies. The pattern that +is recommended is to have a single master group policy for a given group, +and then to create separate group policies with different decision policies +and delegate the desired permissions from the master account to +those "sub-accounts" using the `x/authz` module. + +### Decision Policy + +A decision policy is the mechanism by which members of a group can vote on +proposals, as well as the rules that dictate whether a proposal should pass +or not based on its tally outcome. + +All decision policies generally would have a minimum execution period and a +maximum voting window. The minimum execution period is the minimum amount of time +that must pass after submission in order for a proposal to potentially be executed, and it may +be set to 0. The maximum voting window is the maximum time after submission that a proposal may +be voted on before it is tallied. + +The chain developer also defines an app-wide maximum execution period, which is +the maximum amount of time after a proposal's voting period end where users are +allowed to execute a proposal. + +The current group module comes shipped with two decision policies: threshold +and percentage. Any chain developer can extend upon these two, by creating +custom decision policies, as long as they adhere to the `DecisionPolicy` +interface: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/group/types.go#L27-L45 +``` + +#### Threshold decision policy + +A threshold decision policy defines a threshold of yes votes (based on a tally +of voter weights) that must be achieved in order for a proposal to pass. For +this decision policy, abstain and veto are simply treated as no's. + +This decision policy also has a VotingPeriod window and a MinExecutionPeriod +window. The former defines the duration after proposal submission where members +are allowed to vote, after which tallying is performed. The latter specifies +the minimum duration after proposal submission where the proposal can be +executed. If set to 0, then the proposal is allowed to be executed immediately +on submission (using the `TRY_EXEC` option). Obviously, MinExecutionPeriod +cannot be greater than VotingPeriod+MaxExecutionPeriod (where MaxExecution is +the app-defined duration that specifies the window after voting ended where a +proposal can be executed). + +#### Percentage decision policy + +A percentage decision policy is similar to a threshold decision policy, except +that the threshold is not defined as a constant weight, but as a percentage. +It's more suited for groups where the group members' weights can be updated, as +the percentage threshold stays the same, and doesn't depend on how those member +weights get updated. + +Same as the Threshold decision policy, the percentage decision policy has the +two VotingPeriod and MinExecutionPeriod parameters. + +### Proposal + +Any member(s) of a group can submit a proposal for a group policy account to decide upon. +A proposal consists of a set of messages that will be executed if the proposal +passes as well as any metadata associated with the proposal. + +#### Voting + +There are four choices to choose while voting - yes, no, abstain and veto. Not +all decision policies will take the four choices into account. Votes can contain some optional metadata. +In the current implementation, the voting window begins as soon as a proposal +is submitted, and the end is defined by the group policy's decision policy. + +#### Withdrawing Proposals + +Proposals can be withdrawn any time before the voting period end, either by the +admin of the group policy or by one of the proposers. Once withdrawn, it is +marked as `PROPOSAL_STATUS_WITHDRAWN`, and no more voting or execution is +allowed on it. + +#### Aborted Proposals + +If the group policy is updated during the voting period of the proposal, then +the proposal is marked as `PROPOSAL_STATUS_ABORTED`, and no more voting or +execution is allowed on it. This is because the group policy defines the rules +of proposal voting and execution, so if those rules change during the lifecycle +of a proposal, then the proposal should be marked as stale. + +#### Tallying + +Tallying is the counting of all votes on a proposal. It happens only once in +the lifecycle of a proposal, but can be triggered by two factors, whichever +happens first: + +* either someone tries to execute the proposal (see next section), which can + happen on a `Msg/Exec` transaction, or a `Msg/{SubmitProposal,Vote}` + transaction with the `Exec` field set. When a proposal execution is attempted, + a tally is done first to make sure the proposal passes. +* or on `EndBlock` when the proposal's voting period end just passed. + +If the tally result passes the decision policy's rules, then the proposal is +marked as `PROPOSAL_STATUS_ACCEPTED`, or else it is marked as +`PROPOSAL_STATUS_REJECTED`. In any case, no more voting is allowed anymore, and the tally +result is persisted to state in the proposal's `FinalTallyResult`. + +#### Executing Proposals + +Proposals are executed only when the tallying is done, and the group account's +decision policy allows the proposal to pass based on the tally outcome. They +are marked by the status `PROPOSAL_STATUS_ACCEPTED`. Execution must happen +before a duration of `MaxExecutionPeriod` (set by the chain developer) after +each proposal's voting period end. + +Proposals will not be automatically executed by the chain in this current design, +but rather a user must submit a `Msg/Exec` transaction to attempt to execute the +proposal based on the current votes and decision policy. Any user (not only the +group members) can execute proposals that have been accepted, and execution fees are +paid by the proposal executor. +It's also possible to try to execute a proposal immediately on creation or on +new votes using the `Exec` field of `Msg/SubmitProposal` and `Msg/Vote` requests. +In the former case, proposers signatures are considered as yes votes. +In these cases, if the proposal can't be executed (i.e. it didn't pass the +decision policy's rules), it will still be opened for new votes and +could be tallied and executed later on. + +A successful proposal execution will have its `ExecutorResult` marked as +`PROPOSAL_EXECUTOR_RESULT_SUCCESS`. The proposal will be automatically pruned +after execution. On the other hand, a failed proposal execution will be marked +as `PROPOSAL_EXECUTOR_RESULT_FAILURE`. Such a proposal can be re-executed +multiple times, until it expires after `MaxExecutionPeriod` after voting period +end. + +### Pruning + +Proposals and votes are automatically pruned to avoid state bloat. + +Votes are pruned: + +* either after a successful tally, i.e. a tally whose result passes the decision + policy's rules, which can be triggered by a `Msg/Exec` or a + `Msg/{SubmitProposal,Vote}` with the `Exec` field set, +* or on `EndBlock` right after the proposal's voting period end. This applies to proposals with status `aborted` or `withdrawn` too. + +whichever happens first. + +Proposals are pruned: + +* on `EndBlock` whose proposal status is `withdrawn` or `aborted` on proposal's voting period end before tallying, +* and either after a successful proposal execution, +* or on `EndBlock` right after the proposal's `voting_period_end` + + `max_execution_period` (defined as an app-wide configuration) is passed, + +whichever happens first. + +## State + +The `group` module uses the `orm` package which provides table storage with support for +primary keys and secondary indexes. `orm` also defines `Sequence` which is a persistent unique key generator based on a counter that can be used along with `Table`s. + +Here's the list of tables and associated sequences and indexes stored as part of the `group` module. + +### Group Table + +The `groupTable` stores `GroupInfo`: `0x0 | BigEndian(GroupId) -> ProtocolBuffer(GroupInfo)`. + +#### groupSeq + +The value of `groupSeq` is incremented when creating a new group and corresponds to the new `GroupId`: `0x1 | 0x1 -> BigEndian`. + +The second `0x1` corresponds to the ORM `sequenceStorageKey`. + +#### groupByAdminIndex + +`groupByAdminIndex` allows to retrieve groups by admin address: +`0x2 | len([]byte(group.Admin)) | []byte(group.Admin) | BigEndian(GroupId) -> []byte()`. + +### Group Member Table + +The `groupMemberTable` stores `GroupMember`s: `0x10 | BigEndian(GroupId) | []byte(member.Address) -> ProtocolBuffer(GroupMember)`. + +The `groupMemberTable` is a primary key table and its `PrimaryKey` is given by +`BigEndian(GroupId) | []byte(member.Address)` which is used by the following indexes. + +#### groupMemberByGroupIndex + +`groupMemberByGroupIndex` allows to retrieve group members by group id: +`0x11 | BigEndian(GroupId) | PrimaryKey -> []byte()`. + +#### groupMemberByMemberIndex + +`groupMemberByMemberIndex` allows to retrieve group members by member address: +`0x12 | len([]byte(member.Address)) | []byte(member.Address) | PrimaryKey -> []byte()`. + +### Group Policy Table + +The `groupPolicyTable` stores `GroupPolicyInfo`: `0x20 | len([]byte(Address)) | []byte(Address) -> ProtocolBuffer(GroupPolicyInfo)`. + +The `groupPolicyTable` is a primary key table and its `PrimaryKey` is given by +`len([]byte(Address)) | []byte(Address)` which is used by the following indexes. + +#### groupPolicySeq + +The value of `groupPolicySeq` is incremented when creating a new group policy and is used to generate the new group policy account `Address`: +`0x21 | 0x1 -> BigEndian`. + +The second `0x1` corresponds to the ORM `sequenceStorageKey`. + +#### groupPolicyByGroupIndex + +`groupPolicyByGroupIndex` allows to retrieve group policies by group id: +`0x22 | BigEndian(GroupId) | PrimaryKey -> []byte()`. + +#### groupPolicyByAdminIndex + +`groupPolicyByAdminIndex` allows to retrieve group policies by admin address: +`0x23 | len([]byte(Address)) | []byte(Address) | PrimaryKey -> []byte()`. + +### Proposal Table + +The `proposalTable` stores `Proposal`s: `0x30 | BigEndian(ProposalId) -> ProtocolBuffer(Proposal)`. + +#### proposalSeq + +The value of `proposalSeq` is incremented when creating a new proposal and corresponds to the new `ProposalId`: `0x31 | 0x1 -> BigEndian`. + +The second `0x1` corresponds to the ORM `sequenceStorageKey`. + +#### proposalByGroupPolicyIndex + +`proposalByGroupPolicyIndex` allows to retrieve proposals by group policy account address: +`0x32 | len([]byte(account.Address)) | []byte(account.Address) | BigEndian(ProposalId) -> []byte()`. + +#### ProposalsByVotingPeriodEndIndex + +`proposalsByVotingPeriodEndIndex` allows to retrieve proposals sorted by chronological `voting_period_end`: +`0x33 | sdk.FormatTimeBytes(proposal.VotingPeriodEnd) | BigEndian(ProposalId) -> []byte()`. + +This index is used when tallying the proposal votes at the end of the voting period, and for pruning proposals at `VotingPeriodEnd + MaxExecutionPeriod`. + +### Vote Table + +The `voteTable` stores `Vote`s: `0x40 | BigEndian(ProposalId) | []byte(voter.Address) -> ProtocolBuffer(Vote)`. + +The `voteTable` is a primary key table and its `PrimaryKey` is given by +`BigEndian(ProposalId) | []byte(voter.Address)` which is used by the following indexes. + +#### voteByProposalIndex + +`voteByProposalIndex` allows to retrieve votes by proposal id: +`0x41 | BigEndian(ProposalId) | PrimaryKey -> []byte()`. + +#### voteByVoterIndex + +`voteByVoterIndex` allows to retrieve votes by voter address: +`0x42 | len([]byte(voter.Address)) | []byte(voter.Address) | PrimaryKey -> []byte()`. + +## Msg Service + +### Msg/CreateGroup + +A new group can be created with the `MsgCreateGroup`, which has an admin address, a list of members and some optional metadata. + +The metadata has a maximum length that is chosen by the app developer, and +passed into the group keeper as a config. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L67-L80 +``` + +It's expected to fail if + +* metadata length is greater than `MaxMetadataLen` config +* members are not correctly set (e.g. wrong address format, duplicates, or with 0 weight). + +### Msg/UpdateGroupMembers + +Group members can be updated with the `UpdateGroupMembers`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L88-L102 +``` + +In the list of `MemberUpdates`, an existing member can be removed by setting its weight to 0. + +It's expected to fail if: + +* the signer is not the admin of the group. +* for any one of the associated group policies, if its decision policy's `Validate()` method fails against the updated group. + +### Msg/UpdateGroupAdmin + +The `UpdateGroupAdmin` can be used to update a group admin. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L107-L120 +``` + +It's expected to fail if the signer is not the admin of the group. + +### Msg/UpdateGroupMetadata + +The `UpdateGroupMetadata` can be used to update a group metadata. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L125-L138 +``` + +It's expected to fail if: + +* new metadata length is greater than `MaxMetadataLen` config. +* the signer is not the admin of the group. + +### Msg/CreateGroupPolicy + +A new group policy can be created with the `MsgCreateGroupPolicy`, which has an admin address, a group id, a decision policy and some optional metadata. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L147-L165 +``` + +It's expected to fail if: + +* the signer is not the admin of the group. +* metadata length is greater than `MaxMetadataLen` config. +* the decision policy's `Validate()` method doesn't pass against the group. + +### Msg/CreateGroupWithPolicy + +A new group with policy can be created with the `MsgCreateGroupWithPolicy`, which has an admin address, a list of members, a decision policy, a `group_policy_as_admin` field to optionally set group and group policy admin with group policy address and some optional metadata for group and group policy. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L191-L215 +``` + +It's expected to fail for the same reasons as `Msg/CreateGroup` and `Msg/CreateGroupPolicy`. + +### Msg/UpdateGroupPolicyAdmin + +The `UpdateGroupPolicyAdmin` can be used to update a group policy admin. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L173-L186 +``` + +It's expected to fail if the signer is not the admin of the group policy. + +### Msg/UpdateGroupPolicyDecisionPolicy + +The `UpdateGroupPolicyDecisionPolicy` can be used to update a decision policy. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L226-L241 +``` + +It's expected to fail if: + +* the signer is not the admin of the group policy. +* the new decision policy's `Validate()` method doesn't pass against the group. + +### Msg/UpdateGroupPolicyMetadata + +The `UpdateGroupPolicyMetadata` can be used to update a group policy metadata. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L246-L259 +``` + +It's expected to fail if: + +* new metadata length is greater than `MaxMetadataLen` config. +* the signer is not the admin of the group. + +### Msg/SubmitProposal + +A new proposal can be created with the `MsgSubmitProposal`, which has a group policy account address, a list of proposers addresses, a list of messages to execute if the proposal is accepted and some optional metadata. +An optional `Exec` value can be provided to try to execute the proposal immediately after proposal creation. Proposers signatures are considered as yes votes in this case. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L281-L315 +``` + +It's expected to fail if: + +* metadata, title, or summary length is greater than `MaxMetadataLen` config. +* if any of the proposers is not a group member. + +### Msg/WithdrawProposal + +A proposal can be withdrawn using `MsgWithdrawProposal` which has an `address` (can be either a proposer or the group policy admin) and a `proposal_id` (which has to be withdrawn). + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L323-L333 +``` + +It's expected to fail if: + +* the signer is neither the group policy admin nor proposer of the proposal. +* the proposal is already closed or aborted. + +### Msg/Vote + +A new vote can be created with the `MsgVote`, given a proposal id, a voter address, a choice (yes, no, veto or abstain) and some optional metadata. +An optional `Exec` value can be provided to try to execute the proposal immediately after voting. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L338-L358 +``` + +It's expected to fail if: + +* metadata length is greater than `MaxMetadataLen` config. +* the proposal is not in voting period anymore. + +### Msg/Exec + +A proposal can be executed with the `MsgExec`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L363-L373 +``` + +The messages that are part of this proposal won't be executed if: + +* the proposal has not been accepted by the group policy. +* the proposal has already been successfully executed. + +### Msg/LeaveGroup + +The `MsgLeaveGroup` allows group member to leave a group. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/group/v1/tx.proto#L381-L391 +``` + +It's expected to fail if: + +* the group member is not part of the group. +* for any one of the associated group policies, if its decision policy's `Validate()` method fails against the updated group. + +## Events + +The group module emits the following events: + +### EventCreateGroup + +| Type | Attribute Key | Attribute Value | +| -------------------------------- | ------------- | -------------------------------- | +| message | action | /cosmos.group.v1.Msg/CreateGroup | +| cosmos.group.v1.EventCreateGroup | group_id | {groupId} | + +### EventUpdateGroup + +| Type | Attribute Key | Attribute Value | +| -------------------------------- | ------------- | ---------------------------------------------------------- | +| message | action | /cosmos.group.v1.Msg/UpdateGroup{Admin\|Metadata\|Members} | +| cosmos.group.v1.EventUpdateGroup | group_id | {groupId} | + +### EventCreateGroupPolicy + +| Type | Attribute Key | Attribute Value | +| -------------------------------------- | ------------- | -------------------------------------- | +| message | action | /cosmos.group.v1.Msg/CreateGroupPolicy | +| cosmos.group.v1.EventCreateGroupPolicy | address | {groupPolicyAddress} | + +### EventUpdateGroupPolicy + +| Type | Attribute Key | Attribute Value | +| -------------------------------------- | ------------- | ----------------------------------------------------------------------- | +| message | action | /cosmos.group.v1.Msg/UpdateGroupPolicy{Admin\|Metadata\|DecisionPolicy} | +| cosmos.group.v1.EventUpdateGroupPolicy | address | {groupPolicyAddress} | + +### EventCreateProposal + +| Type | Attribute Key | Attribute Value | +| ----------------------------------- | ------------- | ----------------------------------- | +| message | action | /cosmos.group.v1.Msg/CreateProposal | +| cosmos.group.v1.EventCreateProposal | proposal_id | {proposalId} | + +### EventWithdrawProposal + +| Type | Attribute Key | Attribute Value | +| ------------------------------------- | ------------- | ------------------------------------- | +| message | action | /cosmos.group.v1.Msg/WithdrawProposal | +| cosmos.group.v1.EventWithdrawProposal | proposal_id | {proposalId} | + +### EventVote + +| Type | Attribute Key | Attribute Value | +| ------------------------- | ------------- | ------------------------- | +| message | action | /cosmos.group.v1.Msg/Vote | +| cosmos.group.v1.EventVote | proposal_id | {proposalId} | + +## EventExec + +| Type | Attribute Key | Attribute Value | +| ------------------------- | ------------- | ------------------------- | +| message | action | /cosmos.group.v1.Msg/Exec | +| cosmos.group.v1.EventExec | proposal_id | {proposalId} | +| cosmos.group.v1.EventExec | logs | {logs_string} | + +### EventLeaveGroup + +| Type | Attribute Key | Attribute Value | +| ------------------------------- | ------------- | ------------------------------- | +| message | action | /cosmos.group.v1.Msg/LeaveGroup | +| cosmos.group.v1.EventLeaveGroup | proposal_id | {proposalId} | +| cosmos.group.v1.EventLeaveGroup | address | {address} | + +### EventProposalPruned + +| Type | Attribute Key | Attribute Value | +|-------------------------------------|---------------|---------------------------------| +| message | action | /cosmos.group.v1.Msg/LeaveGroup | +| cosmos.group.v1.EventProposalPruned | proposal_id | {proposalId} | +| cosmos.group.v1.EventProposalPruned | status | {ProposalStatus} | +| cosmos.group.v1.EventProposalPruned | tally_result | {TallyResult} | + + +## Client + +### CLI + +A user can query and interact with the `group` module using the CLI. + +#### Query + +The `query` commands allow users to query `group` state. + +```bash +simd query group --help +``` + +##### group-info + +The `group-info` command allows users to query for group info by given group id. + +```bash +simd query group group-info [id] [flags] +``` + +Example: + +```bash +simd query group group-info 1 +``` + +Example Output: + +```bash +admin: cosmos1.. +group_id: "1" +metadata: AQ== +total_weight: "3" +version: "1" +``` + +##### group-policy-info + +The `group-policy-info` command allows users to query for group policy info by account address of group policy . + +```bash +simd query group group-policy-info [group-policy-account] [flags] +``` + +Example: + +```bash +simd query group group-policy-info cosmos1.. +``` + +Example Output: + +```bash +address: cosmos1.. +admin: cosmos1.. +decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s +group_id: "1" +metadata: AQ== +version: "1" +``` + +##### group-members + +The `group-members` command allows users to query for group members by group id with pagination flags. + +```bash +simd query group group-members [id] [flags] +``` + +Example: + +```bash +simd query group group-members 1 +``` + +Example Output: + +```bash +members: +- group_id: "1" + member: + address: cosmos1.. + metadata: AQ== + weight: "2" +- group_id: "1" + member: + address: cosmos1.. + metadata: AQ== + weight: "1" +pagination: + next_key: null + total: "2" +``` + +##### groups-by-admin + +The `groups-by-admin` command allows users to query for groups by admin account address with pagination flags. + +```bash +simd query group groups-by-admin [admin] [flags] +``` + +Example: + +```bash +simd query group groups-by-admin cosmos1.. +``` + +Example Output: + +```bash +groups: +- admin: cosmos1.. + group_id: "1" + metadata: AQ== + total_weight: "3" + version: "1" +- admin: cosmos1.. + group_id: "2" + metadata: AQ== + total_weight: "3" + version: "1" +pagination: + next_key: null + total: "2" +``` + +##### group-policies-by-group + +The `group-policies-by-group` command allows users to query for group policies by group id with pagination flags. + +```bash +simd query group group-policies-by-group [group-id] [flags] +``` + +Example: + +```bash +simd query group group-policies-by-group 1 +``` + +Example Output: + +```bash +group_policies: +- address: cosmos1.. + admin: cosmos1.. + decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s + group_id: "1" + metadata: AQ== + version: "1" +- address: cosmos1.. + admin: cosmos1.. + decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s + group_id: "1" + metadata: AQ== + version: "1" +pagination: + next_key: null + total: "2" +``` + +##### group-policies-by-admin + +The `group-policies-by-admin` command allows users to query for group policies by admin account address with pagination flags. + +```bash +simd query group group-policies-by-admin [admin] [flags] +``` + +Example: + +```bash +simd query group group-policies-by-admin cosmos1.. +``` + +Example Output: + +```bash +group_policies: +- address: cosmos1.. + admin: cosmos1.. + decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s + group_id: "1" + metadata: AQ== + version: "1" +- address: cosmos1.. + admin: cosmos1.. + decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s + group_id: "1" + metadata: AQ== + version: "1" +pagination: + next_key: null + total: "2" +``` + +##### proposal + +The `proposal` command allows users to query for proposal by id. + +```bash +simd query group proposal [id] [flags] +``` + +Example: + +```bash +simd query group proposal 1 +``` + +Example Output: + +```bash +proposal: + address: cosmos1.. + executor_result: EXECUTOR_RESULT_NOT_RUN + group_policy_version: "1" + group_version: "1" + metadata: AQ== + msgs: + - '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "100000000" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. + proposal_id: "1" + proposers: + - cosmos1.. + result: RESULT_UNFINALIZED + status: STATUS_SUBMITTED + submitted_at: "2021-12-17T07:06:26.310638964Z" + windows: + min_execution_period: 0s + voting_period: 432000s + vote_state: + abstain_count: "0" + no_count: "0" + veto_count: "0" + yes_count: "0" + summary: "Summary" + title: "Title" +``` + +##### proposals-by-group-policy + +The `proposals-by-group-policy` command allows users to query for proposals by account address of group policy with pagination flags. + +```bash +simd query group proposals-by-group-policy [group-policy-account] [flags] +``` + +Example: + +```bash +simd query group proposals-by-group-policy cosmos1.. +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "1" +proposals: +- address: cosmos1.. + executor_result: EXECUTOR_RESULT_NOT_RUN + group_policy_version: "1" + group_version: "1" + metadata: AQ== + msgs: + - '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "100000000" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. + proposal_id: "1" + proposers: + - cosmos1.. + result: RESULT_UNFINALIZED + status: STATUS_SUBMITTED + submitted_at: "2021-12-17T07:06:26.310638964Z" + windows: + min_execution_period: 0s + voting_period: 432000s + vote_state: + abstain_count: "0" + no_count: "0" + veto_count: "0" + yes_count: "0" + summary: "Summary" + title: "Title" +``` + +##### vote + +The `vote` command allows users to query for vote by proposal id and voter account address. + +```bash +simd query group vote [proposal-id] [voter] [flags] +``` + +Example: + +```bash +simd query group vote 1 cosmos1.. +``` + +Example Output: + +```bash +vote: + choice: CHOICE_YES + metadata: AQ== + proposal_id: "1" + submitted_at: "2021-12-17T08:05:02.490164009Z" + voter: cosmos1.. +``` + +##### votes-by-proposal + +The `votes-by-proposal` command allows users to query for votes by proposal id with pagination flags. + +```bash +simd query group votes-by-proposal [proposal-id] [flags] +``` + +Example: + +```bash +simd query group votes-by-proposal 1 +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "1" +votes: +- choice: CHOICE_YES + metadata: AQ== + proposal_id: "1" + submitted_at: "2021-12-17T08:05:02.490164009Z" + voter: cosmos1.. +``` + +##### votes-by-voter + +The `votes-by-voter` command allows users to query for votes by voter account address with pagination flags. + +```bash +simd query group votes-by-voter [voter] [flags] +``` + +Example: + +```bash +simd query group votes-by-voter cosmos1.. +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "1" +votes: +- choice: CHOICE_YES + metadata: AQ== + proposal_id: "1" + submitted_at: "2021-12-17T08:05:02.490164009Z" + voter: cosmos1.. +``` + +### Transactions + +The `tx` commands allow users to interact with the `group` module. + +```bash +simd tx group --help +``` + +#### create-group + +The `create-group` command allows users to create a group which is an aggregation of member accounts with associated weights and +an administrator account. + +```bash +simd tx group create-group [admin] [metadata] [members-json-file] +``` + +Example: + +```bash +simd tx group create-group cosmos1.. "AQ==" members.json +``` + +#### update-group-admin + +The `update-group-admin` command allows users to update a group's admin. + +```bash +simd tx group update-group-admin [admin] [group-id] [new-admin] [flags] +``` + +Example: + +```bash +simd tx group update-group-admin cosmos1.. 1 cosmos1.. +``` + +#### update-group-members + +The `update-group-members` command allows users to update a group's members. + +```bash +simd tx group update-group-members [admin] [group-id] [members-json-file] [flags] +``` + +Example: + +```bash +simd tx group update-group-members cosmos1.. 1 members.json +``` + +#### update-group-metadata + +The `update-group-metadata` command allows users to update a group's metadata. + +```bash +simd tx group update-group-metadata [admin] [group-id] [metadata] [flags] +``` + +Example: + +```bash +simd tx group update-group-metadata cosmos1.. 1 "AQ==" +``` + +#### create-group-policy + +The `create-group-policy` command allows users to create a group policy which is an account associated with a group and a decision policy. + +```bash +simd tx group create-group-policy [admin] [group-id] [metadata] [decision-policy] [flags] +``` + +Example: + +```bash +simd tx group create-group-policy cosmos1.. 1 "AQ==" '{"@type":"/cosmos.group.v1.ThresholdDecisionPolicy", "threshold":"1", "windows": {"voting_period": "120h", "min_execution_period": "0s"}}' +``` + +#### create-group-with-policy + +The `create-group-with-policy` command allows users to create a group which is an aggregation of member accounts with associated weights and an administrator account with decision policy. If the `--group-policy-as-admin` flag is set to `true`, the group policy address becomes the group and group policy admin. + +```bash +simd tx group create-group-with-policy [admin] [group-metadata] [group-policy-metadata] [members-json-file] [decision-policy] [flags] +``` + +Example: + +```bash +simd tx group create-group-with-policy cosmos1.. "AQ==" "AQ==" members.json '{"@type":"/cosmos.group.v1.ThresholdDecisionPolicy", "threshold":"1", "windows": {"voting_period": "120h", "min_execution_period": "0s"}}' +``` + +#### update-group-policy-admin + +The `update-group-policy-admin` command allows users to update a group policy admin. + +```bash +simd tx group update-group-policy-admin [admin] [group-policy-account] [new-admin] [flags] +``` + +Example: + +```bash +simd tx group update-group-policy-admin cosmos1.. cosmos1.. cosmos1.. +``` + +#### update-group-policy-metadata + +The `update-group-policy-metadata` command allows users to update a group policy metadata. + +```bash +simd tx group update-group-policy-metadata [admin] [group-policy-account] [new-metadata] [flags] +``` + +Example: + +```bash +simd tx group update-group-policy-metadata cosmos1.. cosmos1.. "AQ==" +``` + +#### update-group-policy-decision-policy + +The `update-group-policy-decision-policy` command allows users to update a group policy's decision policy. + +```bash +simd tx group update-group-policy-decision-policy [admin] [group-policy-account] [decision-policy] [flags] +``` + +Example: + +```bash +simd tx group update-group-policy-decision-policy cosmos1.. cosmos1.. '{"@type":"/cosmos.group.v1.ThresholdDecisionPolicy", "threshold":"2", "windows": {"voting_period": "120h", "min_execution_period": "0s"}}' +``` + +#### submit-proposal + +The `submit-proposal` command allows users to submit a new proposal. + +```bash +simd tx group submit-proposal [group-policy-account] [proposer[,proposer]*] [msg_tx_json_file] [metadata] [flags] +``` + +Example: + +```bash +simd tx group submit-proposal cosmos1.. cosmos1.. msg_tx.json "AQ==" +``` + +#### withdraw-proposal + +The `withdraw-proposal` command allows users to withdraw a proposal. + +```bash +simd tx group withdraw-proposal [proposal-id] [group-policy-admin-or-proposer] +``` + +Example: + +```bash +simd tx group withdraw-proposal 1 cosmos1.. +``` + +#### vote + +The `vote` command allows users to vote on a proposal. + +```bash +simd tx group vote proposal-id] [voter] [choice] [metadata] [flags] +``` + +Example: + +```bash +simd tx group vote 1 cosmos1.. CHOICE_YES "AQ==" +``` + +#### exec + +The `exec` command allows users to execute a proposal. + +```bash +simd tx group exec [proposal-id] [flags] +``` + +Example: + +```bash +simd tx group exec 1 +``` + +#### leave-group + +The `leave-group` command allows group member to leave the group. + +```bash +simd tx group leave-group [member-address] [group-id] +``` + +Example: + +```bash +simd tx group leave-group cosmos1... 1 +``` + +### gRPC + +A user can query the `group` module using gRPC endpoints. + +#### GroupInfo + +The `GroupInfo` endpoint allows users to query for group info by given group id. + +```bash +cosmos.group.v1.Query/GroupInfo +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"group_id":1}' localhost:9090 cosmos.group.v1.Query/GroupInfo +``` + +Example Output: + +```bash +{ + "info": { + "groupId": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "totalWeight": "3" + } +} +``` + +#### GroupPolicyInfo + +The `GroupPolicyInfo` endpoint allows users to query for group policy info by account address of group policy. + +```bash +cosmos.group.v1.Query/GroupPolicyInfo +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/GroupPolicyInfo +``` + +Example Output: + +```bash +{ + "info": { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows": {"voting_period": "120h", "min_execution_period": "0s"}}, + } +} +``` + +#### GroupMembers + +The `GroupMembers` endpoint allows users to query for group members by group id with pagination flags. + +```bash +cosmos.group.v1.Query/GroupMembers +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"group_id":"1"}' localhost:9090 cosmos.group.v1.Query/GroupMembers +``` + +Example Output: + +```bash +{ + "members": [ + { + "groupId": "1", + "member": { + "address": "cosmos1..", + "weight": "1" + } + }, + { + "groupId": "1", + "member": { + "address": "cosmos1..", + "weight": "2" + } + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### GroupsByAdmin + +The `GroupsByAdmin` endpoint allows users to query for groups by admin account address with pagination flags. + +```bash +cosmos.group.v1.Query/GroupsByAdmin +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"admin":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/GroupsByAdmin +``` + +Example Output: + +```bash +{ + "groups": [ + { + "groupId": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "totalWeight": "3" + }, + { + "groupId": "2", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "totalWeight": "3" + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### GroupPoliciesByGroup + +The `GroupPoliciesByGroup` endpoint allows users to query for group policies by group id with pagination flags. + +```bash +cosmos.group.v1.Query/GroupPoliciesByGroup +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"group_id":"1"}' localhost:9090 cosmos.group.v1.Query/GroupPoliciesByGroup +``` + +Example Output: + +```bash +{ + "GroupPolicies": [ + { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows":{"voting_period": "120h", "min_execution_period": "0s"}}, + }, + { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows":{"voting_period": "120h", "min_execution_period": "0s"}}, + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### GroupPoliciesByAdmin + +The `GroupPoliciesByAdmin` endpoint allows users to query for group policies by admin account address with pagination flags. + +```bash +cosmos.group.v1.Query/GroupPoliciesByAdmin +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"admin":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/GroupPoliciesByAdmin +``` + +Example Output: + +```bash +{ + "GroupPolicies": [ + { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows":{"voting_period": "120h", "min_execution_period": "0s"}}, + }, + { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows":{"voting_period": "120h", "min_execution_period": "0s"}}, + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### Proposal + +The `Proposal` endpoint allows users to query for proposal by id. + +```bash +cosmos.group.v1.Query/Proposal +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' localhost:9090 cosmos.group.v1.Query/Proposal +``` + +Example Output: + +```bash +{ + "proposal": { + "proposalId": "1", + "address": "cosmos1..", + "proposers": [ + "cosmos1.." + ], + "submittedAt": "2021-12-17T07:06:26.310638964Z", + "groupVersion": "1", + "GroupPolicyVersion": "1", + "status": "STATUS_SUBMITTED", + "result": "RESULT_UNFINALIZED", + "voteState": { + "yesCount": "0", + "noCount": "0", + "abstainCount": "0", + "vetoCount": "0" + }, + "windows": { + "min_execution_period": "0s", + "voting_period": "432000s" + }, + "executorResult": "EXECUTOR_RESULT_NOT_RUN", + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"100000000"}],"fromAddress":"cosmos1..","toAddress":"cosmos1.."} + ], + "title": "Title", + "summary": "Summary", + } +} +``` + +#### ProposalsByGroupPolicy + +The `ProposalsByGroupPolicy` endpoint allows users to query for proposals by account address of group policy with pagination flags. + +```bash +cosmos.group.v1.Query/ProposalsByGroupPolicy +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/ProposalsByGroupPolicy +``` + +Example Output: + +```bash +{ + "proposals": [ + { + "proposalId": "1", + "address": "cosmos1..", + "proposers": [ + "cosmos1.." + ], + "submittedAt": "2021-12-17T08:03:27.099649352Z", + "groupVersion": "1", + "GroupPolicyVersion": "1", + "status": "STATUS_CLOSED", + "result": "RESULT_ACCEPTED", + "voteState": { + "yesCount": "1", + "noCount": "0", + "abstainCount": "0", + "vetoCount": "0" + }, + "windows": { + "min_execution_period": "0s", + "voting_period": "432000s" + }, + "executorResult": "EXECUTOR_RESULT_NOT_RUN", + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"100000000"}],"fromAddress":"cosmos1..","toAddress":"cosmos1.."} + ], + "title": "Title", + "summary": "Summary", + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### VoteByProposalVoter + +The `VoteByProposalVoter` endpoint allows users to query for vote by proposal id and voter account address. + +```bash +cosmos.group.v1.Query/VoteByProposalVoter +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1","voter":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/VoteByProposalVoter +``` + +Example Output: + +```bash +{ + "vote": { + "proposalId": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "submittedAt": "2021-12-17T08:05:02.490164009Z" + } +} +``` + +#### VotesByProposal + +The `VotesByProposal` endpoint allows users to query for votes by proposal id with pagination flags. + +```bash +cosmos.group.v1.Query/VotesByProposal +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' localhost:9090 cosmos.group.v1.Query/VotesByProposal +``` + +Example Output: + +```bash +{ + "votes": [ + { + "proposalId": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "submittedAt": "2021-12-17T08:05:02.490164009Z" + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### VotesByVoter + +The `VotesByVoter` endpoint allows users to query for votes by voter account address with pagination flags. + +```bash +cosmos.group.v1.Query/VotesByVoter +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"voter":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/VotesByVoter +``` + +Example Output: + +```bash +{ + "votes": [ + { + "proposalId": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "submittedAt": "2021-12-17T08:05:02.490164009Z" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### REST + +A user can query the `group` module using REST endpoints. + +#### GroupInfo + +The `GroupInfo` endpoint allows users to query for group info by given group id. + +```bash +/cosmos/group/v1/group_info/{group_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_info/1 +``` + +Example Output: + +```bash +{ + "info": { + "id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "total_weight": "3" + } +} +``` + +#### GroupPolicyInfo + +The `GroupPolicyInfo` endpoint allows users to query for group policy info by account address of group policy. + +```bash +/cosmos/group/v1/group_policy_info/{address} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_policy_info/cosmos1.. +``` + +Example Output: + +```bash +{ + "info": { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + } +} +``` + +#### GroupMembers + +The `GroupMembers` endpoint allows users to query for group members by group id with pagination flags. + +```bash +/cosmos/group/v1/group_members/{group_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_members/1 +``` + +Example Output: + +```bash +{ + "members": [ + { + "group_id": "1", + "member": { + "address": "cosmos1..", + "weight": "1", + "metadata": "AQ==" + } + }, + { + "group_id": "1", + "member": { + "address": "cosmos1..", + "weight": "2", + "metadata": "AQ==" + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### GroupsByAdmin + +The `GroupsByAdmin` endpoint allows users to query for groups by admin account address with pagination flags. + +```bash +/cosmos/group/v1/groups_by_admin/{admin} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/groups_by_admin/cosmos1.. +``` + +Example Output: + +```bash +{ + "groups": [ + { + "id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "total_weight": "3" + }, + { + "id": "2", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "total_weight": "3" + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### GroupPoliciesByGroup + +The `GroupPoliciesByGroup` endpoint allows users to query for group policies by group id with pagination flags. + +```bash +/cosmos/group/v1/group_policies_by_group/{group_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_policies_by_group/1 +``` + +Example Output: + +```bash +{ + "group_policies": [ + { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + }, + { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### GroupPoliciesByAdmin + +The `GroupPoliciesByAdmin` endpoint allows users to query for group policies by admin account address with pagination flags. + +```bash +/cosmos/group/v1/group_policies_by_admin/{admin} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_policies_by_admin/cosmos1.. +``` + +Example Output: + +```bash +{ + "group_policies": [ + { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + }, + { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +``` + +#### Proposal + +The `Proposal` endpoint allows users to query for proposal by id. + +```bash +/cosmos/group/v1/proposal/{proposal_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/proposal/1 +``` + +Example Output: + +```bash +{ + "proposal": { + "proposal_id": "1", + "address": "cosmos1..", + "metadata": "AQ==", + "proposers": [ + "cosmos1.." + ], + "submitted_at": "2021-12-17T07:06:26.310638964Z", + "group_version": "1", + "group_policy_version": "1", + "status": "STATUS_SUBMITTED", + "result": "RESULT_UNFINALIZED", + "vote_state": { + "yes_count": "0", + "no_count": "0", + "abstain_count": "0", + "veto_count": "0" + }, + "windows": { + "min_execution_period": "0s", + "voting_period": "432000s" + }, + "executor_result": "EXECUTOR_RESULT_NOT_RUN", + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1..", + "to_address": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "100000000" + } + ] + } + ], + "title": "Title", + "summary": "Summary", + } +} +``` + +#### ProposalsByGroupPolicy + +The `ProposalsByGroupPolicy` endpoint allows users to query for proposals by account address of group policy with pagination flags. + +```bash +/cosmos/group/v1/proposals_by_group_policy/{address} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/proposals_by_group_policy/cosmos1.. +``` + +Example Output: + +```bash +{ + "proposals": [ + { + "id": "1", + "group_policy_address": "cosmos1..", + "metadata": "AQ==", + "proposers": [ + "cosmos1.." + ], + "submit_time": "2021-12-17T08:03:27.099649352Z", + "group_version": "1", + "group_policy_version": "1", + "status": "STATUS_CLOSED", + "result": "RESULT_ACCEPTED", + "vote_state": { + "yes_count": "1", + "no_count": "0", + "abstain_count": "0", + "veto_count": "0" + }, + "windows": { + "min_execution_period": "0s", + "voting_period": "432000s" + }, + "executor_result": "EXECUTOR_RESULT_NOT_RUN", + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1..", + "to_address": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "100000000" + } + ] + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### VoteByProposalVoter + +The `VoteByProposalVoter` endpoint allows users to query for vote by proposal id and voter account address. + +```bash +/cosmos/group/v1/vote_by_proposal_voter/{proposal_id}/{voter} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1beta1/vote_by_proposal_voter/1/cosmos1.. +``` + +Example Output: + +```bash +{ + "vote": { + "proposal_id": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "metadata": "AQ==", + "submitted_at": "2021-12-17T08:05:02.490164009Z" + } +} +``` + +#### VotesByProposal + +The `VotesByProposal` endpoint allows users to query for votes by proposal id with pagination flags. + +```bash +/cosmos/group/v1/votes_by_proposal/{proposal_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/votes_by_proposal/1 +``` + +Example Output: + +```bash +{ + "votes": [ + { + "proposal_id": "1", + "voter": "cosmos1..", + "option": "CHOICE_YES", + "metadata": "AQ==", + "submit_time": "2021-12-17T08:05:02.490164009Z" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### VotesByVoter + +The `VotesByVoter` endpoint allows users to query for votes by voter account address with pagination flags. + +```bash +/cosmos/group/v1/votes_by_voter/{voter} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/votes_by_voter/cosmos1.. +``` + +Example Output: + +```bash +{ + "votes": [ + { + "proposal_id": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "metadata": "AQ==", + "submitted_at": "2021-12-17T08:05:02.490164009Z" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +## Metadata + +The group module has four locations for metadata where users can provide further context about the on-chain actions they are taking. By default all metadata fields have a 255 character length field where metadata can be stored in json format, either on-chain or off-chain depending on the amount of data required. Here we provide a recommendation for the json structure and where the data should be stored. There are two important factors in making these recommendations. First, that the group and gov modules are consistent with one another, note the number of proposals made by all groups may be quite large. Second, that client applications such as block explorers and governance interfaces have confidence in the consistency of metadata structure across chains. + +### Proposal + +Location: off-chain as json object stored on IPFS (mirrors [gov proposal](../gov/README.md#metadata)) + +```json +{ + "title": "", + "authors": [""], + "summary": "", + "details": "", + "proposal_forum_url": "", + "vote_option_context": "", +} +``` + +:::note +The `authors` field is an array of strings, this is to allow for multiple authors to be listed in the metadata. +In v0.46, the `authors` field is a comma-separated string. Frontends are encouraged to support both formats for backwards compatibility. +::: + +### Vote + +Location: on-chain as json within 255 character limit (mirrors [gov vote](../gov/README.md#metadata)) + +```json +{ + "justification": "", +} +``` + +### Group + +Location: off-chain as json object stored on IPFS + +```json +{ + "name": "", + "description": "", + "group_website_url": "", + "group_forum_url": "", +} +``` + +### Decision policy + +Location: on-chain as json within 255 character limit + +```json +{ + "name": "", + "description": "", +} +``` diff --git a/versioned_docs/version-0.52/build/modules/mint/README.md b/versioned_docs/version-0.52/build/modules/mint/README.md new file mode 100644 index 000000000..db3449313 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/mint/README.md @@ -0,0 +1,541 @@ +--- +sidebar_position: 1 +--- + +# `x/mint` + +## Contents + +* [Concepts](#concepts) + * [The Minting Mechanism](#the-minting-mechanism) + * [Inflation](#inflation) + * [Provisions](#provisions) + * [Relation to Inflation](#relation-to-inflation) + * [Usage per Block](#usage-per-block-default-function) + * [Example](#example) + * [Bonding](#bonding) +* [State](#state) + * [Minter](#minter) + * [Params](#params) +* [Minting Methods](#minting-methods) + * [Epoch-based Minting](#epoch-based-minting) + * [Block-based Minting](#block-based-minting) + * [MintFn](#mintfn) + * [Default configuration](#default-configuration) + * [Calculations](#calculations) + * [NextInflationRate](#inflation-rate-calculation) + * [NextAnnualProvisions](#nextannualprovisions) + * [BlockProvision](#blockprovision) +* [Parameters](#parameters) +* [Events](#events) + * [BeginBlocker](#beginblocker) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) + +## Concepts + +### The Minting Mechanism + +The minting mechanism in the x/mint module has been redesigned to offer more flexibility. The `InflationCalculationFn` has been deprecated in favor of `MintFn`, that can be customized by the application developer. The `MintFn` function is passed to the `NewAppModule` function and is used to mint tokens on the configured epoch beginning. This change allows users to define their own minting logic and removes any assumptions on how tokens are minted. + +Key features of the new minting mechanism: + +1. **Customizable Minting Function**: The `MintFn` can be defined by the application to implement any desired minting logic. +2. **Default Implementation**: If no custom function is provided, a default minting function is used. +3. **Epoch-based or Block-based Minting**: The mechanism supports both epoch-based and block-based minting, depending on how the `MintFn` is implemented. +4. **Flexible Inflation**: The inflation rate can be adjusted based on various parameters, not just the bonded ratio. + +The default minting function, if no custom one is provided, is implemented in the `DefaultMintFn`. +This function is called during the `BeginBlocker` and is responsible for minting new tokens, implementation details can be found [here](#default-configuration). + +### Inflation + +Inflation is a key concept in the x/mint module, responsible for the creation of new tokens over time. The inflation rate determines how quickly the total supply of tokens increases. + +Key aspects of inflation in the x/mint module: + +1. **Dynamic Inflation**: The inflation rate can change over time based on various factors, primarily the bonded ratio. +2. **Bounded Inflation**: The inflation rate is typically constrained between a minimum and maximum value to prevent extreme fluctuations. +3. **Inflation Calculation**: The specific method of calculating inflation can be customized using the `MintFn`, allowing for flexible monetary policies. + +In the default implementation, inflation is calculated as follows: + +```plaintext +Inflation = CurrentInflation + (1 - BondedRatio / GoalBonded) * (InflationRateChange / BlocksPerYear) +``` + +### Provisions + +Provisions are the number of tokens generated and distributed in each block. They are directly related to the inflation rate and the current total supply of tokens. The amount of provisions generated per block is calculated based on the annual provisions, which are determined by the inflation rate and the total supply of tokens. + +#### Relation to Inflation + +The inflation rate determines the percentage of the total supply of tokens that will be added as provisions over a year. These annual provisions are divided by the number of blocks in a year to obtain the provisions per block. + +#### Usage per Block (default function) + +Each block uses a fraction of the annual provisions, calculated as: + +```plaintext +Provisions per block = Annual provisions / Number of blocks per year +``` + +These provisions are distributed to validators and delegators as rewards for their participation in the network. + + +##### Example + +For example, if the total supply of tokens is 1,000,000 and the inflation rate is 10%, the annual provisions would be: + +Annual provisions = 1,000,000 * 0.10 = 100,000 tokens + +If there are 3,153,600 blocks per year (one block every 10 seconds), the provisions per block would be: +Provisions per block = 100,000 / 3,153,600 ≈ 0.0317 tokens per block. + +These provisions are then passed to the fee collector. + +### Bonding + +Bonding refers to the process of staking tokens in the network, which plays a crucial role in the Proof of Stake consensus mechanism and affects the minting process. + +Key aspects of bonding in relation to the x/mint module: + +1. **Bonded Ratio**: This is the proportion of the total token supply that is currently staked (bonded) in the network. +2. **Goal Bonded Ratio**: A target percentage of tokens that should ideally be bonded, defined in the module parameters. +3. **Inflation Adjustment**: The bonded ratio is used to adjust the inflation rate, encouraging or discouraging bonding as needed to maintain network security and token liquidity. + +## State + +### Minter + +The minter is a space for holding current inflation information and any other data +related to minting (in the `data` field) + +* Minter: `0x00 -> ProtocolBuffer(minter)` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/mint/proto/cosmos/mint/v1beta1/mint.proto#L11-L29 +``` + +### Params + +The mint module stores its params in state with the prefix of `0x01`, +it can be updated with governance or the address with authority. +**Note:** With the latest update, the addition of the `MaxSupply` parameter allows controlling the maximum supply of tokens minted by the module. +A value of `0` indicates an unlimited supply. + +* Params: `mint/params -> legacy_amino(params)` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/mint/proto/cosmos/mint/v1beta1/mint.proto#L31-L73 +``` + +## Minting Methods + +### Epoch-based Minting + +Epoch-based minting allows for tokens to be minted at specific intervals or epochs, rather than on every block. +To implement epoch-based minting, the `MintFn` should be designed to mint tokens only when a specific epoch ID is received. The epoch ID and number are passed as parameters to the `MintFn`. + +### Block-based Minting + +In addition to minting based on epoch, minting based on block is also possible. This is achieved through calling the `MintFn` in `BeginBlock` with an epochID and epochNumber of `"block"` and `-1`, respectively. + +### MintFn + +The `MintFn` function is called at the beginning of each epoch and is responsible for minting tokens. The function signature is as follows: + +```go +type MintFn func(ctx context.Context, env appmodule.Environment, minter *Minter, epochId string, epochNumber int64) error +``` + +How this function mints tokens is defined by the app developers, meaning they can query state and perform any calculations they deem necessary. [This implementation](https://github.com/cosmos/cosmos-sdk/blob/ace7bca105a8d5363782cfd19c6f169b286cd3b2/simapp/mint_fn.go#L25) in SimApp contains examples of how to use `QueryRouterService` and the Minter's `data`. + +:::warning +Note that BeginBlock will keep calling the MintFn for every block, so it is important to ensure that MintFn returns early if the epoch ID does not match the expected one. +::: + +### Default configuration + +If no `MintFn` is passed to the `NewAppModule` function, the minting logic defaults to block-based minting, corresponding to `mintKeeper.DefaultMintFn(types.DefaultInflationCalculationFn)`. + +The next diagram shows how the `DefaultMintFn` works: + +```mermaid +flowchart TD + A[BeforeEpochStart] --> B[MintFn] + B --> C{epochId == 'block'?} + C -->|No| D[Return without minting] + C -->|Yes| E[Get StakingTokenSupply] + E --> F[Get BondedRatio] + F --> G[Get Parameters] + G --> H[Calculate Inflation] + H --> I[Calculate Annual Provisions] + I --> J[Calculate Block Provision] + J --> K{MaxSupply > 0?} + K -->|No| M[Mint coins] + K -->|Yes| L{TotalSupply + MintedCoins > MaxSupply?} + L -->|No| M + L -->|Yes| N[Adjust minting amount] + N --> O{Difference > 0?} + O -->|No| P[Do not mint] + O -->|Yes| M + M --> Q[Send minted coins to FeeCollector] + Q --> R[Emit events] + R --> S[End] + + subgraph Calculations + H --> |Uses InflationCalculationFn| H + I --> |AnnualProvisions = TotalSupply * Inflation| I + J --> |BlockProvision = AnnualProvisions / BlocksPerYear| J + end + + subgraph MaxSupply Adjustment + N --> |MintedCoins = MaxSupply - TotalSupply| N + end +``` + +### Calculations + +#### Inflation rate calculation + +Inflation rate is calculated using an "inflation calculation function" that's +passed to the `NewAppModule` function. If no function is passed, then the SDK's +default inflation function will be used (`NextInflationRate`). In case a custom +inflation calculation logic is needed, this can be achieved by defining and +passing a function that matches `InflationCalculationFn`'s signature. + +```go +type InflationCalculationFn func(ctx sdk.Context, minter Minter, params Params, bondedRatio math.LegacyDec) math.LegacyDec +``` + +#### NextInflationRate + +The target annual inflation rate is recalculated each block. +The inflation is also subject to a rate change (positive or negative) +depending on the distance from the desired ratio (67%). The maximum rate change +possible is defined to be 5% per year, however, the annual inflation is capped between 0% and 5%. + +```go +NextInflationRate(params Params, bondedRatio math.LegacyDec) (inflation math.LegacyDec) { + inflationRateChangePerYear = (1 - bondedRatio/params.GoalBonded) * params.InflationRateChange + inflationRateChange = inflationRateChangePerYear/blocksPerYr + + // increase the new annual inflation for this next block + inflation += inflationRateChange + if inflation > params.InflationMax { + inflation = params.InflationMax + } + if inflation < params.InflationMin { + inflation = params.InflationMin + } + + return inflation +} +``` + +#### NextAnnualProvisions + +Calculate the annual provisions based on current total supply and inflation +rate. This parameter is calculated once per block. + +```go +NextAnnualProvisions(params Params, totalSupply math.LegacyDec) (provisions math.LegacyDec) { + return Inflation * totalSupply +} +``` + +#### BlockProvision + +Calculate the provisions generated for each block based on current annual provisions. The provisions are then minted by the `mint` module's `ModuleMinterAccount` and then transferred to the `auth`'s `FeeCollector` `ModuleAccount`. + +```go +BlockProvision(params Params) sdk.Coin { + provisionAmt = AnnualProvisions/ params.BlocksPerYear + return sdk.NewCoin(params.MintDenom, provisionAmt.Truncate()) +``` + + +## Parameters + +The minting module contains the following parameters: +Note: `0` indicates unlimited supply for MaxSupply param + +| Key | Type | Example | +|---------------------|------------------|------------------------| +| MintDenom | string | "uatom" | +| InflationRateChange | string (dec) | "0.130000000000000000" | +| InflationMax | string (dec) | "0.200000000000000000" | +| InflationMin | string (dec) | "0.070000000000000000" | +| GoalBonded | string (dec) | "0.670000000000000000" | +| BlocksPerYear | string (uint64) | "6311520" | +| MaxSupply | string (math.Int)| "0" | + + +## Events + +The minting module emits the following events: + +### BeginBlocker + +| Type | Attribute Key | Attribute Value | +|------|-------------------|--------------------| +| mint | bonded_ratio | {bondedRatio} | +| mint | inflation | {inflation} | +| mint | annual_provisions | {annualProvisions} | +| mint | amount | {amount} | + + +## Client + +### CLI + +A user can query and interact with the `mint` module using the CLI. + +#### Query + +The `query` commands allows users to query `mint` state. + +```shell +simd query mint --help +``` + +##### annual-provisions + +The `annual-provisions` command allows users to query the current minting annual provisions value + +```shell +simd query mint annual-provisions [flags] +``` + +Example: + +```shell +simd query mint annual-provisions +``` + +Example Output: + +```shell +22268504368893.612100895088410693 +``` + +##### inflation + +The `inflation` command allows users to query the current minting inflation value + +```shell +simd query mint inflation [flags] +``` + +Example: + +```shell +simd query mint inflation +``` + +Example Output: + +```shell +0.199200302563256955 +``` + +##### params + +The `params` command allows users to query the current minting parameters + +```shell +simd query mint params [flags] +``` + +Example: + +```shell +simd query mint params +``` + +Example Output: + +```yml +blocks_per_year: "4360000" +goal_bonded: "0.670000000000000000" +inflation_max: "0.200000000000000000" +inflation_min: "0.070000000000000000" +inflation_rate_change: "0.130000000000000000" +mint_denom: stake +max_supply: "0" +``` + +#### Transactions + +The `tx` commands allow users to interact with the `mint` module. + +```shell +simd tx mint --help +``` + +##### update-params-proposal + +The `update-params-proposal` command allows users to submit a proposal to update the mint module parameters (Note: the entire params must be provided). + +```shell +simd tx mint update-params-proposal [flags] +``` + +Example: + +```shell +simd tx mint update-params-proposal '{ "mint_denom": "stake" }' +``` + +### gRPC + +A user can query the `mint` module using gRPC endpoints. + +#### AnnualProvisions + +The `AnnualProvisions` endpoint allows users to query the current minting annual provisions value + +```shell +/cosmos.mint.v1beta1.Query/AnnualProvisions +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.mint.v1beta1.Query/AnnualProvisions +``` + +Example Output: + +```json +{ + "annualProvisions": "1432452520532626265712995618" +} +``` + +#### Inflation + +The `Inflation` endpoint allows users to query the current minting inflation value + +```shell +/cosmos.mint.v1beta1.Query/Inflation +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.mint.v1beta1.Query/Inflation +``` + +Example Output: + +```json +{ + "inflation": "130197115720711261" +} +``` + +#### Params + +The `Params` endpoint allows users to query the current minting parameters + +```shell +/cosmos.mint.v1beta1.Query/Params +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.mint.v1beta1.Query/Params +``` + +Example Output: + +```json +{ + "params": { + "mintDenom": "stake", + "inflationRateChange": "130000000000000000", + "inflationMax": "200000000000000000", + "inflationMin": "70000000000000000", + "goalBonded": "670000000000000000", + "blocksPerYear": "6311520", + "maxSupply": "0", + } +} +``` + +### REST + +A user can query the `mint` module using REST endpoints. + +#### annual-provisions + +```shell +/cosmos/mint/v1beta1/annual_provisions +``` + +Example: + +```shell +curl "localhost:1317/cosmos/mint/v1beta1/annual_provisions" +``` + +Example Output: + +```json +{ + "annualProvisions": "1432452520532626265712995618" +} +``` + +#### inflation + +```shell +/cosmos/mint/v1beta1/inflation +``` + +Example: + +```shell +curl "localhost:1317/cosmos/mint/v1beta1/inflation" +``` + +Example Output: + +```json +{ + "inflation": "130197115720711261" +} +``` + +#### params + +```shell +/cosmos/mint/v1beta1/params +``` + +Example: + +```shell +curl "localhost:1317/cosmos/mint/v1beta1/params" +``` + +Example Output: + +```json +{ + "params": { + "mintDenom": "stake", + "inflationRateChange": "130000000000000000", + "inflationMax": "200000000000000000", + "inflationMin": "70000000000000000", + "goalBonded": "670000000000000000", + "blocksPerYear": "6311520", + "maxSupply": "0", + } +} +``` diff --git a/versioned_docs/version-0.52/build/modules/nft/README.md b/versioned_docs/version-0.52/build/modules/nft/README.md new file mode 100644 index 000000000..3bf1b6b41 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/nft/README.md @@ -0,0 +1,117 @@ +--- +sidebar_position: 1 +--- + +# `x/nft` + +## Contents + +## Abstract + +`x/nft` is an implementation of a Cosmos SDK module, per [ADR 43](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-043-nft-module.md), that allows you to create nft classification, create nft, transfer nft, update nft, and support various queries by integrating the module. It is fully compatible with the ERC721 specification. + +* [Concepts](#concepts) + * [Class](#class) + * [NFT](#nft) +* [State](#state) + * [Class](#class-1) + * [NFT](#nft-1) + * [NFTOfClassByOwner](#nftofclassbyowner) + * [Owner](#owner) + * [TotalSupply](#totalsupply) +* [Messages](#messages) + * [MsgSend](#msgsend) +* [Events](#events) +* [Queries](#queries) +* [Keeper Functions](#keeper-functions) + +## Concepts + +### Class + +`x/nft` module defines a struct `Class` to describe the common characteristics of a class of nft, under this class, you can create a variety of nft, which is equivalent to an erc721 contract for Ethereum. The design is defined in the [ADR 043](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-043-nft-module.md). + +### NFT + +The full name of NFT is Non-Fungible Tokens. Because of the irreplaceable nature of NFT, it means that it can be used to represent unique things. The nft implemented by this module is fully compatible with Ethereum ERC721 standard. + +## State + +### Class + +Class is mainly composed of `id`, `name`, `symbol`, `description`, `uri`, `uri_hash`,`data` where `id` is the unique identifier of the class, similar to the Ethereum ERC721 contract address, the others are optional. + +* Class: `0x01 | classID | -> ProtocolBuffer(Class)` + +### NFT + +NFT is mainly composed of `class_id`, `id`, `uri`, `uri_hash` and `data`. Among them, `class_id` and `id` are two-tuples that identify the uniqueness of nft, `uri` and `uri_hash` is optional, which identifies the off-chain storage location of the nft, and `data` is an Any type. Use Any chain of `x/nft` modules can be customized by extending this field + +* NFT: `0x02 | classID | 0x00 | nftID |-> ProtocolBuffer(NFT)` + +### NFTOfClassByOwner + +NFTOfClassByOwner is mainly to realize the function of querying all nfts using classID and owner, without other redundant functions. + +* NFTOfClassByOwner: `0x03 | owner | 0x00 | classID | 0x00 | nftID |-> 0x01` + +### Owner + +Since there is no extra field in NFT to indicate the owner of nft, an additional key-value pair is used to save the ownership of nft. With the transfer of nft, the key-value pair is updated synchronously. + +* OwnerKey: `0x04 | classID | 0x00 | nftID |-> owner` + +### TotalSupply + +TotalSupply is responsible for tracking the number of all nfts under a certain class. Mint operation is performed under the changed class, supply increases by one, burn operation, and supply decreases by one. + +* OwnerKey: `0x05 | classID |-> totalSupply` + +## Messages + +In this section we describe the processing of messages for the NFT module. + +:::warning +The validation of `ClassID` and `NftID` is left to the app developer. +The SDK does not provide any validation for these fields. +::: + +### MsgSend + +You can use the `MsgSend` message to transfer the ownership of nft. This is a function provided by the `x/nft` module. Of course, you can use the `Transfer` method to implement your own transfer logic, but you need to pay extra attention to the transfer permissions. + +The message handling should fail if: + +* provided `ClassID` does not exist. +* provided `Id` does not exist. +* provided `Sender` does not the owner of nft. + +## Events + +The NFT module emits proto events defined in [the Protobuf reference](https://buf.build/cosmos/cosmos-sdk/docs/main:cosmos.nft.v1beta1). + +## Queries + +The `x/nft` module provides several queries to retrieve information about NFTs and classes: + +* `Balance`: Returns the number of NFTs of a given class owned by the owner. +* `Owner`: Returns the owner of an NFT based on its class and ID. +* `Supply`: Returns the number of NFTs from the given class. +* `NFTs`: Queries all NFTs of a given class or owner. +* `NFT`: Returns an NFT based on its class and ID. +* `Class`: Returns an NFT class based on its ID. +* `Classes`: Returns all NFT classes. + +## Keeper Functions + +The Keeper of the `x/nft` module provides several functions to manage NFTs: + +* `Mint`: Mints a new NFT. +* `Burn`: Burns an existing NFT. +* `Update`: Updates an existing NFT. +* `Transfer`: Transfers an NFT from one owner to another. +* `GetNFT`: Retrieves information about a specific NFT. +* `GetNFTsOfClass`: Retrieves all NFTs of a specific class. +* `GetNFTsOfClassByOwner`: Retrieves all NFTs of a specific class belonging to an owner. +* `GetBalance`: Retrieves the balance of NFTs of a specific class for an owner. +* `GetTotalSupply`: Retrieves the total supply of NFTs of a specific class. diff --git a/versioned_docs/version-0.52/build/modules/params/README.md b/versioned_docs/version-0.52/build/modules/params/README.md new file mode 100644 index 000000000..961b1f128 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/params/README.md @@ -0,0 +1,79 @@ +--- +sidebar_position: 1 +--- + +# `x/params` + +> Note: The Params module has been deprecated in favour of each module housing its own parameters. + +## Abstract + +Package params provides a globally available parameter store. + +There are two main types, Keeper and Subspace. Subspace is an isolated namespace for a +paramstore, where keys are prefixed by preconfigured spacename. Keeper has a +permission to access all existing spaces. + +Subspace can be used by the individual keepers, which need a private parameter store +that the other keepers cannot modify. The params Keeper can be used to add a route to `x/gov` router in order to modify any parameter in case a proposal passes. + +The following contents explains how to use params module for master and user modules. + +## Contents + +* [Keeper](#keeper) +* [Subspace](#subspace) + * [Key](#key) + * [KeyTable](#keytable) + * [ParamSet](#paramset) + +## Keeper + +In the app initialization stage, [subspaces](#subspace) can be allocated for other modules' keeper using `Keeper.Subspace` and are stored in `Keeper.spaces`. Then, those modules can have a reference to their specific parameter store through `Keeper.GetSubspace`. + +Example: + +```go +type ExampleKeeper struct { + paramSpace paramtypes.Subspace +} + +func (k ExampleKeeper) SetParams(ctx sdk.Context, params types.Params) { + k.paramSpace.SetParamSet(ctx, ¶ms) +} +``` + +## Subspace + +`Subspace` is a prefixed subspace of the parameter store. Each module which uses the +parameter store will take a `Subspace` to isolate permission to access. + +### Key + +Parameter keys are human readable alphanumeric strings. A parameter for the key +`"ExampleParameter"` is stored under `[]byte("SubspaceName" + "/" + "ExampleParameter")`, + where `"SubspaceName"` is the name of the subspace. + +Subkeys are secondary parameter keys those are used along with a primary parameter key. +Subkeys can be used for grouping or dynamic parameter key generation during runtime. + +### KeyTable + +All of the parameter keys that will be used should be registered at the compile +time. `KeyTable` is essentially a `map[string]attribute`, where the `string` is a parameter key. + +Currently, `attribute` consists of a `reflect.Type`, which indicates the parameter +type to check that provided key and value are compatible and registered, as well as a function `ValueValidatorFn` to validate values. + +Only primary keys have to be registered on the `KeyTable`. Subkeys inherit the +attribute of the primary key. + +### ParamSet + +Modules often define parameters as a proto message. The generated struct can implement +`ParamSet` interface to be used with the following methods: + +* `KeyTable.RegisterParamSet()`: registers all parameters in the struct +* `Subspace.{Get, Set}ParamSet()`: Get to & Set from the struct + +The implementer should be a pointer in order to use `GetParamSet()`. diff --git a/versioned_docs/version-0.52/build/modules/protocolpool/README.md b/versioned_docs/version-0.52/build/modules/protocolpool/README.md new file mode 100644 index 000000000..af5db617e --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/protocolpool/README.md @@ -0,0 +1,206 @@ +--- +sidebar_position: 1 +--- + +# `x/protocolpool` + +## Concepts + +Protopool is a module that handle functionality around community pool funds. This provides a separate module account for community pool making it easier to track the pool assets. We no longer track community pool assets in distribution module, but instead in this protocolpool module. Funds are migrated from the distribution module's community pool to protocolpool's module account. + +The module is also designed with a lazy "claim-based" system, which means that users are required to actively claim allocated funds from allocated budget if any budget proposal has been submitted and passed. The module does not automatically distribute funds to recipients. This design choice allows for more flexibility and control over fund distribution. + +## State Transitions + +### FundCommunityPool + +FundCommunityPool can be called by any valid account to send funds to the protocolpool module account. + +```protobuf + // FundCommunityPool defines a method to allow an account to directly + // fund the community pool. + rpc FundCommunityPool(MsgFundCommunityPool) returns (MsgFundCommunityPoolResponse); +``` + +### CommunityPoolSpend + +CommunityPoolSpend can be called by the module authority (default governance module account) or any account with authorization to spend funds from the protocolpool module account to a receiver address. + +```protobuf + // CommunityPoolSpend defines a governance operation for sending tokens from + // the community pool in the x/protocolpool module to another account, which + // could be the governance module itself. The authority is defined in the + // keeper. + rpc CommunityPoolSpend(MsgCommunityPoolSpend) returns (MsgCommunityPoolSpendResponse); +``` + +### SubmitBudgetProposal + +SubmitBudgetProposal is a message used to propose a budget allocation for a specific recipient. The proposed funds will be distributed periodically over a specified time frame. + +It's the responsibility of users to actively claim their allocated funds based on the terms of the approved budget proposals. + +```protobuf + // SubmitBudgetProposal defines a method to set a budget proposal. + rpc SubmitBudgetProposal(MsgSubmitBudgetProposal) returns (MsgSubmitBudgetProposalResponse); +``` + +### ClaimBudget + +ClaimBudget is a message used to claim funds from a previously submitted budget proposal. When a budget proposal is approved and funds are allocated, recipients can use this message to claim their share of the budget. Funds are distributed in tranches over specific periods, and users can claim their share of budget at the appropriate time. + +```protobuf + // ClaimBudget defines a method to claim the distributed budget. + rpc ClaimBudget(MsgClaimBudget) returns (MsgClaimBudgetResponse); + +``` + +### CreateContinuousFund + +CreateContinuousFund is a message used to initiate a continuous fund for a specific recipient. The proposed percentage of funds will be distributed only on withdraw request for the recipient. The fund distribution continues until expiry time is reached or continuous fund request is canceled. +NOTE: This feature is designed to work with the SDK's default bond denom. + +```protobuf + // CreateContinuousFund defines a method to add funds continuously. + rpc CreateContinuousFund(MsgCreateContinuousFund) returns (MsgCreateContinuousFundResponse); +``` + +### CancelContinuousFund + +CancelContinuousFund is a message used to cancel an existing continuous fund proposal for a specific recipient. Cancelling a continuous fund stops further distribution of funds, and the state object is removed from storage. + +```protobuf + // CancelContinuousFund defines a method for cancelling continuous fund. + rpc CancelContinuousFund(MsgCancelContinuousFund) returns (MsgCancelContinuousFundResponse); +``` + +## Messages + +### MsgFundCommunityPool + +This message sends coins directly from the sender to the community pool. + +:::tip +If you know the protocolpool module account address, you can directly use bank `send` transaction instead. +::: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/proto/cosmos/protocolpool/v1/tx.proto#L43-L53 +``` + +* The msg will fail if the amount cannot be transferred from the sender to the protocolpool module account. + +```go +func (k Keeper) FundCommunityPool(ctx context.Context, amount sdk.Coins, sender sdk.AccAddress) error { + return k.bankKeeper.SendCoinsFromAccountToModule(ctx, sender, types.ModuleName, amount) +} +``` + +### MsgCommunityPoolSpend + +This message distributes funds from the protocolpool module account to the recipient using `DistributeFromCommunityPool` keeper method. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/proto/cosmos/protocolpool/v1/tx.proto#L58-L69 +``` + +The message will fail under the following conditions: + +* The amount cannot be transferred to the recipient from the protocolpool module account. +* The `recipient` address is restricted + +```go +func (k Keeper) DistributeFromCommunityPool(ctx context.Context, amount sdk.Coins, receiveAddr sdk.AccAddress) error { + return k.bankKeeper.SendCoinsFromModuleToAccount(ctx, types.ModuleName, receiveAddr, amount) +} +``` + +### MsgSubmitBudgetProposal + +This message is used to submit a budget proposal to allocate funds for a specific recipient. The proposed funds will be distributed periodically over a specified time frame. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/proto/cosmos/protocolpool/v1/tx.proto#L75-L94 +``` + +The message will fail under the following conditions: + +* The budget per tranche is zero. +* The recipient address is empty or restricted. +* The start time is less than current block time. +* The number of tranches is not a positive integer. +* The period length is not a positive integer. + +:::warning +If two budgets to the same address are created, the budget would be updated with the new budget. +::: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/keeper/msg_server.go#L37-L59 +``` + +### MsgClaimBudget + +This message is used to claim funds from a previously submitted budget proposal. When a budget proposal is passed and funds are allocated, recipients can use this message to claim their share of the budget. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/proto/cosmos/protocolpool/v1/tx.proto#L100-L104 +``` + +The message will fail under the following conditions: + +- The recipient address is empty or restricted. +- The budget proposal for the recipient does not exist. +- The budget proposal has not reached its distribution start time. +- The budget proposal's distribution period has not passed yet. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/keeper/msg_server.go#L28-L35 +``` + +### MsgCreateContinuousFund + +This message is used to create a continuous fund for a specific recipient. The proposed percentage of funds will be distributed only on withdraw request for the recipient. This fund distribution continues until expiry time is reached or continuous fund request is canceled. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/proto/cosmos/protocolpool/v1/tx.proto#L114-L130 +``` + +The message will fail under the following conditions: + +- The recipient address is empty or restricted. +- The percentage is zero/negative/greater than one. +- The Expiry time is less than the current block time. + +:::warning +If two continuous fund proposals to the same address are created, the previous ContinuousFund would be updated with the new ContinuousFund. +::: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/keeper/msg_server.go#L103-L166 +``` + +### MsgCancelContinuousFund + +This message is used to cancel an existing continuous fund proposal for a specific recipient. Once canceled, the continuous fund will no longer distribute funds at each end block, and the state object will be removed. Users should be cautious when canceling continuous funds, as it may affect the planned distribution for the recipient. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/proto/cosmos/protocolpool/v1/tx.proto#L136-L161 +``` + +The message will fail under the following conditions: + +- The recipient address is empty or restricted. +- The ContinuousFund for the recipient does not exist. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/keeper/msg_server.go#L188-L226 +``` + +## Client + +It takes the advantage of `AutoCLI` + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/protocolpool/autocli.go +``` diff --git a/versioned_docs/version-0.52/build/modules/slashing/README.md b/versioned_docs/version-0.52/build/modules/slashing/README.md new file mode 100644 index 000000000..942b3afcc --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/slashing/README.md @@ -0,0 +1,827 @@ +--- +sidebar_position: 1 +--- + +# `x/slashing` + +## Abstract + +This section specifies the slashing module of the Cosmos SDK, which implements functionality +first outlined in the [Cosmos Whitepaper](https://cosmos.network/about/whitepaper) in June 2016. + +The slashing module enables Cosmos SDK-based blockchains to disincentivize any attributable action +by a protocol-recognized actor with value at stake by penalizing them ("slashing"). + +Penalties may include, but are not limited to: + +* Burning some amount of their stake +* Removing their ability to vote on future blocks for a period of time. + +This module will be used by the Cosmos Hub, the first hub in the Cosmos ecosystem. + +## Contents + +* [Concepts](#concepts) + * [States](#states) + * [Tombstone Caps](#tombstone-caps) + * [Infraction Timelines](#infraction-timelines) +* [State](#state) + * [Signing Info (Liveness)](#signing-info-liveness) + * [Params](#params) +* [Messages](#messages) + * [Unjail](#unjail) +* [BeginBlock](#beginblock) + * [Liveness Tracking](#liveness-tracking) +* [Hooks](#hooks) +* [Events](#events) +* [Staking Tombstone](#staking-tombstone) +* [Parameters](#parameters) +* [CLI](#cli) + * [Query](#query) + * [Transactions](#transactions) + * [gRPC](#grpc) + * [REST](#rest) + +## Concepts + +### States + +At any given time, there are any number of validators registered in the state +machine. Each block, the top `MaxValidators` (defined by `x/staking`) validators +who are not jailed become _bonded_, meaning that they may propose and vote on +blocks. Validators who are _bonded_ are _at stake_, meaning that part or all of +their stake and their delegators' stake is at risk if they commit a protocol fault. + +For each of these validators we keep a `ValidatorSigningInfo` record that contains +information pertaining to validator's liveness and other infraction related +attributes. + +### Tombstone Caps + +In order to mitigate the impact of initially likely categories of non-malicious +protocol faults, the Cosmos Hub implements for each validator +a _tombstone_ cap, which only allows a validator to be slashed once for a double +sign fault. For example, if you misconfigure your HSM and double-sign a bunch of +old blocks, you'll only be punished for the first double-sign (and then immediately tombstombed). This will still be quite expensive and desirable to avoid, but tombstone caps +somewhat blunt the economic impact of unintentional misconfiguration. + +Liveness faults do not have caps, as they can't stack upon each other. Liveness bugs are "detected" as soon as the infraction occurs, and the validators are immediately put in jail, so it is not possible for them to commit multiple liveness faults without unjailing in between. + +### Infraction Timelines + +To illustrate how the `x/slashing` module handles submitted evidence through +CometBFT consensus, consider the following examples: + +**Definitions**: + +_[_ : timeline start +_]_ : timeline end +_Cn_ : infraction `n` committed +_Dn_ : infraction `n` discovered +_Vb_ : validator bonded +_Vu_ : validator unbonded + +#### Single Double Sign Infraction + +\[----------C1----D1,Vu-----\] + +A single infraction is committed then later discovered, at which point the +validator is unbonded and slashed at the full amount for the infraction. + +#### Multiple Double Sign Infractions + +\[----------C1--C2---C3---D1,D2,D3Vu-----\] + +Multiple infractions are committed and then later discovered, at which point the +validator is jailed and slashed for only one infraction. Because the validator +is also tombstoned, they can not rejoin the validator set. + +## State + +### Signing Info (Liveness) + +Every block includes a set of precommits by the validators for the previous block, +known as the `LastCommitInfo` provided by CometBFT. A `LastCommitInfo` is valid so +long as it contains precommits from +2/3 of total voting power. + +Proposers are incentivized to include precommits from all validators in the CometBFT `LastCommitInfo` +by receiving additional fees proportional to the difference between the voting +power included in the `LastCommitInfo` and +2/3 (see [fee distribution](../distribution/README.md#begin-block)). + +```go +type LastCommitInfo struct { + Round int32 + Votes []VoteInfo +} +``` + +Validators are penalized for failing to be included in the `LastCommitInfo` for some +number of blocks by being automatically jailed, potentially slashed, and unbonded. + +Information about validator's liveness activity is tracked through `ValidatorSigningInfo`. +It is indexed in the store as follows: + +* ValidatorSigningInfo: `0x01 | ConsAddrLen (1 byte) | ConsAddress -> ProtocolBuffer(ValSigningInfo)` +* MissedBlocksBitArray: `0x02 | ConsAddrLen (1 byte) | ConsAddress | LittleEndianUint64(signArrayIndex) -> VarInt(didMiss)` (varint is a number encoding format) + +The first mapping allows us to easily lookup the recent signing info for a +validator based on the validator's consensus address. + +The second mapping (`MissedBlocksBitArray`) acts +as a bit-array of size `SignedBlocksWindow` that tells us if the validator missed +the block for a given index in the bit-array. The index in the bit-array is given +as little endian uint64. +The result is a `varint` that takes on `0` or `1`, where `0` indicates the +validator did not miss (did sign) the corresponding block, and `1` indicates +they missed the block (did not sign). + +Note that the `MissedBlocksBitArray` is not explicitly initialized up-front. Keys +are added as we progress through the first `SignedBlocksWindow` blocks for a newly +bonded validator. The `SignedBlocksWindow` parameter defines the size +(number of blocks) of the sliding window used to track validator liveness. + +The information stored for tracking validator liveness is as follows: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/slashing/proto/cosmos/slashing/v1beta1/slashing.proto#L13-L35 +``` + +### Params + +The slashing module stores its params in state with the prefix of `0x00`, +it can be updated with governance or the address with authority. + +* Params: `0x00 | ProtocolBuffer(Params)` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/slashing/proto/cosmos/slashing/v1beta1/slashing.proto#L37-L62 +``` + +## Messages + +In this section we describe the processing of messages for the `slashing` module. + +### Unjail + +If a validator was automatically unbonded due to downtime and wishes to come back online & +possibly rejoin the bonded set, it must send `MsgUnjail`: + +```protobuf +// MsgUnjail is an sdk.Msg used for unjailing a jailed validator, thus returning +// them into the bonded validator set, so they can begin receiving provisions +// and rewards again. +message MsgUnjail { + string validator_addr = 1; +} +``` + +Below is a pseudocode of the `MsgSrv/Unjail` RPC: + +```go +unjail(tx MsgUnjail) + validator = getValidator(tx.ValidatorAddr) + if validator == nil + fail with "No validator found" + + if getSelfDelegation(validator) == 0 + fail with "validator must self delegate before unjailing" + + if !validator.Jailed + fail with "Validator not jailed, cannot unjail" + + info = GetValidatorSigningInfo(operator) + if info.Tombstoned + fail with "Tombstoned validator cannot be unjailed" + if block time < info.JailedUntil + fail with "Validator still jailed, cannot unjail until period has expired" + + validator.Jailed = false + setValidator(validator) + + return +``` + +If the validator has enough stake to be in the top `n = MaximumBondedValidators`, it will be automatically rebonded, +and all delegators still delegated to the validator will be rebonded and begin to again collect +provisions and rewards. + +## BeginBlock + +### Liveness Tracking + +At the beginning of each block, we update the `ValidatorSigningInfo` for each +validator and check if they've crossed below the liveness threshold over a +sliding window. This sliding window is defined by `SignedBlocksWindow` and the +index in this window is determined by (`height - SignInfo.StartHeight`). +Notice that the position in the sliding window is incremented every block, +independent of whether the validator signed or not. +Once the index is determined, the `MissedBlocksBitArray` and +`MissedBlocksCounter` are updated accordingly. + +Finally, in order to determine if a validator crosses below the liveness threshold, +we fetch the maximum number of blocks missed, `maxMissed`, which is +`SignedBlocksWindow - (MinSignedPerWindow * SignedBlocksWindow)` and the minimum +height at which we can determine liveness, `minHeight`. If the current block is +greater than `minHeight` and the validator's `MissedBlocksCounter` is greater than +`maxMissed`, they will be slashed by `SlashFractionDowntime`, will be jailed +for `DowntimeJailDuration`, and have the following values reset: +`MissedBlocksBitArray`, `MissedBlocksCounter`, and `IndexOffset`. + +**Note**: Liveness slashes do **NOT** lead to a tombstombing. + +```go +height := block.Height + +for vote in block.LastCommitInfo.Votes { + signInfo := GetValidatorSigningInfo(vote.Validator.Address) + + // This is a relative index, so we counts blocks the validator SHOULD have + // signed. We use the 0-value default signing info if not present. + index := (height - signInfo.StartHeight) % SignedBlocksWindow() + + // Update MissedBlocksBitArray and MissedBlocksCounter. The MissedBlocksCounter + // just tracks the sum of MissedBlocksBitArray. That way we avoid needing to + // read/write the whole array each time. + missedPrevious := GetValidatorMissedBlockBitArray(vote.Validator.Address, index) + missed := !signed + + switch { + case !missedPrevious && missed: + // array index has changed from not missed to missed, increment counter + SetValidatorMissedBlockBitArray(vote.Validator.Address, index, true) + signInfo.MissedBlocksCounter++ + + case missedPrevious && !missed: + // array index has changed from missed to not missed, decrement counter + SetValidatorMissedBlockBitArray(vote.Validator.Address, index, false) + signInfo.MissedBlocksCounter-- + + default: + // array index at this index has not changed; no need to update counter + } + + if missed { + // emit events... + } + + minHeight := signInfo.StartHeight + SignedBlocksWindow() + maxMissed := SignedBlocksWindow() - MinSignedPerWindow() + + // If we are past the minimum height and the validator has missed too many + // jail and slash them. + if height > minHeight && signInfo.MissedBlocksCounter > maxMissed { + validator := ValidatorByConsAddr(vote.Validator.Address) + + // emit events... + + // We need to retrieve the stake distribution which signed the block, so we + // subtract ValidatorUpdateDelay from the block height, and subtract an + // additional 1 since this is the LastCommit. + // + // Note, that this CAN result in a negative "distributionHeight" up to + // -ValidatorUpdateDelay-1, i.e. at the end of the pre-genesis block (none) = at the beginning of the genesis block. + // That's fine since this is just used to filter unbonding delegations & redelegations. + distributionHeight := height - sdk.ValidatorUpdateDelay - 1 + + SlashWithInfractionReason(vote.Validator.Address, distributionHeight, vote.Validator.Power, SlashFractionDowntime(), stakingtypes.Downtime) + Jail(vote.Validator.Address) + + signInfo.JailedUntil = block.Time.Add(DowntimeJailDuration()) + + // We need to reset the counter & array so that the validator won't be + // immediately slashed for downtime upon rebonding. + signInfo.MissedBlocksCounter = 0 + signInfo.IndexOffset = 0 + ClearValidatorMissedBlockBitArray(vote.Validator.Address) + } + + SetValidatorSigningInfo(vote.Validator.Address, signInfo) +} +``` + +## Hooks + +This section contains a description of the module's `hooks`. Hooks are operations that are executed automatically when events are raised. + +### Staking hooks + +The slashing module implements the `StakingHooks` defined in `x/staking` and are used as record-keeping of validators information. During the app initialization, these hooks should be registered in the staking module struct. + +The following hooks impact the slashing state: + +* `AfterValidatorBonded` creates a `ValidatorSigningInfo` instance as described in the following section. +* `AfterValidatorCreated` stores a validator's consensus key. +* `AfterValidatorRemoved` removes a validator's consensus key. +* `AfterConsensusPubKeyUpdate` handles the rotation of signing info and updates the address-pubkey relation after a consensus key update. + +### Validator Bonded + +Upon successful first-time bonding of a new validator, we create a new `ValidatorSigningInfo` structure for the +now-bonded validator, which `StartHeight` of the current block. + +If the validator was out of the validator set and gets bonded again, its new bonded height is set. + +```go +onValidatorBonded(address sdk.ValAddress) + + signingInfo, found = GetValidatorSigningInfo(address) + if !found { + signingInfo = ValidatorSigningInfo { + StartHeight : CurrentHeight, + IndexOffset : 0, + JailedUntil : time.Unix(0, 0), + Tombstone : false, + MissedBloskCounter : 0 + } else { + signingInfo.StartHeight = CurrentHeight + } + + setValidatorSigningInfo(signingInfo) + } + + return +``` + +## Events + +The slashing module emits the following events: + +### MsgServer + +#### MsgUnjail + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ------------------ | +| message | module | slashing | +| message | sender | {validatorAddress} | + +### Keeper + +### BeginBlocker: HandleValidatorSignature + +| Type | Attribute Key | Attribute Value | +| ----- | ------------- | --------------------------- | +| slash | address | {validatorConsensusAddress} | +| slash | power | {validatorPower} | +| slash | reason | {slashReason} | +| slash | jailed [0] | {validatorConsensusAddress} | +| slash | burned coins | {math.Int} | + +* [0] Only included if the validator is jailed. + +| Type | Attribute Key | Attribute Value | +| -------- | ------------- | --------------------------- | +| liveness | address | {validatorConsensusAddress} | +| liveness | missed_blocks | {missedBlocksCounter} | +| liveness | height | {blockHeight} | + +#### Slash + +* same as `"slash"` event from `HandleValidatorSignature`, but without the `jailed` attribute. + +#### Jail + +| Type | Attribute Key | Attribute Value | +| ----- | ------------- | ------------------ | +| slash | jailed | {validatorAddress} | + +## Staking Tombstone + +### Abstract + +In the current implementation of the `slashing` module, when the consensus engine +informs the state machine of a validator's consensus fault, the validator is +partially slashed, and put into a "jail period", a period of time in which they +are not allowed to rejoin the validator set. However, because of the nature of +consensus faults and ABCI, there can be a delay between an infraction occurring, +and evidence of the infraction reaching the state machine (this is one of the +primary reasons for the existence of the unbonding period). + +> Note: The tombstone concept, only applies to faults that have a delay between +> the infraction occurring and evidence reaching the state machine. For example, +> evidence of a validator double signing may take a while to reach the state machine +> due to unpredictable evidence gossip layer delays and the ability of validators to +> selectively reveal double-signatures (e.g. to infrequently-online light clients). +> Liveness slashing, on the other hand, is detected immediately as soon as the +> infraction occurs, and therefore no slashing period is needed. A validator is +> immediately put into jail period, and they cannot commit another liveness fault +> until they unjail. In the future, there may be other types of byzantine faults +> that have delays (for example, submitting evidence of an invalid proposal as a transaction). +> When implemented, it will have to be decided whether these future types of +> byzantine faults will result in a tombstoning (and if not, the slash amounts +> will not be capped by a slashing period). + +In the current system design, once a validator is put in the jail for a consensus +fault, after the `JailPeriod` they are allowed to send a transaction to `unjail` +themselves, and thus rejoin the validator set. + +One of the "design desires" of the `slashing` module is that if multiple +infractions occur before evidence is executed (and a validator is put in jail), +they should only be punished for single worst infraction, but not cumulatively. +For example, if the sequence of events is: + +1. Validator A commits Infraction 1 (worth 30% slash) +2. Validator A commits Infraction 2 (worth 40% slash) +3. Validator A commits Infraction 3 (worth 35% slash) +4. Evidence for Infraction 1 reaches state machine (and validator is put in jail) +5. Evidence for Infraction 2 reaches state machine +6. Evidence for Infraction 3 reaches state machine + +Only Infraction 2 should have its slash take effect, as it is the highest. This +is done, so that in the case of the compromise of a validator's consensus key, +they will only be punished once, even if the hacker double-signs many blocks. +Because, the unjailing has to be done with the validator's operator key, they +have a chance to re-secure their consensus key, and then signal that they are +ready using their operator key. We call this period during which we track only +the max infraction, the "slashing period". + +Once, a validator rejoins by unjailing themselves, we begin a new slashing period; +if they commit a new infraction after unjailing, it gets slashed cumulatively on +top of the worst infraction from the previous slashing period. + +However, while infractions are grouped based off of the slashing periods, because +evidence can be submitted up to an `unbondingPeriod` after the infraction, we +still have to allow for evidence to be submitted for previous slashing periods. +For example, if the sequence of events is: + +1. Validator A commits Infraction 1 (worth 30% slash) +2. Validator A commits Infraction 2 (worth 40% slash) +3. Evidence for Infraction 1 reaches state machine (and Validator A is put in jail) +4. Validator A unjails + +We are now in a new slashing period, however we still have to keep the door open +for the previous infraction, as the evidence for Infraction 2 may still come in. +As the number of slashing periods increase, it creates more complexity as we have +to keep track of the highest infraction amount for every single slashing period. + +> Note: Currently, according to the `slashing` module spec, a new slashing period +> is created every time a validator is unbonded then rebonded. This should probably +> be changed to jailed/unjailed. See issue [#3205](https://github.com/cosmos/cosmos-sdk/issues/3205) +> for further details. For the remainder of this, I will assume that we only start +> a new slashing period when a validator gets unjailed. + +The maximum number of slashing periods is the `len(UnbondingPeriod) / len(JailPeriod)`. +The current defaults in Gaia for the `UnbondingPeriod` and `JailPeriod` are 3 weeks +and 2 days, respectively. This means there could potentially be up to 11 slashing +periods concurrently being tracked per validator. If we set the `JailPeriod >= UnbondingPeriod`, +we only have to track 1 slashing period (i.e not have to track slashing periods). + +Currently, in the jail period implementation, once a validator unjails, all of +their delegators who are delegated to them (haven't unbonded / redelegated away), +stay with them. Given that consensus safety faults are so egregious +(way more so than liveness faults), it is probably prudent to have delegators not +"auto-rebond" to the validator. + +#### Proposal: infinite jail + +We propose setting the "jail time" for a +validator who commits a consensus safety fault, to `infinite` (i.e. a tombstone state). +This essentially kicks the validator out of the validator set and does not allow +them to re-enter the validator set. All of their delegators (including the operator themselves) +have to either unbond or redelegate away. The validator operator can create a new +validator if they would like, with a new operator key and consensus key, but they +have to "re-earn" their delegations back. + +Implementing the tombstone system and getting rid of the slashing period tracking +will make the `slashing` module way simpler, especially because we can remove all +of the hooks defined in the `slashing` module consumed by the `staking` module +(the `slashing` module still consumes hooks defined in `staking`). + +#### Single slashing amount + +Another optimization that can be made is that if we assume that all ABCI faults +for CometBFT consensus are slashed at the same level, we don't have to keep +track of "max slash". Once an ABCI fault happens, we don't have to worry about +comparing potential future ones to find the max. + +Currently the only CometBFT ABCI fault is: + +* Unjustified precommits (double signs) + +It is currently planned to include the following fault in the near future: + +* Signing a precommit when you're in unbonding phase (needed to make light client bisection safe) + +Given that these faults are both attributable byzantine faults, we will likely +want to slash them equally, and thus we can enact the above change. + +> Note: This change may make sense for current CometBFT consensus, but maybe +> not for a different consensus algorithm or future versions of CometBFT that +> may want to punish at different levels (for example, partial slashing). + +## Parameters + +The slashing module contains the following parameters: + +| Key | Type | Example | +| ----------------------- | -------------- | ---------------------- | +| SignedBlocksWindow | string (int64) | "100" | +| MinSignedPerWindow | string (dec) | "0.500000000000000000" | +| DowntimeJailDuration | string (ns) | "600000000000" | +| SlashFractionDoubleSign | string (dec) | "0.050000000000000000" | +| SlashFractionDowntime | string (dec) | "0.010000000000000000" | + +## CLI + +A user can query and interact with the `slashing` module using the CLI. + +### Query + +The `query` commands allow users to query `slashing` state. + +```shell +simd query slashing --help +``` + +#### params + +The `params` command allows users to query genesis parameters for the slashing module. + +```shell +simd query slashing params [flags] +``` + +Example: + +```shell +simd query slashing params +``` + +Example Output: + +```yml +downtime_jail_duration: 600s +min_signed_per_window: "0.500000000000000000" +signed_blocks_window: "100" +slash_fraction_double_sign: "0.050000000000000000" +slash_fraction_downtime: "0.010000000000000000" +``` + +#### signing-info + +The `signing-info` command allows users to query signing-info of the validator using consensus public key. + +```shell +simd query slashing signing-infos [flags] +``` + +Example: + +```shell +simd query slashing signing-info '{"@type":"/cosmos.crypto.ed25519.PubKey","key":"Auxs3865HpB/EfssYOzfqNhEJjzys6jD5B6tPgC8="}' + +``` + +Example Output: + +```yml +address: cosmosvalcons1nrqsld3aw6lh6t082frdqc84uwxn0t958c +index_offset: "2068" +jailed_until: "1970-01-01T00:00:00Z" +missed_blocks_counter: "0" +start_height: "0" +tombstoned: false +``` + +#### signing-infos + +The `signing-infos` command allows users to query signing infos of all validators. + +```shell +simd query slashing signing-infos [flags] +``` + +Example: + +```shell +simd query slashing signing-infos +``` + +Example Output: + +```yml +info: +- address: cosmosvalcons1nrqsld3aw6lh6t082frdqc84uwxn0t958c + index_offset: "2075" + jailed_until: "1970-01-01T00:00:00Z" + missed_blocks_counter: "0" + start_height: "0" + tombstoned: false +pagination: + next_key: null + total: "0" +``` + +### Transactions + +The `tx` commands allow users to interact with the `slashing` module. + +```bash +simd tx slashing --help +``` + +#### unjail + +The `unjail` command allows users to unjail a validator previously jailed for downtime. + +```bash +simd tx slashing unjail --from mykey [flags] +``` + +Example: + +```bash +simd tx slashing unjail --from mykey +``` + +#### update-params-proposal + +The `update-params-proposal` command allows users to submit a governance proposal to update the slashing module parameters: + +```bash +simd tx slashing update-params-proposal [flags] +``` + +Example: + +```bash +simd tx slashing update-params-proposal '{ "signed_blocks_window": "100" }' +``` + +### gRPC + +A user can query the `slashing` module using gRPC endpoints. + +#### Params + +The `Params` endpoint allows users to query the parameters of slashing module. + +```shell +cosmos.slashing.v1beta1.Query/Params +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.slashing.v1beta1.Query/Params +``` + +Example Output: + +```json +{ + "params": { + "signedBlocksWindow": "100", + "minSignedPerWindow": "NTAwMDAwMDAwMDAwMDAwMDAw", + "downtimeJailDuration": "600s", + "slashFractionDoubleSign": "NTAwMDAwMDAwMDAwMDAwMDA=", + "slashFractionDowntime": "MTAwMDAwMDAwMDAwMDAwMDA=" + } +} +``` + +#### SigningInfo + +The SigningInfo queries the signing info of given cons address. + +```shell +cosmos.slashing.v1beta1.Query/SigningInfo +``` + +Example: + +```shell +grpcurl -plaintext -d '{"cons_address":"cosmosvalcons1nrqsld3aw6lh6t082frdqc84uwxn0t958c"}' localhost:9090 cosmos.slashing.v1beta1.Query/SigningInfo +``` + +Example Output: + +```json +{ + "valSigningInfo": { + "address": "cosmosvalcons1nrqsld3aw6lh6t082frdqc84uwxn0t958c", + "indexOffset": "3493", + "jailedUntil": "1970-01-01T00:00:00Z" + } +} +``` + +#### SigningInfos + +The SigningInfos queries signing info of all validators. + +```shell +cosmos.slashing.v1beta1.Query/SigningInfos +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.slashing.v1beta1.Query/SigningInfos +``` + +Example Output: + +```json +{ + "info": [ + { + "address": "cosmosvalcons1nrqslkwd3pz096lh6t082frdqc84uwxn0t958c", + "indexOffset": "2467", + "jailedUntil": "1970-01-01T00:00:00Z" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### REST + +A user can query the `slashing` module using REST endpoints. + +#### Params + +```shell +/cosmos/slashing/v1beta1/params +``` + +Example: + +```shell +curl "localhost:1317/cosmos/slashing/v1beta1/params" +``` + +Example Output: + +```json +{ + "params": { + "signed_blocks_window": "100", + "min_signed_per_window": "0.500000000000000000", + "downtime_jail_duration": "600s", + "slash_fraction_double_sign": "0.050000000000000000", + "slash_fraction_downtime": "0.010000000000000000" +} +``` + +#### signing_info + +```shell +/cosmos/slashing/v1beta1/signing_infos/%s +``` + +Example: + +```shell +curl "localhost:1317/cosmos/slashing/v1beta1/signing_infos/cosmosvalcons1nrqslkwd3pz096lh6t082frdqc84uwxn0t958c" +``` + +Example Output: + +```json +{ + "val_signing_info": { + "address": "cosmosvalcons1nrqslkwd3pz096lh6t082frdqc84uwxn0t958c", + "start_height": "0", + "index_offset": "4184", + "jailed_until": "1970-01-01T00:00:00Z", + "tombstoned": false, + "missed_blocks_counter": "0" + } +} +``` + +#### signing_infos + +```shell +/cosmos/slashing/v1beta1/signing_infos +``` + +Example: + +```shell +curl "localhost:1317/cosmos/slashing/v1beta1/signing_infos +``` + +Example Output: + +```json +{ + "info": [ + { + "address": "cosmosvalcons1nrqslkwd3pz096lh6t082frdqc84uwxn0t958c", + "start_height": "0", + "index_offset": "4169", + "jailed_until": "1970-01-01T00:00:00Z", + "tombstoned": false, + "missed_blocks_counter": "0" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` diff --git a/versioned_docs/version-0.52/build/modules/staking/README.md b/versioned_docs/version-0.52/build/modules/staking/README.md new file mode 100644 index 000000000..9e302bfba --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/staking/README.md @@ -0,0 +1,2952 @@ +--- +sidebar_position: 1 +--- + +# `x/staking` + +## Abstract + +This paper specifies the Staking module of the Cosmos SDK that was first +described in the [Cosmos Whitepaper](https://cosmos.network/about/whitepaper) +in June 2016. + +The module enables Cosmos SDK-based blockchain to support an advanced +Proof-of-Stake (PoS) system. In this system, holders of the native staking token of +the chain can become validators and can delegate tokens to validators, +ultimately determining the effective validator set for the system. + +This module is used in the Cosmos Hub, the first Hub in the Cosmos +network. + +## Contents + +* [State](#state) + * [Pool](#pool) + * [LastTotalPower](#lasttotalpower) + * [UnbondingID](#unbondingid) + * [Params](#params) + * [Validator](#validator) + * [Delegation](#delegation) + * [UnbondingDelegation](#unbondingdelegation) + * [Redelegation](#redelegation) + * [Queues](#queues) + * [ConsPubkeyRotation](#conspubkeyrotation) +* [State Transitions](#state-transitions) + * [Validators](#validators) + * [Delegations](#delegations) + * [Slashing](#slashing) + * [How Shares are calculated](#how-shares-are-calculated) +* [Messages](#messages) + * [MsgCreateValidator](#msgcreatevalidator) + * [MsgEditValidator](#msgeditvalidator) + * [MsgDelegate](#msgdelegate) + * [MsgUndelegate](#msgundelegate) + * [MsgCancelUnbondingDelegation](#msgcancelunbondingdelegation) + * [MsgBeginRedelegate](#msgbeginredelegate) + * [MsgUpdateParams](#msgupdateparams) + * [MsgRotateConsPubkey](#msgrotateconspubkey) +* [End-Block](#end-block) + * [Validator Set Changes](#validator-set-changes) + * [Queues](#queues-1) +* [Hooks](#hooks) +* [Events](#events) + * [EndBlocker](#endblocker) + * [Msg's](#msgs) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) + +## State + +### Pool + +Pool is used for tracking bonded and not-bonded token supply of the bond denomination. + +### LastTotalPower + +LastTotalPower tracks the total amounts of bonded tokens recorded during the previous end block. +Store entries prefixed with "Last" must remain unchanged until EndBlock. + +* LastTotalPower: `0x12 -> ProtocolBuffer(math.Int)` + +### UnbondingID + +UnbondingID stores the ID of the latest unbonding operation. It enables creating unique IDs for unbonding operations, i.e., UnbondingID is incremented every time a new unbonding operation (validator unbonding, unbonding delegation, redelegation) is initiated. + +* UnbondingID: `0x37 -> uint64` + +### Params + +The staking module stores its params in state with the prefix of `0x51`, +it can be updated with governance or the address with authority. + +* Params: `0x51 | ProtocolBuffer(Params)` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L300-L328 +``` + +### Validator + +Validators can have one of three statuses + +* `Unbonded`: The validator is not in the active set. They cannot sign blocks and do not earn + rewards. They can receive delegations. +* `Bonded`: Once the validator receives sufficient bonded tokens they automatically join the + active set during [`EndBlock`](#validator-set-changes) and their status is updated to `Bonded`. + They are signing blocks and receiving rewards. They can receive further delegations. + They can be slashed for misbehavior. Delegators to this validator who unbond their delegation + must wait the duration of the UnbondingTime, a chain-specific param, during which time + they are still slashable for offences of the source validator if those offences were committed + during the period of time that the tokens were bonded. +* `Unbonding`: When a validator leaves the active set, either by choice or due to slashing, jailing or + tombstoning, an unbonding of all their delegations begins. All delegations must then wait the UnbondingTime + before their tokens are moved to their accounts from the `BondedPool`. + +:::warning +Tombstoning is permanent, once tombstoned a validator's consensus key can not be reused within the chain where the tombstoning happened. +::: + +Validators objects should be primarily stored and accessed by the +`OperatorAddr`, an SDK validator address for the operator of the validator. Two +additional indices are maintained per validator object in order to fulfill +required lookups for slashing and validator-set updates. A third special index +(`LastValidatorPower`) is also maintained which however remains constant +throughout each block, unlike the first two indices which mirror the validator +records within a block. + +* Validators: `0x21 | OperatorAddrLen (1 byte) | OperatorAddr -> ProtocolBuffer(validator)` +* ValidatorsByConsAddr: `0x22 | ConsAddrLen (1 byte) | ConsAddr -> OperatorAddr` +* ValidatorsByPower: `0x23 | BigEndian(ConsensusPower) | OperatorAddrLen (1 byte) | OperatorAddr -> OperatorAddr` +* LastValidatorsPower: `0x11 | OperatorAddrLen (1 byte) | OperatorAddr -> ProtocolBuffer(ConsensusPower)` +* ValidatorsByUnbondingID: `0x38 | UnbondingID -> 0x21 | OperatorAddrLen (1 byte) | OperatorAddr` + +`Validators` is the primary index - it ensures that each operator can have only one +associated validator, where the public key of that validator can change in the +future. Delegators can refer to the immutable operator of the validator, without +concern for the changing public key. + +`ValidatorsByUnbondingID` is an additional index that enables lookups for + validators by the unbonding IDs corresponding to their current unbonding. + +`ValidatorByConsAddr` is an additional index that enables lookups for slashing. +When CometBFT reports evidence, it provides the validator address, so this +map is needed to find the operator. Note that the `ConsAddr` corresponds to the +address which can be derived from the validator's `ConsPubKey`. + +`ValidatorsByPower` is an additional index that provides a sorted list of +potential validators to quickly determine the current active set. Here +ConsensusPower is validator.Tokens/10^6 by default. Note that all validators +where `Jailed` is true are not stored within this index. + +`LastValidatorsPower` is a special index that provides a historical list of the +last-block's bonded validators. This index remains constant during a block but +is updated during the validator set update process which takes place in [`EndBlock`](#end-block). + +Each validator's state is stored in a `Validator` struct: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L92-L138 +``` + + +The initial commission rates to be used for creating a validator are stored in a `CommissionRates` struct: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L30-L54 +``` + +### Delegation + +Delegations are identified by combining `DelegatorAddr` (the address of the delegator) +with the `ValidatorAddr` Delegators are indexed in the store as follows: + +* Delegation: `0x31 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorAddr -> ProtocolBuffer(delegation)` + +Stake holders may delegate coins to validators; under this circumstance their +funds are held in a `Delegation` data structure. It is owned by one +delegator, and is associated with the shares for one validator. The sender of +the transaction is the owner of the bond. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L196-L210 +``` + +#### Delegator Shares + +When one delegates tokens to a Validator, they are issued a number of delegator shares based on a +dynamic exchange rate, calculated as follows from the total number of tokens delegated to the +validator and the number of shares issued so far: + +`Shares per Token = validator.TotalShares() / validator.Tokens()` + +Only the number of shares received is stored on the DelegationEntry. When a delegator then +Undelegates, the token amount they receive is calculated from the number of shares they currently +hold and the inverse exchange rate: + +`Tokens per Share = validator.Tokens() / validatorShares()` + +These `Shares` are simply an accounting mechanism. They are not a fungible asset. The reason for +this mechanism is to simplify the accounting around slashing. Rather than iteratively slashing the +tokens of every delegation entry, instead the Validator's total bonded tokens can be slashed, +effectively reducing the value of each issued delegator share. + +### UnbondingDelegation + +Shares in a `Delegation` can be unbonded, but they must for some time exist as +an `UnbondingDelegation`, where shares can be reduced if Byzantine behavior is +detected. + +`UnbondingDelegation` are indexed in the store as: + +* UnbondingDelegation: `0x32 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorAddr -> ProtocolBuffer(unbondingDelegation)` +* UnbondingDelegationsFromValidator: `0x33 | ValidatorAddrLen (1 byte) | ValidatorAddr | DelegatorAddrLen (1 byte) | DelegatorAddr -> nil` +* UnbondingDelegationByUnbondingId: `0x38 | UnbondingId -> 0x32 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorAddr` + `UnbondingDelegation` is used in queries, to lookup all unbonding delegations for + a given delegator. + +`UnbondingDelegationsFromValidator` is used in slashing, to lookup all + unbonding delegations associated with a given validator that need to be + slashed. + + `UnbondingDelegationByUnbondingId` is an additional index that enables + lookups for unbonding delegations by the unbonding IDs of the containing + unbonding delegation entries. + + +A UnbondingDelegation object is created every time an unbonding is initiated. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L214-L253 +``` + +### Redelegation + +The bonded tokens worth of a `Delegation` may be instantly redelegated from a +source validator to a different validator (destination validator). However when +this occurs they must be tracked in a `Redelegation` object, whereby their +shares can be slashed if their tokens have contributed to a Byzantine fault +committed by the source validator. + +`Redelegation` are indexed in the store as: + +* Redelegations: `0x34 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorSrcAddr | ValidatorDstAddr -> ProtocolBuffer(redelegation)` +* RedelegationsBySrc: `0x35 | ValidatorSrcAddrLen (1 byte) | ValidatorSrcAddr | ValidatorDstAddrLen (1 byte) | ValidatorDstAddr | DelegatorAddrLen (1 byte) | DelegatorAddr -> nil` +* RedelegationsByDst: `0x36 | ValidatorDstAddrLen (1 byte) | ValidatorDstAddr | ValidatorSrcAddrLen (1 byte) | ValidatorSrcAddr | DelegatorAddrLen (1 byte) | DelegatorAddr -> nil` +* RedelegationByUnbondingId: `0x38 | UnbondingId -> 0x34 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorSrcAddr | ValidatorDstAddr` + + `Redelegations` is used for queries, to lookup all redelegations for a given + delegator. + + `RedelegationsBySrc` is used for slashing based on the `ValidatorSrcAddr`. + + `RedelegationsByDst` is used for slashing based on the `ValidatorDstAddr` + +The first map here is used for queries, to lookup all redelegations for a given +delegator. The second map is used for slashing based on the `ValidatorSrcAddr`, +while the third map is for slashing based on the `ValidatorDstAddr`. + +`RedelegationByUnbondingId` is an additional index that enables + lookups for redelegations by the unbonding IDs of the containing + redelegation entries. + +A redelegation object is created every time a redelegation occurs. To prevent +"redelegation hopping" redelegations may not occur under the situation that: + +* the (re)delegator already has another immature redelegation in progress + with a destination to a validator (let's call it `Validator X`) +* and, the (re)delegator is attempting to create a _new_ redelegation + where the source validator for this new redelegation is `Validator X`. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L256-L298 +``` + +## ConsPubkeyRotation + +The `ConsPubkey` of a validator will be instantly rotated to the new `ConsPubkey`. The rotation will be tracked to only allow a limited number of rotations within an unbonding period of time. + +`ConsPubkeyRotation` are indexed in the store as: + +ValidatorConsPubKeyRotationHistoryKey: `101 | valAddr | rotatedHeight -> ProtocolBuffer(ConsPubKeyRotationHistory)`555682 + +BlockConsPubKeyRotationHistoryKey (index): `102 | rotatedHeight | valAddr | -> ProtocolBuffer(ConsPubKeyRotationHistory)` + +ValidatorConsensusKeyRotationRecordQueueKey: `103 | format(time) -> ProtocolBuffer(ValAddrsOfRotatedConsKeys)` + +ValidatorConsensusKeyRotationRecordIndexKey:`104 | valAddr | format(time) -> ProtocolBuffer([]Byte{})` + +OldToNewConsAddrMap:`105 | byte(oldConsAddr) -> byte(newConsAddr)` + +ConsAddrToValidatorIdentifierMap:`106 | byte(newConsAddr) -> byte(initialConsAddr)` + +`ConsPubKeyRotationHistory` is used for querying the rotations of a validator + +`ValidatorConsensusKeyRotationRecordQueueKey` is to keep track of the rotation across the unbonding period (waiting period in the queue), this will be pruned after the unbonding period of waiting time. + +`ValidatorConsensusKeyRotationRecordIndexKey` is to keep track of a validator that how many rotations were made inside unbonding period. This will be pruned after the unbonding period of waiting time. + +A `ConsPubKeyRotationHistory` object is created every time a consensus pubkey rotation occurs. + +An entry is added in `OldToNewConsAddrMap` collection for every rotation (Note: this is to handle the evidences when submitted with old cons key). + +An entry is added in `ConsAddrToValidatorIdentifierMap` collection for every rotation, this entry is to block the rotation if the validator is rotating to the cons key which is involved in the history. + +To prevent the spam: + +* There will only limited number of rotations can be done within unbonding period of time. +* A non-negligible fee will be deducted for rotating a consensus key. + +### Queues + +All queue objects are sorted by timestamp. The time used within any queue is +firstly converted to UTC, rounded to the nearest nanosecond then sorted. The sortable time format +used is a slight modification of the RFC3339Nano and uses the format string +`"2006-01-02T15:04:05.000000000"`. Notably this format: + +* right pads all zeros +* drops the time zone info (we already use UTC) + +In all cases, the stored timestamp represents the maturation time of the queue +element. + +#### UnbondingDelegationQueue + +For the purpose of tracking progress of unbonding delegations the unbonding +delegations queue is kept. + +* UnbondingDelegation: `0x41 | format(time) -> []DVPair` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L159-L173 +``` + +#### RedelegationQueue + +For the purpose of tracking progress of redelegations the redelegation queue is +kept. + +* RedelegationQueue: `0x42 | format(time) -> []DVVTriplet` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L175-L191 +``` + +#### ValidatorQueue + +For the purpose of tracking progress of unbonding validators the validator +queue is kept. + +* ValidatorQueueTime: `0x43 | format(time) -> []sdk.ValAddress` + +The stored object by each key is an array of validator operator addresses from +which the validator object can be accessed. Typically it is expected that only +a single validator record will be associated with a given timestamp however it is possible +that multiple validators exist in the queue at the same location. + +#### ValidatorConsensusKeyRotationRecordQueueKey + +For the purpose of tracking progress or consensus pubkey rotations the `ValidatorConsensusKeyRotationRecordQueueKey` kept. + +* ValidatorConsensusKeyRotationRecordQueueKey: `103 | format(time) -> types.ValAddrsOfRotatedConsKeys` + +Here timestamp will be the unique identifier in the queue which is of future time +(which is calculated with the current block time adding with unbonding period), +Whenever the next item with the same waiting time comes to the queue, we will get +the present store info and append the `ValAddress` to the array and set it back in the store. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/staking.proto#L420-L424 +``` + + + + +## State Transitions + +### Validators + +State transitions in validators are performed on every [`EndBlock`](#validator-set-changes) +in order to check for changes in the active `ValidatorSet`. + +A validator can be `Unbonded`, `Unbonding` or `Bonded`. `Unbonded` +and `Unbonding` are collectively called `Not Bonded`. A validator can move +directly between all the states, except for from `Bonded` to `Unbonded`. + +#### Not bonded to Bonded + +The following transition occurs when a validator's ranking in the `ValidatorPowerIndex` surpasses +that of the `LastValidator`. + +* set `validator.Status` to `Bonded` +* send the `validator.Tokens` from the `NotBondedTokens` to the `BondedPool` `ModuleAccount` +* delete the existing record from `ValidatorByPowerIndex` +* add a new updated record to the `ValidatorByPowerIndex` +* update the `Validator` object for this validator +* if it exists, delete any `ValidatorQueue` record for this validator + +#### Bonded to Unbonding + +When a validator begins the unbonding process the following operations occur: + +* send the `validator.Tokens` from the `BondedPool` to the `NotBondedTokens` `ModuleAccount` +* set `validator.Status` to `Unbonding` +* delete the existing record from `ValidatorByPowerIndex` +* add a new updated record to the `ValidatorByPowerIndex` +* update the `Validator` object for this validator +* insert a new record into the `ValidatorQueue` for this validator + +#### Unbonding to Unbonded + +A validator moves from unbonding to unbonded when the `ValidatorQueue` object +moves from bonded to unbonded + +* update the `Validator` object for this validator +* set `validator.Status` to `Unbonded` + +#### Jail/Unjail + +when a validator is jailed it is effectively removed from the CometBFT set. +this process may be also be reversed. the following operations occur: + +* set `Validator.Jailed` and update object +* if jailed delete record from `ValidatorByPowerIndex` +* if unjailed add record to `ValidatorByPowerIndex` + +Jailed validators are not present in any of the following stores: + +* the power store (from consensus power to address) + +### Delegations + +#### Delegate + +When a delegation occurs both the validator and the delegation objects are affected + +* determine the delegators shares based on tokens delegated and the validator's exchange rate +* remove tokens from the sending account +* add shares the delegation object or add them to a created validator object +* add new delegator shares and update the `Validator` object +* transfer the `delegation.Amount` from the delegator's account to the `BondedPool` or the `NotBondedPool` `ModuleAccount` depending if the `validator.Status` is `Bonded` or not +* delete the existing record from `ValidatorByPowerIndex` +* add an new updated record to the `ValidatorByPowerIndex` + +#### Begin Unbonding + +As a part of the Undelegate and Complete Unbonding state transitions Unbond +Delegation may be called. + +* subtract the unbonded shares from delegator +* add the unbonded tokens to an `UnbondingDelegationEntry` +* update the delegation or remove the delegation if there are no more shares +* if the delegation is the operator of the validator and no more shares exist then trigger a jail validator +* update the validator with removed the delegator shares and associated coins +* if the validator state is `Bonded`, transfer the `Coins` worth of the unbonded + shares from the `BondedPool` to the `NotBondedPool` `ModuleAccount` +* remove the validator if it is unbonded and there are no more delegation shares. +* remove the validator if it is unbonded and there are no more delegation shares +* get a unique `unbondingId` and map it to the `UnbondingDelegationEntry` in `UnbondingDelegationByUnbondingId` +* call the `AfterUnbondingInitiated(unbondingId)` hook +* add the unbonding delegation to `UnbondingDelegationQueue` with the completion time set to `UnbondingTime` + +#### Cancel an `UnbondingDelegation` Entry + +When a `cancel unbond delegation` occurs both the `validator`, the `delegation` and an `UnbondingDelegationQueue` state will be updated. + +* if cancel unbonding delegation amount equals to the `UnbondingDelegation` entry `balance`, then the `UnbondingDelegation` entry deleted from `UnbondingDelegationQueue`. +* if the `cancel unbonding delegation amount is less than the `UnbondingDelegation` entry balance, then the `UnbondingDelegation` entry will be updated with new balance in the `UnbondingDelegationQueue`. +* cancel `amount` is [Delegated](#delegations) back to the original `validator`. + +#### Complete Unbonding + +For undelegations which do not complete immediately, the following operations +occur when the unbonding delegation queue element matures: + +* remove the entry from the `UnbondingDelegation` object +* transfer the tokens from the `NotBondedPool` `ModuleAccount` to the delegator `Account` + +#### Begin Redelegation + +Redelegations affect the delegation, source and destination validators. + +* perform an `unbond` delegation from the source validator to retrieve the tokens worth of the unbonded shares +* using the unbonded tokens, `Delegate` them to the destination validator +* if the `sourceValidator.Status` is `Bonded`, and the `destinationValidator` is not, + transfer the newly delegated tokens from the `BondedPool` to the `NotBondedPool` `ModuleAccount` +* otherwise, if the `sourceValidator.Status` is not `Bonded`, and the `destinationValidator` + is `Bonded`, transfer the newly delegated tokens from the `NotBondedPool` to the `BondedPool` `ModuleAccount` +* record the token amount in an new entry in the relevant `Redelegation` + +From when a redelegation begins until it completes, the delegator is in a state of "pseudo-unbonding", and can still be +slashed for infractions that occurred before the redelegation began. + +#### Complete Redelegation + +When a redelegations complete the following occurs: + +* remove the entry from the `Redelegation` object + +#### Consensus pubkey rotation + +When a `ConsPubkeyRotation` occurs the validator and the `ValidatorConsensusKeyRotationRecordQueueKey` are updated: + +* the old consensus pubkey address will be removed from state and new consensus pubkey address will be added in place. +* transfers the voting power to the new consensus pubkey address. +* and triggers the hooks to update the `signing-info` in the `slashing` module +* and triggers the hooks to add the deducted fee to the `community pool` funds + +### Slashing + +#### Slash Validator + +When a Validator is slashed, the following occurs: + +* The total `slashAmount` is calculated as the `slashFactor` (a chain parameter) \* `TokensFromConsensusPower`, + the total number of tokens bonded to the validator at the time of the infraction. +* Every unbonding delegation and pseudo-unbonding redelegation such that the infraction occurred before the unbonding or + redelegation began from the validator are slashed by the `slashFactor` percentage of the initialBalance. +* Each amount slashed from redelegations and unbonding delegations is subtracted from the + total slash amount. +* The `remaingSlashAmount` is then slashed from the validator's tokens in the `BondedPool` or + `NonBondedPool` depending on the validator's status. This reduces the total supply of tokens. + +In the case of a slash due to any infraction that requires evidence to submitted (for example double-sign), the slash +occurs at the block where the evidence is included, not at the block where the infraction occurred. +Put otherwise, validators are not slashed retroactively, only when they are caught. + +#### Slash Unbonding Delegation + +When a validator is slashed, so are those unbonding delegations from the validator that began unbonding +after the time of the infraction. Every entry in every unbonding delegation from the validator +is slashed by `slashFactor`. The amount slashed is calculated from the `InitialBalance` of the +delegation and is capped to prevent a resulting negative balance. Completed (or mature) unbondings are not slashed. + +#### Slash Redelegation + +When a validator is slashed, so are all redelegations from the validator that began after the +infraction. Redelegations are slashed by `slashFactor`. +Redelegations that began before the infraction are not slashed. +The amount slashed is calculated from the `InitialBalance` of the delegation and is capped to +prevent a resulting negative balance. +Mature redelegations (that have completed pseudo-unbonding) are not slashed. + +### How Shares are calculated + +At any given point in time, each validator has a number of tokens, `T`, and has a number of shares issued, `S`. +Each delegator, `i`, holds a number of shares, `S_i`. +The number of tokens is the sum of all tokens delegated to the validator, plus the rewards, minus the slashes. + +The delegator is entitled to a portion of the underlying tokens proportional to their proportion of shares. +So delegator `i` is entitled to `T * S_i / S` of the validator's tokens. + +When a delegator delegates new tokens to the validator, they receive a number of shares proportional to their contribution. +So when delegator `j` delegates `T_j` tokens, they receive `S_j = S * T_j / T` shares. +The total number of tokens is now `T + T_j`, and the total number of shares is `S + S_j`. +`j`s proportion of the shares is the same as their proportion of the total tokens contributed: `(S + S_j) / S = (T + T_j) / T`. + +A special case is the initial delegation, when `T = 0` and `S = 0`, so `T_j / T` is undefined. +For the initial delegation, delegator `j` who delegates `T_j` tokens receive `S_j = T_j` shares. +So a validator that hasn't received any rewards and has not been slashed will have `T = S`. + +## Messages + +In this section we describe the processing of the staking messages and the corresponding updates to the state. All created/modified state objects specified by each message are defined within the [state](#state) section. + +### MsgCreateValidator + +A validator is created using the `MsgCreateValidator` message. +The validator must be created with an initial delegation from the operator. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L20-L21 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L57-L80 +``` + +This message is expected to fail if: + +* another validator with this operator address is already registered +* another validator with this pubkey is already registered +* the initial self-delegation tokens are of a denom not specified as the bonding denom +* the commission parameters are faulty, namely: + * `MaxRate` is either > 1 or < 0 + * the initial `Rate` is either negative or > `MaxRate` + * the initial `MaxChangeRate` is either negative or > `MaxRate` +* the description fields are too large + +This message creates and stores the `Validator` object at appropriate indexes. +Additionally a self-delegation is made with the initial tokens delegation +tokens `Delegation`. The validator always starts as unbonded but may be bonded +in the first end-block. + +### MsgEditValidator + +The `Description`, `CommissionRate` of a validator can be updated using the +`MsgEditValidator` message. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L23-L24 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L85-L104 +``` + +This message is expected to fail if: + +* the initial `CommissionRate` is either negative or > `MaxRate` +* the `CommissionRate` has already been updated within the previous 24 hours +* the `CommissionRate` is > `MaxChangeRate` +* the description fields are too large + +This message stores the updated `Validator` object. + +### MsgDelegate + +Within this message the delegator provides coins, and in return receives +some amount of their validator's (newly created) delegator-shares that are +assigned to `Delegation.Shares`. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L26-L28 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L109-L121 +``` + +This message is expected to fail if: + +* the validator does not exist +* the `Amount` `Coin` has a denomination different than one defined by `params.BondDenom` +* the exchange rate is invalid, meaning the validator has no tokens (due to slashing) but there are outstanding shares +* the amount delegated is less than the minimum allowed delegation + +If an existing `Delegation` object for provided addresses does not already +exist then it is created as part of this message otherwise the existing +`Delegation` is updated to include the newly received shares. + +The delegator receives newly minted shares at the current exchange rate. +The exchange rate is the number of existing shares in the validator divided by +the number of currently delegated tokens. + +The validator is updated in the `ValidatorByPower` index, and the delegation is +tracked in validator object in the `Validators` index. + +It is possible to delegate to a jailed validator, the only difference being it +will not be added to the power index until it is unjailed. + +![Delegation sequence](https://raw.githubusercontent.com/cosmos/cosmos-sdk/release/v0.46.x/docs/uml/svg/delegation_sequence.svg) + +### MsgUndelegate + +The `MsgUndelegate` message allows delegators to undelegate their tokens from +validator. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L34-L36 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L147-L159 +``` + +This message returns a response containing the completion time of the undelegation: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L161-L169 +``` + +This message is expected to fail if: + +* the delegation doesn't exist +* the validator doesn't exist +* the delegation has less shares than the ones worth of `Amount` +* existing `UnbondingDelegation` has maximum entries as defined by `params.MaxEntries` +* the `Amount` has a denomination different than one defined by `params.BondDenom` + +When this message is processed the following actions occur: + +* validator's `DelegatorShares` and the delegation's `Shares` are both reduced by the message `SharesAmount` +* calculate the token worth of the shares remove that amount tokens held within the validator +* with those removed tokens, if the validator is: + * `Bonded` - add them to an entry in `UnbondingDelegation` (create `UnbondingDelegation` if it doesn't exist) with a completion time a full unbonding period from the current time. Update pool shares to reduce BondedTokens and increase NotBondedTokens by token worth of the shares. + * `Unbonding` - add them to an entry in `UnbondingDelegation` (create `UnbondingDelegation` if it doesn't exist) with the same completion time as the validator (`UnbondingMinTime`). + * `Unbonded` - then send the coins the message `DelegatorAddr` +* if there are no more `Shares` in the delegation, then the delegation object is removed from the store + * under this situation if the delegation is the validator's self-delegation then also jail the validator. + +![Unbond sequence](https://raw.githubusercontent.com/cosmos/cosmos-sdk/release/v0.46.x/docs/uml/svg/unbond_sequence.svg) + +### MsgCancelUnbondingDelegation + +The `MsgCancelUnbondingDelegation` message allows delegators to cancel the `unbondingDelegation` entry and delegate back to a previous validator. However, please note that this feature does not support canceling unbond delegations from jailed validators. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L38-L42 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L171-L185 +``` + +This message is expected to fail if: + +* the `unbondingDelegation` entry is already processed. +* the `cancel unbonding delegation` amount is greater than the `unbondingDelegation` entry balance. +* the `cancel unbonding delegation` height doesn't exist in the `unbondingDelegationQueue` of the delegator. +* the `unbondingDelegation` is from a jailed validator. + +When this message is processed the following actions occur: + +* if the `unbondingDelegation` Entry balance is zero + * in this condition `unbondingDelegation` entry will be removed from `unbondingDelegationQueue`. + * otherwise `unbondingDelegationQueue` will be updated with new `unbondingDelegation` entry balance and initial balance +* the validator's `DelegatorShares` and the delegation's `Shares` are both increased by the message `Amount`. + +### MsgBeginRedelegate + +The redelegation command allows delegators to instantly switch validators. Once +the unbonding period has passed, the redelegation is automatically completed in +the EndBlocker. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L30-L32 +``` + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L126-L139 +``` + +This message returns a response containing the completion time of the redelegation: + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L141-L145 +``` + +This message is expected to fail if: + +* the delegation doesn't exist +* the source or destination validators don't exist +* the delegation has less shares than the ones worth of `Amount` +* the source validator has a receiving redelegation which is not matured (aka. the redelegation may be transitive) +* existing `Redelegation` has maximum entries as defined by `params.MaxEntries` +* the `Amount` `Coin` has a denomination different than one defined by `params.BondDenom` + +When this message is processed the following actions occur: + +* the source validator's `DelegatorShares` and the delegations `Shares` are both reduced by the message `SharesAmount` +* calculate the token worth of the shares remove that amount tokens held within the source validator. +* if the source validator is: + * `Bonded` - add an entry to the `Redelegation` (create `Redelegation` if it doesn't exist) with a completion time a full unbonding period from the current time. Update pool shares to reduce BondedTokens and increase NotBondedTokens by token worth of the shares (this may be effectively reversed in the next step however). + * `Unbonding` - add an entry to the `Redelegation` (create `Redelegation` if it doesn't exist) with the same completion time as the validator (`UnbondingMinTime`). + * `Unbonded` - no action required in this step +* Delegate the token worth to the destination validator, possibly moving tokens back to the bonded state. +* if there are no more `Shares` in the source delegation, then the source delegation object is removed from the store + * under this situation if the delegation is the validator's self-delegation then also jail the validator. + +![Begin redelegation sequence](https://raw.githubusercontent.com/cosmos/cosmos-sdk/release/v0.46.x/docs/uml/svg/begin_redelegation_sequence.svg) + + +### MsgUpdateParams + +The `MsgUpdateParams` update the staking module parameters. +The params are updated through a governance proposal where the signer is the gov module account address. +When the `MinCommissionRate` is updated, all validators with a lower (max) commission rate than `MinCommissionRate` will be updated to `MinCommissionRate`. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L192-L203 +``` + +The message handling can fail if: + +* signer is not the authority defined in the staking keeper (usually the gov module account). + +### MsgRotateConsPubKey + +The `MsgRotateConsPubKey` updates the consensus pubkey of a validator +with a new pubkey, the validator must pay rotation fees (default fee 1000000stake) to rotate the consensus pubkey. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/release/v0.52.x/x/staking/proto/cosmos/staking/v1beta1/tx.proto#L211-L222 +``` + +The message handling can fail if: + +* The new pubkey is not a `cryptotypes.PubKey`. +* The new pubkey is already associated with another validator. +* The new pubkey is already present in the cons pubkey rotation history. +* The validator address is not in validators list. +* The `max_cons_pubkey_rotations` limit reached within unbonding period. +* The validator doesn't have enough balance to pay for the rotation. + + +## End-Block + +Each abci end block call, the operations to update queues and validator set +changes are specified to execute. + +### Validator Set Changes + +The staking validator set is updated during this process by state transitions +that run at the end of every block. As a part of this process any updated +validators are also returned back to CometBFT for inclusion in the CometBFT +validator set which is responsible for validating CometBFT messages at the +consensus layer. Operations are as following: + +* the new validator set is taken as the top `params.MaxValidators` number of + validators retrieved from the `ValidatorsByPower` index +* the previous validator set is compared with the new validator set: + * missing validators begin unbonding and their `Tokens` are transferred from the + `BondedPool` to the `NotBondedPool` `ModuleAccount` + * new validators are instantly bonded and their `Tokens` are transferred from the + `NotBondedPool` to the `BondedPool` `ModuleAccount` + +In all cases, any validators leaving or entering the bonded validator set or +changing balances and staying within the bonded validator set incur an update +message reporting their new consensus power which is passed back to CometBFT. + +The `LastTotalPower` and `LastValidatorsPower` hold the state of the total power +and validator power from the end of the last block, and are used to check for +changes that have occurred in `ValidatorsByPower` and the total new power, which +is calculated during `EndBlock`. + +### Queues + +Within staking, certain state-transitions are not instantaneous but take place +over a duration of time (typically the unbonding period). When these +transitions are mature certain operations must take place in order to complete +the state operation. This is achieved through the use of queues which are +checked/processed at the end of each block. + +#### Unbonding Validators + +When a validator is kicked out of the bonded validator set (either through +being jailed, or not having sufficient bonded tokens) it begins the unbonding +process along with all its delegations begin unbonding (while still being +delegated to this validator). At this point the validator is said to be an +"unbonding validator", whereby it will mature to become an "unbonded validator" +after the unbonding period has passed. + +Each block the validator queue is to be checked for mature unbonding validators +(namely with a completion time <= current time and completion height <= current +block height). At this point any mature validators which do not have any +delegations remaining are deleted from state. For all other mature unbonding +validators that still have remaining delegations, the `validator.Status` is +switched from `types.Unbonding` to +`types.Unbonded`. + +Unbonding operations can be put on hold by external modules via the `PutUnbondingOnHold(unbondingId)` method. + As a result, an unbonding operation (e.g., an unbonding delegation) that is on hold, cannot complete + even if it reaches maturity. For an unbonding operation with `unbondingId` to eventually complete + (after it reaches maturity), every call to `PutUnbondingOnHold(unbondingId)` must be matched + by a call to `UnbondingCanComplete(unbondingId)`. + +#### Unbonding Delegations + +Complete the unbonding of all mature `UnbondingDelegations.Entries` within the +`UnbondingDelegations` queue with the following procedure: + +* transfer the balance coins to the delegator's wallet address +* remove the mature entry from `UnbondingDelegation.Entries` +* remove the `UnbondingDelegation` object from the store if there are no + remaining entries. + +#### Redelegations + +Complete the unbonding of all mature `Redelegation.Entries` within the +`Redelegations` queue with the following procedure: + +* remove the mature entry from `Redelegation.Entries` +* remove the `Redelegation` object from the store if there are no + remaining entries. + +#### ConsPubKeyRotations + +After the completion of the unbonding period, matured rotations will be removed from the queues and indexes to unblock the validator for the next iterations. + +* remove the mature entry from state of `ValidatorConsensusKeyRotationRecordQueueKey` +* remove the mature entry form state of +`ValidatorConsensusKeyRotationRecordIndexKey` + +## Hooks + +Other modules may register operations to execute when a certain event has +occurred within staking. These events can be registered to execute either +right `Before` or `After` the staking event (as per the hook name). The +following hooks can registered with staking: + +* `AfterValidatorCreated(Context, ValAddress) error` + * called when a validator is created +* `BeforeValidatorModified(Context, ValAddress) error` + * called when a validator's state is changed +* `AfterValidatorRemoved(Context, ConsAddress, ValAddress) error` + * called when a validator is deleted +* `AfterValidatorBonded(Context, ConsAddress, ValAddress) error` + * called when a validator is bonded +* `AfterValidatorBeginUnbonding(Context, ConsAddress, ValAddress) error` + * called when a validator begins unbonding +* `BeforeDelegationCreated(Context, AccAddress, ValAddress) error` + * called when a delegation is created +* `BeforeDelegationSharesModified(Context, AccAddress, ValAddress) error` + * called when a delegation's shares are modified +* `AfterDelegationModified(Context, AccAddress, ValAddress) error` + * called when a delegation is created or modified +* `BeforeDelegationRemoved(Context, AccAddress, ValAddress) error` + * called when a delegation is removed +* `AfterUnbondingInitiated(Context, UnbondingID)` + * called when an unbonding operation (validator unbonding, unbonding delegation, redelegation) was initiated +* `AfterConsensusPubKeyUpdate(ctx Context, oldpubkey, newpubkey types.PubKey, fee sdk.Coin)` + * called when a consensus pubkey rotation of a validator is initiated. + + +## Events + +The staking module emits the following events: + +### EndBlocker + +| Type | Attribute Key | Attribute Value | +| --------------------- | --------------------- | ------------------------- | +| complete_unbonding | amount | {totalUnbondingAmount} | +| complete_unbonding | validator | {validatorAddress} | +| complete_unbonding | delegator | {delegatorAddress} | +| complete_redelegation | amount | {totalRedelegationAmount} | +| complete_redelegation | source_validator | {srcValidatorAddress} | +| complete_redelegation | destination_validator | {dstValidatorAddress} | +| complete_redelegation | delegator | {delegatorAddress} | + +## Msg's + +### MsgCreateValidator + +| Type | Attribute Key | Attribute Value | +| ---------------- | ------------- | ------------------ | +| create_validator | validator | {validatorAddress} | +| create_validator | amount | {delegationAmount} | +| message | module | staking | +| message | action | create_validator | +| message | sender | {senderAddress} | + +### MsgEditValidator + +| Type | Attribute Key | Attribute Value | +| -------------- | ------------------- | ------------------- | +| edit_validator | commission_rate | {commissionRate} | +| edit_validator | min_self_delegation | {minSelfDelegation} | +| message | module | staking | +| message | action | edit_validator | +| message | sender | {senderAddress} | + +### MsgDelegate + +| Type | Attribute Key | Attribute Value | +| -------- | ------------- | ------------------ | +| delegate | validator | {validatorAddress} | +| delegate | amount | {delegationAmount} | +| message | module | staking | +| message | action | delegate | +| message | sender | {senderAddress} | + +### MsgUndelegate + +| Type | Attribute Key | Attribute Value | +| ------- | ------------------- | ------------------ | +| unbond | validator | {validatorAddress} | +| unbond | amount | {unbondAmount} | +| unbond | completion_time [0] | {completionTime} | +| message | module | staking | +| message | action | begin_unbonding | +| message | sender | {senderAddress} | + +* [0] Time is formatted in the RFC3339 standard + +### MsgCancelUnbondingDelegation + +| Type | Attribute Key | Attribute Value | +| ----------------------------- | ------------------ | ------------------------------------| +| cancel_unbonding_delegation | validator | {validatorAddress} | +| cancel_unbonding_delegation | delegator | {delegatorAddress} | +| cancel_unbonding_delegation | amount | {cancelUnbondingDelegationAmount} | +| cancel_unbonding_delegation | creation_height | {unbondingCreationHeight} | +| message | module | staking | +| message | action | cancel_unbond | +| message | sender | {senderAddress} | + +### MsgBeginRedelegate + +| Type | Attribute Key | Attribute Value | +| ---------- | --------------------- | --------------------- | +| redelegate | source_validator | {srcValidatorAddress} | +| redelegate | destination_validator | {dstValidatorAddress} | +| redelegate | amount | {unbondAmount} | +| redelegate | completion_time [0] | {completionTime} | +| message | module | staking | +| message | action | begin_redelegate | +| message | sender | {senderAddress} | + +* [0] Time is formatted in the RFC3339 standard + +## Parameters + +The staking module contains the following parameters: + +| Key | Type | Example | +|------------------- |------------------|------------------------| +| UnbondingTime | string (time ns) | "259200000000000" | +| MaxValidators | uint16 | 100 | +| KeyMaxEntries | uint16 | 7 | +| HistoricalEntries | uint16 | 3 | +| BondDenom | string | "stake" | +| MinCommissionRate | string | "0.000000000000000000" | +| KeyRotationFee | sdk.Coin | "1000000stake" | +| MaxConsPubkeyRotations | int | 1 | + +:::warning +Manually updating the `MinCommissionRate` parameter will not affect the commission rate of the existing validators. It will only affect the commission rate of the new validators. Update the parameter with `MsgUpdateParams` to affect the commission rate of the existing validators as well. +::: + +## Client + +### CLI + +A user can query and interact with the `staking` module using the CLI. + +#### Query + +The `query` commands allows users to query `staking` state. + +```bash +simd query staking --help +``` + +##### delegation + +The `delegation` command allows users to query delegations for an individual delegator on an individual validator. + +Usage: + +```bash +simd query staking delegation [delegator-addr] [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking delegation cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash +balance: + amount: "10000000000" + denom: stake +delegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + shares: "10000000000.000000000000000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +##### delegations + +The `delegations` command allows users to query delegations for an individual delegator on all validators. + +Usage: + +```bash +simd query staking delegations [delegator-addr] [flags] +``` + +Example: + +```bash +simd query staking delegations cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p +``` + +Example Output: + +```bash +delegation_responses: +- balance: + amount: "10000000000" + denom: stake + delegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + shares: "10000000000.000000000000000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +- balance: + amount: "10000000000" + denom: stake + delegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + shares: "10000000000.000000000000000000" + validator_address: cosmosvaloper1x20lytyf6zkcrv5edpkfkn8sz578qg5sqfyqnp +pagination: + next_key: null + total: "0" +``` + +##### delegations-to + +The `delegations-to` command allows users to query delegations on an individual validator. + +Usage: + +```bash +simd query staking delegations-to [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking delegations-to cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash +- balance: + amount: "504000000" + denom: stake + delegation: + delegator_address: cosmos1q2qwwynhv8kh3lu5fkeex4awau9x8fwt45f5cp + shares: "504000000.000000000000000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +- balance: + amount: "78125000000" + denom: uixo + delegation: + delegator_address: cosmos1qvppl3479hw4clahe0kwdlfvf8uvjtcd99m2ca + shares: "78125000000.000000000000000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +pagination: + next_key: null + total: "0" +``` + +##### historical-info + +The `historical-info` command allows users to query historical information at given height. + +Usage: + +```bash +simd query staking historical-info [height] [flags] +``` + +Example: + +```bash +simd query staking historical-info 10 +``` + +Example Output: + +```bash +header: + app_hash: Lbx8cXpI868wz8sgp4qPYVrlaKjevR5WP/IjUxwp3oo= + chain_id: testnet + consensus_hash: BICRvH3cKD93v7+R1zxE2ljD34qcvIZ0Bdi389qtoi8= + data_hash: 47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU= + evidence_hash: 47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU= + height: "10" + last_block_id: + hash: RFbkpu6pWfSThXxKKl6EZVDnBSm16+U0l0xVjTX08Fk= + part_set_header: + hash: vpIvXD4rxD5GM4MXGz0Sad9I7//iVYLzZsEU4BVgWIU= + total: 1 + last_commit_hash: Ne4uXyx4QtNp4Zx89kf9UK7oG9QVbdB6e7ZwZkhy8K0= + last_results_hash: 47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU= + next_validators_hash: nGBgKeWBjoxeKFti00CxHsnULORgKY4LiuQwBuUrhCs= + proposer_address: mMEP2c2IRPLr99LedSRtBg9eONM= + time: "2021-10-01T06:00:49.785790894Z" + validators_hash: nGBgKeWBjoxeKFti00CxHsnULORgKY4LiuQwBuUrhCs= + version: + app: "0" + block: "11" +valset: +- commission: + commission_rates: + max_change_rate: "0.010000000000000000" + max_rate: "0.200000000000000000" + rate: "0.100000000000000000" + update_time: "2021-10-01T05:52:50.380144238Z" + consensus_pubkey: + '@type': /cosmos.crypto.ed25519.PubKey + key: Auxs3865HpB/EfssYOzfqNhEJjzys2Fo6jD5B8tPgC8= + delegator_shares: "10000000.000000000000000000" + description: + details: "" + identity: "" + moniker: myvalidator + security_contact: "" + website: "" + jailed: false + min_self_delegation: "1" + operator_address: cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc + status: BOND_STATUS_BONDED + tokens: "10000000" + unbonding_height: "0" + unbonding_time: "1970-01-01T00:00:00Z" +``` + +##### params + +The `params` command allows users to query values set as staking parameters. + +Usage: + +```bash +simd query staking params [flags] +``` + +Example: + +```bash +simd query staking params +``` + +Example Output: + +```bash +bond_denom: stake +historical_entries: 10000 +max_entries: 7 +max_validators: 50 +unbonding_time: 1814400s +``` + +##### pool + +The `pool` command allows users to query values for amounts stored in the staking pool. + +Usage: + +```bash +simd q staking pool [flags] +``` + +Example: + +```bash +simd q staking pool +``` + +Example Output: + +```bash +bonded_tokens: "10000000" +not_bonded_tokens: "0" +``` + +##### redelegation + +The `redelegation` command allows users to query a redelegation record based on delegator and a source and destination validator address. + +Usage: + +```bash +simd query staking redelegation [delegator-addr] [src-validator-addr] [dst-validator-addr] [flags] +``` + +Example: + +```bash +simd query staking redelegation cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash +pagination: null +redelegation_responses: +- entries: + - balance: "50000000" + redelegation_entry: + completion_time: "2021-10-24T20:33:21.960084845Z" + creation_height: 2.382847e+06 + initial_balance: "50000000" + shares_dst: "50000000.000000000000000000" + - balance: "5000000000" + redelegation_entry: + completion_time: "2021-10-25T21:33:54.446846862Z" + creation_height: 2.397271e+06 + initial_balance: "5000000000" + shares_dst: "5000000000.000000000000000000" + redelegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + entries: null + validator_dst_address: cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm + validator_src_address: cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm +``` + +##### redelegations + +The `redelegations` command allows users to query all redelegation records for an individual delegator. + +Usage: + +```bash +simd query staking redelegations [delegator-addr] [flags] +``` + +Example: + +```bash +simd query staking redelegation cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "0" +redelegation_responses: +- entries: + - balance: "50000000" + redelegation_entry: + completion_time: "2021-10-24T20:33:21.960084845Z" + creation_height: 2.382847e+06 + initial_balance: "50000000" + shares_dst: "50000000.000000000000000000" + - balance: "5000000000" + redelegation_entry: + completion_time: "2021-10-25T21:33:54.446846862Z" + creation_height: 2.397271e+06 + initial_balance: "5000000000" + shares_dst: "5000000000.000000000000000000" + redelegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + entries: null + validator_dst_address: cosmosvaloper1uccl5ugxrm7vqlzwqr04pjd320d2fz0z3hc6vm + validator_src_address: cosmosvaloper1zppjyal5emta5cquje8ndkpz0rs046m7zqxrpp +- entries: + - balance: "562770000000" + redelegation_entry: + completion_time: "2021-10-25T21:42:07.336911677Z" + creation_height: 2.39735e+06 + initial_balance: "562770000000" + shares_dst: "562770000000.000000000000000000" + redelegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + entries: null + validator_dst_address: cosmosvaloper1uccl5ugxrm7vqlzwqr04pjd320d2fz0z3hc6vm + validator_src_address: cosmosvaloper1zppjyal5emta5cquje8ndkpz0rs046m7zqxrpp +``` + +##### redelegations-from + +The `redelegations-from` command allows users to query delegations that are redelegating _from_ a validator. + +Usage: + +```bash +simd query staking redelegations-from [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking redelegations-from cosmosvaloper1y4rzzrgl66eyhzt6gse2k7ej3zgwmngeleucjy +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "0" +redelegation_responses: +- entries: + - balance: "50000000" + redelegation_entry: + completion_time: "2021-10-24T20:33:21.960084845Z" + creation_height: 2.382847e+06 + initial_balance: "50000000" + shares_dst: "50000000.000000000000000000" + - balance: "5000000000" + redelegation_entry: + completion_time: "2021-10-25T21:33:54.446846862Z" + creation_height: 2.397271e+06 + initial_balance: "5000000000" + shares_dst: "5000000000.000000000000000000" + redelegation: + delegator_address: cosmos1pm6e78p4pgn0da365plzl4t56pxy8hwtqp2mph + entries: null + validator_dst_address: cosmosvaloper1uccl5ugxrm7vqlzwqr04pjd320d2fz0z3hc6vm + validator_src_address: cosmosvaloper1y4rzzrgl66eyhzt6gse2k7ej3zgwmngeleucjy +- entries: + - balance: "221000000" + redelegation_entry: + completion_time: "2021-10-05T21:05:45.669420544Z" + creation_height: 2.120693e+06 + initial_balance: "221000000" + shares_dst: "221000000.000000000000000000" + redelegation: + delegator_address: cosmos1zqv8qxy2zgn4c58fz8jt8jmhs3d0attcussrf6 + entries: null + validator_dst_address: cosmosvaloper10mseqwnwtjaqfrwwp2nyrruwmjp6u5jhah4c3y + validator_src_address: cosmosvaloper1y4rzzrgl66eyhzt6gse2k7ej3zgwmngeleucjy +``` + +##### unbonding-delegation + +The `unbonding-delegation` command allows users to query unbonding delegations for an individual delegator on an individual validator. + +Usage: + +```bash +simd query staking unbonding-delegation [delegator-addr] [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking unbonding-delegation cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash +delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p +entries: +- balance: "52000000" + completion_time: "2021-11-02T11:35:55.391594709Z" + creation_height: "55078" + initial_balance: "52000000" +validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +##### unbonding-delegations + +The `unbonding-delegations` command allows users to query all unbonding-delegations records for one delegator. + +Usage: + +```bash +simd query staking unbonding-delegations [delegator-addr] [flags] +``` + +Example: + +```bash +simd query staking unbonding-delegations cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "0" +unbonding_responses: +- delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + entries: + - balance: "52000000" + completion_time: "2021-11-02T11:35:55.391594709Z" + creation_height: "55078" + initial_balance: "52000000" + validator_address: cosmosvaloper1t8ehvswxjfn3ejzkjtntcyrqwvmvuknzmvtaaa + +``` + +##### unbonding-delegations-from + +The `unbonding-delegations-from` command allows users to query delegations that are unbonding _from_ a validator. + +Usage: + +```bash +simd query staking unbonding-delegations-from [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking unbonding-delegations-from cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "0" +unbonding_responses: +- delegator_address: cosmos1qqq9txnw4c77sdvzx0tkedsafl5s3vk7hn53fn + entries: + - balance: "150000000" + completion_time: "2021-11-01T21:41:13.098141574Z" + creation_height: "46823" + initial_balance: "150000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +- delegator_address: cosmos1peteje73eklqau66mr7h7rmewmt2vt99y24f5z + entries: + - balance: "24000000" + completion_time: "2021-10-31T02:57:18.192280361Z" + creation_height: "21516" + initial_balance: "24000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +##### validator + +The `validator` command allows users to query details about an individual validator. + +Usage: + +```bash +simd query staking validator [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking validator cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash +commission: + commission_rates: + max_change_rate: "0.020000000000000000" + max_rate: "0.200000000000000000" + rate: "0.050000000000000000" + update_time: "2021-10-01T19:24:52.663191049Z" +consensus_pubkey: + '@type': /cosmos.crypto.ed25519.PubKey + key: sIiexdJdYWn27+7iUHQJDnkp63gq/rzUq1Y+fxoGjXc= +delegator_shares: "32948270000.000000000000000000" +description: + details: Witval is the validator arm from Vitwit. Vitwit is into software consulting + and services business since 2015. We are working closely with Cosmos ecosystem + since 2018. We are also building tools for the ecosystem, Aneka is our explorer + for the cosmos ecosystem. + identity: 51468B615127273A + moniker: Witval + security_contact: "" + website: "" +jailed: false +min_self_delegation: "1" +operator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +status: BOND_STATUS_BONDED +tokens: "32948270000" +unbonding_height: "0" +unbonding_time: "1970-01-01T00:00:00Z" +``` + +##### validators + +The `validators` command allows users to query details about all validators on a network. + +Usage: + +```bash +simd query staking validators [flags] +``` + +Example: + +```bash +simd query staking validators +``` + +Example Output: + +```bash +pagination: + next_key: FPTi7TKAjN63QqZh+BaXn6gBmD5/ + total: "0" +validators: +commission: + commission_rates: + max_change_rate: "0.020000000000000000" + max_rate: "0.200000000000000000" + rate: "0.050000000000000000" + update_time: "2021-10-01T19:24:52.663191049Z" +consensus_pubkey: + '@type': /cosmos.crypto.ed25519.PubKey + key: sIiexdJdYWn27+7iUHQJDnkp63gq/rzUq1Y+fxoGjXc= +delegator_shares: "32948270000.000000000000000000" +description: + details: Witval is the validator arm from Vitwit. Vitwit is into software consulting + and services business since 2015. We are working closely with Cosmos ecosystem + since 2018. We are also building tools for the ecosystem, Aneka is our explorer + for the cosmos ecosystem. + identity: 51468B615127273A + moniker: Witval + security_contact: "" + website: "" + jailed: false + min_self_delegation: "1" + operator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj + status: BOND_STATUS_BONDED + tokens: "32948270000" + unbonding_height: "0" + unbonding_time: "1970-01-01T00:00:00Z" +- commission: + commission_rates: + max_change_rate: "0.100000000000000000" + max_rate: "0.200000000000000000" + rate: "0.050000000000000000" + update_time: "2021-10-04T18:02:21.446645619Z" + consensus_pubkey: + '@type': /cosmos.crypto.ed25519.PubKey + key: GDNpuKDmCg9GnhnsiU4fCWktuGUemjNfvpCZiqoRIYA= + delegator_shares: "559343421.000000000000000000" + description: + details: Noderunners is a professional validator in POS networks. We have a huge + node running experience, reliable soft and hardware. Our commissions are always + low, our support to delegators is always full. Stake with us and start receiving + your Cosmos rewards now! + identity: 812E82D12FEA3493 + moniker: Noderunners + security_contact: info@noderunners.biz + website: http://noderunners.biz + jailed: false + min_self_delegation: "1" + operator_address: cosmosvaloper1q5ku90atkhktze83j9xjaks2p7uruag5zp6wt7 + status: BOND_STATUS_BONDED + tokens: "559343421" + unbonding_height: "0" + unbonding_time: "1970-01-01T00:00:00Z" +``` + +#### Transactions + +The `tx` commands allows users to interact with the `staking` module. + +```bash +simd tx staking --help +``` + +##### create-validator + +The command `create-validator` allows users to create new validator initialized with a self-delegation to it. + +Usage: + +```bash +simd tx staking create-validator [path/to/validator.json] [flags] +``` + +Example: + +```bash +simd tx staking create-validator /path/to/validator.json \ + --chain-id="name_of_chain_id" \ + --gas="auto" \ + --gas-adjustment="1.2" \ + --gas-prices="0.025stake" \ + --from=mykey +``` + +where `validator.json` contains: + +```json +{ + "pubkey": {"@type":"/cosmos.crypto.ed25519.PubKey","key":"BnbwFpeONLqvWqJb3qaUbL5aoIcW3fSuAp9nT3z5f20="}, + "amount": "1000000stake", + "moniker": "my-moniker", + "website": "https://myweb.site", + "security": "security-contact@gmail.com", + "details": "description of your validator", + "commission-rate": "0.10", + "commission-max-rate": "0.20", + "commission-max-change-rate": "0.01", + "min-self-delegation": "1" +} +``` + +and pubkey can be obtained by using `simd tendermint show-validator` command. + +##### delegate + +The command `delegate` allows users to delegate liquid tokens to a validator. + +Usage: + +```bash +simd tx staking delegate [validator-addr] [amount] [flags] +``` + +Example: + +```bash +simd tx staking delegate cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm 1000stake --from mykey +``` + +##### edit-validator + +The command `edit-validator` allows users to edit an existing validator account. + +Usage: + +```bash +simd tx staking edit-validator [flags] +``` + +Example: + +```bash +simd tx staking edit-validator --moniker "new_moniker_name" --website "new_website_url" --from mykey +``` + +##### redelegate + +The command `redelegate` allows users to redelegate illiquid tokens from one validator to another. + +Usage: + +```bash +simd tx staking redelegate [src-validator-addr] [dst-validator-addr] [amount] [flags] +``` + +Example: + +```bash +simd tx staking redelegate cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm 100stake --from mykey +``` + +##### unbond + +The command `unbond` allows users to unbond shares from a validator. + +Usage: + +```bash +simd tx staking unbond [validator-addr] [amount] [flags] +``` + +Example: + +```bash +simd tx staking unbond cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj 100stake --from mykey +``` + +##### cancel unbond + +The command `cancel-unbond` allow users to cancel the unbonding delegation entry and delegate back to the original validator. + +Usage: + +```bash +simd tx staking cancel-unbond [validator-addr] [amount] [creation-height] +``` + +Example: + +```bash +simd tx staking cancel-unbond cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj 100stake 123123 --from mykey +``` + +##### rotate cons pubkey + +The command `rotate-cons-pubkey` allows validators to rotate the associated consensus pubkey to the new consensus pubkey. + +Usage: + +```bash +simd tx staking rotate-cons-pubkey [validator-address] [new-pubkey] [flags] +``` + +Example: + +```bash +simd tx staking rotate-cons-pubkey myvalidator {"@type":"/cosmos.crypto.ed25519.PubKey","key":"oWg2ISpLF405Jcm2vXV+2v4fnjodh6aafuIdeoW+rUw="} +``` + +### gRPC + +A user can query the `staking` module using gRPC endpoints. + +#### Validators + +The `Validators` endpoint queries all validators that match the given status. + +```bash +cosmos.staking.v1beta1.Query/Validators +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.staking.v1beta1.Query/Validators +``` + +Example Output: + +```bash +{ + "validators": [ + { + "operatorAddress": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "consensusPubkey": {"@type":"/cosmos.crypto.ed25519.PubKey","key":"Auxs3865HpB/EfssYOzfqNhEJjzys2Fo6jD5B8tPgC8="}, + "status": "BOND_STATUS_BONDED", + "tokens": "10000000", + "delegatorShares": "10000000000000000000000000", + "description": { + "moniker": "myvalidator" + }, + "unbondingTime": "1970-01-01T00:00:00Z", + "commission": { + "commissionRates": { + "rate": "100000000000000000", + "maxRate": "200000000000000000", + "maxChangeRate": "10000000000000000" + }, + "updateTime": "2021-10-01T05:52:50.380144238Z" + }, + "minSelfDelegation": "1" + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### Validator + +The `Validator` endpoint queries validator information for given validator address. + +```bash +cosmos.staking.v1beta1.Query/Validator +``` + +Example: + +```bash +grpcurl -plaintext -d '{"validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/Validator +``` + +Example Output: + +```bash +{ + "validator": { + "operatorAddress": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "consensusPubkey": {"@type":"/cosmos.crypto.ed25519.PubKey","key":"Auxs3865HpB/EfssYOzfqNhEJjzys2Fo6jD5B8tPgC8="}, + "status": "BOND_STATUS_BONDED", + "tokens": "10000000", + "delegatorShares": "10000000000000000000000000", + "description": { + "moniker": "myvalidator" + }, + "unbondingTime": "1970-01-01T00:00:00Z", + "commission": { + "commissionRates": { + "rate": "100000000000000000", + "maxRate": "200000000000000000", + "maxChangeRate": "10000000000000000" + }, + "updateTime": "2021-10-01T05:52:50.380144238Z" + }, + "minSelfDelegation": "1" + } +} +``` + +#### ValidatorDelegations + +The `ValidatorDelegations` endpoint queries delegate information for given validator. + +```bash +cosmos.staking.v1beta1.Query/ValidatorDelegations +``` + +Example: + +```bash +grpcurl -plaintext -d '{"validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/ValidatorDelegations +``` + +Example Output: + +```bash +{ + "delegationResponses": [ + { + "delegation": { + "delegatorAddress": "cosmos1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgy3ua5t", + "validatorAddress": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "shares": "10000000000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "10000000" + } + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### ValidatorUnbondingDelegations + +The `ValidatorUnbondingDelegations` endpoint queries delegate information for given validator. + +```bash +cosmos.staking.v1beta1.Query/ValidatorUnbondingDelegations +``` + +Example: + +```bash +grpcurl -plaintext -d '{"validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/ValidatorUnbondingDelegations +``` + +Example Output: + +```bash +{ + "unbonding_responses": [ + { + "delegator_address": "cosmos1z3pzzw84d6xn00pw9dy3yapqypfde7vg6965fy", + "validator_address": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "entries": [ + { + "creation_height": "25325", + "completion_time": "2021-10-31T09:24:36.797320636Z", + "initial_balance": "20000000", + "balance": "20000000" + } + ] + }, + { + "delegator_address": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", + "validator_address": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "entries": [ + { + "creation_height": "13100", + "completion_time": "2021-10-30T12:53:02.272266791Z", + "initial_balance": "1000000", + "balance": "1000000" + } + ] + }, + ], + "pagination": { + "next_key": null, + "total": "8" + } +} +``` + +#### Delegation + +The `Delegation` endpoint queries delegate information for given validator delegator pair. + +```bash +cosmos.staking.v1beta1.Query/Delegation +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/Delegation +``` + +Example Output: + +```bash +{ + "delegation_response": + { + "delegation": + { + "delegator_address":"cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", + "validator_address":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "shares":"25083119936.000000000000000000" + }, + "balance": + { + "denom":"stake", + "amount":"25083119936" + } + } +} +``` + +#### UnbondingDelegation + +The `UnbondingDelegation` endpoint queries unbonding information for given validator delegator. + +```bash +cosmos.staking.v1beta1.Query/UnbondingDelegation +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/UnbondingDelegation +``` + +Example Output: + +```bash +{ + "unbond": { + "delegator_address": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", + "validator_address": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "entries": [ + { + "creation_height": "136984", + "completion_time": "2021-11-08T05:38:47.505593891Z", + "initial_balance": "400000000", + "balance": "400000000" + }, + { + "creation_height": "137005", + "completion_time": "2021-11-08T05:40:53.526196312Z", + "initial_balance": "385000000", + "balance": "385000000" + } + ] + } +} +``` + +#### DelegatorDelegations + +The `DelegatorDelegations` endpoint queries all delegations of a given delegator address. + +```bash +cosmos.staking.v1beta1.Query/DelegatorDelegations +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/DelegatorDelegations +``` + +Example Output: + +```bash +{ + "delegation_responses": [ + {"delegation":{"delegator_address":"cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77","validator_address":"cosmosvaloper1eh5mwu044gd5ntkkc2xgfg8247mgc56fww3vc8","shares":"25083339023.000000000000000000"},"balance":{"denom":"stake","amount":"25083339023"}} + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### DelegatorUnbondingDelegations + +The `DelegatorUnbondingDelegations` endpoint queries all unbonding delegations of a given delegator address. + +```bash +cosmos.staking.v1beta1.Query/DelegatorUnbondingDelegations +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/DelegatorUnbondingDelegations +``` + +Example Output: + +```bash +{ + "unbonding_responses": [ + { + "delegator_address": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", + "validator_address": "cosmosvaloper1sjllsnramtg3ewxqwwrwjxfgc4n4ef9uxyejze", + "entries": [ + { + "creation_height": "136984", + "completion_time": "2021-11-08T05:38:47.505593891Z", + "initial_balance": "400000000", + "balance": "400000000" + }, + { + "creation_height": "137005", + "completion_time": "2021-11-08T05:40:53.526196312Z", + "initial_balance": "385000000", + "balance": "385000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### Redelegations + +The `Redelegations` endpoint queries redelegations of given address. + +```bash +cosmos.staking.v1beta1.Query/Redelegations +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1ld5p7hn43yuh8ht28gm9pfjgj2fctujp2tgwvf", "src_validator_addr" : "cosmosvaloper1j7euyj85fv2jugejrktj540emh9353ltgppc3g", "dst_validator_addr" : "cosmosvaloper1yy3tnegzmkdcm7czzcy3flw5z0zyr9vkkxrfse"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/Redelegations +``` + +Example Output: + +```bash +{ + "redelegation_responses": [ + { + "redelegation": { + "delegator_address": "cosmos1ld5p7hn43yuh8ht28gm9pfjgj2fctujp2tgwvf", + "validator_src_address": "cosmosvaloper1j7euyj85fv2jugejrktj540emh9353ltgppc3g", + "validator_dst_address": "cosmosvaloper1yy3tnegzmkdcm7czzcy3flw5z0zyr9vkkxrfse", + "entries": null + }, + "entries": [ + { + "redelegation_entry": { + "creation_height": 135932, + "completion_time": "2021-11-08T03:52:55.299147901Z", + "initial_balance": "2900000", + "shares_dst": "2900000.000000000000000000" + }, + "balance": "2900000" + } + ] + } + ], + "pagination": null +} +``` + +#### DelegatorValidators + +The `DelegatorValidators` endpoint queries all validators information for given delegator. + +```bash +cosmos.staking.v1beta1.Query/DelegatorValidators +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1ld5p7hn43yuh8ht28gm9pfjgj2fctujp2tgwvf"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/DelegatorValidators +``` + +Example Output: + +```bash +{ + "validators": [ + { + "operator_address": "cosmosvaloper1eh5mwu044gd5ntkkc2xgfg8247mgc56fww3vc8", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "UPwHWxH1zHJWGOa/m6JB3f5YjHMvPQPkVbDqqi+U7Uw=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "347260647559", + "delegator_shares": "347260647559.000000000000000000", + "description": { + "moniker": "BouBouNode", + "identity": "", + "website": "https://boubounode.com", + "security_contact": "", + "details": "AI-based Validator. #1 AI Validator on Game of Stakes. Fairly priced. Don't trust (humans), verify. Made with BouBou love." + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.061000000000000000", + "max_rate": "0.300000000000000000", + "max_change_rate": "0.150000000000000000" + }, + "update_time": "2021-10-01T15:00:00Z" + }, + "min_self_delegation": "1" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### DelegatorValidator + +The `DelegatorValidator` endpoint queries validator information for given delegator validator + +```bash +cosmos.staking.v1beta1.Query/DelegatorValidator +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1eh5mwu044gd5ntkkc2xgfg8247mgc56f3n8rr7", "validator_addr": "cosmosvaloper1eh5mwu044gd5ntkkc2xgfg8247mgc56fww3vc8"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/DelegatorValidator +``` + +Example Output: + +```bash +{ + "validator": { + "operator_address": "cosmosvaloper1eh5mwu044gd5ntkkc2xgfg8247mgc56fww3vc8", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "UPwHWxH1zHJWGOa/m6JB3f5YjHMvPQPkVbDqqi+U7Uw=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "347262754841", + "delegator_shares": "347262754841.000000000000000000", + "description": { + "moniker": "BouBouNode", + "identity": "", + "website": "https://boubounode.com", + "security_contact": "", + "details": "AI-based Validator. #1 AI Validator on Game of Stakes. Fairly priced. Don't trust (humans), verify. Made with BouBou love." + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.061000000000000000", + "max_rate": "0.300000000000000000", + "max_change_rate": "0.150000000000000000" + }, + "update_time": "2021-10-01T15:00:00Z" + }, + "min_self_delegation": "1" + } +} +``` + +#### Pool + +The `Pool` endpoint queries the pool information. + +```bash +cosmos.staking.v1beta1.Query/Pool +``` + +Example: + +```bash +grpcurl -plaintext -d localhost:9090 cosmos.staking.v1beta1.Query/Pool +``` + +Example Output: + +```bash +{ + "pool": { + "not_bonded_tokens": "369054400189", + "bonded_tokens": "15657192425623" + } +} +``` + +#### Params + +The `Params` endpoint queries the pool information. + +```bash +cosmos.staking.v1beta1.Query/Params +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.staking.v1beta1.Query/Params +``` + +Example Output: + +```bash +{ + "params": { + "unbondingTime": "1814400s", + "maxValidators": 100, + "maxEntries": 7, + "historicalEntries": 10000, + "bondDenom": "stake" + } +} +``` + +### REST + +A user can query the `staking` module using REST endpoints. + +#### DelegatorDelegations + +The `DelegtaorDelegations` REST endpoint queries all delegations of a given delegator address. + +```bash +/cosmos/staking/v1beta1/delegations/{delegatorAddr} +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/delegations/cosmos1vcs68xf2tnqes5tg0khr0vyevm40ff6zdxatp5" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "delegation_responses": [ + { + "delegation": { + "delegator_address": "cosmos1vcs68xf2tnqes5tg0khr0vyevm40ff6zdxatp5", + "validator_address": "cosmosvaloper1quqxfrxkycr0uzt4yk0d57tcq3zk7srm7sm6r8", + "shares": "256250000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "256250000" + } + }, + { + "delegation": { + "delegator_address": "cosmos1vcs68xf2tnqes5tg0khr0vyevm40ff6zdxatp5", + "validator_address": "cosmosvaloper194v8uwee2fvs2s8fa5k7j03ktwc87h5ym39jfv", + "shares": "255150000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "255150000" + } + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### Redelegations + +The `Redelegations` REST endpoint queries redelegations of given address. + +```bash +/cosmos/staking/v1beta1/delegators/{delegatorAddr}/redelegations +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/delegators/cosmos1thfntksw0d35n2tkr0k8v54fr8wxtxwxl2c56e/redelegations?srcValidatorAddr=cosmosvaloper1lzhlnpahvznwfv4jmay2tgaha5kmz5qx4cuznf&dstValidatorAddr=cosmosvaloper1vq8tw77kp8lvxq9u3c8eeln9zymn68rng8pgt4" \ +-H "accept: application/json" +``` + +Example Output: + +```bash +{ + "redelegation_responses": [ + { + "redelegation": { + "delegator_address": "cosmos1thfntksw0d35n2tkr0k8v54fr8wxtxwxl2c56e", + "validator_src_address": "cosmosvaloper1lzhlnpahvznwfv4jmay2tgaha5kmz5qx4cuznf", + "validator_dst_address": "cosmosvaloper1vq8tw77kp8lvxq9u3c8eeln9zymn68rng8pgt4", + "entries": null + }, + "entries": [ + { + "redelegation_entry": { + "creation_height": 151523, + "completion_time": "2021-11-09T06:03:25.640682116Z", + "initial_balance": "200000000", + "shares_dst": "200000000.000000000000000000" + }, + "balance": "200000000" + } + ] + } + ], + "pagination": null +} +``` + +#### DelegatorUnbondingDelegations + +The `DelegatorUnbondingDelegations` REST endpoint queries all unbonding delegations of a given delegator address. + +```bash +/cosmos/staking/v1beta1/delegators/{delegatorAddr}/unbonding_delegations +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/delegators/cosmos1nxv42u3lv642q0fuzu2qmrku27zgut3n3z7lll/unbonding_delegations" \ +-H "accept: application/json" +``` + +Example Output: + +```bash +{ + "unbonding_responses": [ + { + "delegator_address": "cosmos1nxv42u3lv642q0fuzu2qmrku27zgut3n3z7lll", + "validator_address": "cosmosvaloper1e7mvqlz50ch6gw4yjfemsc069wfre4qwmw53kq", + "entries": [ + { + "creation_height": "2442278", + "completion_time": "2021-10-12T10:59:03.797335857Z", + "initial_balance": "50000000000", + "balance": "50000000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### DelegatorValidators + +The `DelegatorValidators` REST endpoint queries all validators information for given delegator address. + +```bash +/cosmos/staking/v1beta1/delegators/{delegatorAddr}/validators +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/delegators/cosmos1xwazl8ftks4gn00y5x3c47auquc62ssune9ppv/validators" \ +-H "accept: application/json" +``` + +Example Output: + +```bash +{ + "validators": [ + { + "operator_address": "cosmosvaloper1xwazl8ftks4gn00y5x3c47auquc62ssuvynw64", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "5v4n3px3PkfNnKflSgepDnsMQR1hiNXnqOC11Y72/PQ=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "21592843799", + "delegator_shares": "21592843799.000000000000000000", + "description": { + "moniker": "jabbey", + "identity": "", + "website": "https://twitter.com/JoeAbbey", + "security_contact": "", + "details": "just another dad in the cosmos" + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.100000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.100000000000000000" + }, + "update_time": "2021-10-09T19:03:54.984821705Z" + }, + "min_self_delegation": "1" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### DelegatorValidator + +The `DelegatorValidator` REST endpoint queries validator information for given delegator validator pair. + +```bash +/cosmos/staking/v1beta1/delegators/{delegatorAddr}/validators/{validatorAddr} +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/delegators/cosmos1xwazl8ftks4gn00y5x3c47auquc62ssune9ppv/validators/cosmosvaloper1xwazl8ftks4gn00y5x3c47auquc62ssuvynw64" \ +-H "accept: application/json" +``` + +Example Output: + +```bash +{ + "validator": { + "operator_address": "cosmosvaloper1xwazl8ftks4gn00y5x3c47auquc62ssuvynw64", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "5v4n3px3PkfNnKflSgepDnsMQR1hiNXnqOC11Y72/PQ=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "21592843799", + "delegator_shares": "21592843799.000000000000000000", + "description": { + "moniker": "jabbey", + "identity": "", + "website": "https://twitter.com/JoeAbbey", + "security_contact": "", + "details": "just another dad in the cosmos" + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.100000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.100000000000000000" + }, + "update_time": "2021-10-09T19:03:54.984821705Z" + }, + "min_self_delegation": "1" + } +} +``` + +#### Parameters + +The `Parameters` REST endpoint queries the staking parameters. + +```bash +/cosmos/staking/v1beta1/params +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/params" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "params": { + "unbonding_time": "2419200s", + "max_validators": 100, + "max_entries": 7, + "historical_entries": 10000, + "bond_denom": "stake" + } +} +``` + +#### Pool + +The `Pool` REST endpoint queries the pool information. + +```bash +/cosmos/staking/v1beta1/pool +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/pool" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "pool": { + "not_bonded_tokens": "432805737458", + "bonded_tokens": "15783637712645" + } +} +``` + +#### Validators + +The `Validators` REST endpoint queries all validators that match the given status. + +```bash +/cosmos/staking/v1beta1/validators +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/validators" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "validators": [ + { + "operator_address": "cosmosvaloper1q3jsx9dpfhtyqqgetwpe5tmk8f0ms5qywje8tw", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "N7BPyek2aKuNZ0N/8YsrqSDhGZmgVaYUBuddY8pwKaE=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "383301887799", + "delegator_shares": "383301887799.000000000000000000", + "description": { + "moniker": "SmartNodes", + "identity": "D372724899D1EDC8", + "website": "https://smartnodes.co", + "security_contact": "", + "details": "Earn Rewards with Crypto Staking & Node Deployment" + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.050000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.100000000000000000" + }, + "update_time": "2021-10-01T15:51:31.596618510Z" + }, + "min_self_delegation": "1" + }, + { + "operator_address": "cosmosvaloper1q5ku90atkhktze83j9xjaks2p7uruag5zp6wt7", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "GDNpuKDmCg9GnhnsiU4fCWktuGUemjNfvpCZiqoRIYA=" + }, + "jailed": false, + "status": "BOND_STATUS_UNBONDING", + "tokens": "1017819654", + "delegator_shares": "1017819654.000000000000000000", + "description": { + "moniker": "Noderunners", + "identity": "812E82D12FEA3493", + "website": "http://noderunners.biz", + "security_contact": "info@noderunners.biz", + "details": "Noderunners is a professional validator in POS networks. We have a huge node running experience, reliable soft and hardware. Our commissions are always low, our support to delegators is always full. Stake with us and start receiving your cosmos rewards now!" + }, + "unbonding_height": "147302", + "unbonding_time": "2021-11-08T22:58:53.718662452Z", + "commission": { + "commission_rates": { + "rate": "0.050000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.100000000000000000" + }, + "update_time": "2021-10-04T18:02:21.446645619Z" + }, + "min_self_delegation": "1" + } + ], + "pagination": { + "next_key": "FONDBFkE4tEEf7yxWWKOD49jC2NK", + "total": "2" + } +} +``` + +#### Validator + +The `Validator` REST endpoint queries validator information for given validator address. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr} +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q" \ +-H "accept: application/json" +``` + +Example Output: + +```bash +{ + "validator": { + "operator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "sIiexdJdYWn27+7iUHQJDnkp63gq/rzUq1Y+fxoGjXc=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "33027900000", + "delegator_shares": "33027900000.000000000000000000", + "description": { + "moniker": "Witval", + "identity": "51468B615127273A", + "website": "", + "security_contact": "", + "details": "Witval is the validator arm from Vitwit. Vitwit is into software consulting and services business since 2015. We are working closely with Cosmos ecosystem since 2018. We are also building tools for the ecosystem, Aneka is our explorer for the cosmos ecosystem." + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.050000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.020000000000000000" + }, + "update_time": "2021-10-01T19:24:52.663191049Z" + }, + "min_self_delegation": "1" + } +} +``` + +#### ValidatorDelegations + +The `ValidatorDelegations` REST endpoint queries delegate information for given validator. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr}/delegations +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q/delegations" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "delegation_responses": [ + { + "delegation": { + "delegator_address": "cosmos190g5j8aszqhvtg7cprmev8xcxs6csra7xnk3n3", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "31000000000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "31000000000" + } + }, + { + "delegation": { + "delegator_address": "cosmos1ddle9tczl87gsvmeva3c48nenyng4n56qwq4ee", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "628470000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "628470000" + } + }, + { + "delegation": { + "delegator_address": "cosmos10fdvkczl76m040smd33lh9xn9j0cf26kk4s2nw", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "838120000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "838120000" + } + }, + { + "delegation": { + "delegator_address": "cosmos1n8f5fknsv2yt7a8u6nrx30zqy7lu9jfm0t5lq8", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "500000000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "500000000" + } + }, + { + "delegation": { + "delegator_address": "cosmos16msryt3fqlxtvsy8u5ay7wv2p8mglfg9hrek2e", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "61310000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "61310000" + } + } + ], + "pagination": { + "next_key": null, + "total": "5" + } +} +``` + +#### Delegation + +The `Delegation` REST endpoint queries delegate information for given validator delegator pair. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr}/delegations/{delegatorAddr} +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q/delegations/cosmos1n8f5fknsv2yt7a8u6nrx30zqy7lu9jfm0t5lq8" \ +-H "accept: application/json" +``` + +Example Output: + +```bash +{ + "delegation_response": { + "delegation": { + "delegator_address": "cosmos1n8f5fknsv2yt7a8u6nrx30zqy7lu9jfm0t5lq8", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "500000000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "500000000" + } + } +} +``` + +#### UnbondingDelegation + +The `UnbondingDelegation` REST endpoint queries unbonding information for given validator delegator pair. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr}/delegations/{delegatorAddr}/unbonding_delegation +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu/delegations/cosmos1ze2ye5u5k3qdlexvt2e0nn0508p04094ya0qpm/unbonding_delegation" \ +-H "accept: application/json" +``` + +Example Output: + +```bash +{ + "unbond": { + "delegator_address": "cosmos1ze2ye5u5k3qdlexvt2e0nn0508p04094ya0qpm", + "validator_address": "cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu", + "entries": [ + { + "creation_height": "153687", + "completion_time": "2021-11-09T09:41:18.352401903Z", + "initial_balance": "525111", + "balance": "525111" + } + ] + } +} +``` + +#### ValidatorUnbondingDelegations + +The `ValidatorUnbondingDelegations` REST endpoint queries unbonding delegations of a validator. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr}/unbonding_delegations +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu/unbonding_delegations" \ +-H "accept: application/json" +``` + +Example Output: + +```bash +{ + "unbonding_responses": [ + { + "delegator_address": "cosmos1q9snn84jfrd9ge8t46kdcggpe58dua82vnj7uy", + "validator_address": "cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu", + "entries": [ + { + "creation_height": "90998", + "completion_time": "2021-11-05T00:14:37.005841058Z", + "initial_balance": "24000000", + "balance": "24000000" + } + ] + }, + { + "delegator_address": "cosmos1qf36e6wmq9h4twhdvs6pyq9qcaeu7ye0s3dqq2", + "validator_address": "cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu", + "entries": [ + { + "creation_height": "47478", + "completion_time": "2021-11-01T22:47:26.714116854Z", + "initial_balance": "8000000", + "balance": "8000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` diff --git a/versioned_docs/version-0.52/build/modules/upgrade/README.md b/versioned_docs/version-0.52/build/modules/upgrade/README.md new file mode 100644 index 000000000..9e9c41e8c --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/upgrade/README.md @@ -0,0 +1,678 @@ +--- +sidebar_position: 1 +--- + +# `x/upgrade` + +## Abstract + +`x/upgrade` is an implementation of a Cosmos SDK module that facilitates smoothly +upgrading a live Cosmos chain to a new (breaking) software version. It accomplishes this by +providing a `PreBlocker` hook that prevents the blockchain state machine from +proceeding once a pre-defined upgrade block height has been reached. + +The module does not prescribe anything regarding how governance decides to do an +upgrade, but just the mechanism for coordinating the upgrade safely. Without software +support for upgrades, upgrading a live chain is risky because all of the validators +need to pause their state machines at exactly the same point in the process. If +this is not done correctly, there can be state inconsistencies which are hard to +recover from. + +* [Concepts](#concepts) +* [State](#state) +* [Events](#events) +* [Client](#client) + * [CLI](#cli) + * [REST](#rest) + * [gRPC](#grpc) +* [Resources](#resources) + +## Concepts + +### Plan + +The `x/upgrade` module defines a `Plan` type in which a live upgrade is scheduled +to occur. A `Plan` can be scheduled at a specific block height. +A `Plan` is created once a (frozen) release candidate along with an appropriate upgrade +`Handler` (see below) is agreed upon, where the `Name` of a `Plan` corresponds to a +specific `Handler`. Typically, a `Plan` is created through a governance proposal +process, where if voted upon and passed, will be scheduled. The `Info` of a `Plan` +may contain various metadata about the upgrade, typically application specific +upgrade info to be included on-chain such as a git commit that validators could +automatically upgrade to. + +```go +type Plan struct { + Name string + Height int64 + Info string +} +``` + +#### Sidecar Process + +If an operator running the application binary also runs a sidecar process to assist +in the automatic download and upgrade of a binary, the `Info` allows this process to +be seamless. This tool is [Cosmovisor](https://github.com/cosmos/cosmos-sdk/tree/main/tools/cosmovisor#readme). + +### Handler + +The `x/upgrade` module facilitates upgrading from major version X to major version Y. To +accomplish this, node operators must first upgrade their current binary to a new +binary that has a corresponding `Handler` for the new version Y. It is assumed that +this version has fully been tested and approved by the community at large. This +`Handler` defines what state migrations need to occur before the new binary Y +can successfully run the chain. Naturally, this `Handler` is application specific +and not defined on a per-module basis. Registering a `Handler` is done via +`Keeper#SetUpgradeHandler` in the application. + +```go +type UpgradeHandler func(Context, Plan, VersionMap) (VersionMap, error) +``` + +During each `EndBlock` execution, the `x/upgrade` module checks if there exists a +`Plan` that should execute (is scheduled at that height). If so, the corresponding +`Handler` is executed. If the `Plan` is expected to execute but no `Handler` is registered +or if the binary was upgraded too early, the node will gracefully panic and exit. + +### StoreLoader + +The `x/upgrade` module also facilitates store migrations as part of the upgrade. The +`StoreLoader` sets the migrations that need to occur before the new binary can +successfully run the chain. This `StoreLoader` is also application specific and +not defined on a per-module basis. Registering this `StoreLoader` is done via +`app#SetStoreLoader` in the application. + +```go +func UpgradeStoreLoader (upgradeHeight int64, storeUpgrades *store.StoreUpgrades) baseapp.StoreLoader +``` + +If there's a planned upgrade and the upgrade height is reached, the old binary writes `Plan` to the disk before panicking. + +This information is critical to ensure the `StoreUpgrades` happens smoothly at correct height and +expected upgrade. It eliminiates the chances for the new binary to execute `StoreUpgrades` multiple +times every time on restart. Also if there are multiple upgrades planned on same height, the `Name` +will ensure these `StoreUpgrades` takes place only in planned upgrade handler. + +**Note:** The `StoreLoader` helper function for StoreUpgrades in v2 is not part of the `x/upgrade` module; +instead, you can find it in the runtime v2 module. + +### Proposal + +Typically, a `Plan` is proposed and submitted through governance via a proposal +containing a `MsgSoftwareUpgrade` message. +This proposal prescribes to the standard governance process. If the proposal passes, +the `Plan`, which targets a specific `Handler`, is persisted and scheduled. The +upgrade can be delayed or hastened by updating the `Plan.Height` in a new proposal. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/upgrade/proto/cosmos/upgrade/v1beta1/tx.proto#L29-L40 +``` + +#### Cancelling Upgrade Proposals + +Upgrade proposals can be cancelled. There exists a gov-enabled `MsgCancelUpgrade` +message type, which can be embedded in a proposal, voted on and, if passed, will +remove the scheduled upgrade `Plan`. +Of course this requires that the upgrade was known to be a bad idea well before the +upgrade itself, to allow time for a vote. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/upgrade/proto/cosmos/upgrade/v1beta1/tx.proto#L47-L55 +``` + +If such a possibility is desired, the upgrade height is to be +`2 * (VotingPeriod + DepositPeriod) + (SafetyDelta)` from the beginning of the +upgrade proposal. The `SafetyDelta` is the time available from the success of an +upgrade proposal and the realization it was a bad idea (due to external social consensus). + +A `MsgCancelUpgrade` proposal can also be made while the original +`MsgSoftwareUpgrade` proposal is still being voted upon, as long as the `VotingPeriod` +ends after the `MsgSoftwareUpgrade` proposal. + +## State + +The internal state of the `x/upgrade` module is relatively minimal and simple. The +state contains the currently active upgrade `Plan` (if one exists) by key +`0x0` and if a `Plan` is marked as "done" by key `0x1`. The state +contains the consensus versions of all app modules in the application. The versions +are stored as big endian `uint64`, and can be accessed with prefix `0x2` appended +by the corresponding module name of type `string`. The state maintains a +`Protocol Version` which can be accessed by key `0x3`. + +* Plan: `0x0 -> Plan` +* Done: `0x1 | byte(plan name) -> BigEndian(Block Height)` +* ConsensusVersion: `0x2 | byte(module name) -> BigEndian(Module Consensus Version)` +* ProtocolVersion: `0x3 -> BigEndian(Protocol Version)` + +The `x/upgrade` module contains no genesis state. + +## Events + +The `x/upgrade` does not emit any events by itself. Any and all proposal related +events are emitted through the `x/gov` module. + +## Client + +### CLI + +A user can query and interact with the `upgrade` module using the CLI. + +#### Query + +The `query` commands allow users to query `upgrade` state. + +```bash +simd query upgrade --help +``` + +##### applied + +The `applied` command allows users to query the block header for height at which a completed upgrade was applied. + +```bash +simd query upgrade applied [upgrade-name] [flags] +``` + +If upgrade-name was previously executed on the chain, this returns the header for the block at which it was applied. +This helps a client determine which binary was valid over a given range of blocks, as well as more context to understand past migrations. + +Example: + +```bash +simd query upgrade applied "test-upgrade" +``` + +Example Output: + +```bash +"block_id": { + "hash": "A769136351786B9034A5F196DC53F7E50FCEB53B48FA0786E1BFC45A0BB646B5", + "parts": { + "total": 1, + "hash": "B13CBD23011C7480E6F11BE4594EE316548648E6A666B3575409F8F16EC6939E" + } + }, + "block_size": "7213", + "header": { + "version": { + "block": "11" + }, + "chain_id": "testnet-2", + "height": "455200", + "time": "2021-04-10T04:37:57.085493838Z", + "last_block_id": { + "hash": "0E8AD9309C2DC411DF98217AF59E044A0E1CCEAE7C0338417A70338DF50F4783", + "parts": { + "total": 1, + "hash": "8FE572A48CD10BC2CBB02653CA04CA247A0F6830FF19DC972F64D339A355E77D" + } + }, + "last_commit_hash": "DE890239416A19E6164C2076B837CC1D7F7822FC214F305616725F11D2533140", + "data_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855", + "validators_hash": "A31047ADE54AE9072EE2A12FF260A8990BA4C39F903EAF5636B50D58DBA72582", + "next_validators_hash": "A31047ADE54AE9072EE2A12FF260A8990BA4C39F903EAF5636B50D58DBA72582", + "consensus_hash": "048091BC7DDC283F77BFBF91D73C44DA58C3DF8A9CBC867405D8B7F3DAADA22F", + "app_hash": "28ECC486AFC332BA6CC976706DBDE87E7D32441375E3F10FD084CD4BAF0DA021", + "last_results_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855", + "evidence_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855", + "proposer_address": "2ABC4854B1A1C5AA8403C4EA853A81ACA901CC76" + }, + "num_txs": "0" +} +``` + +##### module versions + +The `module_versions` command gets a list of module names and their respective consensus versions. + +Following the command with a specific module name will return only +that module's information. + +```bash +simd query upgrade module_versions [optional module_name] [flags] +``` + +Example: + +```bash +simd query upgrade module_versions +``` + +Example Output: + +```bash +module_versions: +- name: auth + version: "2" +- name: authz + version: "1" +- name: bank + version: "2" +- name: distribution + version: "2" +- name: evidence + version: "1" +- name: feegrant + version: "1" +- name: genutil + version: "1" +- name: gov + version: "2" +- name: ibc + version: "2" +- name: mint + version: "1" +- name: params + version: "1" +- name: slashing + version: "2" +- name: staking + version: "2" +- name: transfer + version: "1" +- name: upgrade + version: "1" +- name: vesting + version: "1" +``` + +Example: + +```bash +regen query upgrade module_versions ibc +``` + +Example Output: + +```bash +module_versions: +- name: ibc + version: "2" +``` + +##### plan + +The `plan` command gets the currently scheduled upgrade plan, if one exists. + +```bash +regen query upgrade plan [flags] +``` + +Example: + +```bash +simd query upgrade plan +``` + +Example Output: + +```bash +height: "130" +info: "" +name: test-upgrade +time: "0001-01-01T00:00:00Z" +upgraded_client_state: null +``` + +##### authority + +The `authority` command allows users to query the address that is authorized to submit upgrade proposals. + +```bash +simd query upgrade authority [flags] +``` + +This command returns the bech32-encoded address of the account that has the authority to submit upgrade proposals. + +Example: + +```bash +simd query upgrade authority +``` + +Example Output: + +```bash +cosmos10d07y265gmmuvt4z0w9aw880jnsr700j6zn9kn +``` + +#### Transactions + +The upgrade module supports the following transactions: + +* `software-proposal` - submits an upgrade proposal: + +```bash +simd tx upgrade software-upgrade v2 --title="Test Proposal" --summary="testing" --deposit="100000000stake" --upgrade-height 1000000 \ +--upgrade-info '{ "binaries": { "linux/amd64":"https://example.com/simd.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f" } }' --from cosmos1.. +``` + +* `cancel-upgrade-proposal` - cancels a previously submitted upgrade proposal: + +```bash +simd tx upgrade cancel-upgrade-proposal --title="Test Proposal" --summary="testing" --deposit="100000000stake" --from cosmos1.. +``` + +### REST + +A user can query the `upgrade` module using REST endpoints. + +#### Applied Plan + +`AppliedPlan` queries a previously applied upgrade plan by its name. + +```bash +/cosmos/upgrade/v1beta1/applied_plan/{name} +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/upgrade/v1beta1/applied_plan/v2.0-upgrade" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "height": "30" +} +``` + +#### Current Plan + +`CurrentPlan` queries the current upgrade plan. + +```bash +/cosmos/upgrade/v1beta1/current_plan +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/upgrade/v1beta1/current_plan" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "plan": "v2.1-upgrade" +} +``` + +#### Module versions + +`ModuleVersions` queries the list of module versions from state. + +```bash +/cosmos/upgrade/v1beta1/module_versions +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/upgrade/v1beta1/module_versions" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "module_versions": [ + { + "name": "auth", + "version": "2" + }, + { + "name": "authz", + "version": "1" + }, + { + "name": "bank", + "version": "2" + }, + { + "name": "distribution", + "version": "2" + }, + { + "name": "evidence", + "version": "1" + }, + { + "name": "feegrant", + "version": "1" + }, + { + "name": "genutil", + "version": "1" + }, + { + "name": "gov", + "version": "2" + }, + { + "name": "ibc", + "version": "2" + }, + { + "name": "mint", + "version": "1" + }, + { + "name": "params", + "version": "1" + }, + { + "name": "slashing", + "version": "2" + }, + { + "name": "staking", + "version": "2" + }, + { + "name": "transfer", + "version": "1" + }, + { + "name": "upgrade", + "version": "1" + }, + { + "name": "vesting", + "version": "1" + } + ] +} +``` + +#### Authority + +`Authority` queries the address that is authorized to submit upgrade proposals. + +```bash +/cosmos/upgrade/v1beta1/authority +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/upgrade/v1beta1/authority" -H "accept: application/json" +``` + +Example Output: + +```json +{ +"address": "cosmos10d07y265gmmuvt4z0w9aw880jnsr700j6zn9kn" +} +``` + +### gRPC + +A user can query the `upgrade` module using gRPC endpoints. + +#### Applied Plan + +`AppliedPlan` queries a previously applied upgrade plan by its name. + +```bash +cosmos.upgrade.v1beta1.Query/AppliedPlan +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"name":"v2.0-upgrade"}' \ + localhost:9090 \ + cosmos.upgrade.v1beta1.Query/AppliedPlan +``` + +Example Output: + +```bash +{ + "height": "30" +} +``` + +#### Current Plan + +`CurrentPlan` queries the current upgrade plan. + +```bash +cosmos.upgrade.v1beta1.Query/CurrentPlan +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.upgrade.v1beta1.Query/CurrentPlan +``` + +Example Output: + +```bash +{ + "plan": "v2.1-upgrade" +} +``` + +#### Module versions + +`ModuleVersions` queries the list of module versions from state. + +```bash +cosmos.upgrade.v1beta1.Query/ModuleVersions +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.upgrade.v1beta1.Query/ModuleVersions +``` + +Example Output: + +```bash +{ + "module_versions": [ + { + "name": "auth", + "version": "2" + }, + { + "name": "authz", + "version": "1" + }, + { + "name": "bank", + "version": "2" + }, + { + "name": "distribution", + "version": "2" + }, + { + "name": "evidence", + "version": "1" + }, + { + "name": "feegrant", + "version": "1" + }, + { + "name": "genutil", + "version": "1" + }, + { + "name": "gov", + "version": "2" + }, + { + "name": "ibc", + "version": "2" + }, + { + "name": "mint", + "version": "1" + }, + { + "name": "params", + "version": "1" + }, + { + "name": "slashing", + "version": "2" + }, + { + "name": "staking", + "version": "2" + }, + { + "name": "transfer", + "version": "1" + }, + { + "name": "upgrade", + "version": "1" + }, + { + "name": "vesting", + "version": "1" + } + ] +} +``` + +#### Authority + +`Authority` queries the address that is authorized to submit upgrade proposals. + +```bash +cosmos.upgrade.v1beta1.Query/Authority +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.upgrade.v1beta1.Query/Authority +``` + +Example Output: + +```json +{ + "address": "cosmos10d07y265gmmuvt4z0w9aw880jnsr700j6zn9kn" +} +``` + +## Resources + +A list of (external) resources to learn more about the `x/upgrade` module. + +* [Cosmos Dev Series: Cosmos Blockchain Upgrade](https://medium.com/web3-surfers/cosmos-dev-series-cosmos-sdk-based-blockchain-upgrade-b5e99181554c) - The blog post that explains how software upgrades work in detail. diff --git a/versioned_docs/version-0.52/build/modules/validate/README.md b/versioned_docs/version-0.52/build/modules/validate/README.md new file mode 100644 index 000000000..f077102a0 --- /dev/null +++ b/versioned_docs/version-0.52/build/modules/validate/README.md @@ -0,0 +1,35 @@ +# x/validate + +:::tip +This module is only required when using runtime and runtime v2 and you want to make use of the pre-defined ante/poste handlers or tx validators. +::: + +The `x/validate` is an app module solely there to setup ante/post handlers on a runtime app (via baseapp options) and the global tx validators on a runtime/v2 app (via app module). Depinject will automatically inject the ante/post handlers and tx validators into the app. Module specific tx validators should be registered on their own modules. + +## Extra TxValidators + +It is possible to add extra tx validators to the app. This is useful when you want to add extra tx validators that do not belong to one specific module. For example, you can add a tx validator that checks if the tx is signed by a specific address. + +In your `app.go`, when using runtime/v2, supply the extra tx validators using `depinject`: + +```go +appConfig = depinject.Configs( + AppConfig(), + depinject.Supply( + []appmodulev2.TxValidator[transaction.Tx]{ + // Add extra tx validators here + } + ), +) +``` + +## Storage + +This module has no store key. Do not forget to add the module name in the `SkipStoreKeys` runtime config present in the app config. + +```go +SkipStoreKeys: []string{ + authtxconfig.DepinjectModuleName, + validate.ModuleName, +}, +``` diff --git a/versioned_docs/version-0.52/build/packages/01-depinject.md b/versioned_docs/version-0.52/build/packages/01-depinject.md new file mode 100644 index 000000000..de58b49e2 --- /dev/null +++ b/versioned_docs/version-0.52/build/packages/01-depinject.md @@ -0,0 +1,205 @@ +--- +sidebar_position: 1 +--- + +# Depinject + +> **DISCLAIMER**: This is a **beta** package. The SDK team is actively working on this feature and we are looking for feedback from the community. Please try it out and let us know what you think. + +## Overview + +`depinject` is a dependency injection (DI) framework for the Cosmos SDK, designed to streamline the process of building and configuring blockchain applications. It works in conjunction with the `core/appconfig` module to replace the majority of boilerplate code in `app.go` with a configuration file in Go, YAML, or JSON format. + +`depinject` is particularly useful for developing blockchain applications: + +* With multiple interdependent components, modules, or services. Helping manage their dependencies effectively. +* That require decoupling of these components, making it easier to test, modify, or replace individual parts without affecting the entire system. +* That are wanting to simplify the setup and initialisation of modules and their dependencies by reducing boilerplate code and automating dependency management. + +By using `depinject`, developers can achieve: + +* Cleaner and more organised code. +* Improved modularity and maintainability. +* A more maintainable and modular structure for their blockchain applications, ultimately enhancing development velocity and code quality. + +* [Go Doc](https://pkg.go.dev/cosmossdk.io/depinject) + +## Usage + +The `depinject` framework, based on dependency injection concepts, streamlines the management of dependencies within your blockchain application using its Configuration API. This API offers a set of functions and methods to create easy to use configurations, making it simple to define, modify, and access dependencies and their relationships. + +A core component of the [Configuration API](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/depinject#Config) is the `Provide` function, which allows you to register provider functions that supply dependencies. Inspired by constructor injection, these provider functions form the basis of the dependency tree, enabling the management and resolution of dependencies in a structured and maintainable manner. Additionally, `depinject` supports interface types as inputs to provider functions, offering flexibility and decoupling between components, similar to interface injection concepts. + +By leveraging `depinject` and its Configuration API, you can efficiently handle dependencies in your blockchain application, ensuring a clean, modular, and well-organised codebase. + +Example: + +```go +package main + +import ( + "fmt" + + "cosmossdk.io/depinject" +) + +type AnotherInt int + +func GetInt() int { return 1 } +func GetAnotherInt() AnotherInt { return 2 } + +func main() { + var ( + x int + y AnotherInt + ) + + fmt.Printf("Before (%v, %v)\n", x, y) + depinject.Inject( + depinject.Provide( + GetInt, + GetAnotherInt, + ), + &x, + &y, + ) + fmt.Printf("After (%v, %v)\n", x, y) +} +``` + +In this example, `depinject.Provide` registers two provider functions that return `int` and `AnotherInt` values. The `depinject.Inject` function is then used to inject these values into the variables `x` and `y`. + +Provider functions serve as the basis for the dependency tree. They are analysed to identify their inputs as dependencies and their outputs as dependents. These dependents can either be used by another provider function or be stored outside the DI container (e.g., `&x` and `&y` in the example above). Provider functions must be exported. + +### Interface type resolution + +`depinject` supports the use of interface types as inputs to provider functions, which helps decouple dependencies between modules. This approach is particularly useful for managing complex systems with multiple modules, such as the Cosmos SDK, where dependencies need to be flexible and maintainable. + +For example, `x/bank` expects an [AccountKeeper](https://pkg.go.dev/cosmossdk.io/x/bank/types#AccountKeeper) interface as [input to ProvideModule](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/bank/module.go#L208-L260). `SimApp` uses the implementation in `x/auth`, but the modular design allows for easy changes to the implementation if needed. + +Consider the following example: + +```go +package duck + +type Duck interface { + quack() +} + +type AlsoDuck interface { + quack() +} + +type Mallard struct{} +type Canvasback struct{} + +func (duck Mallard) quack() {} +func (duck Canvasback) quack() {} + +type Pond struct { + Duck AlsoDuck +} +``` + +And the following provider functions: + +```go +func GetMallard() duck.Mallard { + return Mallard{} +} + +func GetPond(duck Duck) Pond { + return Pond{Duck: duck} +} + +func GetCanvasback() Canvasback { + return Canvasback{} +} +``` + +In this example, there's a `Pond` struct that has a `Duck` field of type `AlsoDuck`. The `depinject` framework can automatically resolve the appropriate implementation when there's only one available, as shown below: + +```go +var pond Pond + +depinject.Inject( + depinject.Provide( + GetMallard, + GetPond, + ), + &pond) +``` + +This code snippet results in the `Duck` field of `Pond` being implicitly bound to the `Mallard` implementation because it's the only implementation of the `Duck` interface in the container. + +However, if there are multiple implementations of the `Duck` interface, as in the following example, you'll encounter an error: + +```go +var pond Pond + +depinject.Inject( + depinject.Provide( + GetMallard, + GetCanvasback, + GetPond, + ), + &pond) +``` + +A specific binding preference for `Duck` is required. + +#### `BindInterface` API + +In the above situation registering a binding for a given interface binding may look like: + +```go +depinject.Inject( + depinject.Configs( + depinject.BindInterface( + "duck/duck.Duck", + "duck/duck.Mallard", + ), + depinject.Provide( + GetMallard, + GetCanvasback, + GetPond, + ), + ), + &pond) +``` + +Now `depinject` has enough information to provide `Mallard` as an input to `APond`. + +### Full example in real app + +:::warning +When using `depinject.Inject`, the injected types must be pointers. +::: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app_v2.go#L219-L244 +``` + +## Debugging + +Issues with resolving dependencies in the container can be done with logs and [Graphviz](https://graphviz.org) renderings of the container tree. +By default, whenever there is an error, logs will be printed to stderr and a rendering of the dependency graph in Graphviz DOT format will be saved to `debug_container.dot`. + +Here is an example Graphviz rendering of a successful build of a dependency graph: +![Graphviz Example](https://raw.githubusercontent.com/cosmos/cosmos-sdk/ff39d243d421442b400befcd959ec3ccd2525154/depinject/testdata/example.svg) + +Rectangles represent functions, ovals represent types, rounded rectangles represent modules and the single hexagon +represents the function which called `Build`. Black-colored shapes mark functions and types that were called/resolved +without an error. Gray-colored nodes mark functions and types that could have been called/resolved in the container but +were left unused. + +Here is an example Graphviz rendering of a dependency graph build which failed: +![Graphviz Error Example](https://raw.githubusercontent.com/cosmos/cosmos-sdk/ff39d243d421442b400befcd959ec3ccd2525154/depinject/testdata/example_error.svg) + +Graphviz DOT files can be converted into SVG's for viewing in a web browser using the `dot` command-line tool, ex: + +```txt +dot -Tsvg debug_container.dot > debug_container.svg +``` + +Many other tools including some IDEs support working with DOT files. diff --git a/versioned_docs/version-0.52/build/packages/README.md b/versioned_docs/version-0.52/build/packages/README.md new file mode 100644 index 000000000..cbb150f50 --- /dev/null +++ b/versioned_docs/version-0.52/build/packages/README.md @@ -0,0 +1,53 @@ +--- +sidebar_position: 0 +--- + +# Packages + +The Cosmos SDK is a collection of Go modules. This section provides documentation on various packages that can used when developing a Cosmos SDK chain. +It lists all standalone Go modules that are part of the Cosmos SDK. + +:::tip +For more information on SDK modules, see the [SDK Modules](https://docs.cosmos.network/main/modules) section. +For more information on SDK tooling, see the [Tooling](https://docs.cosmos.network/main/build/tooling) section. +::: + +## Core + +* [Core](https://pkg.go.dev/cosmossdk.io/core) - Core library defining SDK modules and Server core interfaces ([ADR-063](https://docs.cosmos.network/main/architecture/adr-063-core-module-api)) +* [API](https://pkg.go.dev/cosmossdk.io/api) - API library containing generated SDK Pulsar API +* [Store](https://pkg.go.dev/cosmossdk.io/store) - Implementation of the Cosmos SDK store +* [Store/v2](https://pkg.go.dev/cosmossdk.io/store/v2) - Implementation of the Cosmos SDK store + +## V2 + +* [Server/v2/stf](https://pkg.go.dev/cosmossdk.io/server/v2/stf) - State Transition Function (STF) library for Cosmos SDK v2 +* [Server/v2/appmanager](https://pkg.go.dev/cosmossdk.io/server/v2/appmanager) - App coordinator for Cosmos SDK v2 +* [runtime/v2](https://pkg.go.dev/cosmossdk.io/runtime/v2) - Runtime library for Cosmos SDK v2 +* [Server/v2](https://pkg.go.dev/cosmossdk.io/server/v2) - Global server library for Cosmos SDK v2 +* [Server/v2/cometbft](https://pkg.go.dev/cosmossdk.io/server/v2/cometbft) - CometBFT Server implementation for Cosmos SDK v2 + +## State Management + +* [Collections](./02-collections.md) - State management library +* [ORM](./03-orm.md) - State management library +* [Schema](https://pkg.go.dev/cosmossdk.io/schema) - Logical representation of module state schemas +* [PostgreSQL indexer](https://pkg.go.dev/cosmossdk.io/indexer/postgres) - PostgreSQL indexer for Cosmos SDK modules + +## UX + +* [Depinject](./01-depinject.md) - Dependency injection framework +* [Client/v2](https://pkg.go.dev/cosmossdk.io/client/v2) - Library powering [AutoCLI](https://docs.cosmos.network/main/core/autocli) + +## Utilities + +* [Core/Testing](https://pkg.go.dev/cosmossdk.io/core/testing) - Mocking library for SDK modules +* [Log](https://pkg.go.dev/cosmossdk.io/log) - Logging library +* [Errors](https://pkg.go.dev/cosmossdk.io/errors) - Error handling library +* [Errors/v2](https://pkg.go.dev/cosmossdk.io/errors/v2) - Error handling library +* [Math](https://pkg.go.dev/cosmossdk.io/math) - Math library for SDK arithmetic operations + +## Example + +* [SimApp v2](https://pkg.go.dev/cosmossdk.io/simapp/v2) - SimApp/v2 is **the** sample Cosmos SDK v2 chain. This package should not be imported in your application. +* [SimApp](https://pkg.go.dev/cosmossdk.io/simapp) - SimApp is **the** sample Cosmos SDK chain. This package should not be imported in your application. diff --git a/versioned_docs/version-0.52/build/packages/_category_.json b/versioned_docs/version-0.52/build/packages/_category_.json new file mode 100644 index 000000000..5ed885eb2 --- /dev/null +++ b/versioned_docs/version-0.52/build/packages/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Packages", + "position": 4, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/rfc/PROCESS.md b/versioned_docs/version-0.52/build/rfc/PROCESS.md new file mode 100644 index 000000000..a4946cb2a --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/PROCESS.md @@ -0,0 +1,62 @@ +# RFC Creation Process + +1. Copy the `rfc-template.md` file. Use the following filename pattern: `rfc-next_number-title.md` +2. Create a draft Pull Request if you want to get early feedback. +3. Make sure the context and solution are clear and well-documented. +4. Add an entry to the list in the [README](./README.md) file. +5. Create a Pull Request to propose a new RFC. + +## What is an RFC? + +An RFC is a sort of async whiteboarding session. It is meant to replace the need for a distributed team to come together to decide. Currently, the Cosmos SDK team and contributors are distributed around the world. The team conducts working groups to have a synchronous discussion, and an RFC can be used to capture the discussion for a wider audience to better understand the changes that are coming to the software. + +The main difference the Cosmos SDK defines between RFCs and ADRs is that an RFC is to come to consensus and circulate information about a potential change or feature. An ADR is used if there is already consensus on a feature or change and there is no need to articulate the change coming to the software. An ADR will articulate the changes and require less communication. + +## RFC life cycle + +RFC creation is an **iterative** process. An RFC is meant as a distributed collaboration session, it may have many comments and is usually the by-product of no working group or synchronous communication. + +1. Proposals could start with a new GitHub Issue, be a result of existing Issues or a discussion. + +2. An RFC doesn't have to arrive to `main` with an _accepted_ status in a single PR. If the motivation is clear and the solution is sound, we SHOULD be able to merge it and keep a _proposed_ status. It's preferable to have an iterative approach rather than long, not merged Pull Requests. + +3. If a _proposed_ RFC is merged, then it should clearly document outstanding issues either in the RFC document notes or in a GitHub Issue. + +4. The PR SHOULD always be merged. In the case of a faulty RFC, we still prefer to merge it with a _rejected_ status. The only time the RFC SHOULD NOT be merged is if the author abandons it. + +5. Merged RFCs SHOULD NOT be pruned. + +6. If there is consensus and enough feedback then the RFC can be accepted. + +> Note: An RFC is written when there is no working group or team session on the problem. RFCs are meant as a distributed whiteboarding session. If there is a working group on the proposal, there is no need to have an RFC as there is synchronous whiteboarding going on. + +### RFC status + +Status has two components: + +```text +{CONSENSUS STATUS} +``` + +#### Consensus Status + +```text +DRAFT -> PROPOSED -> LAST CALL yyyy-mm-dd -> ACCEPTED | REJECTED -> SUPERSEDED by RFC-xxx + \ | + \ | + v v + ABANDONED +``` + +* `DRAFT`: [optional] an RFC which is work in progress, not being ready for a general review. This is to present early work and get early feedback in a Draft Pull Request form. +* `PROPOSED`: an RFC covering a full solution architecture and still in review - project stakeholders haven't reached an agreement yet. +* `LAST CALL `: [optional] clearly notify that we are close to accepting updates. Changing a status to `LAST CALL` means that social consensus (of Cosmos SDK maintainers) has been reached, and we still want to give it time to let the community react or analyze. +* `ACCEPTED`: RFC which will represent a currently implemented or to be implemented architecture design. +* `REJECTED`: RFC can go from PROPOSED or ACCEPTED to rejected if the consensus among project stakeholders decides so. +* `SUPERSEDED by RFC-xxx`: RFC which has been superseded by a new RFC. +* `ABANDONED`: the RFC is no longer pursued by the original authors. + +## Language used in RFC + +* The background/goal should be written in the present tense. +* Avoid using a first-person form. diff --git a/versioned_docs/version-0.52/build/rfc/README.md b/versioned_docs/version-0.52/build/rfc/README.md new file mode 100644 index 000000000..940d36107 --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/README.md @@ -0,0 +1,42 @@ +--- +sidebar_position: 1 +--- + +# Requests for Comments + +A Request for Comments (RFC) is a record of discussion on an open-ended topic +related to the design and implementation of the Cosmos SDK, for which no +immediate decision is required. + +The purpose of an RFC is to serve as a historical record of a high-level +discussion that might otherwise only be recorded in an ad-hoc way (for example, +via gists or Google docs) that are difficult to discover for someone after the +fact. An RFC _may_ give rise to more specific architectural _decisions_ for +the Cosmos SDK, but those decisions must be recorded separately in +[Architecture Decision Records (ADR)](../architecture). + +As a rule of thumb, if you can articulate a specific question that needs to be +answered, write an ADR. If you need to explore the topic and get input from +others to know what questions need to be answered, an RFC may be appropriate. + +## RFC Content + +An RFC should provide: + +* A **changelog**, documenting when and how the RFC has changed. +* An **abstract**, briefly summarizing the topic so the reader can quickly tell + whether it is relevant to their interest. +* Any **background** a reader will need to understand and participate in the + substance of the discussion (links to other documents are fine here). +* The **discussion**, the primary content of the document. + +The [rfc-template.md](./rfc-template.md) file includes placeholders for these +sections. + +## Table of Contents + + +* [RFC-001: Tx Validation](./rfc-001-tx-validation.md) +* [RFC-002: Zero Copy Encoding](./rfc-002-zero-copy-encoding.md) +* [RFC-004: Accounts](./rfc-004-accounts.md) +* [RFC-005: Optimistic Execution](./rfc-005-optimistic-execution.md) diff --git a/versioned_docs/version-0.52/build/rfc/_category_.json b/versioned_docs/version-0.52/build/rfc/_category_.json new file mode 100644 index 000000000..a5712bdae --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "RFC", + "position": 7, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/rfc/rfc-001-tx-validation.md b/versioned_docs/version-0.52/build/rfc/rfc-001-tx-validation.md new file mode 100644 index 000000000..c8abb8cb6 --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/rfc-001-tx-validation.md @@ -0,0 +1,25 @@ +# RFC 001: Transaction Validation + +## Changelog + +* 2023-03-12: Proposed + +## Background + +Transaction Validation is crucial to a functioning state machine. Within the Cosmos SDK there are two validation flows, one is outside the message server and the other within. The flow outside of the message server is the `ValidateBasic` function. It is called in the antehandler on both `CheckTx` and `DeliverTx`. There is an overhead and sometimes duplication of validation within these two flows. This extra validation provides an additional check before entering the mempool. + +With the deprecation of [`GetSigners`](https://github.com/cosmos/cosmos-sdk/issues/11275) we have the optionality to remove [sdk.Msg](https://github.com/cosmos/cosmos-sdk/blob/16a5404f8e00ddcf8857c8a55dca2f7c109c29bc/types/tx_msg.go#L16) and the `ValidateBasic` function. + +With the separation of CometBFT and Cosmos-SDK, there is a lack of control of what transactions get broadcasted and included in a block. This extra validation in the antehandler is meant to help in this case. In most cases the transaction is or should be simulated against a node for validation. With this flow transactions will be treated the same. + +## Proposal + +The acceptance of this RFC would move validation within `ValidateBasic` to the message server in modules, update tutorials and docs to remove mention of using `ValidateBasic` in favour of handling all validation for a message where it is executed. + +We can and will still support the `Validatebasic` function for users and provide an extension interface of the function once `sdk.Msg` is deprecated. + +> Note: This is how messages are handled in VMs like Ethereum and CosmWasm. + +### Consequences + +The consequence of updating the transaction flow is that transaction that may have failed before with the `ValidateBasic` flow will now be included in a block and fees charged. diff --git a/versioned_docs/version-0.52/build/rfc/rfc-002-zero-copy-encoding.md b/versioned_docs/version-0.52/build/rfc/rfc-002-zero-copy-encoding.md new file mode 100644 index 000000000..aeef7308d --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/rfc-002-zero-copy-encoding.md @@ -0,0 +1,406 @@ +# RFC 002: Zero-Copy Encoding + +## Changelog + +* 2022-03-08: Initial draft + +## Background + +When the SDK originally migrated to [protobuf state encoding](./../architecture/adr-019-protobuf-state-encoding.md), +zero-copy encodings such as [Cap'n Proto](https://capnproto.org/) +and [FlatBuffers](https://google.github.io/flatbuffers/) +were considered. We considered how a zero-copy encoding could be beneficial for interoperability with modules +and scripts in other languages and VMs. However, protobuf was still chosen because the maturity of its ecosystem and +tooling was much higher and the client experience and performance were considered the highest priorities. + +In [ADR 033: Protobuf-based Inter-Module Communication](./../architecture/adr-033-protobuf-inter-module-comm.md), the +idea of cross-language/VM inter-module +communication was considered again. And in the discussions surrounding [ADR 054: Semver Compatible SDK Modules](./../architecture/adr-054-semver-compatible-modules.md), +it was determined that multi-language/VM support in the SDK is a near term priority. + +While we could do cross-language/VM inter-module communication with protobuf binary or even JSON, the performance +overhead is deemed to be too high because: +* we are proposing replacing keeper calls with inter-module message calls and the overhead of even the inter-module +routing checks has come into question by some SDK users without even considering the possible overhead of encoding. +Effectively we would be replacing function calls with encoding. One of the SDK's primary objectives currently is +improving performance, and we want to avoid inter-module calls from becoming a big step backward. +* we want Rust code to be able to operate in highly resource constrained virtual machines so whatever we can do to +reduce performance overhead as well as the size of generated code will make it easier and more feasible to deploy +first-class integrations with these virtual machines. + +Thus, the agreement when the [ADR 054](./../architecture/adr-054-semver-compatible-modules.md) working group concluded +was to pursue a performant zero-copy encoding which is suitable for usage in highly resource constrained environments. + +## Proposal + +This RFC proposes a zero-copy encoding that is derived from the schema definitions defined in .proto files in the SDK +and all app chains. This would result in a new code generator for that supports both this zero-copy encoding as well as +the existing protobuf binary and JSON encodings as well as the google.golang.org/protobuf API. To make this zero-copy +encoding work, a number of changes are needed to how we manage the versioning of protobuf messages that should +address other concerns raised in [ADR 054](./../architecture/adr-054-semver-compatible-modules.md). The API for using +protobuf in golang would also change and this will be described in the [code generation](#generated-code) section +along with a proposed Rust code generator. + +An alternative approach to building a zero-copy encoding based on protobuf schemas would be to switch to FlatBuffers +or Cap'n Proto directly. However, this would require a complete rewrite of the SDK and all app chains. Places this +burden on the ecosystem would not be a wise choice when creating a zero-copy encoding compatible with all our +existing types and schemas is feasible. In the future, we may consider a native schema language for this encoding +that is more natural and succinct for its rules, but for now we are assuming that it is best to continue supporting +the existing protobuf based workflow. + +Also, we are not proposing a new encoding for transactions or gRPC query servers. From a client API perspective nothing +would change. The SDK would be capable of marshaling any message to and from protobuf binary and this zero-copy encoding +as needed. + +Furthermore, migrating to the **new golang generated code would be 100% opt-in** because the inter-module router will +simply marshal existing gogo proto generated types to/from the zero-copy encoding when needed. So migrating to the new +code generator would provide a performance benefit, but would not be required. + +In addition to supporting first-class Cosmos SDK modules defined in other languages and VMs, this encoding is intended +to be useful for user-defined code executing in a VM. To satisfy this, this encoding is designed to enable proper bounds +checking on all memory access at the expense of introducing some error return values in generated code. + +### New Protobuf Linting and Breaking Change Rules + +This zero-copy encoding places some additional requirements on the definition and maintenance of protobuf schemas. + +#### No New Fields Can Be Added To Existing Messages + +The biggest change is that it will be invalid to add a new field to an existing message and a breaking change detector +will need to be created which augments [buf breaking](https://docs.buf.build/breaking/overview) to detect this. + +The reasons for this are two-fold: + +1) from an API compatibility perspective, adding a new field to an existing message is actually a state machine breaking + change which in [ADR 020](../architecture/adr-020-protobuf-transaction-encoding.md) required us to add an unknown + field detector. Furthermore, in [ADR 054](../architecture/adr-054-semver-compatible-modules.md) this "feature" of protobuf + poses one of the biggest problems for correct forward compatibility between different versions of the same module. +2) not allowing new fields in existing messages makes the generated code in languages like Rust (which is currently our + highest priority target), much simpler and more performant because we can assume a fixed size struct gets allocated. + If new fields can be added to existing messages, we need to encode the number of fields into the message and then + do runtime checks. So this both increases memory layers and requires another layout of indirection. With the encoding + proposed below, "plain old Rust structs" (used with some special field types) can be used. + +Instead of adding new fields to existing messages, APIs can add new messages to existing packages or create new packages +with new versions of the messages. Also, we are not restricting the addition of cases to `oneof`s or values to `enum`s. +All of these cases are easier to detect at runtime with standard `switch` statements than the addition of new fields. + +#### Additional Linting Rules + +The following additional rules will be enforced by a linter that +complements [buf lint](https://docs.buf.build/lint/overview): + +* all message fields must be specified in continuous ascending order starting from `1` +* all enums must be specified in continuous ascending order starting from `0` - otherwise it is too complex to check at + runtime whether an enum value is unknown. An alternative would be to make adding new values to existing enums breaking +* all enum values must be `<= 255`. Any enum in a blockchain application which needs more than 256 values is probably + doing something very wrong. +* all oneof's must be the *only* element in their containing message and must start at field number `1` and be added in + continuous ascending order - this makes it possible to quickly check for unknown values +* all `oneof` field numbers must be `<= 255`. Any `oneof` which needs more field cases is probably doing something very + wrong. + +These requirements make the encoding and generated code simpler. + +### Encoding + +#### Buffers and Memory Management + +By default, this encoding attempts to use a single fixed size encoding buffer of 64kb. This imposes a limit on the +maximum size of a message that can be encoded. In the context of a message passing protocol for blockchains, this +is generally a reasonable limit and the only known valid use case for exceeding it is to store user-uploaded byte +code for execution in VMs. To accommodate this, large `string` and `bytes` values can be encoded in additional +standalone buffers if needed. Still, the body of a message included all scalar and message fields +must fit inside the 64kb buffer. + +While this design decision greatly simplifies the encoding and decoding logic, as well as the complexity of +generated code, it does mean that APIs will need to do proper bounds checking when writing data that is not fixed +size and return errors. + +The term `Root` is used to refer to the main 64kb buffer plus any additional large `string`/`bytes` buffers that are +allocated. + +#### Scalar Encoding + +* `bool`s are encoded as 1 byte - `0` or `1` +* `uint32`, `int32`, `sint32`, `fixed32`, `sfixed32` are encoded as 4 bytes by default +* `uint64`, `int64`, `sint64`, `fixed64`, `sfixed64` are encoded as 8 bytes by default +* `enum`s are encoded as 1 byte and values *MUST* be in the range of `0` to `255`. +* all scalars declared as `optional` are prefixed with 1 additional byte whose value is `0` or `1` to indicate presence + +All multibyte integers are encoded as little-endian which is by far the most common native byte order for modern +CPUs. Signed integers always use two's complement encoding. + +#### Message Encoding + +By default, messages field are encoded inline as structs. Meaning that if a message struct takes 8 bytes then its inline +field in another struct will add 8 bytes to that struct size. + +`optional` message fields will be prefixed by 1 byte to indicate presence. (Alternatively, we could encode optional +message fields as pointers (see below) if the desire is to save memory when they are rarely used needed.) + +#### Oneof’s + +`oneof`s are encoded as a combination of a `uint8` discriminant field and memory that is as large as the largest member +field. `oneof` field numbers *MUST* be between `1` and `255`. + +```protobuf +message Foo { + oneof sum { + bool x = 1; + int32 y = 2; + } +} +``` + +A discriminant of `0` indicates that the field is not set. + +#### Pointer Types: Bytes and Strings and Repeated fields + +A pointer is an 16-bit unsigned integer that points to an offset in the current memory buffer or to another memory +buffer. If the bit mask `0xFF00` on the is unset, then the pointer points to an offset in the main 64kb memory buffer. +If that bit mask is set, then the pointer points to a large `string` or `bytes` buffer. Up to 256 such buffers +can be referenced in a single `Root`. The pointer `0` indicates that a field is not defined. + +`bytes`, `string`s and repeated fields are encoded as pointers to a memory location that is prefixed with the +length of the `bytes`, `string` or repeated field value. If the referenced memory location is in the main 64kb memory +buffer, then this length prefix will be a 16-bit unsigned integer. If the referenced memory location is a large +`string` or `bytes` buffer, then this length prefix will be a 32-bit unsigned integer. + +#### `Any`s + +`Any`s are encoded as a pointer to the type URL string and a pointer to the start of the message +specified by the type URL. + +#### Maps + +Maps are not supported. + +#### Extended Encoding Options + +We may choose to allow customizing the encoding of fields so that they take up less space. + +For example, we could allow 8-bit or 16-bit integers: +`int32 x = 1 [(cosmos_proto.int16) = true]` would indicate that the field only needs 2 bytes + +Or we could allow `string`, `bytes` or `repeated` fields to have a fixed size rather than being encoding as +pointers to a variable-length value: +`string y = 2 [(cosmos_proto.fixed_size) = 3]` could indicate that this is a fixed width 3 byte string + +If we choose to enable these encoding options, changing these options would be a breaking change that needs to be +prevented by the breaking change detector. + +### Generated Code + +We will describe the generated Go and Rust code using this example protobuf file: + +```protobuf +message Foo { + int32 x = 1; + optional uint32 y = 2; + string z = 3; + Bar bar = 4; + repeated Bar bars = 5; +} + + +message Bar { + ABC abc = 1; + Baz baz = 2; + repeated uint32 xs = 3; +} + +message Baz { + oneof sum { + uint32 x = 1; + string y = 2; + } +} + +enum ABC { + A = 0; + B = 1; + C = 2; + D = 3; +} +``` + +#### Go + +In golang, the generated code would not expose any exported struct fields, but rather getters and setters as an +interface +or struct methods, ex: + +```go +type Foo interface { + X() int32 + SetX(int32) + Y() zpb.Option[uint32] + SetY(zpb.Option[uint32]) + Z() (string, error) + SetZ(string) error + Bar() Bar + Bars() (zpb.Array[Bar], error) +} + +type Bar interface { + Abc() ABC + SetAbc(ABC) Bar + Baz() Baz + Xs() (zpb.ScalarArray[uint32], error) +} + +type Baz interface { + Case() Baz_case + GetX() uint32 + SetX(uint32) + GetY() (string, error) + SetY(string) +} + +type Baz_case int32 +const ( + Baz_X Baz_case = 0 + Baz_Y Baz_case = 1 +) + +type ABC int32 +const ( + ABC_A ABC = 0 + ABC_B ABC = 1 + ABC_C ABC = 2 + ABC_D ABC = 3 +) +``` + +Special types `zpb.Option`, `zpb.Array` and `zpb.ScalarArray` are used to represent `optional` and repeated fields +respectively. These types would be included in the runtime library (called `zpb` here for zero-copy protobuf) and would +have an API like this: + +```go +type Option[T] interface { + IsSet() bool + Value() T +} + +type Array[T] interface { + InitWithLength(int) error + Len() int + Get(int) T +} + +type ScalarArray[T] interface { + Array[T] + Set(int, T) +} +``` + +Arrays in particular would not be resizable, but would be initialized with a fixed length. This is to ensure that arrays +can be written to the underlying buffer in a linear way. + +In golang, buffers would be managed transparently under the hood by the first message initialized, and usage of this +generated code might look like this: + +```go +foo := NewFoo() +foo.SetX(1) +foo.SetY(zpb.NewOption[uint32](2)) +err := foo.SetZ("hello") +if err != nil { + panic(err) +} + +bar := foo.Bar() +bar.Baz().SetX(3) + +xs, err = bar.Xs() +if err != nil { + panic(err) +} +xs.InitWithLength(2) +xs.Set(0, 0) +xs.Set(1, 2) + +bars, err = foo.Bars() +if err != nil { + panic(err) +} +bars.InitWithLength(3) +bars.Get(0).Baz().SetY("hello") +bars.Get(1).SetAbc(ABC_B) +bars.Get(2).Baz().SetX(4) +``` + +Under the hood the generated code would manage memory buffers on its own. The usage of `oneof`s is a bit easier than +the existing go generated code (as with `bar.Baz()` above). And rather than using setters on embedded messages, we +simply get the field (already allocated) and set its fields (as in the case of `foo.Bar()` above or the repeated +field `foo.Bars()`). Whenever a field is stored with a pointer (`string`, `bytes`, and `repeated` fields), there is +always an error returned on the getter to do proper bounds checking on the buffer. + +#### Rust + +This encoding should allow generating native structs in Rust that are annotated with `#[repr(C, align(1))]`. It should +be fairly natural to use from Rust with a key difference that memory buffers (called `Root`s) must be manually allocated +and passed into any pointer type. + +Here is some example code that uses library types `Option`, `Enum`, `String`, `OneOf` and `Repeated` +as well as little-endian integer types from [rend](https://lib.rs/crates/rend): + +```rust! +#[repr(C, align(1))] +struct Foo { + x: rend:i32_le, + y: cosmos_proto::Option, + z: cosmos_proto::String, // String wraps a pointer to a string + bar: Bar +} + +#[repr(C, align(1))] +struct Bar { + abc: cosmos_proto::Enum, // the Enum wrapper allows us to distinguish undefined and defined values of ABC at runtime. 3 is specified as the max value of ABC. + baz: cosmos_proto::OneOf, // the OneOf wrapper allows distinguished undefined values of Baz at runtime. 2 is specified as the max field value of Baz. + xs: cosmos_proto::Repeated // Repeated wraps a pointer to repeated fields +} + +#[repr(u8)] +enum ABC { + A = 0, + B = 1, + C = 2, + D = 3, +} + +#[repr(C, u8)] +enum Baz { + Empty, // all oneof's have a case for Empty if they are unset + X(rend::u32_le), + Y(cosmos_proto::String) +} +``` + +Example usage (which does the exact same thing as the go example above) would be: + +```rust! +let mut root = Root::new(); +let mut foo = root.get_mut(); +foo.x = 1.into(); +foo.y = Some(2.into()); +foo.z.set(root.new_string("hello")?); // could return an allocation error + +foo.bar.baz = Baz::X(3.into()); + +foo.bar.xs.init_with_size(&mut root, 2)?; // could return an allocation error +foo.bar.xs[0] = 0.into(); +foo.bar.xs[1] = 2.into(); + +foo.bars.init_with_size(&mut root, 3)?; // could return an allocation error +foo.bars[0].baz = Baz::Y(root.new_string("hello")?); // could return an allocation error +foo.bars[1].abc = ABC::B; +foo.bars[2].baz = Baz::X(4.into()); +``` + +## Abandoned Ideas (Optional) + +## References + +## Discussion diff --git a/versioned_docs/version-0.52/build/rfc/rfc-004-accounts.md b/versioned_docs/version-0.52/build/rfc/rfc-004-accounts.md new file mode 100644 index 000000000..c465508ef --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/rfc-004-accounts.md @@ -0,0 +1,330 @@ +# RFC 004: Accounts + +## Changelog + +* 17/03/2023: DRAFT +* 09/05/2023: DRAFT 2 + +## Context + +The current implementation of accounts in the Cosmos SDK is limiting in terms of functionality, extensibility, and overall +architecture. This RFC aims to address the following issues with the current account system: + +### 1. Accounts Representation and Authentication Mechanism + +The current SDK accounts are represented as `google.Protobuf.Any`, which are then encapsulated into the account interface. +This account interface essentially represents the authentication mechanism, as it implements methods such as `GetNumber` +and `GetSequence` that serve as abstractions over the authentication system. However, this approach restricts the scope and +functionality of accounts within the SDK. + +### 2. Limited Account Interface + +The account interface in its current form is not versatile enough to accommodate more advanced account functionalities, +such as implementing vesting capabilities or more complex authentication and authorization systems. + +### 3. Multiple Implementations of the Account Interface + +There are several implementations of the account interface, like `ModuleAccount`, but the existing abstraction does not +allow for meaningful differentiation between them. This hinders the ability to create specialized accounts that cater to +specific use cases. + +### 4. Primitive Authorization System + +The authorization system in the `x/auth` module is basic and defines authorizations solely for the functionalities of the +`x/bank` module. Consequently, although the state transition authorization system is defined in `x/auth`, it only covers the +use cases of `x/bank`, limiting the system's overall scope and adaptability. + +### 5. Cyclic Dependencies and Abstraction Leaks + +The current account system leads to cyclic dependencies and abstraction leaks throughout the Cosmos SDK. For instance, +the `Vesting` functionality belongs to the `x/auth` module, which depends on the `x/bank` module. However, +the `x/bank` module depends on the `x/auth` module again to identify the account type (either `Vesting` or `Base`) during +a coin transfer. This dependency structure creates architectural issues and complicates the overall design of the SDK. + +## Proposal + +This proposal aims to transform the way accounts are managed within the Cosmos SDK by introducing significant changes to +their structure and functionality. + +### Rethinking Account Representation and Business Logic + +Instead of representing accounts as simple `google.Protobuf.Any` structures stored in state with no business logic +attached, this proposal suggests a more sophisticated account representation that is closer to module entities. +In fact, accounts should be able to receive messages and process them in the same way modules do, and be capable of storing +state in a isolated (prefixed) portion of state belonging only to them, in the same way as modules do. + +### Account Message Reception + +We propose that accounts should be able to receive messages in the same way modules can, allowing them to manage their +own state modifications without relying on other modules. This change would enable more advanced account functionality, such as the +`VestingAccount` example, where the x/bank module previously needed to change the vestingState by casting the abstracted +account to `VestingAccount` and triggering the `TrackDelegation` call. Accounts are already capable of sending messages when +a state transition, originating from a transaction, is executed. + +When accounts receive messages, they will be able to identify the sender of the message and decide how to process the +state transition, if at all. + +### Consequences + +These changes would have significant implications for the Cosmos SDK, resulting in a system of actors that are equal from +the runtime perspective. The runtime would only be responsible for propagating messages between actors and would not +manage the authorization system. Instead, actors would manage their own authorizations. For instance, there would be no +need for the `x/auth` module to manage minting or burning of coins permissions, as it would fall within the scope of the +`x/bank` module. + +The key difference between accounts and modules would lie in the origin of the message (state transition). Accounts +(ExternallyOwnedAccount), which have credentials (e.g., a public/private key pairing), originate state transitions from +transactions. In contrast, module state transitions do not have authentication credentials backing them and can be +caused by two factors: either as a consequence of a state transition coming from a transaction or triggered by a scheduler +(e.g., the runtime's Begin/EndBlock). + +By implementing these proposed changes, the Cosmos SDK will benefit from a more extensible, versatile, and efficient account +management system that is better suited to address the requirements of the Cosmos ecosystem. + +#### Standardization + +With `x/accounts` allowing a modular api there becomes a need for standardization of accounts or the interfaces wallets and other clients should expect to use. For this reason we will be using the [`CIP` repo](https://github.com/cosmos/cips) in order to standardize interfaces in order for wallets to know what to expect when interacting with accounts. + +## Implementation + +### Account Definition + +We define the new `Account` type, which is what an account needs to implement to be treated as such. +An `Account` type is defined at APP level, so it cannot be dynamically loaded as the chain is running without upgrading the +node code, unless we create something like a `CosmWasmAccount` which is an account backed by an `x/wasm` contract. + +```go +// Account is what the developer implements to define an account. +type Account[InitMsg proto.Message] interface { + // Init is the function that initialises an account instance of a given kind. + // InitMsg is used to initialise the initial state of an account. + Init(ctx *Context, msg InitMsg) error + // RegisterExecuteHandlers registers an account's execution messages. + RegisterExecuteHandlers(executeRouter *ExecuteRouter) + // RegisterQueryHandlers registers an account's query messages. + RegisterQueryHandlers(queryRouter *QueryRouter) + // RegisterMigrationHandlers registers an account's migration messages. + RegisterMigrationHandlers(migrationRouter *MigrationRouter) +} +``` + +### The InternalAccount definition + +The public `Account` interface implementation is then converted by the runtime into an `InternalAccount` implementation, +which contains all the information and business logic needed to operate the account. + +```go +type Schema struct { + state StateSchema // represents the state of an account + init InitSchema // represents the init msg schema + exec ExecSchema // represents the multiple execution msg schemas, containing also responses + query QuerySchema // represents the multiple query msg schemas, containing also responses + migrate *MigrateSchema // represents the multiple migrate msg schemas, containing also responses, it's optional +} + +type InternalAccount struct { + init func(ctx *Context, msg proto.Message) (*InitResponse, error) + execute func(ctx *Context, msg proto.Message) (*ExecuteResponse, error) + query func(ctx *Context, msg proto.Message) (proto.Message, error) + schema func() *Schema + migrate func(ctx *Context, msg proto.Message) (*MigrateResponse, error) +} +``` + +This is an internal view of the account as intended by the system. It is not meant to be what developers implement. An +example implementation of the `InternalAccount` type can be found in [this](https://github.com/testinginprod/accounts-poc/blob/main/examples/recover/recover.go) +example of account whose credentials can be recovered. In fact, even if the `Internal` implementation is untyped (with +respect to `proto.Message`), the concrete implementation is fully typed. + +During any of the execution methods of `InternalAccount`, `schema` excluded, the account is given a `Context` which provides: + +* A namespaced `KVStore` for the account, which isolates the account state from others (NOTE: no `store keys` needed, + the account address serves as `store key`). +* Information regarding itself (its address) +* Information regarding the sender. +* ... + +#### Init + +Init defines the entrypoint that allows for a new account instance of a given kind to be initialised. +The account is passed some opaque protobuf message which is then interpreted and contains the instructions that +constitute the initial state of an account once it is deployed. + +An `Account` code can be deployed multiple times through the `Init` function, similar to how a `CosmWasm` contract code +can be deployed (Instantiated) multiple times. + +#### Execute + +Execute defines the entrypoint that allows an `Account` to process a state transition, the account can decide then how to +process the state transition based on the message provided and the sender of the transition. + +#### Query + +Query defines a read-only entrypoint that provides a stable interface that links an account with its state. The reason for +which `Query` is still being preferred as an addition to raw state reflection is to: + +* Provide a stable interface for querying (state can be optimised and change more frequently than a query) +* Provide a way to define an account `Interface` with respect to its `Read/Write` paths. +* Provide a way to query information that cannot be processed from raw state reflection, ex: compute information from lazy + state that has not been yet concretely processed (eg: balances with respect to lazy inputs/outputs) + +#### Schema + +Schema provides the definition of an account from `API` perspective, and it's the only thing that should be taken into account +when interacting with an account from another account or module, for example: an account is an `authz-interface` account if +it has the following message in its execution messages `MsgProxyStateTransition{ state_transition: google.Protobuf.Any }`. + +### Migrate + +Migrate defines the entrypoint that allows an `Account` to migrate its state from a previous version to a new one. Migrations +can be initiated only by the account itself, concretely this means that the migrate action sender can only be the account address +itself, if the account wants to allow another address to migrate it on its behalf then it could create an execution message +that makes the account migrate itself. + +### x/accounts module + +In order to create accounts we define a new module `x/accounts`, note that `x/accounts` deploys account with no authentication +credentials attached to it which means no action of an account can be incepted from a TX, we will later explore how the +`x/authn` module uses `x/accounts` to deploy authenticated accounts. + +This also has another important implication for which account addresses are now fully decoupled from the authentication mechanism +which makes in turn off-chain operations a little more complex, as the chain becomes the real link between account identifier +and credentials. + +We could also introduce a way to deterministically compute the account address. + +Note, from the transaction point of view, the `init_message` and `execute_message` are opaque `google.Protobuf.Any`. + +The module protobuf definition for `x/accounts` are the following: + +```protobuf +// Msg defines the Msg service. +service Msg { + rpc Deploy(MsgDeploy) returns (MsgDeployResponse); + rpc Execute(MsgExecute) returns (MsgExecuteResponse); + rpc Migrate(MsgMigrate) returns (MsgMigrateResponse); +} + +message MsgDeploy { + string sender = 1; + string kind = 2; + google.Protobuf.Any init_message = 3; + repeated google.Protobuf.Any authorize_messages = 4 [(gogoproto.nullable) = false]; +} + +message MsgDeployResponse { + string address = 1; + uint64 id = 2; + google.Protobuf.Any data = 3; +} + +message MsgExecute { + string sender = 1; + string address = 2; + google.Protobuf.Any message = 3; + repeated google.Protobuf.Any authorize_messages = 4 [(gogoproto.nullable) = false]; +} + +message MsgExecuteResponse { + google.Protobuf.Any data = 1; +} + +message MsgMigrate { + string sender = 1; + string new_account_kind = 2; + google.Protobuf.Any migrate_message = 3; +} + +message MsgMigrateResponse { + google.Protobuf.Any data = 1; +} + +``` + +#### MsgDeploy + +Deploys a new instance of the given account `kind` with initial settings represented by the `init_message` which is a `google.Protobuf.Any`. +Of course the `init_message` can be empty. A response is returned containing the account ID and humanised address, alongside some response +that the account instantiation might produce. + +#### Address derivation + +In order to decouple public keys from account addresses, we introduce a new address derivation mechanism which is + + +#### MsgExecute + +Sends a `StateTransition` execution request, where the state transition is represented by the `message` which is a `google.Protobuf.Any`. +The account can then decide if to process it or not based on the `sender`. + +### MsgMigrate + +Migrates an account to a new version of itself, the new version is represented by the `new_account_kind`. The state transition +can only be incepted by the account itself, which means that the `sender` must be the account address itself. During the migration +the account current state is given to the new version of the account, which then executes the migration logic using the `migrate_message`, +it might change state or not, it's up to the account to decide. The response contains possible data that the account might produce +after the migration. + +#### Authorize Messages + +The `Deploy` and `Execute` messages have a field in common called `authorize_messages`, these messages are messages that the account +can execute on behalf of the sender. For example, in case an account is expecting some funds to be sent from the sender, +the sender can attach a `MsgSend` that the account can execute on the sender's behalf. These authorizations are short-lived, +they live only for the duration of the `Deploy` or `Execute` message execution, or until they are consumed. + +An alternative would have been to add a `funds` field, like it happens in cosmwasm, which guarantees the called contract that +the funds are available and sent in the context of the message execution. This would have been a simpler approach, but it would +have been limited to the context of `MsgSend` only, where the asset is `sdk.Coins`. The proposed generic way, instead, allows +the account to execute any message on behalf of the sender, which is more flexible, it could include NFT send execution, or +more complex things like `MsgMultiSend` or `MsgDelegate`, etc. + + +### Further discussion + +#### Sub-accounts + +We could provide a way to link accounts to other accounts. Maybe during deployment the sender could decide to link the +newly created to its own account, although there might be use-cases for which the deployer is different from the account +that needs to be linked, in this case a handshake protocol on linking would need to be defined. + +#### Predictable address creation + +We need to provide a way to create an account with a predictable address, this might serve a lot of purposes, like accounts +wanting to generate an address that: + +* nobody else can claim besides the account used to generate the new account +* is predictable + +For example: + +```protobuf + +message MsgDeployPredictable { + string sender = 1; + uint32 nonce = 2; + ... +} +``` + +And then the address becomes `bechify(concat(sender, nonce))` + +`x/accounts` would still use the monotonically increasing sequence as account number. + +#### Joining Multiple Accounts + +As developers are building new kinds of accounts, it becomes necessary to provide a default way to combine the +functionalities of different account types. This allows developers to avoid duplicating code and enables end-users to +create or migrate to accounts with multiple functionalities without requiring custom development. + +To address this need, we propose the inclusion of a default account type called "MultiAccount". The MultiAccount type is +designed to merge the functionalities of other accounts by combining their execution, query, and migration APIs. +The account joining process would only fail in the case of API (intended as non-state Schema APIs) conflicts, ensuring +compatibility and consistency. + +With the introduction of the MultiAccount type, users would have the option to either migrate their existing accounts to +a MultiAccount type or extend an existing MultiAccount with newer APIs. This flexibility empowers users to leverage +various account functionalities without compromising compatibility or resorting to manual code duplication. + +The MultiAccount type serves as a standardized solution for combining different account functionalities within the +cosmos-sdk ecosystem. By adopting this approach, developers can streamline the development process and users can benefit +from a modular and extensible account system. diff --git a/versioned_docs/version-0.52/build/rfc/rfc-005-optimistic-execution.md b/versioned_docs/version-0.52/build/rfc/rfc-005-optimistic-execution.md new file mode 100644 index 000000000..997cc35aa --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/rfc-005-optimistic-execution.md @@ -0,0 +1,124 @@ +# RFC 005: Optimistic Execution + +## Changelog + +- 2023-06-07: Refactor for Cosmos SDK (@facundomedica) +- 2022-08-16: Initial draft by Sei Network + +## Background + +Before ABCI++, the first and only time a CometBFT blockchain's application layer would know about a block proposal is after the voting period, at which point CometBFT would invoke `BeginBlock`, `DeliverTx`, `EndBlock`, and `Commit` ABCI methods of the application, with the block proposal contents passed in. + +With the introduction of ABCI++, the application layer now receives the block proposal before the voting period commences. This can be used to optimistically execute the block proposal in parallel with the voting process, thus reducing the block time. + +## Proposal + +Given that the application receives the block proposal in an earlier stage (`ProcessProposal`), it can be executed in the background so when `FinalizeBlock` is called the response can returned instantly. + +## Decision + +The newly introduced ABCI method `ProcessProposal` is called after a node receives the full block proposal of the current height but before prevote starts. CometBFT states that preemptively executing the block proposal is a potential use case for it: + +> - **Usage**: +> - Contains all information on the proposed block needed to fully execute it. +> - The Application may fully execute the block as though it was handling +> `RequestFinalizeBlock`. +> - However, any resulting state changes must be kept as _candidate state_, +> and the Application should be ready to discard it in case another block is decided. +> - The Application MAY fully execute the block — immediate execution + +Nevertheless, synchronously executing the proposal preemptively would not improve block time because it would just change the order of events (so the time we would like to save will be spent at `ProcessProposal` instead of `FinalizeBlock`). + +Instead, we need to make block execution asynchronous by starting a goroutine in `ProcessProposal` (whose termination signal is kept in the application context) and returning a response immediately. That way, the actual block execution would happen at the same time as voting. When voting finishes and `FinalizeBlock` is called, the application handler can wait for the previously started goroutine to finish, and commit the resulting cache store if the block hash matches. + +Assuming average voting period takes `P` and average block execution takes `Q`, this would reduce the average block time by `P + Q - max(P, Q)`. + +Sei Network reported `P=~600ms` and `Q=~300ms` during a load test, meaning that optimistic execution could cut the block time by ~300ms. + +The following diagram illustrates the intended flow: + +```mermaid +flowchart LR; + Propose-->ProcessProposal; + ProcessProposal-->Prevote; + Precommit-->FinalizeBlock; + FinalizeBlock-->Commit; + subgraph CometBFT + Propose + Prevote-->Precommit; + Commit + end + subgraph CosmosSDK + OEG[Optimistic Execution Goroutine]-.->FinalizeBlock + ProcessProposal-.->OEG + FinalizeBlock + end +``` + +Some considerations: + +- In the case that a proposal is being rejected by the local node, this node won't attempt to execute the proposal. +- The app must drop any previous Optimistic Execution data if `ProcessProposal` is called, in other words, abort and restart. + +## Implementation + +The execution context needs to have the following information: + +- The block proposal (`abci.RequestFinalizeBlock`) +- Termination and completion signal for the OE goroutine + +The OE goroutine would run on top of a cached branch of the root multi-store (which is the default behavior for `FinalizeBlock` as we only write to the underlying store once we've reached the end). + +The OE goroutine would periodically check if a termination signal has been sent to it, and stops if so. Once the OE goroutine finishes the execution it will set the completion signal. + +The application developers will have the option to enable or disable Optimistic Execution, being disabled by default. To enable it, the application will need to pass the `SetOptimisticExecution` option to `NewBaseApp`. This is because this feature will be considered experimental until it's proven to be stable. Note that individual nodes should not enable/disable this feature on their own, as it could lead to inconsistent behavior. + +Upon receiving a `ProcessProposal` call, the SDK would adopt the following procedure if OE is enabled: + +``` +abort any running OE goroutine and wait for goroutine exit +if height > initial height: + set OE fields + kick off an OP goroutine that optimistically process the proposal +else: + do nothing +respond to CometBFT +``` + +OE won't be enabled during the first block of the chain, regardless of the configuration. This is because during the first block `InitChain` writes to `finalizeBlockState` so `FinalizeBlock` starts with an already initialized state. In the case that an abort would be needed it would mean we need to run `InitChain` again or discard/recover the state; either way complicates the implementation too much just for a single block. + +Upon receiving a `FinalizeBlock` call, the SDK would adopt the following procedure: + +``` +if OE is enabled: + abort if the currently executing OE's hash doesn't match the proposal's hash + if aborted: + process the proposal synchronously + else: + wait for the OE goroutine to finish + respond to CometBFT +``` + +## Consequences + +### Backwards Compatibility + +This can only be implemented on SDK versions using ABCI++, meaning v0.50.x and above. + +### Positive + +- Shorter block times for the same amount of transactions + +### Negative + +- Adds some complexity to the SDK +- Multiple rounds on a block that contains long running transactions could be problematic + +### Neutral + +- Each node can decide whether to use this feature or not. + +### References + +- [Original RFC by Sei Network](https://github.com/sei-protocol/sei-chain/blob/81b8af7980df722a63a910cc35ff96e60a94cbfe/docs/rfc/rfc-000-optimistic-proposal-processing.md) +- [ABCI++ methods in CometBFT](https://github.com/cometbft/cometbft/blob/a09f5d33ecd8846369b93cae9063291eb8abc3a0/spec/abci/abci%2B%2B_methods.md) diff --git a/versioned_docs/version-0.52/build/rfc/rfc-006-handlers.md b/versioned_docs/version-0.52/build/rfc/rfc-006-handlers.md new file mode 100644 index 000000000..a22992ccd --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/rfc-006-handlers.md @@ -0,0 +1,282 @@ +# RFC 006: Handlers + +## Changelog + +* January 26, 2024: Initialized + +## Background + +The Cosmos SDK has a very powerful and flexible module system that has been tested +and proven to be very good in production. The design of how messages are handled +is built around Protobuf services and gRPC. This design was proposed and implemented +during a time when we migrated from Amino to Protocol Buffers. This design has +fulfilled the needs of users today. While this design is useful it has caused a +elevated learning curve to be adopted by users. Today, these services are the +only way to write a module. This RFC proposes a new design that simplifies the +design and enables new use cases we are seeing today. + +Taking a step back, we have seen the emergence of rollups and proving technologies. +These technologies enable new use cases and new methods of achieving various goals. +When we look at things like proving we look to [TinyGo](https://TinyGo.org/). When +we have attempted to use TinyGo with existing modules we have run into a hiccup, +the use of [gRPC](https://github.com/TinyGo-org/TinyGo/issues/2814) within modules. +This has led us to look at a design which would allow the usage of TinyGo and +other technologies. + +We looked at TinyGo for our first target in order to compile down to a 32 bit environment which could be used with +things like [Risc-0](https://www.risczero.com/), [Fluent](https://fluentlabs.xyz/) and other technologies. When speaking with the teams behind these technologies +we found that they were interested in using the Cosmos SDK but were unable to due to being unable to use TinyGo or the +Cosmos SDK go code in a 32 bit environment. + +The Cosmos SDK team has been hard at work over the last few months designing and implementing a modular core layer, with +the idea that proving can be enabled later on. This design allows us to push the design of what can be done with the +Cosmos SDK to the next level. In the future when we have proving tools and technologies integrated parts of the new core +layer will be able to be used in conjunction with proving technologies without the need to rewrite the stack. + + +## Proposal + +This proposal is around enabling modules to be compiled to an environment in which they can be used with TinyGo and/or +different proving technologies. + +> Note the usage of handlers in modules is optional, modules can still use the existing design. This design is meant to +> be a new way to write modules, with proving in mind, and is not meant to replace the existing design. + +This proposal is for server/v2. Baseapp will continue to work in the same way as it does today. + +### Pre and Post Message Handlers + +In the Cosmos SDK, there exists hooks on messages and execution of function calls. Separating the two we will focus on +message hooks. When a message is implemented it can be unknown if others will use the module and if a message will need +hooks. When hooks are needed before or after a message, users are required to fork the module. This is not ideal as it +leads to a lot of forks of modules and a lot of code duplication. + +Pre and Post message handlers solve this issue. Where we allow modules to register listeners for messages in order to +execute something before and/or after the message. Although hooks can be bypassed by doing keeper function calls, we can +assume that as we shift the communication design of the SDK to use messages instead of keeper calls it will be safe to +assume that the bypass surface of hooks will reduce to zero. + +If an application developer would like to check the sender of funds before the message is executed they can +register a pre message handler. If the message is called by a user the pre message handler will be called with the custom +logic. If the sender is not allowed to send funds the pre-message handler can return an error and the message will not be +executed. + +> Note: This is different from the ante-handler and post-handler we have today. These will still exist in the same form. + +A module can register handlers for any or all message(s), this allows for modules to be extended without the need to fork. + +A module will implement the below for a pre-message hook: + +```golang +package core_appmodule + +import ( + "context" + + "google.golang.org/protobuf/runtime/protoiface" +) + +type PreMsgHandlerRouter interface { + // RegisterGlobalPreMsgHandler will register a pre msg handler that hooks before any message executes. + // Handler will be called before ANY message executes. + RegisterGlobalPreMsgHandler(handler func(ctx context.Context, msg transaction.Msg) error) + // RegisterPreMsgHandler will register a pre msg handler that hooks before the provided message + // with the given message name executes. Handler will be called before the message is executed + // by the module. + RegisterPreMsgHandler(msgName string, handler func(ctx context.Context, msg transaction.Msg) error) +} + +type HasPreMsgHandler interface { + RegisterPreMsgHandler(router PreMsgHandlerRouter) +} +``` + +A module will implement the below for a postmessage hook: + +```go +package core_appmodule + +import ( + "context" + + "google.golang.org/protobuf/runtime/protoiface" +) + +type PostMsgHandlerRouter interface { + // RegisterGlobalPostMsgHandler will register a post msg handler that hooks after any message executes. + // Handler will be called after ANY message executes, alongside the response. + RegisterGlobalPostMsgHandler(handler func(ctx context.Context, msg, msgResp transaction.Msg) error) + // RegisterPostMsgHandler will register a pre msg handler that hooks after the provided message + // with the given message name executes. Handler will be called after the message is executed + // by the module, alongside the response returned by the module. + RegisterPostMsgHandler(msgName string, handler func(ctx context.Context, msg, msgResp transaction.Msg) error) +} + +type HasPostMsgHandler interface { + RegisterPostMsgHandler(router PostMsgHandlerRouter) +} +``` + +We note the following behaviors: + +* A pre msg handler returning an error will yield to a state transition revert. +* A post msg handler returning an error will yield to a state transition revert. +* A post msg handler will not be called if the execution handler (message handler) fails. +* A post msg handler will be called only if the execution handler succeeds alongside the response provided by the execution handler. + +### Message and Query Handlers + +Similar to the above design, message handlers will allow the application developer to replace existing gRPC based services +with handlers. This enables the module to be compiled down to TinyGo, and abandon the gRPC dependency. As mentioned +upgrading the modules immediately is not mandatory, module developers can do so in a gradual way. Application developers have the option to use the existing gRPC services or the new handlers. + +For message handlers we propose the introduction of the following core/appmodule interfaces and functions: + +```go +package core_appmodule + +import ( + "context" + + "google.golang.org/protobuf/runtime/protoiface" +) + +type MsgHandlerRouter interface { + RegisterMsgHandler(msgName string, handler func(ctx context.Context, msg transaction.Msg) (msgResp transaction.Msg, err error)) +} + +type HasMsgHandler interface { + RegisterMsgHandlers(router MsgHandlerRouter) +} + +// RegisterMsgHandler is a helper function to retain type safety when creating handlers, so we do not need to cast messages. +func RegisterMsgHandler[Req, Resp transaction.Msg](router MsgHandlerRouter, handler func(ctx context.Context, req Req) (resp Resp, err error)) { + // impl detail +} +``` + +Example + +```go +package bank + +func (b BankKeeper) Send(ctx context.Context, msg bank.MsgSend) (bank.MsgSendResponse, error) { + // logic +} + +func (b BankModule) RegisterMsgHandlers(router core_appmodule.MsgHandlerRouter) { + // the RegisterMsgHandler function takes care of doing type casting and conversions, ensuring we retain type safety + core_appmodule.RegisterMsgHandler(router, b.Send) +} + +``` + +This change is fully state machine compatible as, even if we were using gRPC, messages were +routed using the message type name and not the gRCP method name. + +We apply the same principles of MsgHandlers to QueryHandlers, by introducing a new core/appmodule interface: + +```go +package core_appmodule + +import ( + "context" + + "google.golang.org/protobuf/runtime/protoiface" +) + +type QueryHandlerRouter interface { + RegisterQueryHandler(msgName string, handler func(ctx context.Context, req transaction.Msg) (resp transaction.Msg, err error)) +} + +type HasQueryHandler interface { + RegisterQueryHandlers(router QueryHandlerRouter) +} + +// RegisterQueryHandler is a helper function to retain type safety when creating handlers, so we do not need to cast messages. +func RegisterQueryHandler[Req, Resp transaction.Msg](router QueryHandlerRouter, handler func(ctx context.Context, req Req) (resp Resp, err error)) { + // impl detail +} + +``` + +The difference between gRPC handlers and query handlers is that we expect query handlers to be deterministic and usable +in consensus by other modules. Non consensus queries should be registered outside of the state machine itself, and we will +provide guidelines on how to do so with serverv2. + +As a consequence queries would be now mapped by their message name. + +We can provide JSON exposure of the Query APIs following this rest API format: + +``` +method: POST +path: /msg_name +ReqBody: protojson.Marshal(msg) +---- +RespBody: protojson.Marshal(msgResp) +``` + +### Consensus Messages + +Similar to the above design, consensus messages will allow the underlying consensus engine to speak to the modules. Today we get consensus related information from `sdk.Context`. In server/v2 we are unable to continue with this design due to the forced dependency leakage of comet throughout the repo. Secondly, while we already have `cometInfo` if we were to put this on the new execution client we would be tying CometBFT to the application manager and STF. + +In the case of CometBFT, consensus would register handlers for consensus messages for evidence, voteinfo and consensus params. This would allow the consensus engine to speak to the modules. + + +```go +package consensus + +func (b ConsensusKeeper) ConsensusParams(ctx context.Context, msg bank.MsgConsensusParams) (bank.MsgConsensusParamsResponse, error) { + // logic +} + +func (b CircuitModule) RegisterConsensusHandlers(router core_appmodule.MsgHandlerRouter) { + // the RegisterConsensusHandler function takes care of doing type casting and conversions, ensuring we retain type safety + core_appmodule.RegisterConsensusHandler(router, b.Send) +} + +``` + + +## Consequences + +* REST endpoints for message and queries change due to lack of services and gRPC gateway annotations. +* When using gRPC directly, one must query a schema endpoint in order to see all possible messages and queries. + +### Backwards Compatibility + +The way to interact with modules changes, REST and gRPC will still be available. + +### Positive + +* Allows modules to be compiled to TinyGo. +* Reduces the cosmos-sdk's learning curve, since understanding gRPC semantics is not a must anymore. +* Allows other modules to extend existing modules behaviour using pre and post msg handlers, without forking. +* The system becomes overall more simple as gRPC is not anymore a hard dependency and requirement for the state machine. +* Reduces the need on sdk.Context +* Concurrently safe +* Reduces public interface of modules + +### Negative + +* Pre, Post and Consensus msg handlers are a new concept that module developers need to learn (although not immediately). + +### Neutral + +> {neutral consequences} + +### References + +> Links to external materials needed to follow the discussion may be added here. +> +> In addition, if the discussion in a request for comments leads to any design +> decisions, it may be helpful to add links to the ADR documents here after the +> discussion has settled. + +## Discussion + +> This section contains the core of the discussion. +> +> There is no fixed format for this section, but ideally changes to this +> section should be updated before merging to reflect any discussion that took +> place on the PR that made those changes. diff --git a/versioned_docs/version-0.52/build/rfc/rfc-template.md b/versioned_docs/version-0.52/build/rfc/rfc-template.md new file mode 100644 index 000000000..f4e79fbb4 --- /dev/null +++ b/versioned_docs/version-0.52/build/rfc/rfc-template.md @@ -0,0 +1,83 @@ +# RFC {RFC-NUMBER}: {TITLE} + +## Changelog + +* {date}: {changelog} + +## Background + +> The next section is the "Background" section. This section should be at least two paragraphs and can take up to a whole +> page in some cases. The guiding goal of the background section is: as a newcomer to this project (new employee, team +> transfer), can I read the background section and follow any links to get the full context of why this change is +> necessary? +> +> If you can't show a random engineer the background section and have them acquire nearly full context on the necessity +> for the RFC, then the background section is not full enough. To help achieve this, link to prior RFCs, discussions, and +> more here as necessary to provide context so you don't have to simply repeat yourself. + + +## Proposal + +> The next required section is "Proposal" or "Goal". Given the background above, this section proposes a solution. +> This should be an overview of the "how" for the solution, but for details further sections will be used. + + +## Abandoned Ideas (Optional) + +> As RFCs evolve, it is common that there are ideas that are abandoned. Rather than simply deleting them from the +> document, you should try to organize them into sections that make it clear they're abandoned while explaining why they +> were abandoned. +> +> When sharing your RFC with others or having someone look back on your RFC in the future, it is common to walk the same +> path and fall into the same pitfalls that we've since matured from. Abandoned ideas are a way to recognize that path +> and explain the pitfalls and why they were abandoned. + +## Decision + +> This section describes alternative designs to the chosen design. This section +> is important and if an adr does not have any alternatives then it should be +> considered that the ADR was not thought through. + +## Consequences (optional) + +> This section describes the resulting context, after applying the decision. All +> consequences should be listed here, not just the "positive" ones. A particular +> decision may have positive, negative, and neutral consequences, but all of them +> affect the team and project in the future. + +### Backwards Compatibility + +> All ADRs that introduce backwards incompatibilities must include a section +> describing these incompatibilities and their severity. The ADR must explain +> how the author proposes to deal with these incompatibilities. ADR submissions +> without a sufficient backwards compatibility treatise may be rejected outright. + +### Positive + +> {positive consequences} + +### Negative + +> {negative consequences} + +### Neutral + +> {neutral consequences} + + + +### References + +> Links to external materials needed to follow the discussion may be added here. +> +> In addition, if the discussion in a request for comments leads to any design +> decisions, it may be helpful to add links to the ADR documents here after the +> discussion has settled. + +## Discussion + +> This section contains the core of the discussion. +> +> There is no fixed format for this section, but ideally changes to this +> section should be updated before merging to reflect any discussion that took +> place on the PR that made those changes. diff --git a/versioned_docs/version-0.52/build/spec/README.md b/versioned_docs/version-0.52/build/spec/README.md new file mode 100644 index 000000000..91f347a8e --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/README.md @@ -0,0 +1,25 @@ +--- +sidebar_position: 1 +--- + +# Specifications + +This directory contains specifications for the modules of the Cosmos SDK as well as Interchain Standards (ICS) and other specifications. + +Cosmos SDK applications hold this state in a Merkle store. Updates to +the store may be made during transactions and at the beginning and end of every +block. + +## Cosmos SDK specifications + +* [Store](./store) - The core Merkle store that holds the state. +* [Bech32](./addresses/bech32.md) - Address format for Cosmos SDK applications. + +## Modules specifications + +Go the [module directory](https://docs.cosmos.network/main/modules) + +## CometBFT + +For details on the underlying blockchain and p2p protocols, see +the [CometBFT specification](https://github.com/cometbft/cometbft/tree/main/spec). diff --git a/versioned_docs/version-0.52/build/spec/SPEC_MODULE.md b/versioned_docs/version-0.52/build/spec/SPEC_MODULE.md new file mode 100644 index 000000000..275d17a5c --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/SPEC_MODULE.md @@ -0,0 +1,60 @@ +# Specification of Modules + +This file intends to outline the common structure for specifications within +this directory. + +## Tense + +For consistency, specs should be written in passive present tense. + +## Pseudo-Code + +Generally, pseudo-code should be minimized throughout the spec. Often, simple +bulleted-lists which describe a function's operations are sufficient and should +be considered preferable. In certain instances, due to the complex nature of +the functionality being described pseudo-code may the most suitable form of +specification. In these cases use of pseudo-code is permissible, but should be +presented in a concise manner, ideally restricted to only the complex +element as a part of a larger description. + +## Common Layout + +The following generalized `README` structure should be used to breakdown +specifications for modules. The following list is nonbinding and all sections are optional. + +* `# {Module Name}` - overview of the module +* `## Concepts` - describe specialized concepts and definitions used throughout the spec +* `## State` - specify and describe structures expected to be marshalled into the store, and their keys +* `## State Transitions` - standard state transition operations triggered by hooks, messages, etc. +* `## Messages` - specify message structure(s) and expected state machine behaviour(s) +* `## Begin Block` - specify any begin-block operations +* `## End Block` - specify any end-block operations +* `## Hooks` - describe available hooks to be called by/from this module +* `## Events` - list and describe event tags used +* `## Client` - list and describe CLI commands and gRPC and REST endpoints +* `## Params` - list all module parameters, their types (in JSON) and examples +* `## Future Improvements` - describe future improvements of this module +* `## Tests` - acceptance tests +* `## Appendix` - supplementary details referenced elsewhere within the spec + +### Notation for key-value mapping + +Within `## State` the following notation `->` should be used to describe key to +value mapping: + +```text +key -> value +``` + +to represent byte concatenation the `|` may be used. In addition, encoding +type may be specified, for example: + +```text +0x00 | addressBytes | address2Bytes -> amino(value_object) +``` + +Additionally, index mappings may be specified by mapping to the `nil` value, for example: + +```text +0x01 | address2Bytes | addressBytes -> nil +``` diff --git a/versioned_docs/version-0.52/build/spec/SPEC_STANDARD.md b/versioned_docs/version-0.52/build/spec/SPEC_STANDARD.md new file mode 100644 index 000000000..51797dfaf --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/SPEC_STANDARD.md @@ -0,0 +1,121 @@ +# What is an SDK standard? + +An SDK standard is a design document describing a particular protocol, standard, or feature expected to be used by the Cosmos SDK. A SDK standard should list the desired properties of the standard, explain the design rationale, and provide a concise but comprehensive technical specification. The primary author is responsible for pushing the proposal through the standardization process, soliciting input and support from the community, and communicating with relevant stakeholders to ensure (social) consensus. + +## Sections + +A SDK standard consists of: + +* a synopsis, +* overview and basic concepts, +* technical specification, +* history log, and +* copyright notice. + +All top-level sections are required. References should be included inline as links, or tabulated at the bottom of the section if necessary. Included sub-sections should be listed in the order specified below. + +### Table Of Contents + +Provide a table of contents at the top of the file to assist readers. + +### Synopsis + +The document should include a brief (~200 word) synopsis providing a high-level description of and rationale for the specification. + +### Overview and basic concepts + +This section should include a motivation sub-section and a definitions sub-section if required: + +* *Motivation* - A rationale for the existence of the proposed feature, or the proposed changes to an existing feature. +* *Definitions* - A list of new terms or concepts utilized in the document or required to understand it. + +### System model and properties + +This section should include an assumptions sub-section if any, the mandatory properties sub-section, and a dependencies sub-section. Note that the first two sub-section are tightly coupled: how to enforce a property will depend directly on the assumptions made. This sub-section is important to capture the interactions of the specified feature with the "rest-of-the-world", i.e., with other features of the ecosystem. + +* *Assumptions* - A list of any assumptions made by the feature designer. It should capture which features are used by the feature under specification, and what do we expect from them. +* *Properties* - A list of the desired properties or characteristics of the feature specified, and expected effects or failures when the properties are violated. In case it is relevant, it can also include a list of properties that the feature does not guarantee. +* *Dependencies* - A list of the features that use the feature under specification and how. + +### Technical specification + +This is the main section of the document, and should contain protocol documentation, design rationale, required references, and technical details where appropriate. +The section may have any or all of the following sub-sections, as appropriate to the particular specification. The API sub-section is especially encouraged when appropriate. + +* *API* - A detailed description of the feature's API. +* *Technical Details* - All technical details including syntax, diagrams, semantics, protocols, data structures, algorithms, and pseudocode as appropriate. The technical specification should be detailed enough such that separate correct implementations of the specification without knowledge of each other are compatible. +* *Backwards Compatibility* - A discussion of compatibility (or lack thereof) with previous feature or protocol versions. +* *Known Issues* - A list of known issues. This sub-section is specially important for specifications of already in-use features. +* *Example Implementation* - A concrete example implementation or description of an expected implementation to serve as the primary reference for implementers. + +### History + +A specification should include a history section, listing any inspiring documents and a plaintext log of significant changes. + +See an example history section [below](#history-1). + +### Copyright + +A specification should include a copyright section waiving rights via [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). + +## Formatting + +### General + +Specifications must be written in GitHub-flavoured Markdown. + +For a GitHub-flavoured Markdown cheat sheet, see [here](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). For a local Markdown renderer, see [here](https://github.com/joeyespo/grip). + +### Language + +Specifications should be written in Simple English, avoiding obscure terminology and unnecessary jargon. For excellent examples of Simple English, please see the [Simple English Wikipedia](https://simple.wikipedia.org/wiki/Main_Page). + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in specifications are to be interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). + +### Pseudocode + +Pseudocode in specifications should be language-agnostic and formatted in a simple imperative standard, with line numbers, variables, simple conditional blocks, for loops, and +English fragments where necessary to explain further functionality such as scheduling timeouts. LaTeX images should be avoided because they are difficult to review in diff form. + +Pseudocode for structs can be written in a simple language like Typescript or golang, as interfaces. + +Example Golang pseudocode struct: + +```go +type CacheKVStore interface { + cache: map[Key]Value + parent: KVStore + deleted: Key +} +``` + +Pseudocode for algorithms should be written in simple Golang, as functions. + +Example pseudocode algorithm: + +```go +func get( + store CacheKVStore, + key Key) Value { + + value = store.cache.get(Key) + if (value !== null) { + return value + } else { + value = store.parent.get(key) + store.cache.set(key, value) + return value + } +} +``` + +## History + +This specification was significantly inspired by and derived from IBC's [ICS](https://github.com/cosmos/ibc/blob/main/spec/ics-001-ics-standard/README.md), which +was in turn derived from Ethereum's [EIP 1](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1.md). + +Nov 24, 2022 - Initial draft finished and submitted as a PR + +## Copyright + +All content herein is licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). diff --git a/versioned_docs/version-0.52/build/spec/_category_.json b/versioned_docs/version-0.52/build/spec/_category_.json new file mode 100644 index 000000000..5c2ccf7d4 --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Specifications", + "position": 8, + "link": null +} \ No newline at end of file diff --git a/versioned_docs/version-0.52/build/spec/_ics/README.md b/versioned_docs/version-0.52/build/spec/_ics/README.md new file mode 100644 index 000000000..803e0c89c --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/_ics/README.md @@ -0,0 +1,3 @@ +# Cosmos ICS + +* [ICS030 - Signed Messages](./ics-030-signed-messages.md) diff --git a/versioned_docs/version-0.52/build/spec/_ics/ics-030-signed-messages.md b/versioned_docs/version-0.52/build/spec/_ics/ics-030-signed-messages.md new file mode 100644 index 000000000..6ea8ccfaa --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/_ics/ics-030-signed-messages.md @@ -0,0 +1,192 @@ +# ICS 030: Cosmos Signed Messages + +>TODO: Replace with valid ICS number and possibly move to new location. + +* [Changelog](#changelog) +* [Abstract](#abstract) +* [Preliminary](#preliminary) +* [Specification](#specification) +* [Future Adaptations](#future-adaptations) +* [API](#api) +* [References](#references) + +## Status + +Proposed. + +## Changelog + +## Abstract + +Having the ability to sign messages off-chain has proven to be a fundamental aspect +of nearly any blockchain. The notion of signing messages off-chain has many +added benefits such as saving on computational costs and reducing transaction +throughput and overhead. Within the context of the Cosmos, some of the major +applications of signing such data includes, but is not limited to, providing a +cryptographic secure and verifiable means of proving validator identity and +possibly associating it with some other framework or organization. In addition, +having the ability to sign Cosmos messages with a Ledger or similar HSM device. + +A standardized protocol for hashing, signing, and verifying messages that can be +implemented by the Cosmos SDK and other third-party organizations is needed. Such a +standardized protocol subscribes to the following: + +* Contains a specification of human-readable and machine-verifiable typed structured data +* Contains a framework for deterministic and injective encoding of structured data +* Utilizes cryptographic secure hashing and signing algorithms +* A framework for supporting extensions and domain separation +* Is invulnerable to chosen ciphertext attacks +* Has protection against potentially signing transactions a user did not intend to + +This specification is only concerned with the rationale and the standardized +implementation of Cosmos signed messages. It does **not** concern itself with the +concept of replay attacks as that will be left up to the higher-level application +implementation. If you view signed messages in the means of authorizing some +action or data, then such an application would have to either treat this as +idempotent or have mechanisms in place to reject known signed messages. + +## Preliminary + +The Cosmos message signing protocol will be parameterized with a cryptographic +secure hashing algorithm `SHA-256` and a signing algorithm `S` that contains +the operations `sign` and `verify` which provide a digital signature over a set +of bytes and verification of a signature respectively. + +Note, our goal here is not to provide context and reasoning about why necessarily +these algorithms were chosen apart from the fact they are the defacto algorithms +used in CometBFT and the Cosmos SDK and that they satisfy our needs for such +cryptographic algorithms such as having resistance to collision and second +pre-image attacks, as well as being [deterministic](https://en.wikipedia.org/wiki/Hash_function#Determinism) and [uniform](https://en.wikipedia.org/wiki/Hash_function#Uniformity). + +## Specification + +CometBFT has a well established protocol for signing messages using a canonical +JSON representation as defined [here](https://github.com/cometbft/cometbft/blob/master/types/canonical.go). + +An example of such a canonical JSON structure is CometBFT's vote structure: + +```go +type CanonicalJSONVote struct { + ChainID string `json:"@chain_id"` + Type string `json:"@type"` + BlockID CanonicalJSONBlockID `json:"block_id"` + Height int64 `json:"height"` + Round int `json:"round"` + Timestamp string `json:"timestamp"` + VoteType byte `json:"type"` +} +``` + +With such canonical JSON structures, the specification requires that they include +meta fields: `@chain_id` and `@type`. These meta fields are reserved and must be +included. They are both of type `string`. In addition, fields must be ordered +in lexicographically ascending order. + +For the purposes of signing Cosmos messages, the `@chain_id` field must correspond +to the Cosmos chain identifier. The user-agent should **refuse** signing if the +`@chain_id` field does not match the currently active chain! The `@type` field +must equal the constant `"message"`. The `@type` field corresponds to the type of +structure the user will be signing in an application. For now, a user is only +allowed to sign bytes of valid ASCII text ([see here](https://github.com/cometbft/cometbft/blob/v0.38.0/libs/strings/string.go#L57-L67)). +However, this will change and evolve to support additional application-specific +structures that are human-readable and machine-verifiable. + +Thus, we can have a canonical JSON structure for signing Cosmos messages using +the [JSON schema](http://json-schema.org/) specification as such: + +```json +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "$id": "cosmos/signing/typeData/schema", + "title": "The Cosmos signed message typed data schema.", + "type": "object", + "properties": { + "@chain_id": { + "type": "string", + "description": "The corresponding Cosmos chain identifier.", + "minLength": 1 + }, + "@type": { + "type": "string", + "description": "The message type. It must be 'message'.", + "enum": [ + "message" + ] + }, + "text": { + "type": "string", + "description": "The valid ASCII text to sign.", + "pattern": "^[\\x20-\\x7E]+$", + "minLength": 1 + } + }, + "required": [ + "@chain_id", + "@type", + "text" + ] +} +``` + +e.g. + +```json +{ + "@chain_id": "1", + "@type": "message", + "text": "Hello, you can identify me as XYZ on keybase." +} +``` + +## Future Adaptations + +As applications can vary greatly in domain, it will be vital to support both +domain separation and human-readable and machine-verifiable structures. + +Domain separation will allow for application developers to prevent collisions of +otherwise identical structures. It should be designed to be unique per application +use and should directly be used in the signature encoding itself. + +Human-readable and machine-verifiable structures will allow end users to sign +more complex structures, apart from just string messages, and still be able to +know exactly what they are signing (opposed to signing a bunch of arbitrary bytes). + +Thus, in the future, the Cosmos signing message specification will be expected +to expand upon it's canonical JSON structure to include such functionality. + +## API + +Application developers and designers should formalize a standard set of APIs that +adhere to the following specification: + +----- + +### **cosmosSignBytes** + +Params: + +* `data`: the Cosmos signed message canonical JSON structure +* `address`: the Bech32 Cosmos account address to sign data with + +Returns: + +* `signature`: the Cosmos signature derived using signing algorithm `S` + +----- + +### Examples + +Using the `secp256k1` as the DSA, `S`: + +```javascript +data = { + "@chain_id": "1", + "@type": "message", + "text": "I hereby claim I am ABC on Keybase!" +} + +cosmosSignBytes(data, "cosmos1pvsch6cddahhrn5e8ekw0us50dpnugwnlfngt3") +> "0x7fc4a495473045022100dec81a9820df0102381cdbf7e8b0f1e2cb64c58e0ecda1324543742e0388e41a02200df37905a6505c1b56a404e23b7473d2c0bc5bcda96771d2dda59df6ed2b98f8" +``` + +## References diff --git a/versioned_docs/version-0.52/build/spec/addresses/README.md b/versioned_docs/version-0.52/build/spec/addresses/README.md new file mode 100644 index 000000000..61db3aa93 --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/addresses/README.md @@ -0,0 +1,3 @@ +# Addresses spec + +* [Bech32](./bech32.md) diff --git a/versioned_docs/version-0.52/build/spec/addresses/bech32.md b/versioned_docs/version-0.52/build/spec/addresses/bech32.md new file mode 100644 index 000000000..626dfd11d --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/addresses/bech32.md @@ -0,0 +1,21 @@ +# Bech32 on Cosmos + +The Cosmos network prefers to use the Bech32 address format wherever users must handle binary data. Bech32 encoding provides robust integrity checks on data and the human readable part (HRP) provides contextual hints that can assist UI developers with providing informative error messages. + +In the Cosmos network, keys and addresses may refer to a number of different roles in the network like accounts, validators etc. + +## HRP table + +| HRP | Definition | +| ---------------- | ------------------------------------- | +| cosmos | Cosmos Account Address | +| cosmosvalcons | Cosmos Validator Consensus Address | +| cosmosvaloper | Cosmos Validator Operator Address | + +## Encoding + +While all user facing interfaces to Cosmos software should exposed Bech32 interfaces, many internal interfaces encode binary value in hex or base64 encoded form. + +To convert between other binary representation of addresses and keys, it is important to first apply the Amino encoding process before Bech32 encoding. + +A complete implementation of the Amino serialization format is unnecessary in most cases. Simply prepending bytes from this [table](https://github.com/cometbft/cometbft/blob/main/spec/blockchain/encoding.md) to the byte string payload before Bech32 encoding will sufficient for compatible representation. diff --git a/versioned_docs/version-0.52/build/spec/fee_distribution/f1_fee_distr.pdf b/versioned_docs/version-0.52/build/spec/fee_distribution/f1_fee_distr.pdf new file mode 100644 index 000000000..b99953869 Binary files /dev/null and b/versioned_docs/version-0.52/build/spec/fee_distribution/f1_fee_distr.pdf differ diff --git a/versioned_docs/version-0.52/build/spec/fee_distribution/f1_fee_distr.tex b/versioned_docs/version-0.52/build/spec/fee_distribution/f1_fee_distr.tex new file mode 100644 index 000000000..6f230a286 --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/fee_distribution/f1_fee_distr.tex @@ -0,0 +1,245 @@ +\documentclass[]{article} +\usepackage{hyperref} + +%opening +\title{F1 Fee Distribution Draft-02} +\author{Dev Ojha} + +\begin{document} + +\maketitle + +\begin{abstract} + In a proof of stake blockchain, validators need to split the rewards gained from transaction fees each block. Furthermore, these fees must be fairly distributed to each of a validator's constituent delegators. They accrue this reward throughout the entire time they are delegated, and they have a special operation to withdraw accrued rewards. + + The F1 fee distribution scheme works for any algorithm to split funds between validators each block, with minimal iteration, and the only approximations being due to finite decimal precision. Per block there is a single iteration over the validator set, to enable reward algorithms that differ by validator. No iteration is required to delegate, or withdraw. The state usage is one state update per validator per block, and one state entry per active delegation. It can optionally handle arbitrary inflation schemes, and auto-bonding of rewards. +\end{abstract} + +\section{F1 Fee Distribution} + +\subsection{Context} +In a proof of stake blockchain, each validator has an associated stake. +Transaction fees get rewarded to validators based on the incentive scheme of the underlying proof of stake model. +The fee distribution problem occurs in proof of stake blockchains supporting delegation, as there is a need to distribute a validator's fee rewards to its delegators. +The trivial solution of just giving the rewards to each delegator every block is too expensive to perform on-chain. +So instead fee distribution algorithms have delegators perform a withdraw action, which when performed yields the same total amount of fees as if they had received them at every block. + +This details F1, an approximation-free, slash-tolerant fee distribution algorithm which allows validator commission-rates, inflation rates, and fee proportions, which can all efficiently change per validator, every block. +The algorithm requires iterating over the bonded validators every block, and withdraws require no iteration. +This is cheap, due to staking logic already requiring iteration over all validators, which causes the expensive state-reads to be cached. + +The key point of how F1 works is that it tracks how much rewards a delegator with 1 stake for a given validator would be entitled to if it had bonded at block 0 until the latest block. +When a delegator bonds at block $b$, the amount of rewards a delegator with 1 stake would have if bonded at block 0 until block $b$ is also persisted to state. +When the delegator withdraws, they receive the difference of these two values. +Since rewards are distributed according to stake-weighting, this amount of rewards can be scaled by the amount of stake a delegator had. +Section 1.2 describes this in more detail, with an argument for it being approximation free. +Section 2 details how to adapt this algorithm to handle commission rates, slashing, and inflation. + +\subsection{Base algorithm} +In this section, we show that the F1 base algorithm gives each delegator rewards identical to that which they'd receive in the naive and correct fee distribution algorithm that iterated over all delegators every block. + +Even distribution of a validators rewards amongst its validators weighted by stake means the following: +Suppose a delegator delegates $x$ stake to a validator $v$ at block $h$. +Let the amount of stake the validator has at block $i$ be $s_i$ and the amount of fees they receive at this height be $f_i$. +Then if a delegator contributing $x$ stake decides to withdraw at block $n$, the rewards they receive are +$$\sum_{i = h}^{n} \frac{x}{s_i}f_i = x \sum_{i = h}^{n} \frac{f_i}{s_i}$$ + +Note that $s_i$ does not change every block, +it only changes if the validator gets slashed, +or if any delegator alters the amount they have delegated. +We'll relegate handling of slashes to \autoref{ssec:slashing}, +and only consider the case with no slashing here. +We can change the iteration from being over every block, to instead being over the set of blocks between two changes in validator $v$'s total stake. +Let each of these set of blocks be called a period. +A new period begins every time that validator's total stake changes. +Let the total amount of stake for the validator in period $p$ be $n_p$. +Let $T_p$ be the total fees that validator $v$ accrued in period $p$. +Let $h$ be the start of period $p_{init}$, and height $n$ be the end of $p_{final}$. +It follows that +$$x \sum_{i = h}^{n} \frac{f_i}{s_i} = x \sum_{p = p_{init}}^{p_{final}} \frac{T_p}{n_p}$$ + +Let $p_0$ represent the period which begins when the validator first bonds. +The central idea to the F1 model is that at the end of the $k$th period, +the following is stored at a state location indexable by $k$: $\sum_{i=0}^{k}\frac{T_i}{n_i}$. +Let the index of the current period be $f$. +When a delegator wants to delegate or withdraw their reward, they first create a new entry in state to end the current period. +Then this entry is created using the previous entry as follows: +$$Entry_f = \sum_{i=0}^{f}\frac{T_i}{n_i} = \sum_{i=0}^{f-1}\frac{T_i}{n_i} + \frac{T_f}{n_f} = Entry_{f-1} + \frac{T_f}{n_f}$$ +Where $T_f$ is the fees the validator has accrued in period $f$, and $n_f$ is the validators total amount of stake in period $f$. + +The withdrawer's delegation object has the index $k$ for the period which they ended by bonding. (They start receiving rewards for period $k + 1$) +The reward they should receive when withdrawing is: + +$$x \sum_{i = k + 1}^{f} \frac{T_i}{n_i} = x\left(\left(\sum_{i=0}^{f}\frac{T_i}{n_i}\right) - \left(\sum_{i=0}^{k}\frac{T_i}{n_i}\right)\right) = x\left(Entry_f - Entry_k\right)$$ + +It is clear from the equations that this payout mechanism maintains correctness, and requires no iterations. It just needed the two state reads for these entries. + +$T_f$ is a separate variable in state for the amount of fees this validator has accrued since the last update to its power. +This variable is incremented at every block by however much fees this validator received that block. +On the update to the validators power, this variable is used to create the entry in state at $f$, and is then reset to 0. + +This fee distribution proposal is agnostic to how all of the blocks fees are divied up between validators. +This creates many nice properties, for example it is possible to only rewarding validators who signed that block. + +\section{Additional add-ons} +\subsection{Commission Rates} +Commission rates are the idea that a validator can take a fixed $x\%$ cut of all of their received fees, before redistributing evenly to the constituent delegators. +This can easily be done as follows: + +In block $h$ a validator receives $f_h$ fees. +Instead of incrementing that validators ``total accrued fees this period variable" by $f_h$, it is instead incremented by $(1 - commission\_rate) * f_p$. +Then $commission\_rate * f_p$ is deposited directly to the validator's account. +This allows for efficient updates to a validator's commission rate every block if desired. +More generally, each validator could have a function which takes their fees as input, and outputs a set of outputs to pay these fees too. (i.e. x\% going to themselves, y\% to delegators, z\% burnt) + +\subsection{Slashing} +\label{ssec:slashing} +Slashing is distinct from withdrawals, since it lowers the stake of all of the delegator's by a fixed percentage. +Since no one is charged gas for slashes, a slash cannot iterate over all delegators. +Thus we can no longer just multiply by $x$ over the difference in stake. +This section describes a simple solution that should suffice for most chains needs. An asymptotically optimal solution is provided in section 2.4. +TODO: Consider removing this section in favor of just using the current section 2.4? + +The solution here is to instead store each period created by a slash in the validators state. +Then when withdrawing, you must iterate over all slashes between when you started and ended. +Suppose you delegated at period $0$, a y\% slash occurred at period $2$, and your withdrawal creates period $4$. +Then you receive funds from periods $0$ to $2$ as normal. +The equations for funds you receive for periods $2$ to $4$ now uses $(1 - y)x$ for your stake instead of just $x$ stake. +When there are multiple slashes, you just account for the accumulated slash factor. + +In practice this will not really be an efficiency hit, as the number of slashes is expected to be 0 or 1 for most validators. +Validators that get slashed more will naturally lose their delegators. +A malicious validator that gets itself slashed many times would increase the gas to withdraw linearly, but the economic loss of funds due to the slashes is expected to far out-weigh the extra overhead the honest withdrawer must pay for due to the gas. +(TODO: frame that above sentence in terms of griefing factors, as that's more correct) + +\subsection{Inflation} +Inflation is the idea that we want every staked coin to create more staking tokens as time progresses. +The purpose being to drive down the relative worth of unstaked tokens. +Each block, every staked token should produce $x$ staking tokens as inflation, where $x$ is calculated from a function $inflation$ which takes state and the block information as input. +Let $x_i$ represent the evaluation of $inflation$ in the $i$th block. +The goal of this section is to auto-bond inflation in the fee distribution model without iteration. +This is done by preserving the invariant that every state entry contains the rewards one would have if they had bonded one stake at genesis until that corresponding block. + +In state a variable should be kept for the number of tokens one would have now due to inflation, +given that they bonded one token at genesis. +This is $\prod_{0}^{now} (1 + x_i)$. +Each period now stores this total inflation product along with what it already stores per-period. + +Let $R_i$ be the fee rewards in block $i$, and $n_i$ be the total amount bonded to that validator in that block. +The correct amount of rewards which 1 token at genesis should have now is: +$$Reward(now) = \sum_{i = 0}^{now}\left(\prod_{j = 0}^{i} 1 + x_j \right) * \frac{R_i}{n_i}$$ +The term in the sum is the amount of stake one stake becomes due to inflation, multiplied by the amount of fees per stake. + +Now we cast this into the period frame of view. +Recall that we build the rewards by creating a state entry for the rewards of the previous period, and keeping track of the rewards within this period. +Thus we first define the correct amount of rewards for each successive period, proving correctness of this via induction. +We then show that the state entry that gets efficiently built up block by block is equal to this value for the latest period. + +Let $start, end$ denote the start/end of a period. + +Suppose that $\forall f > 0$, $Reward(end(f))$ is correctly constructed as +$$Reward(end(f)) = Reward(end(f-1)) + \sum_{i = start(f)}^{end(f)}\left(\prod_{j = 0}^{i} 1 + x_j \right) \frac{R_i}{n_i}$$ +and that for $f = 0$, $Reward(end(0)) = 0$. +(With period 1 being defined as the period that has the first bond into it) +It must be shown that assuming the supposition $\forall f \leq f_0$, $$Reward(end(f_0 + 1)) = Reward(end(f_0)) + \sum_{i = start(f_0 + 1)}^{end(f_0 + 1)}\left(\prod_{j = 0}^{i} 1 + x_j \right) \frac{R_i}{n_i}$$ +Using the definition of $Reward$, it follows that: +$$\sum_{i = 0}^{end(f_0 + 1)}\left(\prod_{j = 0}^{i} 1 + x_j \right) * \frac{R_i}{n_i} = \sum_{i = 0}^{end(f_0)}\left(\prod_{j = 0}^{i} 1 + x_j \right) * \frac{R_i}{n_i} + \sum_{i = start(f_0 + 1)}^{end(f_0 + 1)}\left(\prod_{j = 0}^{i} 1 + x_j \right) \frac{R_i}{n_i}$$ + +Since the first summation on the right hand side is $Reward(end(f_0))$, the supposition is proven true. +Consequently, the reward for just period $f$ adjusted for the amount of inflation 1 token at genesis would produce, is: +$$\sum_{i = start(f)}^{end(f)}\left(\prod_{j = 0}^{i} 1 + x_j \right) \frac{R_i}{n_i}$$ + +TODO: make this proof + pre-amble less verbose, and just wrap up into a lemma. +Maybe just leave this proof or the last part to the reader, since it easily follows from summation bounds. + +Now note that +$$\sum_{i = start(f)}^{end(f)}\left(\prod_{j = 0}^{i} 1 + x_j \right) \frac{R_i}{n_i} = \left(\prod_{j = 0}^{end(f - 1)} 1 + x_j \right)\sum_{i = start(f)}^{end(f)}\left(\prod_{j = start(f)}^{i} 1 + x_j \right) \frac{R_i}{n_i}$$ +By definition of period, and inflation being applied every block, \\ +$n_i = n_{start(f)}\left(\prod_{j = start(f)}^{i} 1 + x_j \right)$. This cancels out the product in the summation, therefore +$$\sum_{i = start(f)}^{end(f)}\left(\prod_{j = 0}^{i} 1 + x_j \right) \frac{R_i}{n_i} = \left(\prod_{j = 0}^{end(f - 1)} 1 + x_j \right)\frac{\sum_{i = start(f)}^{end(f)}R_i}{n_{start(f)}}$$ + +Thus every block, each validator just has to add the total amount of fees (The $R_i$ term) that goes to delegates to some per-period term. +When creating a new period, $n_{start(f)}$ can be cached in state, and the product is already stored in the previous periods state entry. +You then get the next period's $n_{start(f)}$ from the consensus' power entry for this validator. +This is thus extremely efficient per block. + +When withdrawing, you take the difference as before, +which yields the amount of rewards you would have obtained with $(\prod_0^{begin\ bonding\ period}1 + x)$ stake from the block you began bonding at until now. +$(\prod_0^{begin\ bonding\ period}1 + x)$ is known, since its included in the state entry for when you bonded. +You then divide the entitled fees by $(\prod_0^{begin\ bonding\ period}1 + x)$ to normalize it to being the amount of rewards you're entitled to from 1 stake at that block to now. +Then as before, you multiply by the amount of stake you had initially bonded. +\\TODO: (Does the difference equating to that make sense, or should it be shown explicitly) +\\TODO: Does this need to explain how the originally bonded tokens are refunded, or is that clear? + +The inflation function could vary per block, +and per validator if ever a need rose. +If the inflation rate is the same for everyone then there can be a single global store for the entries corresponding to the product of inflations. +Inflation creation can trivially be epoched as long as inflation isn't required within the epoch, through changes to the $inflation$ function. + +\subsection{Withdrawing with no iteration over slashes} +Notice that a slash is the same as a negative inflation rate for a validator in one block. +For example a $20\%$ slash is equivalent to a $-20\%$ inflation for a validator in a block. +Given correctness of auto-bonding inflation with different inflation rates per-validator, +it follows that handling slashes can be correctly done by simply subtracting the validators inflation factor in that block to be the negative of the slash factor. +This significantly simplifies the withdrawal procedure. + +\subsection{Auto bonding fees} +TODO: Fill this out. +Core idea: you use the same mechanism as previously, but you just don't take that optimization with $n_{i}$ and the $n_{start}$ relation. +Fairly simple to do. + +\subsection{Delegation updates} +Updating your delegation amount is equivalent to withdrawing earned rewards and a fully independent new delegation occurring in the same block. +The same applies for redelegation. +From the view of fee distribution, partial redelegation is the same as a delegation update + a new delegation. + +\subsection{Jailing / being kicked out of the validator set} +This basically requires no change. +In each block you only iterate over the currently bonded validators. +So you simply don't update the "total accrued fees this period" variable for jailed / non-bonded validators. +Withdrawing requires \textit{no} special casing here! + +\section{State Requirements} +State entries can be pruned quite effectively. +Suppose for the sake of exposition that there is at most one delegation / withdrawal to a particular validator in any given block. +Then each delegation is responsible for one addition to state. +Only the next period, and this delegator's withdrawal could depend on this entry. Thus once this delegator withdraws, this state entry can be pruned. +For the entry created by the delegator's withdrawal, that is only required by the creation of the next period. +Thus once the next period is created, that withdrawal's period can be deleted. + +This can be easily adapted to the case where there are multiple delegations / withdrawals per block, by maintaining a reference count in each period starting state entry. + +The slash entries for a validator can only be pruned when all of that validator's delegators have their bonding period starting after the slash. +This seems ineffective to keep track of, thus it is not worth it. +Each slash should instead remain in state until the validator unbonds and all delegators have their fees withdrawn. + +\section{Implementers Considerations} +TODO: Convert this section into a proper conclusion + +This is an extremely simple scheme with many nice benefits. +\begin{itemize} + \item The overhead per block is a simple iteration over the bonded validator set, which occurs anyway. (Thus it can be implemented ``for-free" with an optimized code-base) + \item Withdrawing earned fees only requires iterating over slashes since when you bonded. (Which is a negligible iteration) + \item There are no approximations in any of the calculations. (modulo minor errata resulting from fixed precision decimals used in divisions) + \item Supports arbitrary inflation models. (Thus could even vary upon block signers) + \item Supports arbitrary fee distribution amongst the validator set. (Thus can account for things like only online validators get fees, which has important incentivization impacts) + \item The above two can change on a live chain with no issues. + \item Validator commission rates can be changed every block + \item The simplicity of this scheme lends itself well to implementation +\end{itemize} + +Thus this scheme has efficiency improvements, simplicity improvements, and expressiveness improvements over the currently proposed schemes. With a correct fee distribution amongst the validator set, this solves the existing problem where one could withhold their signature for risk-free gain. + +\section{TO DOs} + +\begin{itemize} + \item A global fee pool can be described. + \item Mention storage optimization for how to prune slashing entries in the uniform inflation and iteration over slashing case + \item Add equation numbers + \item perhaps re-organize so that the no iteration + \item Section on decimal precision considerations (would unums help?), and mitigating errors in calculation with floats and decimals. -- This probably belongs in a corollary markdown file in the implementation + \item Consider indicating that the withdraw action need not be a tx type and could instead happen 'transparently' when more coins are needed, if a chain desired this for UX / p2p efficiency. +\end{itemize} + + +\end{document} diff --git a/versioned_docs/version-0.52/build/spec/store/README.md b/versioned_docs/version-0.52/build/spec/store/README.md new file mode 100644 index 000000000..706ee4d6c --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/store/README.md @@ -0,0 +1,235 @@ +# Store + +The store package defines the interfaces, types and abstractions for Cosmos SDK +modules to read and write to Merkleized state within a Cosmos SDK application. +The store package provides many primitives for developers to use in order to +work with both state storage and state commitment. Below we describe the various +abstractions. + +## Types + +### `Store` + +The bulk of the store interfaces are defined [here](https://github.com/cosmos/cosmos-sdk/blob/main/store/store.go), +where the base primitive interface, for which other interfaces build off of, is +the `Store` type. The `Store` interface defines the ability to tell the type of +the implementing store and the ability to cache wrap via the `CacheWrapper` interface. + +### `CacheWrapper` & `CacheWrap` + +One of the most important features a store has the ability to perform is the +ability to cache wrap. Cache wrapping is essentially the underlying store wrapping +itself within another store type that performs caching for both reads and writes +with the ability to flush writes via `Write()`. + +### `KVStore` & `CacheKVStore` + +One of the most important interfaces that both developers and modules interface +with, which also provides the basis of most state storage and commitment operations, +is the `KVStore`. The `KVStore` interface provides basic CRUD abilities and +prefix-based iteration, including reverse iteration. + +Typically, each module has its own dedicated `KVStore` instance, which it can +get access to via the `sdk.Context` and the use of a pointer-based named key -- +`KVStoreKey`. The `KVStoreKey` provides pseudo-OCAP. How a exactly a `KVStoreKey` +maps to a `KVStore` will be illustrated below through the `CommitMultiStore`. + +Note, a `KVStore` cannot directly commit state. Instead, a `KVStore` can be wrapped +by a `CacheKVStore` which extends a `KVStore` and provides the ability for the +caller to execute `Write()` which commits state to the underlying state storage. +Note, this doesn't actually flush writes to disk as writes are held in memory +until `Commit()` is called on the `CommitMultiStore`. + +### `CommitMultiStore` + +The `CommitMultiStore` interface exposes the top-level interface that is used +to manage state commitment and storage by an SDK application and abstracts the +concept of multiple `KVStore`s which are used by multiple modules. Specifically, +it supports the following high-level primitives: + +* Allows for a caller to retrieve a `KVStore` by providing a `KVStoreKey`. +* Exposes pruning mechanisms to remove state pinned against a specific height/version + in the past. +* Allows for loading state storage at a particular height/version in the past to + provide current head and historical queries. +* Provides the ability to rollback state to a previous height/version. +* Provides the ability to load state storage at a particular height/version + while also performing store upgrades, which are used during live hard-fork + application state migrations. +* Provides the ability to commit all current accumulated state to disk and performs + Merkle commitment. + +## Implementation Details + +While there are many interfaces that the `store` package provides, there is +typically a core implementation for each main interface that modules and +developers interact with that are defined in the Cosmos SDK. + +### `iavl.Store` + +The `iavl.Store` provides the core implementation for state storage and commitment +by implementing the following interfaces: + +* `KVStore` +* `CommitStore` +* `CommitKVStore` +* `Queryable` +* `StoreWithInitialVersion` + +It allows for all CRUD operations to be performed along with allowing current +and historical state queries, prefix iteration, and state commitment along with +Merkle proof operations. The `iavl.Store` also provides the ability to remove +historical state from the state commitment layer. + +An overview of the IAVL implementation can be found [here](https://github.com/cosmos/iavl/blob/master/docs/overview.md). +It is important to note that the IAVL store provides both state commitment and +logical storage operations, which comes with drawbacks as there are various +performance impacts, some of which are very drastic, when it comes to the +operations mentioned above. + +When dealing with state management in modules and clients, the Cosmos SDK provides +various layers of abstractions or "store wrapping", where the `iavl.Store` is the +bottom most layer. When requesting a store to perform reads or writes in a module, +the typical abstraction layer in order is defined as follows: + +```text +iavl.Store <- cachekv.Store <- gaskv.Store <- cachemulti.Store <- rootmulti.Store +``` + +### Concurrent use of IAVL store + +The tree under `iavl.Store` is not safe for concurrent use. It is the +responsibility of the caller to ensure that concurrent access to the store is +not performed. + +The main issue with concurrent use is when data is written at the same time as +it's being iterated over. Doing so will cause a irrecoverable fatal error because +of concurrent reads and writes to an internal map. + +Although it's not recommended, you can iterate through values while writing to +it by disabling "FastNode" **without guarantees that the values being written will +be returned during the iteration** (if you need this, you might want to reconsider +the design of your application). This is done by setting `iavl-disable-fastnode` +to `true` in the config TOML file. + +### `cachekv.Store` + +The `cachekv.Store` store wraps an underlying `KVStore`, typically a `iavl.Store` +and contains an in-memory cache for storing pending writes to underlying `KVStore`. +`Set` and `Delete` calls are executed on the in-memory cache, whereas `Has` calls +are proxied to the underlying `KVStore`. + +One of the most important calls to a `cachekv.Store` is `Write()`, which ensures +that key-value pairs are written to the underlying `KVStore` in a deterministic +and ordered manner by sorting the keys first. The store keeps track of "dirty" +keys and uses these to determine what keys to sort. In addition, it also keeps +track of deleted keys and ensures these are also removed from the underlying +`KVStore`. + +The `cachekv.Store` also provides the ability to perform iteration and reverse +iteration. Iteration is performed through the `cacheMergeIterator` type and uses +both the dirty cache and underlying `KVStore` to iterate over key-value pairs. + +Note, all calls to CRUD and iteration operations on a `cachekv.Store` are thread-safe. + +### `gaskv.Store` + +The `gaskv.Store` store provides a simple implementation of a `KVStore`. +Specifically, it just wraps an existing `KVStore`, such as a cache-wrapped +`iavl.Store`, and incurs configurable gas costs for CRUD operations via +`ConsumeGas()` calls defined on the `GasMeter` which exists in a `sdk.Context` +and then proxies the underlying CRUD call to the underlying store. Note, the +`GasMeter` is reset on each block. + +### `cachemulti.Store` & `rootmulti.Store` + +The `rootmulti.Store` acts as an abstraction around a series of stores. Namely, +it implements the `CommitMultiStore` an `Queryable` interfaces. Through the +`rootmulti.Store`, an SDK module can request access to a `KVStore` to perform +state CRUD operations and queries by holding access to a unique `KVStoreKey`. + +The `rootmulti.Store` ensures these queries and state operations are performed +through cached-wrapped instances of `cachekv.Store` which is described above. The +`rootmulti.Store` implementation is also responsible for committing all accumulated +state from each `KVStore` to disk and returning an application state Merkle root. + +Queries can be performed to return state data along with associated state +commitment proofs for both previous heights/versions and the current state root. +Queries are routed based on store name, i.e. a module, along with other parameters +which are defined in `abci.QueryRequest`. + +The `rootmulti.Store` also provides primitives for pruning data at a given +height/version from state storage. When a height is committed, the `rootmulti.Store` +will determine if other previous heights should be considered for removal based +on the operator's pruning settings defined by `PruningOptions`, which defines +how many recent versions to keep on disk and the interval at which to remove +"staged" pruned heights from disk. During each interval, the staged heights are +removed from each `KVStore`. Note, it is up to the underlying `KVStore` +implementation to determine how pruning is actually performed. The `PruningOptions` +are defined as follows: + +```go +type PruningOptions struct { + // KeepRecent defines how many recent heights to keep on disk. + KeepRecent uint64 + + // Interval defines when the pruned heights are removed from disk. + Interval uint64 + + // Strategy defines the kind of pruning strategy. See below for more information on each. + Strategy PruningStrategy +} +``` + +The Cosmos SDK defines a preset number of pruning "strategies": `default`, `everything` +`nothing`, and `custom`. + +It is important to note that the `rootmulti.Store` considers each `KVStore` as a +separate logical store. In other words, they do not share a Merkle tree or +comparable data structure. This means that when state is committed via +`rootmulti.Store`, each store is committed in sequence and thus is not atomic. + +In terms of store construction and wiring, each Cosmos SDK application contains +a `BaseApp` instance which internally has a reference to a `CommitMultiStore` +that is implemented by a `rootmulti.Store`. The application then registers one or +more `KVStoreKey` that pertain to a unique module and thus a `KVStore`. Through +the use of an `sdk.Context` and a `KVStoreKey`, each module can get direct access +to it's respective `KVStore` instance. + +Example: + +```go +func NewApp(...) Application { + // ... + + bApp := baseapp.NewBaseApp(appName, logger, db, txConfig.TxDecoder(), baseAppOptions...) + bApp.SetCommitMultiStoreTracer(traceStore) + bApp.SetVersion(version.Version) + bApp.SetInterfaceRegistry(interfaceRegistry) + + // ... + + keys := sdk.NewKVStoreKeys(...) + transientKeys := sdk.NewTransientStoreKeys(...) + memKeys := sdk.NewMemoryStoreKeys(...) + + // ... + + // initialize stores + app.MountKVStores(keys) + app.MountTransientStores(transientKeys) + app.MountMemoryStores(memKeys) + + // ... +} +``` + +The `rootmulti.Store` itself can be cache-wrapped which returns an instance of a +`cachemulti.Store`. For each block, `BaseApp` ensures that the proper abstractions +are created on the `CommitMultiStore`, i.e. ensuring that the `rootmulti.Store` +is cached-wrapped and uses the resulting `cachemulti.Store` to be set on the +`sdk.Context` which is then used for block and transaction execution. As a result, +all state mutations due to block and transaction execution are actually held +ephemerally until `Commit()` is called by the ABCI client. This concept is further +expanded upon when the AnteHandler is executed per transaction to ensure state +is not committed for transactions that failed CheckTx. diff --git a/versioned_docs/version-0.52/build/spec/store/interblock-cache.md b/versioned_docs/version-0.52/build/spec/store/interblock-cache.md new file mode 100644 index 000000000..afe070761 --- /dev/null +++ b/versioned_docs/version-0.52/build/spec/store/interblock-cache.md @@ -0,0 +1,289 @@ +# Inter-block Cache + +* [Inter-block Cache](#inter-block-cache) + * [Synopsis](#synopsis) + * [Overview and basic concepts](#overview-and-basic-concepts) + * [Motivation](#motivation) + * [Definitions](#definitions) + * [System model and properties](#system-model-and-properties) + * [Assumptions](#assumptions) + * [Properties](#properties) + * [Thread safety](#thread-safety) + * [Crash recovery](#crash-recovery) + * [Iteration](#iteration) + * [Technical specification](#technical-specification) + * [General design](#general-design) + * [API](#api) + * [CommitKVCacheManager](#commitkvcachemanager) + * [CommitKVStoreCache](#commitkvstorecache) + * [Implementation details](#implementation-details) + * [History](#history) + * [Copyright](#copyright) + +## Synopsis + +The inter-block cache is an in-memory cache storing (in-most-cases) immutable state that modules need to read in between blocks. When enabled, all sub-stores of a multi store, e.g., `rootmulti`, are wrapped. + +## Overview and basic concepts + +### Motivation + +The goal of the inter-block cache is to allow SDK modules to have fast access to data that it is typically queried during the execution of every block. This is data that do not change often, e.g. module parameters. The inter-block cache wraps each `CommitKVStore` of a multi store such as `rootmulti` with a fixed size, write-through cache. Caches are not cleared after a block is committed, as opposed to other caching layers such as `cachekv`. + +### Definitions + +* `Store key` uniquely identifies a store. +* `KVCache` is a `CommitKVStore` wrapped with a cache. +* `Cache manager` is a key component of the inter-block cache responsible for maintaining a map from `store keys` to `KVCaches`. + +## System model and properties + +### Assumptions + +This specification assumes that there exists a cache implementation accessible to the inter-block cache feature. + +> The implementation uses adaptive replacement cache (ARC), an enhancement over the standard last-recently-used (LRU) cache in that tracks both frequency and recency of use. + +The inter-block cache requires that the cache implementation to provide methods to create a cache, add a key/value pair, remove a key/value pair and retrieve the value associated to a key. In this specification, we assume that a `Cache` feature offers this functionality through the following methods: + +* `NewCache(size int)` creates a new cache with `size` capacity and returns it. +* `Get(key string)` attempts to retrieve a key/value pair from `Cache.` It returns `(value []byte, success bool)`. If `Cache` contains the key, it `value` contains the associated value and `success=true`. Otherwise, `success=false` and `value` should be ignored. +* `Add(key string, value []byte)` inserts a key/value pair into the `Cache`. +* `Remove(key string)` removes the key/value pair identified by `key` from `Cache`. + +The specification also assumes that `CommitKVStore` offers the following API: + +* `Get(key string)` attempts to retrieve a key/value pair from `CommitKVStore`. +* `Set(key, string, value []byte)` inserts a key/value pair into the `CommitKVStore`. +* `Delete(key string)` removes the key/value pair identified by `key` from `CommitKVStore`. + +> Ideally, both `Cache` and `CommitKVStore` should be specified in a different document and referenced here. + +### Properties + +#### Thread safety + +Accessing the `cache manager` or a `KVCache` is not thread-safe: no method is guarded with a lock. +Note that this is true even if the cache implementation is thread-safe. + +> For instance, assume that two `Set` operations are executed concurrently on the same key, each writing a different value. After both are executed, the cache and the underlying store may be inconsistent, each storing a different value under the same key. + +#### Crash recovery + +The inter-block cache transparently delegates `Commit()` to its aggregate `CommitKVStore`. If the +aggregate `CommitKVStore` supports atomic writes and use them to guarantee that the store is always in a consistent state in disk, the inter-block cache can be transparently moved to a consistent state when a failure occurs. + +> Note that this is the case for `IAVLStore`, the preferred `CommitKVStore`. On commit, it calls `SaveVersion()` on the underlying `MutableTree`. `SaveVersion` writes to disk are atomic via batching. This means that only consistent versions of the store (the tree) are written to the disk. Thus, in case of a failure during a `SaveVersion` call, on recovery from disk, the version of the store will be consistent. + +#### Iteration + +Iteration over each wrapped store is supported via the embedded `CommitKVStore` interface. + +## Technical specification + +### General design + +The inter-block cache feature is composed by two components: `CommitKVCacheManager` and `CommitKVCache`. + +`CommitKVCacheManager` implements the cache manager. It maintains a mapping from a store key to a `KVStore`. + +```go +type CommitKVStoreCacheManager interface{ + cacheSize uint + caches map[string]CommitKVStore +} +``` + +`CommitKVStoreCache` implements a `KVStore`: a write-through cache that wraps a `CommitKVStore`. This means that deletes and writes always happen to both the cache and the underlying `CommitKVStore`. Reads on the other hand first hit the internal cache. During a cache miss, the read is delegated to the underlying `CommitKVStore` and cached. + +```go +type CommitKVStoreCache interface{ + store CommitKVStore + cache Cache +} +``` + +To enable inter-block cache on `rootmulti`, one needs to instantiate a `CommitKVCacheManager` and set it by calling `SetInterBlockCache()` before calling one of `LoadLatestVersion()`, `LoadLatestVersionAndUpgrade(...)`, `LoadVersionAndUpgrade(...)` and `LoadVersion(version)`. + +### API + +#### CommitKVCacheManager + +The method `NewCommitKVStoreCacheManager` creates a new cache manager and returns it. + +| Name | Type | Description | +| ------------- | ---------|------- | +| size | integer | Determines the capacity of each of the KVCache maintained by the manager | + +```go +func NewCommitKVStoreCacheManager(size uint) CommitKVStoreCacheManager { + manager = CommitKVStoreCacheManager{size, make(map[string]CommitKVStore)} + return manager +} +``` + +`GetStoreCache` returns a cache from the CommitStoreCacheManager for a given store key. If no cache exists for the store key, then one is created and set. + +| Name | Type | Description | +| ------------- | ---------|------- | +| manager | `CommitKVStoreCacheManager` | The cache manager | +| storeKey | string | The store key of the store being retrieved | +| store | `CommitKVStore` | The store that it is cached in case the manager does not have any in its map of caches | + +```go +func GetStoreCache( + manager CommitKVStoreCacheManager, + storeKey string, + store CommitKVStore) CommitKVStore { + + if manager.caches.has(storeKey) { + return manager.caches.get(storeKey) + } else { + cache = CommitKVStoreCacheManager{store, manager.cacheSize} + manager.set(storeKey, cache) + return cache + } +} +``` + +`Unwrap` returns the underlying CommitKVStore for a given store key. + +| Name | Type | Description | +| ------------- | ---------|------- | +| manager | `CommitKVStoreCacheManager` | The cache manager | +| storeKey | string | The store key of the store being unwrapped | + +```go +func Unwrap( + manager CommitKVStoreCacheManager, + storeKey string) CommitKVStore { + + if manager.caches.has(storeKey) { + cache = manager.caches.get(storeKey) + return cache.store + } else { + return nil + } +} +``` + +`Reset` resets the manager's map of caches. + +| Name | Type | Description | +| ------------- | ---------|------- | +| manager | `CommitKVStoreCacheManager` | The cache manager | + +```go +function Reset(manager CommitKVStoreCacheManager) { + + for (let storeKey of manager.caches.keys()) { + manager.caches.delete(storeKey) + } +} +``` + +#### CommitKVStoreCache + +`NewCommitKVStoreCache` creates a new `CommitKVStoreCache` and returns it. + +| Name | Type | Description | +| ------------- | ---------|------- | +| store | CommitKVStore | The store to be cached | +| size | string | Determines the capacity of the cache being created | + +```go +func NewCommitKVStoreCache( + store CommitKVStore, + size uint) CommitKVStoreCache { + KVCache = CommitKVStoreCache{store, NewCache(size)} + return KVCache +} +``` + +`Get` retrieves a value by key. It first looks in the cache. If the key is not in the cache, the query is delegated to the underlying `CommitKVStore`. In the latter case, the key/value pair is cached. The method returns the value. + +| Name | Type | Description | +| ------------- | ---------|------- | +| KVCache | `CommitKVStoreCache` | The `CommitKVStoreCache` from which the key/value pair is retrieved | +| key | string | Key of the key/value pair being retrieved | + +```go +func Get( + KVCache CommitKVStoreCache, + key string) []byte { + valueCache, success := KVCache.cache.Get(key) + if success { + // cache hit + return valueCache + } else { + // cache miss + valueStore = KVCache.store.Get(key) + KVCache.cache.Add(key, valueStore) + return valueStore + } +} +``` + +`Set` inserts a key/value pair into both the write-through cache and the underlying `CommitKVStore`. + +| Name | Type | Description | +| ------------- | ---------|------- | +| KVCache | `CommitKVStoreCache` | The `CommitKVStoreCache` to which the key/value pair is inserted | +| key | string | Key of the key/value pair being inserted | +| value | []byte | Value of the key/value pair being inserted | + +```go +func Set( + KVCache CommitKVStoreCache, + key string, + value []byte) { + + KVCache.cache.Add(key, value) + KVCache.store.Set(key, value) +} +``` + +`Delete` removes a key/value pair from both the write-through cache and the underlying `CommitKVStore`. + +| Name | Type | Description | +| ------------- | ---------|------- | +| KVCache | `CommitKVStoreCache` | The `CommitKVStoreCache` from which the key/value pair is deleted | +| key | string | Key of the key/value pair being deleted | + +```go +func Delete( + KVCache CommitKVStoreCache, + key string) { + + KVCache.cache.Remove(key) + KVCache.store.Delete(key) +} +``` + +`CacheWrap` wraps a `CommitKVStoreCache` with another caching layer (`CacheKV`). + +> It is unclear whether there is a use case for `CacheWrap`. + +| Name | Type | Description | +| ------------- | ---------|------- | +| KVCache | `CommitKVStoreCache` | The `CommitKVStoreCache` being wrapped | + +```go +func CacheWrap( + KVCache CommitKVStoreCache) { + + return CacheKV.NewStore(KVCache) +} +``` + +### Implementation details + +The inter-block cache implementation uses a fixed-sized adaptive replacement cache (ARC) as cache. [The ARC implementation](https://github.com/hashicorp/golang-lru/blob/main/arc/arc.go) is thread-safe. ARC is an enhancement over the standard LRU cache in that tracks both frequency and recency of use. This avoids a burst in access to new entries from evicting the frequently used older entries. It adds some additional tracking overhead to a standard LRU cache, computationally it is roughly `2x` the cost, and the extra memory overhead is linear with the size of the cache. The default cache size is `1000`. + +## History + +Dec 20, 2022 - Initial draft finished and submitted as a PR + +## Copyright + +All content herein is licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). diff --git a/versioned_docs/version-0.52/build/tooling/00-protobuf.md b/versioned_docs/version-0.52/build/tooling/00-protobuf.md new file mode 100644 index 000000000..849a7974e --- /dev/null +++ b/versioned_docs/version-0.52/build/tooling/00-protobuf.md @@ -0,0 +1,94 @@ +--- +sidebar_position: 1 +--- + +# Protocol Buffers + +Cosmos SDK uses protocol buffers extensively, this document is meant to provide a guide on how it is used in the cosmos-sdk. + +To generate the proto file, the Cosmos SDK uses a docker image, this image is provided to all to use as well. The latest version is `ghcr.io/cosmos/proto-builder:0.15.x` + +Below is the example of the Cosmos SDK's commands for generating, linting, and formatting protobuf files that can be reused in any applications makefile. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/scripts/build/protobuf.mk#L1-L10 +``` + +The [`protocgen.sh`](https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/scripts/protocgen.sh) script used to generate the protobuf files via buf can be found in the `scripts/` directory. + +## Buf + +[Buf](https://buf.build) is a protobuf tool that abstracts the needs to use the complicated `protoc` toolchain on top of various other things that ensure you are using protobuf in accordance with the majority of the ecosystem. Within the cosmos-sdk repository there are a few files that have a buf prefix. Lets start with the top level and then dive into the various directories. + +### Workspace + +At the root level directory a workspace is defined using [buf workspaces](https://docs.buf.build/configuration/v1/buf-work-yaml). This helps if there are one or more protobuf containing directories in your project. + +Cosmos SDK example: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/buf.work.yaml#L6-L9 +``` + +### Proto Directory + +The `proto/` directory where all of global protobuf files live. +In here there are many different buf files defined each serving a different purpose. +Modules proto files are defined in their respective module directories (in the SDK `x/{moduleName}/proto`). + +```bash +├── README.md +├── buf.gen.gogo.yaml +├── buf.gen.pulsar.yaml +├── buf.gen.swagger.yaml +├── buf.lock +├── buf.md +├── buf.yaml +├── cosmos +└── tendermint +``` + +#### `buf.gen.gogo.yaml` + +`buf.gen.gogo.yaml` defines how the protobuf files should be generated for use with in the module. This file uses [gogoproto](https://github.com/gogo/protobuf), a separate generator from the google go-proto generator that makes working with various objects more ergonomic, and it has more performant encode and decode steps + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.gogo.yaml#L1-L9 +``` + +#### `buf.gen.pulsar.yaml` + +`buf.gen.pulsar.yaml` defines how protobuf files should be generated using the [new golang apiv2 of protobuf](https://go.dev/blog/protobuf-apiv2). This generator is used instead of the google go-proto generator because it has some extra helpers for Cosmos SDK applications and will have more performant encode and decode than the google go-proto generator. You can follow the development of this generator [here](https://github.com/cosmos/cosmos-proto). + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.pulsar.yaml#L1-L18 +``` + +#### `buf.gen.swagger.yaml` + +`buf.gen.swagger.yaml` generates the swagger documentation for the query and messages of the chain. This will only define the REST API end points that were defined in the query and msg servers. You can find examples of this [here](https://github.com/cosmos/cosmos-sdk/blob/main/x/bank/proto/cosmos/bank/v1beta1/query.proto) + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.swagger.yaml#L1-L6 +``` + +#### `buf.lock` + +This is an autogenerated file based off the dependencies required by the `.gen` files. There is no need to copy the current one. If you depend on cosmos-sdk proto definitions a new entry for the Cosmos SDK will need to be provided. The dependency you will need to use is `buf.build/cosmos/cosmos-sdk`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.lock#L1-L16 +``` + +#### `buf.yaml` + +`buf.yaml` defines the [name of your package](https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.yaml#L3), which [breakage checker](https://buf.build/docs/tutorials/getting-started-with-buf-cli#detect-breaking-changes) to use and how to [lint your protobuf files](https://buf.build/docs/tutorials/getting-started-with-buf-cli#lint-your-api). + +It is advised to use a tagged version of the buf modules corresponding to the version of the Cosmos SDK being are used. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.yaml#L1-L24 +``` + +We use a variety of linters for the Cosmos SDK protobuf files. The repo also checks this in ci. +A reference to the github actions can be found [here](https://github.com/cosmos/cosmos-sdk/blob/main/.github/workflows/proto.yml#L1-L32) diff --git a/versioned_docs/version-0.52/build/tooling/01-cosmovisor.md b/versioned_docs/version-0.52/build/tooling/01-cosmovisor.md new file mode 100644 index 000000000..3b5f722c5 --- /dev/null +++ b/versioned_docs/version-0.52/build/tooling/01-cosmovisor.md @@ -0,0 +1,411 @@ +--- +sidebar_position: 1 +--- + +# Cosmovisor + +`cosmovisor` is a process manager for Cosmos SDK application binaries that automates application binary switch at chain upgrades. +It polls the `upgrade-info.json` file that is created by the x/upgrade module at upgrade height, and then can automatically download the new binary, stop the current binary, switch from the old binary to the new one, and finally restart the node with the new binary. + +* [Design](#design) +* [Contributing](#contributing) +* [Setup](#setup) + * [Installation](#installation) + * [Command Line Arguments And Environment Variables](#command-line-arguments-and-environment-variables) + * [Folder Layout](#folder-layout) +* [Usage](#usage) + * [Initialization](#initialization) + * [Detecting Upgrades](#detecting-upgrades) + * [Adding Upgrade Binary](#adding-upgrade-binary) + * [Auto-Download](#auto-download) + * [Preparing for an Upgrade](#preparing-for-an-upgrade) +* [Example: SimApp Upgrade](#example-simapp-upgrade) + * [Chain Setup](#chain-setup) + * [Prepare Cosmovisor and Start the Chain](#prepare-cosmovisor-and-start-the-chain) + * [Update App](#update-app) + +## Design + +Cosmovisor is designed to be used as a wrapper for a `Cosmos SDK` app: + +* it will pass arguments to the associated app (configured by `DAEMON_NAME` env variable). + Running `cosmovisor run arg1 arg2 ....` will run `app arg1 arg2 ...`; +* it will manage an app by restarting and upgrading if needed; +* it is configured using environment variables, not positional arguments. + +*Note: If new versions of the application are not set up to run in-place store migrations, migrations will need to be run manually before restarting `cosmovisor` with the new binary. For this reason, we recommend applications adopt in-place store migrations.* + +:::tip +Only the latest version of cosmovisor is actively developed/maintained. +::: + +:::warning +Versions prior to v1.0.0 have a vulnerability that could lead to a DOS. Please upgrade to the latest version. +::: + +## Contributing + +Cosmovisor is part of the Cosmos SDK monorepo, but it's a separate module with it's own release schedule. + +Release branches have the following format `release/cosmovisor/vA.B.x`, where A and B are a number (e.g. `release/cosmovisor/v1.3.x`). Releases are tagged using the following format: `cosmovisor/vA.B.C`. + +## Setup + +### Installation + +You can download Cosmovisor from the [GitHub releases](https://github.com/cosmos/cosmos-sdk/releases/tag/cosmovisor%2Fv1.5.0). + +To install the latest version of `cosmovisor`, run the following command: + +```shell +go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@latest +``` + +To install a specific version, you can specify the version: + +```shell +go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@v1.5.0 +``` + +Run `cosmovisor version` to check the cosmovisor version. + +Alternatively, for building from source, simply run `make cosmovisor`. The binary will be located in `tools/cosmovisor`. + +:::warning +Installing cosmovisor using `go install` will display the correct `cosmovisor` version. +Building from source (`make cosmovisor`) or installing `cosmovisor` by other means won't display the correct version. +::: + +### Command Line Arguments And Environment Variables + +The first argument passed to `cosmovisor` is the action for `cosmovisor` to take. Options are: + +* `help`, `--help`, or `-h` - Output `cosmovisor` help information and check your `cosmovisor` configuration. +* `run` - Run the configured binary using the rest of the provided arguments. +* `version` - Output the `cosmovisor` version and also run the binary with the `version` argument. +* `config` - Display the current `cosmovisor` configuration, that means displaying the environment variables value that `cosmovisor` is using. +* `add-upgrade` - Add an upgrade manually to `cosmovisor`. This command allow you to easily add the binary corresponding to an upgrade in cosmovisor. + +All arguments passed to `cosmovisor run` will be passed to the application binary (as a subprocess). `cosmovisor` will return `/dev/stdout` and `/dev/stderr` of the subprocess as its own. For this reason, `cosmovisor run` cannot accept any command-line arguments other than those available to the application binary. + +`cosmovisor` reads its configuration from environment variables, or its configuration file (use `--cosmovisor-config `): + +* `DAEMON_HOME` is the location where the `cosmovisor/` directory is kept that contains the genesis binary, the upgrade binaries, and any additional auxiliary files associated with each binary (e.g. `$HOME/.gaiad`, `$HOME/.regend`, `$HOME/.simd`, etc.). +* `DAEMON_NAME` is the name of the binary itself (e.g. `gaiad`, `regend`, `simd`, etc.). +* `DAEMON_ALLOW_DOWNLOAD_BINARIES` (*optional*), if set to `true`, will enable auto-downloading of new binaries (for security reasons, this is intended for full nodes rather than validators). By default, `cosmovisor` will not auto-download new binaries. +* `DAEMON_DOWNLOAD_MUST_HAVE_CHECKSUM` (*optional*, default = `false`), if `true` cosmovisor will require that a checksum is provided in the upgrade plan for the binary to be downloaded. If `false`, cosmovisor will not require a checksum to be provided, but still check the checksum if one is provided. +* `DAEMON_RESTART_AFTER_UPGRADE` (*optional*, default = `true`), if `true`, restarts the subprocess with the same command-line arguments and flags (but with the new binary) after a successful upgrade. Otherwise (`false`), `cosmovisor` stops running after an upgrade and requires the system administrator to manually restart it. Note restart is only after the upgrade and does not auto-restart the subprocess after an error occurs. +* `DAEMON_RESTART_DELAY` (*optional*, default none), allow a node operator to define a delay between the node halt (for upgrade) and backup by the specified time. The value must be a duration (e.g. `1s`). +* `DAEMON_SHUTDOWN_GRACE` (*optional*, default none), if set, send interrupt to binary and wait the specified time to allow for cleanup/cache flush to disk before sending the kill signal. The value must be a duration (e.g. `1s`). +* `DAEMON_POLL_INTERVAL` (*optional*, default 300 milliseconds), is the interval length for polling the upgrade plan file. The value must be a duration (e.g. `1s`). +* `DAEMON_DATA_BACKUP_DIR` option to set a custom backup directory. If not set, `DAEMON_HOME` is used. +* `UNSAFE_SKIP_BACKUP` (defaults to `false`), if set to `true`, upgrades directly without performing a backup. Otherwise (`false`, default) backs up the data before trying the upgrade. The default value of false is useful and recommended in case of failures and when a backup needed to rollback. We recommend using the default backup option `UNSAFE_SKIP_BACKUP=false`. +* `DAEMON_PREUPGRADE_MAX_RETRIES` (defaults to `0`). The maximum number of times to call [`pre-upgrade`](https://docs.cosmos.network/main/build/building-apps/app-upgrade#pre-upgrade-handling) in the application after exit status of `31`. After the maximum number of retries, Cosmovisor fails the upgrade. +* `COSMOVISOR_DISABLE_LOGS` (defaults to `false`). If set to true, this will disable Cosmovisor logs (but not the underlying process) completely. This may be useful, for example, when a Cosmovisor subcommand you are executing returns a valid JSON you are then parsing, as logs added by Cosmovisor make this output not a valid JSON. +* `COSMOVISOR_COLOR_LOGS` (defaults to `true`). If set to true, this will colorise Cosmovisor logs (but not the underlying process). +* `COSMOVISOR_TIMEFORMAT_LOGS` (defaults to `kitchen`). If set to a value (`layout|ansic|unixdate|rubydate|rfc822|rfc822z|rfc850|rfc1123|rfc1123z|rfc3339|rfc3339nano|kitchen`), this will add timestamp prefix to Cosmovisor logs (but not the underlying process). +* `COSMOVISOR_CUSTOM_PREUPGRADE` (defaults to ``). If set, this will run $DAEMON_HOME/cosmovisor/$COSMOVISOR_CUSTOM_PREUPGRADE prior to upgrade with the arguments [ upgrade.Name, upgrade.Height ]. Executes a custom script (separate and prior to the chain daemon pre-upgrade command) +* `COSMOVISOR_DISABLE_RECASE` (defaults to `false`). If set to true, the upgrade directory will expected to match the upgrade plan name without any case changes + +### Folder Layout + +`$DAEMON_HOME/cosmovisor` is expected to belong completely to `cosmovisor` and the subprocesses that are controlled by it. The folder content is organized as follows: + +```text +. +├── current -> genesis or upgrades/ +├── genesis +│   └── bin +│   └── $DAEMON_NAME +└── upgrades +│ └── +│ ├── bin +│ │   └── $DAEMON_NAME +│ └── upgrade-info.json +└── preupgrade.sh (optional) +``` + +The `cosmovisor/` directory includes a subdirectory for each version of the application (i.e. `genesis` or `upgrades/`). Within each subdirectory is the application binary (i.e. `bin/$DAEMON_NAME`) and any additional auxiliary files associated with each binary. `current` is a symbolic link to the currently active directory (i.e. `genesis` or `upgrades/`). The `name` variable in `upgrades/` is the lowercased URI-encoded name of the upgrade as specified in the upgrade module plan. Note that the upgrade name path are normalized to be lowercased: for instance, `MyUpgrade` is normalized to `myupgrade`, and its path is `upgrades/myupgrade`. + +Please note that `$DAEMON_HOME/cosmovisor` only stores the *application binaries*. The `cosmovisor` binary itself can be stored in any typical location (e.g. `/usr/local/bin`). The application will continue to store its data in the default data directory (e.g. `$HOME/.simapp`) or the data directory specified with the `--home` flag. `$DAEMON_HOME` is dependent of the data directory and must be set to the same directory as the data directory, you will end up with a configuration like the following: + +```text +.simapp +├── config +├── data +└── cosmovisor +``` + +## Usage + +The system administrator is responsible for: + +* installing the `cosmovisor` binary +* configuring the host's init system (e.g. `systemd`, `launchd`, etc.) +* appropriately setting the environmental variables +* creating the `/cosmovisor` directory +* creating the `/cosmovisor/genesis/bin` folder +* creating the `/cosmovisor/upgrades//bin` folders +* placing the different versions of the `` executable in the appropriate `bin` folders. + +`cosmovisor` will set the `current` link to point to `genesis` at first start (i.e. when no `current` link exists) and then handle switching binaries at the correct points in time so that the system administrator can prepare days in advance and relax at upgrade time. + +In order to support downloadable binaries, a tarball for each upgrade binary will need to be packaged up and made available through a canonical URL. Additionally, a tarball that includes the genesis binary and all available upgrade binaries can be packaged up and made available so that all the necessary binaries required to sync a fullnode from start can be easily downloaded. + +The `DAEMON` specific code and operations (e.g. cometBFT config, the application db, syncing blocks, etc.) all work as expected. The application binaries' directives such as command-line flags and environment variables also work as expected. + +### Initialization + +The `cosmovisor init ` command creates the folder structure required for using cosmovisor. + +It does the following: + +* creates the `/cosmovisor` folder if it doesn't yet exist +* creates the `/cosmovisor/genesis/bin` folder if it doesn't yet exist +* copies the provided executable file to `/cosmovisor/genesis/bin/` +* creates the `current` link, pointing to the `genesis` folder + +It uses the `DAEMON_HOME` and `DAEMON_NAME` environment variables for folder location and executable name. + +The `cosmovisor init` command is specifically for initializing cosmovisor, and should not be confused with a chain's `init` command (e.g. `cosmovisor run init`). + +### Detecting Upgrades + +`cosmovisor` is polling the `$DAEMON_HOME/data/upgrade-info.json` file for new upgrade instructions. The file is created by the x/upgrade module in `BeginBlocker` when an upgrade is detected and the blockchain reaches the upgrade height. +The following heuristic is applied to detect the upgrade: + +* When starting, `cosmovisor` doesn't know much about currently running upgrade, except the binary which is `current/bin/`. It tries to read the `current/update-info.json` file to get information about the current upgrade name. +* If neither `cosmovisor/current/upgrade-info.json` nor `data/upgrade-info.json` exist, then `cosmovisor` will wait for `data/upgrade-info.json` file to trigger an upgrade. +* If `cosmovisor/current/upgrade-info.json` doesn't exist but `data/upgrade-info.json` exists, then `cosmovisor` assumes that whatever is in `data/upgrade-info.json` is a valid upgrade request. In this case `cosmovisor` tries immediately to make an upgrade according to the `name` attribute in `data/upgrade-info.json`. +* Otherwise, `cosmovisor` waits for changes in `upgrade-info.json`. As soon as a new upgrade name is recorded in the file, `cosmovisor` will trigger an upgrade mechanism. + +When the upgrade mechanism is triggered, `cosmovisor` will: + +1. if `DAEMON_ALLOW_DOWNLOAD_BINARIES` is enabled, start by auto-downloading a new binary into `cosmovisor//bin` (where `` is the `upgrade-info.json:name` attribute); +2. update the `current` symbolic link to point to the new directory and save `data/upgrade-info.json` to `cosmovisor/current/upgrade-info.json`. + +### Adding Upgrade Binary + +`cosmovisor` has an `add-upgrade` command that allows to easily link a binary to an upgrade. It creates a new folder in `cosmovisor/upgrades/` and copies the provided executable file to `cosmovisor/upgrades//bin/`. + +Using the `--upgrade-height` flag allows to specify at which height the binary should be switched, without going via a gorvernance proposal. +This enables support for an emergency coordinated upgrades where the binary must be switched at a specific height, but there is no time to go through a governance proposal. + +:::warning +`--upgrade-height` creates an `upgrade-info.json` file. This means if a chain upgrade via governance proposal is executed before the specified height with `--upgrade-height`, the governance proposal will overwrite the `upgrade-info.json` plan created by `add-upgrade --upgrade-height `. +Take this into consideration when using `--upgrade-height`. +::: + +### Auto-Download + +Generally, `cosmovisor` requires that the system administrator place all relevant binaries on disk before the upgrade happens. However, for people who don't need such control and want an automated setup (maybe they are syncing a non-validating fullnode and want to do little maintenance), there is another option. + +**NOTE: we don't recommend using auto-download** because it doesn't verify in advance if a binary is available. If there will be any issue with downloading a binary, the cosmovisor will stop and won't restart an App (which could lead to a chain halt). + +If `DAEMON_ALLOW_DOWNLOAD_BINARIES` is set to `true`, and no local binary can be found when an upgrade is triggered, `cosmovisor` will attempt to download and install the binary itself based on the instructions in the `info` attribute in the `data/upgrade-info.json` file. The files is constructed by the x/upgrade module and contains data from the upgrade `Plan` object. The `Plan` has an info field that is expected to have one of the following two valid formats to specify a download: + +1. Store an os/architecture -> binary URI map in the upgrade plan info field as JSON under the `"binaries"` key. For example: + + ```json + { + "binaries": { + "linux/amd64":"https://example.com/gaia.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f" + } + } + ``` + + You can include multiple binaries at once to ensure more than one environment will receive the correct binaries: + + ```json + { + "binaries": { + "linux/amd64":"https://example.com/gaia.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", + "linux/arm64":"https://example.com/gaia.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", + "darwin/amd64":"https://example.com/gaia.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f" + } + } + ``` + + When submitting this as a proposal ensure there are no spaces. An example command using `gaiad` could look like: + + ```shell + > gaiad tx upgrade software-upgrade Vega \ + --title Vega \ + --deposit 100uatom \ + --upgrade-height 7368420 \ + --upgrade-info '{"binaries":{"linux/amd64":"https://github.com/cosmos/gaia/releases/download/v6.0.0-rc1/gaiad-v6.0.0-rc1-linux-amd64","linux/arm64":"https://github.com/cosmos/gaia/releases/download/v6.0.0-rc1/gaiad-v6.0.0-rc1-linux-arm64","darwin/amd64":"https://github.com/cosmos/gaia/releases/download/v6.0.0-rc1/gaiad-v6.0.0-rc1-darwin-amd64"}}' \ + --summary "upgrade to Vega" \ + --gas 400000 \ + --from user \ + --chain-id test \ + --home test/val2 \ + --node tcp://localhost:36657 \ + --yes + ``` + +2. Store a link to a file that contains all information in the above format (e.g. if you want to specify lots of binaries, changelog info, etc. without filling up the blockchain). For example: + + ```text + https://example.com/testnet-1001-info.json?checksum=sha256:deaaa99fda9407c4dbe1d04bd49bab0cc3c1dd76fa392cd55a9425be074af01e + ``` + +When `cosmovisor` is triggered to download the new binary, `cosmovisor` will parse the `"binaries"` field, download the new binary with [go-getter](https://github.com/hashicorp/go-getter), and unpack the new binary in the `upgrades/` folder so that it can be run as if it was installed manually. + +Note that for this mechanism to provide strong security guarantees, all URLs should include a SHA 256/512 checksum. This ensures that no false binary is run, even if someone hacks the server or hijacks the DNS. `go-getter` will always ensure the downloaded file matches the checksum if it is provided. `go-getter` will also handle unpacking archives into directories (in this case the download link should point to a `zip` file of all data in the `bin` directory). + +To properly create a sha256 checksum on linux, you can use the `sha256sum` utility. For example: + +```shell +sha256sum ./testdata/repo/zip_directory/autod.zip +``` + +The result will look something like the following: `29139e1381b8177aec909fab9a75d11381cab5adf7d3af0c05ff1c9c117743a7`. + +You can also use `sha512sum` if you would prefer to use longer hashes, or `md5sum` if you would prefer to use broken hashes. Whichever you choose, make sure to set the hash algorithm properly in the checksum argument to the URL. + +### Preparing for an Upgrade + +To prepare for an upgrade, use the `prepare-upgrade` command: + +```shell +cosmovisor prepare-upgrade +``` + +This command performs the following actions: + +1. Retrieves upgrade information directly from the blockchain about the next scheduled upgrade. +2. Downloads the new binary specified in the upgrade plan. +3. Verifies the binary's checksum (if required by configuration). +4. Places the new binary in the appropriate directory for Cosmovisor to use during the upgrade. + +The `prepare-upgrade` command provides detailed logging throughout the process, including: + +* The name and height of the upcoming upgrade +* The URL from which the new binary is being downloaded +* Confirmation of successful download and verification +* The path where the new binary has been placed + +Example output: + +```bash +INFO Preparing for upgrade name=v1.0.0 height=1000000 +INFO Downloading upgrade binary url=https://example.com/binary/v1.0.0?checksum=sha256:339911508de5e20b573ce902c500ee670589073485216bee8b045e853f24bce8 +INFO Upgrade preparation complete name=v1.0.0 height=1000000 +``` + +*Note: The current way of downloading manually and placing the binary at the right place would still work.* + +## Example: SimApp Upgrade + +The following instructions provide a demonstration of `cosmovisor` using the simulation application (`simapp`) shipped with the Cosmos SDK's source code. The following commands are to be run from within the `cosmos-sdk` repository. + +### Chain Setup + +Let's create a new chain using the `v0.47.4` version of simapp (the Cosmos SDK demo app): + +```shell +git checkout v0.47.4 +make build +``` + +Clean `~/.simapp` (never do this in a production environment): + +```shell +./build/simd tendermint unsafe-reset-all +``` + +Set up app config: + +```shell +./build/simd config chain-id test +./build/simd config keyring-backend test +./build/simd config broadcast-mode sync +``` + +Initialize the node and overwrite any previous genesis file (never do this in a production environment): + +```shell +./build/simd init test --chain-id test --overwrite +``` + +For the sake of this demonstration, amend `voting_period` in `genesis.json` to a reduced time of 20 seconds (`20s`): + +```shell +cat <<< $(jq '.app_state.gov.params.voting_period = "20s"' $HOME/.simapp/config/genesis.json) > $HOME/.simapp/config/genesis.json +``` + +Create a validator, and setup genesis transaction: + +```shell +./build/simd keys add validator +./build/simd genesis add-genesis-account validator 1000000000stake --keyring-backend test +./build/simd genesis gentx validator 1000000stake --chain-id test +./build/simd genesis collect-gentxs +``` + +#### Prepare Cosmovisor and Start the Chain + +Set the required environment variables: + +```shell +export DAEMON_NAME=simd +export DAEMON_HOME=$HOME/.simapp +``` + +Set the optional environment variable to trigger an automatic app restart: + +```shell +export DAEMON_RESTART_AFTER_UPGRADE=true +``` + +Initialize cosmovisor with the current binary: + +```shell +cosmovisor init ./build/simd +``` + +Now you can run cosmovisor with simapp v0.47.4: + +```shell +cosmovisor run start +``` + +### Update App + +Update app to the latest version (e.g. v0.50.0). + +:::note + +Migration plans are defined using the `x/upgrade` module and described in [In-Place Store Migrations](https://github.com/cosmos/cosmos-sdk/blob/main/docs/learn/advanced/15-upgrade.md). Migrations can perform any deterministic state change. + +The migration plan to upgrade the simapp from v0.47 to v0.50 is defined in `simapp/upgrade.go`. + +::: + +Build the new version `simd` binary: + +```shell +make build +``` + +Add the new `simd` binary and the upgrade name: + +:::warning + +The migration name must match the one defined in the migration plan. + +::: + +```shell +cosmovisor add-upgrade v047-to-v050 ./build/simd +``` + +Open a new terminal window and submit an upgrade proposal along with a deposit and a vote (these commands must be run within 20 seconds of each other): + +```shell +./build/simd tx upgrade software-upgrade v047-to-v050 --title upgrade --summary upgrade --upgrade-height 200 --upgrade-info "{}" --no-validate --from validator --yes +./build/simd tx gov deposit 1 10000000stake --from validator --yes +./build/simd tx gov vote 1 yes --from validator --yes +``` + +The upgrade will occur automatically at height 200. Note: you may need to change the upgrade height in the snippet above if your test play takes more time. diff --git a/versioned_docs/version-0.52/build/tooling/02-confix.md b/versioned_docs/version-0.52/build/tooling/02-confix.md new file mode 100644 index 000000000..00851ede1 --- /dev/null +++ b/versioned_docs/version-0.52/build/tooling/02-confix.md @@ -0,0 +1,156 @@ +--- +sidebar_position: 1 +--- + +# Confix + +`Confix` is a configuration management tool that allows you to manage your configuration via CLI. + +It is based on the [CometBFT RFC 019](https://github.com/cometbft/cometbft/blob/5013bc3f4a6d64dcc2bf02ccc002ebc9881c62e4/docs/rfc/rfc-019-config-version.md). + +## Installation + +### Add Config Command + +To add the confix tool, it's required to add the `ConfigCommand` to your application's root command file (e.g. `/cmd/root.go`). + +Import the `confixCmd` package: + +```go +import "cosmossdk.io/tools/confix/cmd" +``` + +Find the following line: + +```go +initRootCmd(rootCmd, moduleManager) +``` + +After that line, add the following: + +```go +rootCmd.AddCommand( + confixcmd.ConfigCommand(), +) +``` + +The `ConfixCommand` function builds the `config` root command and is defined in the `confixCmd` package (`cosmossdk.io/tools/confix/cmd`). +An implementation example can be found in `simapp`. + +The command will be available as `simd config`. + +:::tip +Using confix directly in the application can have less features than using it standalone. +This is because confix is versioned with the SDK, while `latest` is the standalone version. +::: + +### Using Confix Standalone + +To use Confix standalone, without having to add it in your application, install it with the following command: + +```bash +go install cosmossdk.io/tools/confix/cmd/confix@latest +``` + +Alternatively, for building from source, simply run `make confix`. The binary will be located in `tools/confix`. + +## Usage + +Use standalone: + +```shell +confix --help +``` + +Use in simd: + +```shell +simd config fix --help +``` + +### Get + +Get a configuration value, e.g.: + +```shell +simd config get app pruning # gets the value pruning from app.toml +simd config get client chain-id # gets the value chain-id from client.toml +``` + +```shell +confix get ~/.simapp/config/app.toml pruning # gets the value pruning from app.toml +confix get ~/.simapp/config/client.toml chain-id # gets the value chain-id from client.toml +``` + +### Set + +Set a configuration value, e.g.: + +```shell +simd config set app pruning "enabled" # sets the value pruning from app.toml +simd config set client chain-id "foo-1" # sets the value chain-id from client.toml +``` + +```shell +confix set ~/.simapp/config/app.toml pruning "enabled" # sets the value pruning from app.toml +confix set ~/.simapp/config/client.toml chain-id "foo-1" # sets the value chain-id from client.toml +``` + +### Migrate + +Migrate a configuration file to a new version, config type defaults to `app.toml`, if you want to change it to `client.toml`, please indicate it by adding the optional parameter, e.g.: + +```shell +simd config migrate v0.50 # migrates defaultHome/config/app.toml to the latest v0.50 config +simd config migrate v0.50 --client # migrates defaultHome/config/client.toml to the latest v0.50 config +``` + +```shell +confix migrate v0.50 ~/.simapp/config/app.toml # migrate ~/.simapp/config/app.toml to the latest v0.50 config +confix migrate v0.50 ~/.simapp/config/client.toml --client # migrate ~/.simapp/config/client.toml to the latest v0.50 config +``` + +### Diff + +Get the diff between a given configuration file and the default configuration file, e.g.: + +```shell +simd config diff v0.47 # gets the diff between defaultHome/config/app.toml and the latest v0.47 config +simd config diff v0.47 --client # gets the diff between defaultHome/config/client.toml and the latest v0.47 config +``` + +```shell +confix diff v0.47 ~/.simapp/config/app.toml # gets the diff between ~/.simapp/config/app.toml and the latest v0.47 config +confix diff v0.47 ~/.simapp/config/client.toml --client # gets the diff between ~/.simapp/config/client.toml and the latest v0.47 config +``` + +### View + +View a configuration file, e.g: + +```shell +simd config view client # views the current app client config +``` + +```shell +confix view ~/.simapp/config/client.toml # views the current app client conf +``` + +### Maintainer + +At each SDK modification of the default configuration, add the default SDK config under `data/vXX-app.toml`. +This allows users to use the tool standalone. + +### Compatibility + +The recommended standalone version is `latest`, which is using the latest development version of the Confix. + +| SDK Version | Confix Version | +| ----------- | -------------- | +| v0.50 | v0.1.x | +| v0.52 | v0.2.x | +| v2 | v0.2.x | + +## Credits + +This project is based on the [CometBFT RFC 019](https://github.com/cometbft/cometbft/blob/5013bc3f4a6d64dcc2bf02ccc002ebc9881c62e4/docs/rfc/rfc-019-config-version.md) and their never released own implementation of [confix](https://github.com/cometbft/cometbft/blob/v0.36.x/scripts/confix/confix.go). diff --git a/versioned_docs/version-0.52/build/tooling/03-hubl.md b/versioned_docs/version-0.52/build/tooling/03-hubl.md new file mode 100644 index 000000000..9088abfd0 --- /dev/null +++ b/versioned_docs/version-0.52/build/tooling/03-hubl.md @@ -0,0 +1,73 @@ +--- +sidebar_position: 1 +--- + +# Hubl + +`Hubl` is a tool that allows you to query any Cosmos SDK based blockchain. +It takes advantage of the new [AutoCLI](https://docs.cosmos.network/main/learn/advanced/autocli) feature of the Cosmos SDK. + +## Installation + +Hubl can be installed using `go install`: + +```shell +go install cosmossdk.io/tools/hubl/cmd/hubl@latest +``` + +Or build from source: + +```shell +git clone --depth=1 https://github.com/cosmos/cosmos-sdk +make hubl +``` + +The binary will be located in `tools/hubl`. + +## Usage + +```shell +hubl --help +``` + +### Add chain + +To configure a new chain just run this command using the --init flag and the name of the chain as it's listed in the chain registry (). + +If the chain is not listed in the chain registry, you can use any unique name. + +```shell +hubl init [chain-name] +hubl init regen +``` + +The chain configuration is stored in `~/.hubl/config.toml`. + +:::tip + +When using an insecure gRPC endpoint, change the `insecure` field to `true` in the config file. + +```toml +[chains] +[chains.regen] +[[chains.regen.trusted-grpc-endpoints]] +endpoint = 'localhost:9090' +insecure = true +``` + +Or use the `--insecure` flag: + +```shell +hubl init regen --insecure +``` + +::: + +### Query + +To query a chain, you can use the `query` command. +Then specify which module you want to query and the query itself. + +```shell +hubl regen query auth module-accounts +``` diff --git a/versioned_docs/version-0.52/build/tooling/README.md b/versioned_docs/version-0.52/build/tooling/README.md new file mode 100644 index 000000000..d9125ff3c --- /dev/null +++ b/versioned_docs/version-0.52/build/tooling/README.md @@ -0,0 +1,26 @@ +--- +sidebar_position: 0 +--- + +# Tools + +This section provides documentation on various tooling maintained by the SDK team. +This includes tools for development, operating a node, and ease of use of a Cosmos SDK chain. + +## CLI Tools + +* [Cosmovisor](./01-cosmovisor.md) +* [Confix](./02-confix.md) +* [Hubl](./03-hubl.md) +* [Rosetta](https://docs.cosmos.network/main/run-node/rosetta) + +## Other Tools + +* [Protocol Buffers](./00-protobuf.md) + +## External Tools + +This section highlights tools that are not maintained by the SDK team, but are useful for Cosmos SDK development. + +* [Ignite](https://docs.ignite.com) +* [Spawn](https://github.com/rollchains/spawn) diff --git a/versioned_docs/version-0.52/build/tooling/_category_.json b/versioned_docs/version-0.52/build/tooling/_category_.json new file mode 100644 index 000000000..eb57cb8a5 --- /dev/null +++ b/versioned_docs/version-0.52/build/tooling/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Tooling", + "position": 5, + "link": null +} \ No newline at end of file