-
Notifications
You must be signed in to change notification settings - Fork 12
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Testing Docs] Add initial testing docs (#871)
## Summary Adds the following pages to a new "Develop" > "Testing" section: - Testing Levels - Module Integration Tests - App Integration Tests - App Integration Suites - In-Memory Network Integration Tests - End-to-End Tests ## Issue N/A ## Type of change Select one or more from the following: - [ ] New feature, functionality or library - [ ] Consensus breaking; add the `consensus-breaking` label if so. See #791 for details - [ ] Bug fix - [ ] Code health or cleanup - [x] Documentation - [ ] Other (specify) ## Testing - [x] **Documentation**: `make docusaurus_start`; only needed if you make doc changes - [ ] **Unit Tests**: `make go_develop_and_test` - [ ] **LocalNet E2E Tests**: `make test_e2e` - [ ] **DevNet E2E Tests**: Add the `devnet-test-e2e` label to the PR. ## Sanity Checklist - [ ] I have tested my changes using the available tooling - [ ] I have commented my code - [x] I have performed a self-review of my own code; both comments & source code - [ ] I create and reference any new tickets, if applicable - [x] I have left TODOs throughout the codebase, if applicable --------- Co-authored-by: Daniel Olshansky <[email protected]>
- Loading branch information
1 parent
0663679
commit d038470
Showing
8 changed files
with
427 additions
and
14 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
135 changes: 135 additions & 0 deletions
135
docusaurus/docs/develop/developer_guide/testing/app_integration.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,135 @@ | ||
--- | ||
sidebar_position: 3 | ||
title: App Integration Tests | ||
--- | ||
|
||
// TODO(@bryanchriswhite): Replace github source links with godocs links once available. | ||
|
||
## Table Of Contents <!-- omit in toc --> | ||
|
||
- [Overview](#overview) | ||
- [Using `integration.App`](#using-integrationapp) | ||
- [Constructors](#constructors) | ||
- [Customizing `integration.App` Configuration](#customizing-integrationapp-configuration) | ||
- [Module Configuration](#module-configuration) | ||
- [Setting Module Genesis State](#setting-module-genesis-state) | ||
- [Message / Transaction / Block Processing](#message--transaction--block-processing) | ||
- [Example Test](#example-test) | ||
|
||
## Overview | ||
|
||
[**App integration level**](testing_levels#app-integration-tests) tests leverage a custom construction of the poktroll appchain (for testing only). | ||
|
||
This construction integrates all the poktroll modules (and their cosmos-sdk dependencies) and exercises the appchain's message routing/handling and transaction processing logic. | ||
|
||
Tests in this level conventionally use the `testutil/integration` package's `App` structure and constructors to set up the appchain, execute messages, and make assertions against the resulting appchain state. | ||
|
||
:::info | ||
See [App Integration Suites](integration_suites) for organizing larger or higher-level app integration tests. | ||
::: | ||
|
||
## Using `integration.App` | ||
|
||
### Constructors | ||
|
||
To create a new instance of the `IntegrationApp` for your tests, use the `NewCompleteIntegrationApp` constructor, which handles the setup of all modules, multistore, base application, etc.: | ||
|
||
```go | ||
// NewCompleteIntegrationApp creates a new instance of the App, abstracting out | ||
// all the internal details and complexities of the application setup. | ||
func NewCompleteIntegrationApp(t *testing.T, opts ...IntegrationAppOptionFn) *App | ||
|
||
// IntegrationAppOptionFn is a function that receives and has the opportunity to | ||
// modify the IntegrationAppConfig. It is intended to be passed during integration | ||
// App construction to modify the behavior of the integration App. | ||
type IntegrationAppOptionFn func(*IntegrationAppConfig) | ||
``` | ||
|
||
If more granular control over the application configuration is required, the more verbose `NewIntegrationApp` constructor exposes additional parameters: | ||
|
||
```go | ||
// NewIntegrationApp creates a new instance of the App with the provided details | ||
// on how the modules should be configured. | ||
func NewIntegrationApp( | ||
t *testing.T, | ||
sdkCtx sdk.Context, | ||
cdc codec.Codec, | ||
txCfg client.TxConfig, | ||
registry codectypes.InterfaceRegistry, | ||
bApp *baseapp.BaseApp, | ||
logger log.Logger, | ||
authority sdk.AccAddress, | ||
modules map[string]appmodule.AppModule, | ||
keys map[string]*storetypes.KVStoreKey, | ||
msgRouter *baseapp.MsgServiceRouter, | ||
queryHelper *baseapp.QueryServiceTestHelper, | ||
opts ...IntegrationAppOptionFn, | ||
) *App { | ||
``` | ||
#### Customizing `integration.App` Configuration | ||
If the existing [`IntegrationAppConfig`](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/options.go#L13) is insufficient, it may be extended with additional fields, corresponding logic, and `IntegrationAppOptionFn`s to set them. | ||
### Module Configuration | ||
Integrated modules can be configured using `IntegrationAppOptionFn` typed option functions. | ||
Example use cases include: | ||
- Setting custom genesis states one or more modules (see [`integration.WithModuleGenesisState()`](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/options.go#L40)). | ||
- Setting up a faucet account (see: [`newFaucetInitChainerFn()`](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/app.go#L985)). | ||
- Collecting module info (see: [`newInitChainerCollectModuleNames()`](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/suites/base.go#L157)). | ||
#### Setting Module Genesis State | ||
```go | ||
supplierGenesisState := &suppliertypes.GenesisState{ | ||
// ... | ||
} | ||
app := NewCompleteIntegrationApp(t, | ||
WithModuleGenesisState[suppliermodule.AppModule](supplierGenesisState), | ||
) | ||
``` | ||
### Message / Transaction / Block Processing | ||
The `IntegrationApp` provides several methods to manage the lifecycle of transactions and blocks during tests: | ||
- `RunMsg`/`RunMsgs`: Processes one or more messages by: | ||
- calling their respective handlers | ||
- packaging them into a transaction | ||
- finalizing the block | ||
- committing the state | ||
- advancing the block height | ||
- returning the message responses | ||
- `NextBlock`/`NextBlocks`: Only advances the blockchain state to subsequent blocks. | ||
## Example Test | ||
Here's a simple example of how to create a new integration app instance and run a message using the helper functions: | ||
```go | ||
func TestAppIntegrationExample(t *testing.T) { | ||
// Initialize a new complete integration app with default options. | ||
app := NewCompleteIntegrationApp(t) | ||
|
||
// Example message to be processed | ||
msg := banktypes.NewMsgSend(fromAddr, toAddr, sdk.NewCoins(sdk.NewInt64Coin("upokt", 100))) | ||
|
||
// Run the message in the integration app | ||
res, err := app.RunMsg(t, msg) | ||
require.NoError(t, err) | ||
|
||
// Check the result | ||
require.NotNil(t, res, "Expected a valid response for the message") | ||
|
||
// Type assert the result to the message response type | ||
sendRes, ok := res.(*banktypes.MsgSendResponse) | ||
|
||
require.True(t, ok) | ||
require.NotNil(t, sendRes) | ||
} | ||
``` | ||
This example initializes the app, processes a bank message, and validates the result. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
--- | ||
sidebar_position: 7 | ||
title: End-to-End Tests | ||
--- | ||
|
||
// TODO_DOCUMENT(@bryanchriswhite) |
6 changes: 6 additions & 0 deletions
6
docusaurus/docs/develop/developer_guide/testing/in_memory_integration.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
--- | ||
sidebar_position: 6 | ||
title: In-Memory Network Integration Tests | ||
--- | ||
|
||
// TODO_DOCUMENT(@bryanchriswhite): Explain cosmos-sdk in-memory network and its appropriate usage. |
114 changes: 114 additions & 0 deletions
114
docusaurus/docs/develop/developer_guide/testing/integration_suites.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
--- | ||
sidebar_position: 5 | ||
title: App Integration Suites | ||
--- | ||
|
||
// TODO(@bryanchriswhite): Replace github source links with godocs links once available. | ||
|
||
## Table of Contents <!-- omit in toc --> | ||
|
||
- [Overview](#overview) | ||
- [When to Use Test Suites](#when-to-use-test-suites) | ||
- [Using an Existing Integration Suite](#using-an-existing-integration-suite) | ||
- [Example (`ParamsSuite`)](#example-paramssuite) | ||
- [Implementing a Test Suite](#implementing-a-test-suite) | ||
- [Test Suite Gotchas](#test-suite-gotchas) | ||
|
||
## Overview | ||
|
||
The [`suites` package](https://github.com/pokt-network/poktroll/tree/main/testutil/integration/suites) provides interfaces and base implementations for creating and managing **app integration test** suites. | ||
|
||
The foundational components are: | ||
|
||
- [**`IntegrationSuite`**](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/suites/interface.go#L14): An interface defining common methods for interacting with an integration app. | ||
- [**`BaseIntegrationSuite`**](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/suites/base.go#L26): A base implementation of the `IntegrationSuite` interface that can be extended by embedding in other test suites. | ||
|
||
## When to Use Test Suites | ||
|
||
- **Complex Integration Tests**: Testing interactions between several modules; suites facilitate encapsulation and decomposition. | ||
- **Complex Scenarios**: Simulating real-world scenarios that involve several transactions, state changes, and/or complex assertion logic. | ||
- **Reusable Components**: To DRY (Don't Repeat Yourself) up common test helpers which can be embedded in other test suites (object oriented). | ||
|
||
## Using an Existing Integration Suite | ||
|
||
The `testutil/integration/suites` package contains multiple **app integration suites** which are intended to be embedded in [**app integration level**](testing_levels#app-integration-tests) test suites. | ||
|
||
### Example (`ParamsSuite`) | ||
|
||
The following example shows a test suite which embeds `suites.ParamsSuite`, in order to set on-chain module params as part of its `SetupTest()` method: | ||
|
||
```go | ||
package suites | ||
|
||
import ( | ||
"testing" | ||
"github.com/stretchr/testify/require" | ||
"github.com/stretchr/testify/suite" | ||
cosmostypes "github.com/cosmos/cosmos-sdk/types" | ||
banktypes "github.com/cosmos/cosmos-sdk/x/bank/types" | ||
) | ||
|
||
type ExampleTestSuite struct { | ||
suites.ParamsSuite | ||
} | ||
|
||
// SetupTest is called before each test method in the suite. | ||
func (s *ExampleTestSuite) SetupTest() { | ||
// Initialize a new app instance for each test. | ||
s.app = NewApp(s.T()) | ||
|
||
// Setup the authz accounts and grants for updating parameters. | ||
s.SetupTestAuthzAccounts() | ||
s.SetupTestAuthzGrants() | ||
|
||
// Set the module params using the ParamsSuite. | ||
s.RunUpdateParam(s.T(), | ||
sharedtypes.ModuleName, | ||
string(sharedtypes.KeyNumBlocksPerSession), | ||
9001, | ||
) | ||
} | ||
|
||
func (s *ExampleTestSuite) TestExample() { | ||
// Query module params using the ParamsSuite. | ||
sharedParams, err := s.QueryModuleParams(s.T(), sharedtypes.ModuleName) | ||
require.NoError(s.T(), err) | ||
|
||
// Utilize other BaseIntegrationSuite methods to interact with the app... | ||
|
||
fundAmount := int64(1000) | ||
fundAddr, err := cosmostypes.AccAddressFromBech32("cosmos1exampleaddress...") | ||
require.NoError(s.T(), err) | ||
|
||
// Fund an address using the suite's FundAddress method. | ||
s.FundAddress(s.T(), fundAddr, fundAmount) | ||
|
||
// Use the bank query client to verify the balance. | ||
bankQueryClient := s.GetBankQueryClient() | ||
balRes, err := bankQueryClient.Balance(s.SdkCtx(), &banktypes.QueryBalanceRequest{ | ||
Address: fundAddr.String(), | ||
Denom: "upokt", | ||
}) | ||
|
||
// Validate the balance. | ||
require.NoError(s.T(), err) | ||
require.Equal(s.T(), fundAmount, balRes.GetBalance().Amount.Int64()) | ||
} | ||
|
||
// Run the ExampleIntegrationSuite. | ||
func TestExampleTestSuite(t *testing.T) { | ||
suite.Run(t, new(ExampleTestSuite)) | ||
} | ||
``` | ||
|
||
## Implementing a Test Suite | ||
|
||
// TODO_DOCUMENT(@bryanchriswhite) | ||
|
||
### Test Suite Gotchas | ||
|
||
- **Setup**: You MAY need to call `SetupXXX()`: check embedded suites for any required setup and copy-paste | ||
- **Accessing Test State**: Avoid using `s.T()` in methods of suites which are intended to be embedded in other suites; pass a `*testing.T` argument instead. | ||
- **Inheritance**: Inheriting multiple suites is hard since only one can be embedded anonymously: others will have to accessed via a named field. | ||
|
||
// TODO_DOCUMENT(@bryanchriswhite): Add a `testutil/integration/suites.doc.go` with testable examples. |
6 changes: 6 additions & 0 deletions
6
docusaurus/docs/develop/developer_guide/testing/module_integration.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
--- | ||
sidebar_position: 2 | ||
title: Module Integration Tests | ||
--- | ||
|
||
// TODO_DOCUMENT(@bryanchriswhite): Explain `testkeeper.NewTokenomicsKeepers()` (to be renamed) and its appropriate usage. |
Oops, something went wrong.