-
Notifications
You must be signed in to change notification settings - Fork 26
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
97 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
# Starship End-to-End Testing Guide | ||
Welcome to the Starship E2E Testing Guide. | ||
This guide outlines how to set up, run, and add test cases for the Starship project. | ||
The following instructions will enable you to execute tests locally, customize configurations, and integrate tests into GitHub Actions for continuous integration. | ||
|
||
## Table of Contents | ||
* [Overview](#overview) | ||
* [Setup](#setup) | ||
* [Running Tests Locally](#running-tests-locally) | ||
* [Setup the Kubernetes Cluster](#setup-the-kubernetes-cluster) | ||
* [Install the Helm Chart](#install-the-helm-chart) | ||
* [Port Forward Services](#port-forward-services) | ||
* [Run Tests](#run-tests) | ||
* [Available Configurations](#available-configurations) | ||
* [GitHub Actions Integration](#github-actions-integration) | ||
* [Adding Tests to PR Workflow](#adding-tests-to-pr-workflow) | ||
|
||
## Overview | ||
This guide is intended for developers working on the Starship project to run E2E tests locally and contribute test cases. | ||
Tests can be run multiple times to ensure idempotency. A variety of configurations in the `configs/` directory allows testing different setups. | ||
|
||
## Setup | ||
Ensure you have the following prerequisites: | ||
* Kubernetes setup (recommended: Docker Desktop with kubernetes support for local setups): [Docker Desktop](https://www.docker.com/products/docker-desktop/ | ||
* `kubectl`: [Installation Guide](https://kubernetes.io/docs/tasks/tools/) | ||
* `helm`: [Installation Guide](https://helm.sh/docs/intro/install/) | ||
* `yq`: [Installation Guide](https://mikefarah.gitbook.io/yq/v3.x) | ||
* `jq`: [Installation Guide](https://jqlang.github.io/jq/download/) | ||
|
||
## Running Tests Locally | ||
1. Setup the Kubernetes Cluster: | ||
|
||
To start a local Kubernetes cluster with Docker Desktop follow steps: | ||
* From the Docker Dashboard, select the Settings. | ||
* Select Kubernetes from the left sidebar. | ||
* Next to Enable Kubernetes, select the checkbox. | ||
* Select Apply & Restart to save the settings and then click Install to confirm. | ||
|
||
2. Install the Helm Chart: | ||
|
||
Once the cluster is ready, install the Helm chart with a specific configuration file, for example, `configs/two-chain.yaml`: | ||
```bash | ||
make install HELM_FILE=configs/two-chain.yaml | ||
``` | ||
|
||
3. Check pod statues: | ||
|
||
Check the status of the pods to ensure everything is running before proceeding: | ||
```bash | ||
kubectl get pods | ||
``` | ||
|
||
4. Port Forward Services: | ||
|
||
After confirming the pods are in the Running state, forward the necessary ports: | ||
```bash | ||
make port-forward HELM_FILE=configs/two-chain.yaml | ||
``` | ||
|
||
5. Run Tests: | ||
|
||
Run the test suite with the following command: | ||
```bash | ||
make test HELM_FILE=configs/two-chain.yaml | ||
``` | ||
These tests are designed to be idempotent, meaning you can run make test multiple times as needed. | ||
|
||
## Available Configurations | ||
In the `configs/` directory, you’ll find various configuration files that specify different test setups. | ||
Some examples include: | ||
``` | ||
one-chain.yaml | ||
multi-validator.yaml | ||
simapp.yaml | ||
two-chain.yaml | ||
``` | ||
|
||
Feel free to test with any of these configurations by adjusting the `HELM_FILE` parameter. | ||
|
||
## GitHub Actions Integration | ||
### Adding Tests to PR Workflow | ||
The workflow for PR tests is defined in `.github/workflows/pr-tests.yaml`. | ||
This workflow runs on pull requests to the `main` and `release/*` branches, as well as on a daily schedule. | ||
|
||
### Resource Limitations | ||
Since GitHub Actions runners have limited resources, be cautious when adding resource-intensive configurations to the PR tests. | ||
Only the following configurations are currently enabled in the PR tests: | ||
``` | ||
one-chain-custom-scripts.yaml | ||
one-chain.yaml | ||
multi-validator.yaml | ||
simapp.yaml | ||
multi-validator-starship-faucet.yaml | ||
``` | ||
|
||
To add a new test configuration, simply include it in the `matrix.config-file` section of the `pr-test` job in `.github/workflows/pr-tests.yaml`. | ||
Ensure the configuration is efficient enough for GitHub Actions runners. |