Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add readme to e2e #585

Merged
merged 1 commit into from
Nov 7, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions starship/tests/e2e/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Starship End-to-End Testing Guide
Welcome to the Starship E2E Testing Guide.
This guide outlines how to set up, run, and add test cases for the Starship project.
The following instructions will enable you to execute tests locally, customize configurations, and integrate tests into GitHub Actions for continuous integration.

## Table of Contents
* [Overview](#overview)
* [Setup](#setup)
* [Running Tests Locally](#running-tests-locally)
* [Setup the Kubernetes Cluster](#setup-the-kubernetes-cluster)
* [Install the Helm Chart](#install-the-helm-chart)
* [Port Forward Services](#port-forward-services)
* [Run Tests](#run-tests)
* [Available Configurations](#available-configurations)
* [GitHub Actions Integration](#github-actions-integration)
* [Adding Tests to PR Workflow](#adding-tests-to-pr-workflow)

## Overview
This guide is intended for developers working on the Starship project to run E2E tests locally and contribute test cases.
Tests can be run multiple times to ensure idempotency. A variety of configurations in the `configs/` directory allows testing different setups.

## Setup
Ensure you have the following prerequisites:
* Kubernetes setup (recommended: Docker Desktop with kubernetes support for local setups): [Docker Desktop](https://www.docker.com/products/docker-desktop/
* `kubectl`: [Installation Guide](https://kubernetes.io/docs/tasks/tools/)
* `helm`: [Installation Guide](https://helm.sh/docs/intro/install/)
* `yq`: [Installation Guide](https://mikefarah.gitbook.io/yq/v3.x)
* `jq`: [Installation Guide](https://jqlang.github.io/jq/download/)

## Running Tests Locally
1. Setup the Kubernetes Cluster:

To start a local Kubernetes cluster with Docker Desktop follow steps:
* From the Docker Dashboard, select the Settings.
* Select Kubernetes from the left sidebar.
* Next to Enable Kubernetes, select the checkbox.
* Select Apply & Restart to save the settings and then click Install to confirm.

2. Install the Helm Chart:

Once the cluster is ready, install the Helm chart with a specific configuration file, for example, `configs/two-chain.yaml`:
```bash
make install HELM_FILE=configs/two-chain.yaml
```

3. Check pod statues:

Check the status of the pods to ensure everything is running before proceeding:
```bash
kubectl get pods
```

4. Port Forward Services:

After confirming the pods are in the Running state, forward the necessary ports:
```bash
make port-forward HELM_FILE=configs/two-chain.yaml
```

5. Run Tests:

Run the test suite with the following command:
```bash
make test HELM_FILE=configs/two-chain.yaml
```
These tests are designed to be idempotent, meaning you can run make test multiple times as needed.

## Available Configurations
In the `configs/` directory, you’ll find various configuration files that specify different test setups.
Some examples include:
```
one-chain.yaml
multi-validator.yaml
simapp.yaml
two-chain.yaml
```

Feel free to test with any of these configurations by adjusting the `HELM_FILE` parameter.

## GitHub Actions Integration
### Adding Tests to PR Workflow
The workflow for PR tests is defined in `.github/workflows/pr-tests.yaml`.
This workflow runs on pull requests to the `main` and `release/*` branches, as well as on a daily schedule.

### Resource Limitations
Since GitHub Actions runners have limited resources, be cautious when adding resource-intensive configurations to the PR tests.
Only the following configurations are currently enabled in the PR tests:
```
one-chain-custom-scripts.yaml
one-chain.yaml
multi-validator.yaml
simapp.yaml
multi-validator-starship-faucet.yaml
```

To add a new test configuration, simply include it in the `matrix.config-file` section of the `pr-test` job in `.github/workflows/pr-tests.yaml`.
Ensure the configuration is efficient enough for GitHub Actions runners.
Loading