Skip to content

Commit

Permalink
chore: add comprehensive testing guidelines (#99)
Browse files Browse the repository at this point in the history
## Description
This PR introduces a documentation outlining MetaMask's approach to
testing, including unit, integration, and end-to-end (e2e) tests. This
document provides aims to provide contributors with a clear
understanding of MetaMask's testing practices, encouraging comprehensive
test coverage and contributing to the robustness of the application.

## Changes
* Added `test_guidelines.md` to the docs folder in the contributor docs
GitHub repo.
* The document includes sections on integration tests, unit tests, and
e2e tests, each detailing the purpose, guidelines, and recommended
practices for these types of tests.
  • Loading branch information
cryptotavares authored Oct 1, 2024
1 parent 0a471eb commit dd3c581
Show file tree
Hide file tree
Showing 7 changed files with 79 additions and 2 deletions.
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,7 @@ This is a living repository — nothing is set in stone! If you're member of Met
- [Redux Guidelines](./docs/redux.md)
- [Secure Development Lifecycle Policy](./docs/sdlc.md)
- [TypeScript Guidelines](./docs/typescript.md)
- [Unit Testing Guidelines](./docs/unit-testing.md)
- [E2E Testing Guidelines](./docs/e2e-testing.md)
- [Testing Guidelines](./docs/testing/overview.md)
- [Unit Testing Guidelines](./docs/testing/unit-testing.md)
- [UI Integration Testing Guidelines](./docs/testing/ui-integration-testing.md)
- [E2E Testing Guidelines](./docs/testing/e2e-testing.md)
File renamed without changes.
File renamed without changes.
File renamed without changes.
44 changes: 44 additions & 0 deletions docs/testing/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Testing Overview

At MetaMask, we are committed to delivering high-quality software. A key aspect of achieving this goal is through a robust testing strategy that includes both end-to-end (e2e), integration and unit tests. This document provides comprehensive guidelines on each testing type, emphasizing the importance of choosing the right test for the right scenario and detailing the specific practices for writing and organizing these tests.

## What are unit tests?

Unit testing is a conceptually vague practice, as there is no consensus on how they should be written and what they should test. But in general, [as outlined by Martin Fowler](https://martinfowler.com/articles/2021-test-shapes.html), there are two common approaches: solitary unit tests and sociable unit tests. At MetaMask, we employ both of these types of unit tests (see [discussion on the use cases of each type](https://github.com/MetaMask/core/pull/3827#discussion_r1469377179)).
Unit tests examine individual components or functions in isolation or within their immediate context. At MetaMask we encourage a flexible approach to unit testing, recognizing the spectrum between sociable and solitary unit tests.

### When to write unit tests?

All components or functions should be validated through unit tests. These tests focus on the component's or function's internal logic and functionality.
If UI integration tests are available, those are preferred for page-level components. However, developers may choose to write sociable unit tests for specific scenarios where UI integration tests might not be necessary or practical.

**Best Practices**: Follow the [unit test best practices](./unit-testing.md).

## What are UI integration tests?

UI integration tests evaluate the interaction between multiple components within the application, ensuring they work together as intended. For MetaMask, we are aiming to make UI integration tests particularly focused on page-level components and are conducted in the context of the full UI application.

### When to write UI integration tests?

UI integration tests should be written for page-level components. These tests should simulate user journeys and validate the application's behavior in a full app context.

**Best Practices**: Follow the [ui integration tests best practices](./ui-integration-testing.md).

## What are end-to-end tests?

End-to-end tests simulate real user scenarios from start to finish, testing the application as a whole. In the Extension, end-to-end tests are tests that strive to test a real extension app (including `background`, `contentscript` and `ui` processes) in a controlled environment from a user perspective.

### When to write end-to-end tests?

End-to-end tests exercise an application's integration with external systems or services through critical user journeys. This includes any interactions between the UI, background process, dapp, blockchain, etc. End-to-end tests should closely mimic real user actions and flows.

**Best Practices**: Follow the [end-to-end test best practices](./e2e-testing.md).

## Generic testing considerations

- **Code fencing:** Code fencing is available in MetaMask Extension and Mobile codebases. Though useful for its purposes, this practice is very problematic for unit/UI integration testing. We should avoid it.

## Conclusion

To sum up, understanding the distinction between end-to-end, UI integration and unit tests and applying them appropriately is crucial for maintaining MetaMask's code quality and reliability. UI integration tests offer a broad view of user interactions and system integration for page-level components, while unit tests provide detailed insights into the functionality of individual components, and end-to-end tests validate the application's overall behavior, ensuring readiness for production.
By following these guidelines, developers can ensure comprehensive test coverage, contributing to the robustness and user satisfaction of the application
27 changes: 27 additions & 0 deletions docs/testing/ui-integration-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# UI Integration Testing Guidelines

## Use Jest

Just like for unit tests, for UI integration tests, we use Jest as the test runner and testing framework. Jest provides a rich set of features that make it easy to write and maintain tests. UI integration tests should be written following the same conventions as unit tests.

## Focus on User Journeys

UI integration tests should focus on user journeys and interactions within the application. Design tests to mimic actual user actions and flows. This helps in identifying issues that could affect the user experience.

### Provide Full App Context

Always test page-level components in the context of the full application. This approach ensures that tests reflect real user scenarios and interactions within the app.

UI integration tests should not be written for components other than page-level components. Other components should be tested using unit tests.

## Keep Tests Separate from Implementation

Place integration test files within a `test/integration` directory. This centralized location helps manage tests that span multiple pages and components, improving maintenance and discoverability. This approach differs from unit tests, where co-location with the component is standard.

## Avoid Snapshot testing

Refrain from using snapshot testing in UI integration tests. They tend to be less meaningful in the context of full app testing and can lead to brittle tests.

## Don't Mock UI Components

Keep mocking to minimum. Ideally only the background connection (MetaMask Extension), or any network request (fired from the UI) should be mocked. Avoid mocking any UI components or hooks. For mocking the background connection we can use jest, for mocking network requests we can use `nock` (or `msw`).
4 changes: 4 additions & 0 deletions docs/unit-testing.md → docs/testing/unit-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -1071,6 +1071,10 @@ Jest incorporates most of the features of Sinon with a slimmer API:
- `jest.spyOn(object, method)` can be used in place of `sinon.spy(object, method)` or `sinon.stub(object, method)` (with the caveat that the method being spied upon will still be called by default).
- `jest.useFakeTimers()` can be used in place of `sinon.useFakeTimers()` (though note that Jest's "clock" object had fewer features than Sinon's prior to Jest v29.5).

## Avoid general manual mocks:

According to Jest's documentation `Manual mocks are defined by writing a module in a __mocks__/ subdirectory immediately`. These types of mocks are automatically picked up by Jest for all tests. We should be very careful when writing this types of mocks as they will be shared across all tests (including UI integration tests).

## Snapshots

Jest snapshots are not testing the validity of the value tested against a snapshot.
Expand Down

0 comments on commit dd3c581

Please sign in to comment.