Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Adds how to report coverage on test failures #317

Merged
merged 3 commits into from
Dec 17, 2023
Merged

docs: Adds how to report coverage on test failures #317

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
239db7e
docs: Adds how to report coverage on test failures
davelosert Dec 17, 2023
99c6372
docs: Improve general wording an structure
davelosert Dec 17, 2023
b5e7307
docs: Improve general wording an structure
davelosert Dec 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 30 additions & 26 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# vitest-coverage-report-action

A GitHub Action to report [vitest](https://vitest.dev/) coverage results as a GitHub step-summary and pull request comment.
This GitHub Action reports [vitest](https://vitest.dev/) coverage results as a GitHub step-summary and as a comment on a pull request.

![Coverage Report as Step Summary](./docs/coverage-report.png)

It will create a high-level coverage summary for all coverage-category as well as a file-based report linking to the files itself and the uncovered lines for easy discovery.
The action generates a high-level coverage summary for all coverage categories, as well as a detailed, file-based report. The report includes links to the files themselves and the uncovered lines for easy reference.

## Usage

This action requires you to use `vitest` to create a coverage report with the following reporters:
To use this action, you need to configure `vitest` to create a coverage report with the following reporters:

- `json-summary` (required): Will add a high-level summary of your overall coverage
- `json` (optional): If provided, will add file-specific coverage reports for any file of your project
- `json-summary` (required): This reporter generates a high-level summary of your overall coverage.
- `json` (optional): If provided, this reporter generates file-specific coverage reports for each file in your project.

You can configure the reporters within the `vitest.config.js` file likes this:
You can configure the reporters in your Vite configuration file (e.g., `vite.config.js`) as follows:

```js
import { defineConfig } from 'vite';
Expand All @@ -23,6 +23,8 @@ export default defineConfig({
coverage: {
// you can include other reporters, but 'json-summary' is required, json is recommended
reporter: ['text', 'json-summary', 'json'],
// If you want a coverage reports even if your tests are failing, include the reportOnFailure option
reportOnFailure: true,
}
}
});
Expand Down Expand Up @@ -58,28 +60,30 @@ jobs:
- name: 'Test'
run: npx vitest --coverage
- name: 'Report Coverage'
if: always() # Also generate the report if tests are failing
# Set if: always() to also generate the report if tests are failing
# Only works if you set `reportOnFailure: true` in your vite config as specified above
if: always()
uses: davelosert/vitest-coverage-report-action@v2
```

> [!NOTE]
> If you want to have comments on pull requests coming from forks, you will have to use the configuration as provided in the section [Working with pull requests from forks](#working-with-pull-requests-from-forks)
> To enable comments on pull requests originating from forks, please refer to the configuration provided in the [Working with Pull Requests from Forks](#working-with-pull-requests-from-forks) section.

### Required Permissions

This action requires permissions set to `pull-request: write` in order for it to be able to add a comment to your pull-request. If you are using the default `GITHUB_TOKEN`, make sure to include the permissions together with `contents: read` to the the job, so that the `actions/checkout` action is allowed to checkout the repository. This is especially important for new repositories created after [GitHub's announcement](https://github.blog/changelog/2023-02-02-github-actions-updating-the-default-github_token-permissions-to-read-only/) to change the default permissions to `read-only` for all new `GITHUB_TOKEN`s.
This action requires the `pull-request: write` permission to add a comment to your pull request. If you're using the default `GITHUB_TOKEN`, ensure that you include both `pull-request: write` and `contents: read` permissions in the job. The `contents: read` permission is necessary for the `actions/checkout` action to checkout the repository. This is particularly important for new repositories created after GitHub's [announcement](https://github.blog/changelog/2023-02-02-github-actions-updating-the-default-github_token-permissions-to-read-only/) to change the default permissions to `read-only` for all new `GITHUB_TOKEN`s.

### Options

| Option | Description | Default |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------- |
| `json-summary-path` | The path to the json summary file. | `./coverage/coverage-summary.json` |
| `json-final-path` | The path to the json final file. | `./coverage/coverage-final.json` |
| `vite-config-path` | The path to the vite config file. Will check the same paths as vite and vitest | Checks pattern `vite[st].config.{t\|mt\|ct\|j\|mj\|cj}s` |
| `github-token` | A GitHub access token with permissions to write to issues (defaults to `secrets.GITHUB_TOKEN`). | `${{ github.token }}` |
| `working-directory` | Run action within a custom directory (for monorepos). | `./` |
| `name` | Give the report a name. This is useful if you want multiple reports for different test suites within the same PR. Needs to be unique. | '' |
| `file-coverage-mode` | Defines how file-based coverage is reported. Possible values are `all`, `changes` or `none`. | `changes` |
| Option | Description | Default |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| `working-directory` | The main path to search for coverage- and configuration files (adjusting this is especially useful in monorepos). | `./` |
| `json-summary-path` | The path to the json summary file. | `${working-directory}/coverage/coverage-summary.json` |
| `json-final-path` | The path to the json final file. | `${working-directory}/coverage/coverage-final.json` |
| `vite-config-path` | The path to the vite config file. Will check the same paths as vite and vitest | Checks pattern `${working-directory}/vite[st].config.{t\|mt\|ct\|j\|mj\|cj}s` |
| `github-token` | A GitHub access token with permissions to write to issues (defaults to `secrets.GITHUB_TOKEN`). | `${{ github.token }}` |
| `file-coverage-mode` | Defines how file-based coverage is reported. Possible values are `all`, `changes` or `none`. | `changes` |
| `name` | Give the report a custom name. This is useful if you want multiple reports for different test suites within the same PR. Needs to be unique. | '' |

#### File Coverage Mode

Expand All @@ -89,7 +93,7 @@ This action requires permissions set to `pull-request: write` in order for it to

#### Name

If you have multiple test-suites but want to report the coverage in a single PR, you have to provide a unique `name` for each action-step that parses a summary-report, e.g.:
If your project includes multiple test suites and you want to consolidate their coverage reports into a single pull request comment, you must assign a unique `name` to each action step that parses a summary report. For example:

```yml
## ...
Expand All @@ -111,9 +115,9 @@ If you have multiple test-suites but want to report the coverage in a single PR,

### Coverage Thresholds

This action will read the coverage thresholds defined in the `coverage`-property of the `vite.config.js`-file and mark the status of the generated report accordingly.
This action reads the coverage thresholds specified in the `coverage` property of the Vite configuration file. It then uses these thresholds to determine the status of the generated report.

E.g. with a config like this:
For instance, consider the following configuration:

```typescript
import { defineConfig } from 'vite';
Expand All @@ -130,17 +134,17 @@ export default defineConfig({
});
```

the report would look like this:
With the above configuration, the report would appear as follows:

![Coverage Threshold Report](./docs/coverage-report-threshold.png)

If there are no thresholds defined, the status will be '🔵'.
If no thresholds are defined, the status will display as '🔵'.

### Workspaces

If you are using a monorepo with [Vitest Workspaces](https://vitest.dev/guide/workspace.html) and you run Vitest from the root of your project, Vitest will ignore the `coverage`-property of the individual project-level `vite.config.js`-files. This is because some of the [configuration options](https://vitest.dev/guide/workspace.html#configuration) are not allowed in a project config, for example coverage is done for the whole workspace.
If you're using a monorepo with [Vitest Workspaces](https://vitest.dev/guide/workspace.html) and running Vitest from your project's root, Vitest will disregard the `coverage` property in individual project-level Vite configuration files. This is because some [configuration options](https://vitest.dev/guide/workspace.html#configuration), such as coverage, apply to the entire workspace and are not allowed in a project config.

In this case, you can create a `vite.config.js`-file in the root of your project next to your `vitest.workspace.js`-file to configure coverage for the whole workspace:
In such cases, you can create a Vite configuration file at the root of your project, alongside your `vitest.workspace.js` file, to configure coverage for the entire workspace:

```js
import { defineConfig } from 'vite';
Expand All @@ -155,7 +159,7 @@ export default defineConfig({
});
```

Alternatively, you can provide [coverage options](https://vitest.dev/config/#coverage) to CLI with dot notation:
Alternatively, you can supply [coverage options](https://vitest.dev/config/#coverage) directly to the CLI using dot notation:

```sh
npx vitest --coverage.enabled --coverage.provider=v8 --coverage.reporter=json-summary --coverage.reporter=json
Expand Down