Ok To Test

Example workflow configuration showing how to use GitHub Actions secrets in pull requests from forks 🍴🔑

Summary

An Ok To Test workflow is configured so that when someone with write access to this repository comments ok-to-test sha=<head-sha> on a pull request from a fork, a "privileged" Integration tests workflow needing secrets is triggered. In parallel, a "non-privileged" Unit tests workflow not needing secrets is triggered on any pull request.

About

GitHub Actions purposely limits the secrets available to pull requests from forks for security reasons:

• GITHUB_TOKEN is read-only
• Other secrets aren't available at all

Though this provides peace of mind, many projects depend on the fork pull request model. If you've configured a GitHub Actions test workflow to trigger on pull requests, and those tests require secrets, the secrets aren't available and the workflow fails.

No longer with this workaround, which shows an example Prow-like /ok-to-test sha=<head-sha> slash command configuration! 🥳

This project is not affiliated with GitHub.

Setup

This is a template repository with three example workflows. Start by creating a new repository ("Use this template"). Then, consider for your use case:

1. Which type of token you'll use to emit the repository_dispatch event in Ok To Test. Set the secrets in your repository accordingly, e.g. I used a GitHub App and had to save secrets called APP_ID and PRIVATE_KEY. Remember: if you also choose GitHub App authentication (preferred), you must create and install it on the repo(s) in which this configuration will run. See Creating A GitHub App for a basic overview of how to do this.
2. Which workflow(s) need secrets. In this example, it's Integration tests, and I would need to fill in my tests here.
3. Which workflow(s) do not need secrets. In this example, it's Unit tests. These types of workflows can simply trigger on pull request.
4. The Permissions required for your GITHUB_TOKEN. The workflows used to implement ok-to-test require the ability to: add reactions to your pull request comments, and update the status of your pull request checks. Currently GitHub Actions' built-in GITHUB_TOKEN is read-only by default. The example workflows in this repo explicitly grant the necessary write permissions to the jobs that require them. You can read more about this in the GitHub Docs, which also describe how to update the defaults.

Usage

As someone with write access, comment /ok-to-test sha=<head-sha> on an incoming pull request to set off this Rube Goldberg machine 😄. The head sha is the first seven characters of the most recent commit of the incoming pull request. For example, /ok-to-test sha=742c71a.

Example

1. A fork pull request is opened.
2. A unit test workflow runs. Secrets are not available to this workflow.
3. Someone with write access looks over the pull request code. ⚠️ Before proceeding, they should be sure the code isn't doing anything malicious like secret logging. ⚠️
4. They comment /ok-to-test sha=<head-sha> on the pull request.
5. A repository_dispatch API request is sent to this repository. See guidance below on how to authenticate.
6. An integration test workflow runs, checking out the merge commit if the head sha hasn't changed since the comment was made. Secrets are available to this workflow! 💫
7. The pull request status check is updated to reflect the success or failure of the integration test workflow.

Note that this sequence also works for branch based pull requests, as you'd expect!

Authentication

Choose one of these authentication methods for the repository_dispatch helper action, peter-evans/slash-command-dispatch, in ok-to-test.yml:

• Personal access token with repo scope
• OAuth "app" token with repo scope
• ⭐️ Preferred: GitHub App installation access token with contents: write and metadata: read permissions. See Creating A GitHub App for a basic overview of how to do this.

GitHub Apps have distinct identities on GitHub – no seat taken up by a machine account, no potential for leaking your personal credentials, and no rate limit sharing!

Creating a GitHub App

Here we are using a GitHub App as an authentication entity. Below are some brief instructions on how to setup a GitHub App for this purpose, note that there are other methods of creating a GitHub App such as with a manifest file (e.g. one similar to app.yml). (The instructions below are for setting up an app within your user, but you can also do it for your organization.)

1. Go to Settings > Developer Settings > GitHub Apps, and select New GitHub App.
2. Enter a name for your app (this needs to be unique across GitHub), and fill in the required URL fields. You can fill in these URLs with fake values - they do not need to resolve, so you can use:
   • Homepage URL = http://example.com
3. You can ignore the 'Callback URL' field, and untick 'Webook' > Active
4. Under Repository Permissions, set:
   • Contents = 'Read and Write'
   • Metadata = 'Read-only'
5. Click 'Create GitHub app'
6. Click 'Generate Private key' (this will be downloaded to your computer), and take a note of the App ID field
7. Install the GitHub App into your user or organization, by clicking 'Install' under the 'Install App' tab, and choose whether you want to give the app access to all of your user's / org's repositories, or just specific ones
8. Go to the repository that you want to use ok-to-test with, and then Settings > Secrets and variables > Actions and create two new secrets:
   • APP_ID with the value for the App ID field that you noted earlier
   • PRIVATE_KEY, copying and pasting in the full contents of the Private Key file that you generated and downloaded earlier

Credits

• Prow for the idea for ok-to-test
• A few handy community actions, peter-evans/slash-command-dispatch, tibdex/github-app-token, and actions/github-script

Contributing

Pull requests are welcome!

License

MIT