Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smoke Test Improvements #448

Open
rye opened this issue Oct 22, 2020 · 3 comments
Open

Smoke Test Improvements #448

rye opened this issue Oct 22, 2020 · 3 comments
Assignees
Labels
ci This issue or pull request concerns our Continuous Integration setup enhancement New feature or request

Comments

@rye
Copy link
Member

rye commented Oct 22, 2020

Take the changes to the smoke tests from #344 into their own branch.

We should test every API route, and known values for those API routes, as well as identify passing criteria for those API routes. Currently (on the master branch) our "smoke test" just boots up the server; it's important to verify that the routes we expect to work do work, especially when evaluating the efficacy of PRs like #344 and #447, which would otherwise silently break production.

This'll also give us a canary mechanism for nightly or weekly tests.

@frog-pond/core, I think it might be wise to use Python or something else (maybe even Rust 😉 ?) to combinatorially generate templates (e.g. the Cartesian products of {gzip, no-gzip} x {[various headers]} x {api routes}, etc.) to test during smoke testing. Any opinions on doing that? Obviously we don't want to spam our upstream sources, too, so we might have to cache responses from that.

@rye rye added enhancement New feature or request ci This issue or pull request concerns our Continuous Integration setup labels Oct 22, 2020
@rye rye self-assigned this Oct 22, 2020
@drewvolz
Copy link
Member

+1 to extracting them into their own branch.
+1 to creating a passing criteria for canary tests.
+1 to either Python or Rust!

@hawkrives
Copy link
Member

Thanks to @drewvolz, we now have that /routes endpoint, so we could probably do this programmatically!

@hawkrives
Copy link
Member

Most of this is now done.

Still to-do:

  • Add example parameters to the endpoints that need parameters
  • Add a Google auth token to allow testing the calendar endpoints in CI
  • Think about moving the smoke tests to Jest + https://testcontainers.com (so we're actually running the tests against the produced Docker container)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci This issue or pull request concerns our Continuous Integration setup enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants