-
Notifications
You must be signed in to change notification settings - Fork 62
Testing
We use Jest as our primary testing framework, for unit testing throughout the repo, and backend end-to-end testing (with the help of light-my-request). For frontend component and E2E testing we use Cypress.
Due to the volunteer nature of this project our test coverage isn't as high as we'd like. We have good E2E testing on the backend, along with some convenient utilities for writing E2E tests concisely and validating outgoing DTOs. We don't aim for wide unit test coverage, but heavily test authentication and various systems that wouldn't lend well to E2E testing.
Testing can of course be tedious. We appreciate that for those who've just spent considerable time learning the whole stack and working on a first contribution, having to then write tests can feel like a demoralizing additional hurdle. That being said, if you're experienced with writing tests for other languages/frameworks already, this is easy to pick up. If you're not, being able to write good tests is a vital part of a programmer's repertoire; tests are a completely fundamental part of good web systems. Try to keep tests in mind as you code (or, write the tests first!) and you'll often find yourself writing better code in general, with greater consideration for potential bugs and difficult edge cases.
For contributors, E2E testing the handling of all query parameters and other potential inputs is a hard requirement. If service logic is too complicated to test sufficiently with E2E, unit tests are also expected. We don't bother unit testing trivial methods like NestJS controllers.
Unit tests live in .spec.ts files inside the backend project.
As above, unit testing is currently lacking, but examples can be found in the auth service. We take a very conventional approach here, the Nest docs page should be a good reference.
We include the jest-mock-extended package which can be used to make type-safe mocks of any service. Here's a good example!
To run unit tests, run
nx test backendAs an aside, one of the strongest areas of AI programming currently (in my opinion) is unit testing. GPT-4 (Microsoft's Bing chat thing provides access for free) can often be given entire methods/functions and will produce decent tests. I suggest you do not do this unless you're already confident in your ability to write tests, but if you are at that point, you may find it a major time-save.
E2E tests live in the backend-e2e project, structured based on the outermost endpoint name (e.g. maps, user, users).
To run E2E tests, run
nx e2e backend-e2eE2E tests can be run with the
TEST_LOG_DEBUG=trueto make the exception filter emit more detailed logs, including the error messages for any 4XX/5XX responses. This can be helpful for finding bugs without debugging, but spams stdout. Best to use for individual tests!
They heavily use the backend-test-utils library, which contains the following tools
Provides a wrapper around Fastify's light-my-request tool. This is designed to vastly improve the amount of boilerplate in our old tests, and allow constructing queries, passing access tokens, etc. easily.
In particular, it uses our custom toBeValidDto decorator heavily, which uses class-transformer and class-validator to reconstruct DTO classes from the JSON the API responds with, essentially performing DTO validation on outgoing data, rather than just incoming.
Handles the database queries we're constantly making as we setup and teardown tests. The vast majority of endpoints on our API require a login to use, and every test suite starts with a clean DB, so this utility provides functions for creating those users, as well as maps, runs, etc.
Since we need a login for each endpoint, this utility makes use of the fact we obviously know the JWT the backend uses, so creates JWTs in exactly the same way the backend does.
Mostly E2E tests check stuff in DB, but s3.util is provided for testing any endpoints that do file storage related actions.
Let's run through a simple end-to-end test using this system. We'll test some of the expected functionality of the /api/v1/user PATCH endpoint, which allows logged-in users to update their alias and bio. It's quite simple, so here's the whole thing (db, req and prisma are assigned further up in the file, see user.e2e-spec.ts):
describe("user", () => {
describe('PATCH', () => {
it("should update the authenticated user's alias", async () => {
const [user, token] = await db.createAndLoginUser();
const newAlias = 'Donkey Kong';
await req.patch({
url: 'user',
body: { alias: newAlias },
token: token,
status: 204
});
const updatedUser = await prisma.user.findFirst({
where: { id: user.id },
include: { profile: true }
});
expect(updatedUser.alias).toBe(newAlias);
});
});
});Since this lives in a large test file we use blocks of describe calls to Jest - don't worry about this.
Then, we declare our test with it (note, in Jest this is identical to test - God knows why), taking the test name as a string then the actual test in a (async) function.
Inside, we call db.util to create a user in the database, then generate a JWT access token that the API will accept. We await the returned promise, which returns a tuple of the Prisma User type, and a token string, which we deconstruct into the user and token variables.
"Donkey Kong" is just a constant we're going to use to test the alias update (Donkey Kong was a major inspiration for Momentum Mod). We could just use the string literal "Donkey Kong" everywhere, but defining it in one place is just a useful way to avoid future bugs.
Now, req.patch is doing a lot of heavy lifting for us - this is one of the utilities we use to reduce endless boilerplate in these tests. Let's go through each options:
-
url: 'user'means this will be a PATCH request toapi/v1/user(req.patchhandles the start of the URL,v1is the defaultversionoption) -
body: { alias: newAlias }is the actual request body.light-my-requesthandles serializing this Javascript object to JSON. -
token: token(justtokenis actually equivalent here but let's not get too nerdy):req.patchwill attach this to the requestAuthorizationhead for us -
status: 204this is a little different from the rest: this is a test condition. It's saying the response should have status code204, and to fail if it's anything else
Okay, if runtime execution reaches this point, our request was successful, since it must have responded with 204. Now we check that the database was actually updated with the new user. (You could be pedantic here and say that an E2E test should just be testing input and output on each "end" and not care about the contents of the database. Maybe we should test the database with a separate HTTP query? To this I say: go away this is useful leave me alone) prisma is just an instance of the Prisma client (but running outside of the backend instance) we can use to do easy DB checks for us. So we use a simple Prisma query to grab the updated user, using the user's id which obviously we know.
Finally, have dug out the user, we check that the alias was indeed updated using Jest's expect and toBe matcher.
Due to time constraints and developer availability, we don't currently require testing for frontend work, besides critical errors like authentication. That being said, contributors are very welcome to write tests, which will always get run in CI, by with Jest and Cypress.
Whilst we barely use Cypress at the moment, it's an extremely powerful tool, and seems a lot more fun that backend stuff - it puppets a browser instance and everything, try it out!
To run unit tests, run
nx test frontendCypress is disabled in CI for now until we actually start doing some frontend E2E testing (maybe never)
For these we use Cypress, an exceptionally powerful tool for interaction pages and components in an actual web browser.
To run E2E tests, run
nx e2e frontend-e2eIf you want to really have some fun, run in watch mode to bring up Cypress's interactive UI. This will let you see tests being run and elements of the page being interacted with in the browser window Cypress is puppeting.