-
Notifications
You must be signed in to change notification settings - Fork 62
Testing
We use Jest as our primary testing framework, for unit testing throughout the repo, and backend E2E testing (with the help of light-my-request). For frontend component and E2E testing we use Cypress.
Due to the volunteer nature of this project our test coverage isn't as high as we'd like. Currently, we have good E2E testing on the backend, along with some convenient utilities for writing E2E tests concisely and validating outgoing DTOs. In some places, we have E2E tests where they could be written more simply as unit tests - for future development we should focus more on unit testing specific service logic.
Testing can of course be tedious, and we appreciate that for those who've just spent considerable time learning the main parts of the stack to work on a first contribution, having to then write tests can feel like a demoralizing additional hurdle. That being said, if you're experienced with writing tests for other languages/frameworks already, this is easy to pick up. If you're not, being able to write good tests is a vital part of a programmer's repertoire; tests are a completely fundamental part of good web systems. Try to keep tests in mind as you write your code (or, write the tests first!) and you'll often find it to keep the purpose of your code in mind, as well as possible edge cases it should handle.
For contributors, E2E testing the handling of all query parameters and other potential inputs is a must, as well as any complex service logic (ideally unit tests, though you can follow our current misguided approach of using E2E if you really want).
Unit tests live in .spec.ts
files inside the backend
project.
As above, unit testing is currently lacking, but examples can be found in the auth service. We take a very conventional approach here, the Nest docs page should be a good reference.
We include the @golevelup/ts-jest package which makes mocking extremely convenient.
To run unit tests, run
nx test backend
As an aside, one of the strongest areas of AI programming currently (in my opinion) is unit testing. GPT-4 (Microsoft's Bing chat thing provides access for free) can often be given entire methods/functions and will produce decent tests. I suggest you do not do this unless you're already confident in your ability to write tests, but if you are at that point, you may find it a major time-save.
E2E tests live in the backend-e2e
project, structured based on the outermost endpoint name (e.g. maps, user, users).
To run E2E tests, run
nx e2e backend-e2e
E2E tests can be run with the
TEST_LOG_DEBUG=true
to make the exception filter emit more detailed logs, including the error messages for any 4XX/5XX responses. This can be helpful for finding bugs without debugging, but spams stdout. Best to use for individual tests!
They heavily use the backend-test-utils
library, which contains the following tools
Provides a wrapper around Fastify's light-my-request
tool. This is designed to vastly improve the amount of boilerplate in our old tests, and allow constructing queries, passing access tokens, etc. easily.
In particular, it uses our custom toBeValidDto
decorator heavily, which uses class-transformer
and class-validator
to reconstruct DTO classes from the JSON the API responds with, essentially performing DTO validation on outgoing data, rather than just incoming.
Handles the database queries we're constantly making as we setup and teardown tests. The vast majority of endpoints on our API require a login to use, and every test suite starts with a clean DB, so this utility provides functions for creating those users, as well as maps, runs, etc.
Since we need a login for each endpoint, this utility makes use of the fact we obviously know the JWT the backend uses, so creates JWTs in exactly the same way the backend does.
Mostly E2E tests check stuff in DB, but s3.util
is provided for testing any endpoints that do file storage related actions.
Let's run through a simple end-to-end test using this system. We'll test some of the expected functionality of the /api/v1/user PATCH
endpoint, which allows logged-in users to update their alias and bio. It's quite simple, so here's the whole thing (db
, req
and prisma
are assigned further up in the file, see user.e2e-spec.ts
):
describe("user", () => {
describe('PATCH', () => {
it("should update the authenticated user's alias", async () => {
const [user, token] = await db.createAndLoginUser();
const newAlias = 'Donkey Kong';
await req.patch({
url: 'user',
body: { alias: newAlias },
token: token,
status: 204
});
const updatedUser = await prisma.user.findFirst({
where: { id: user.id },
include: { profile: true }
});
expect(updatedUser.alias).toBe(newAlias);
});
});
});
Since this lives in a large test file we use blocks of describe
calls to Jest - don't worry about this.
Then, we declare our test with it
(note, in Jest this is identical to test
- God knows why), taking the test name as a string then the actual test in a (async) function.
Inside, we call db.util
to create a user in the database, then generate a JWT access token that the API will accept. We await the returned promise, which returns a tuple of the Prisma User
type, and a token string, which we deconstruct into the user
and token
variables.
"Donkey Kong"
is just a constant we're going to use to test the alias update (Donkey Kong was a major inspiration for Momentum Mod). We could just use the string literal "Donkey Kong"
everywhere, but defining it in one place is just a useful way to avoid future bugs.
Now, req.patch
is doing a lot of heavy lifting for us - this is one of the utilities we use to reduce endless boilerplate in these tests. Let's go through each options:
-
url: 'user'
means this will be a PATCH request toapi/v1/user
(req.patch
handles the start of the URL,v1
is the defaultversion
option) -
body: { alias: newAlias }
is the actual request body.light-my-request
handles serializing this Javascript object to JSON. -
token: token
(justtoken
is actually equivalent here but let's not get too nerdy):req.patch
will attach this to the requestAuthorization
head for us -
status: 204
this is a little different from the rest: this is a test condition. It's saying the response should have status code204
, and to fail if it's anything else
Okay, if runtime execution reaches this point, our request was successful, since it must have responded with 204
. Now we check that the database was actually updated with the new user. yuou could be pedantic here and say that an E2E test should just be testing each "end" and not care about the contents of the database. maybe we should test the database with a separate HTTP query? to this I say: go away this is useful leave me alone) prisma
is just an instance of the Prisma client (but running outside of the backend instance) we can use to do easy DB checks for us. So we use a simple Prisma query to grab the updated user, using the user
's id which obviously we know.
Finally, have dug out the user, we check that the alias was indeed updated using Jest's expect
and toBe
matcher.
On the frontend (which hasn't been subject to a rewrite), tests are currently quite lacking.
The data layer and some components have unit tests, but we have no significant component or E2E testing (the Cypress setup is essentially stubbed).
Tests are not a hard requirement for contributors, though we hope to do far more in the future. Cypress was only just added, which should make E2E testing a lot easier (it's not like a dreary backend E2E where it's just looking at JSON, it puppets a browser instance and everything, it's really fun!)
To run unit tests, run
nx test frontend
For these we use Cypress, an exceptionally powerful tool for interaction pages and components in an actual web browser. To run E2E tests, run
nx e2e frontend-e2e
If you want to really have some fun, run in watch mode to bring up Cypress's interactive UI. This will let you see tests being run and elements of the page being interacted with in the browser window Cypress is puppeting.