Automated Testing

[jacksteamdev]

As your Chrome extension grows, it will become more difficult to guard against regressions. Manual testing isn't enough, but it's hard to know where to start with automated testing.

• What has worked?
• What hasn't worked?
• What testing suite do you use?
• How do you structure your extension to make it testable?

Please share your experience testing a Chrome extension!

[jacksteamdev]

I've been using Playwright to do end-to-end testing for CRXJS. You can check out their Chrome extension guide here: https://playwright.dev/docs/chrome-extensions

Jest vs Vitest

Until recently CRXJS has used Jest for testing. Jest uses CommonJS under the hood, and Vite 3 is an ESM only package. After trying to get things working with Jest's experimental ESM support, I found that Jest just wouldn't work with Vite. The people from Vite have created a test framework called Vitest, and OFC it works with Vite 3.

The next release of CRXJS uses Vitest. The CRXJS tests were flaky with Jest. Some e2e tests would randomly stall, especially in CI. I've found Vitest to be faster and more reliable than Jest for my purposes. Vitest's feature set isn't as complete as Jest, but it has most of the features you'd need for automated testing.

Test Setup

I suggest starting with something like this in either Vitest or Jest:

1. Build the extension in a global setup file. We don't need to build the extension before each test.
2. Use Playwright to load the extension into Chromium. You could abstract this step into a function, since it's going to be the same for each test.
3. Use the Playwright API to open pages and make assertions on the UI elements.
4. Use jest-image-snapshot to compare screenshots for visual regression testing. This works with Vitest as well as Jest.

Playwright also provides mocking for network requests, which is useful for testing background events. I haven't explored testing the service worker in MV3, so I'd love to hear your experience with this.

Other Options

Another option is the Playwright test framework. I haven't tried this myself, but the documentation looks promising.