add basic e2e testing setup

.github/workflows/e2e-tests.yml

@@ -0,0 +1,59 @@
+name: E2E Tests
+
+# suppress warning raised by https://github.com/jupyter/jupyter_core/pull/292
+env:
+  JUPYTER_PLATFORM_DIRS: '1'
+
+on:
+  push:
+    branches: main
+  pull_request:
+    branches: '*'
+
+jobs:
+  e2e-tests:
+    name: Linux
+    runs-on: ubuntu-latest
+
+    env:
+      PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/pw-browsers
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+
+      - name: Base Setup
+        uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
+
+      - name: Install extension dependencies and build the extension
+        run: ./scripts/install.sh
+
+      - name: Install ui-tests dependencies
+        working-directory: ui-tests
+        env:
+          PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
+        run: jlpm install
+
+      - name: Set up browser cache
+        uses: actions/cache@v2
+        with:
+          path: |
+            ${{ github.workspace }}/pw-browsers
+          key: ${{ runner.os }}-${{ hashFiles('ui-tests/yarn.lock') }}
+
+      - name: Install browser
+        working-directory: ui-tests
+        run: jlpm install-chromium
+
+      - name: Execute integration tests
+        working-directory: ui-tests
+        run: jlpm test
+
+      - name: Upload Playwright Test report
+        if: always()
+        uses: actions/upload-artifact@v2
+        with:
+          name: jupyter-ai-playwright-tests-linux
+          path: |
+            ui-tests/test-results
+            ui-tests/playwright-report

ui-tests/README.md

@@ -0,0 +1,167 @@
+# Integration Testing
+
+This folder contains the integration tests of the extension.
+
+They are defined using [Playwright](https://playwright.dev/docs/intro) test runner
+and [Galata](https://github.com/jupyterlab/jupyterlab/tree/master/galata) helper.
+
+The Playwright configuration is defined in [playwright.config.js](./playwright.config.js).
+
+The JupyterLab server configuration to use for the integration test is defined
+in [jupyter_server_test_config.py](./jupyter_server_test_config.py).
+
+The default configuration will produce video for failing tests and an HTML report.
+
+> There is a new experimental UI mode that you may fall in love with; see [that video](https://www.youtube.com/watch?v=jF0yA-JLQW0).
+
+## Run the tests
+
+> All commands are assumed to be executed from the root directory
+
+To run the tests, you need to:
+
+1. Compile the extension:
+
+```sh
+jlpm install
+jlpm build:prod
+```
+
+> Check the extension is installed in JupyterLab.
+
+2. Install test dependencies (needed only once):
+
+```sh
+cd ./ui-tests
+jlpm install
+jlpm playwright install
+cd ..
+```
+
+3. Execute the [Playwright](https://playwright.dev/docs/intro) tests:
+
+```sh
+cd ./ui-tests
+jlpm playwright test
+```
+
+Test results will be shown in the terminal. In case of any test failures, the test report
+will be opened in your browser at the end of the tests execution; see
+[Playwright documentation](https://playwright.dev/docs/test-reporters#html-reporter)
+for configuring that behavior.
+
+## Update the tests snapshots
+
+> All commands are assumed to be executed from the root directory
+
+If you are comparing snapshots to validate your tests, you may need to update
+the reference snapshots stored in the repository. To do that, you need to:
+
+1. Compile the extension:
+
+```sh
+jlpm install
+jlpm build:prod
+```
+
+> Check the extension is installed in JupyterLab.
+
+2. Install test dependencies (needed only once):
+
+```sh
+cd ./ui-tests
+jlpm install
+jlpm playwright install
+cd ..
+```
+
+3. Execute the [Playwright](https://playwright.dev/docs/intro) command:
+
+```sh
+cd ./ui-tests
+jlpm playwright test -u
+```
+
+> Some discrepancy may occurs between the snapshots generated on your computer and
+> the one generated on the CI. To ease updating the snapshots on a PR, you can
+> type `please update playwright snapshots` to trigger the update by a bot on the CI.
+> Once the bot has computed new snapshots, it will commit them to the PR branch.
+
+## Create tests
+
+> All commands are assumed to be executed from the root directory
+
+To create tests, the easiest way is to use the code generator tool of playwright:
+
+1. Compile the extension:
+
+```sh
+jlpm install
+jlpm build:prod
+```
+
+> Check the extension is installed in JupyterLab.
+
+2. Install test dependencies (needed only once):
+
+```sh
+cd ./ui-tests
+jlpm install
+jlpm playwright install
+cd ..
+```
+
+3. Start the server:
+
+```sh
+cd ./ui-tests
+jlpm start
+```
+
+4. Execute the [Playwright code generator](https://playwright.dev/docs/codegen) in **another terminal**:
+
+```sh
+cd ./ui-tests
+jlpm playwright codegen localhost:8888
+```
+
+## Debug tests
+
+> All commands are assumed to be executed from the root directory
+
+To debug tests, a good way is to use the inspector tool of playwright:
+
+1. Compile the extension:
+
+```sh
+jlpm install
+jlpm build:prod
+```
+
+> Check the extension is installed in JupyterLab.
+
+2. Install test dependencies (needed only once):
+
+```sh
+cd ./ui-tests
+jlpm install
+jlpm playwright install
+cd ..
+```
+
+3. Execute the Playwright tests in [debug mode](https://playwright.dev/docs/debug):
+
+```sh
+cd ./ui-tests
+jlpm playwright test --debug
+```
+
+## Upgrade Playwright and the browsers
+
+To update the web browser versions, you must update the package `@playwright/test`:
+
+```sh
+cd ./ui-tests
+jlpm up "@playwright/test"
+jlpm playwright install
+```

ui-tests/jupyter_server_test_config.py

@@ -0,0 +1,12 @@
+"""Server configuration for integration tests.
+
+!! Never use this configuration in production because it
+opens the server to the world and provide access to JupyterLab
+JavaScript objects through the global window variable.
+"""
+from jupyterlab.galata import configure_jupyter_server
+
+configure_jupyter_server(c)
+
+# Uncomment to set server log level to debug level
+# c.ServerApp.log_level = "DEBUG"

ui-tests/package.json

@@ -0,0 +1,16 @@
+{
+  "name": "jupyter-ai-e2e-tests",
+  "version": "1.0.0",
+  "description": "Jupyter AI E2E tests",
+  "private": true,
+  "scripts": {
+    "start": "jupyter lab --config jupyter_server_test_config.py",
+    "install-chromium": "jlpm playwright install chromium",
+    "test": "jlpm playwright test",
+    "test:update": "jlpm playwright test --update-snapshots"
+  },
+  "devDependencies": {
+    "@jupyterlab/galata": "^5.0.5",
+    "@playwright/test": "^1.37.0"
+  }
+}

ui-tests/playwright.config.js

@@ -0,0 +1,14 @@
+/**
+ * Configuration for Playwright using default from @jupyterlab/galata
+ */
+const baseConfig = require('@jupyterlab/galata/lib/playwright-config');
+
+module.exports = {
+  ...baseConfig,
+  webServer: {
+    command: 'jlpm start',
+    url: 'http://localhost:8888/lab',
+    timeout: 120 * 1000,
+    reuseExistingServer: !process.env.CI
+  }
+};

ui-tests/tests/jupyter-ai.spec.ts

@@ -0,0 +1,13 @@
+import { expect, test } from '@jupyterlab/galata';
+
+/**
+ * Don't load JupyterLab webpage before running the tests.
+ * This is required to ensure we capture all log messages.
+ */
+test.use({ autoGoto: false });
+
+test.describe('Jupyter AI', () => {
+  test('one equals one', async () => {
+    expect(1 === 1);
+  });
+});