Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jupyterlab extension #234

Merged
merged 103 commits into from
Oct 13, 2023
Merged
Show file tree
Hide file tree
Changes from 76 commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
bc3db54
initial jupyterlab extension from https://github.com/jupyterlab/exten…
mmarchetti Sep 12, 2023
89505d8
ignore local env/
mmarchetti Sep 12, 2023
1b00a91
update instructions
mmarchetti Sep 12, 2023
289efc3
add yarn.lock
mmarchetti Sep 12, 2023
47bacb6
add to notebook toolbar
mmarchetti Sep 14, 2023
7100d21
handler cleanup
mmarchetti Sep 14, 2023
7fdd29b
get python path and version and call API
mmarchetti Sep 15, 2023
dd39606
cleanup
mmarchetti Sep 19, 2023
9788b7d
wip
mmarchetti Sep 20, 2023
1fb7f40
disable gitignore until we get a handle on performance
mmarchetti Sep 21, 2023
ca3130f
remove debug code
mmarchetti Sep 21, 2023
7b68f76
add profiling option
mmarchetti Sep 21, 2023
61f85e8
launch the agent
mmarchetti Sep 21, 2023
ac7d737
add .ipynb_checkpoints to default exclusions
mmarchetti Sep 21, 2023
df0f722
restore default listen address
mmarchetti Sep 21, 2023
ff6deb8
set SameSite:None
mmarchetti Sep 21, 2023
9aa8ffd
use agent auth
mmarchetti Sep 21, 2023
d3fd612
ignore .posit by default
mmarchetti Sep 21, 2023
4578da8
only visit ancestor gitignores
mmarchetti Sep 21, 2023
9a4396f
requirements file for example notebook
mmarchetti Sep 21, 2023
1f6cab0
log errors in console
mmarchetti Sep 25, 2023
c3915af
python formatting
mmarchetti Sep 25, 2023
966eac5
ensure process.stdout
mmarchetti Sep 25, 2023
b826425
set black line length for vscode
mmarchetti Sep 25, 2023
7fb5ccc
add justfile with python checks
mmarchetti Sep 25, 2023
37c5f01
type annotations
mmarchetti Sep 25, 2023
72bac84
frontend validation
mmarchetti Sep 25, 2023
0743690
add CI code check
mmarchetti Sep 25, 2023
8b5bb4d
install python tools in CI
mmarchetti Sep 25, 2023
ca0d8f4
fix indentation in GHA
mmarchetti Sep 25, 2023
f9f1f37
test
mmarchetti Sep 25, 2023
ba0820b
Merge branch 'main' into mm-jupyterlab
mmarchetti Sep 25, 2023
3dee592
setup just
mmarchetti Sep 25, 2023
d89c663
install python dependencies
mmarchetti Sep 26, 2023
f853754
remove unused functions from gitignore
mmarchetti Sep 26, 2023
8917800
terminate git search
mmarchetti Sep 26, 2023
72d2f3c
preinstall jupyterlab
mmarchetti Sep 26, 2023
81ded71
check error from gitignore
mmarchetti Sep 26, 2023
62f5f32
install jupyterlab[dev]
mmarchetti Sep 26, 2023
d85a33d
run in virtualenv
mmarchetti Sep 26, 2023
188ccd8
proxy to the agent
mmarchetti Sep 26, 2023
43fac6e
rewrite redirects and cookies
mmarchetti Sep 26, 2023
75c00b2
update yarn.lock
mmarchetti Sep 26, 2023
782a232
lint
mmarchetti Sep 26, 2023
f95a5ab
suppress mypy warning
mmarchetti Sep 26, 2023
bebadf0
run lint with jlpm
mmarchetti Sep 26, 2023
884b191
set yarn version
mmarchetti Sep 26, 2023
cdb7ddc
set base URL for UI and API
mmarchetti Sep 26, 2023
9d3aa9a
Revert "set yarn version"
mmarchetti Sep 26, 2023
aa708a0
use jlpm to install
mmarchetti Sep 26, 2023
d159b0e
sidebar wip
mmarchetti Sep 27, 2023
50c2d0a
cleanup
mmarchetti Sep 28, 2023
ceb1fe5
theme
mmarchetti Sep 28, 2023
5dd19a1
use SameSite:Lax
mmarchetti Sep 28, 2023
6f21a37
improve error handling
mmarchetti Sep 28, 2023
8500ea5
retain open iframes
mmarchetti Sep 28, 2023
756a069
lint
mmarchetti Sep 29, 2023
d2b9cac
run python tests
mmarchetti Sep 29, 2023
5410dfa
make conda environment elsewhere
mmarchetti Sep 29, 2023
1fd4ab7
install test deps
mmarchetti Sep 29, 2023
211e987
run frontend tests
mmarchetti Sep 29, 2023
d411dea
initial python tests
mmarchetti Sep 29, 2023
2805450
example non-python notebook
mmarchetti Sep 29, 2023
efc01d3
error test
mmarchetti Sep 29, 2023
c28df30
proxy tests
mmarchetti Sep 29, 2023
ea020b9
no typing for jupyter_server_proxy
mmarchetti Oct 2, 2023
ebadc3e
Merge branch 'main' into mm-jupyterlab
mmarchetti Oct 2, 2023
9b69fdc
reuse existing agents
mmarchetti Oct 2, 2023
6bd48de
tests
mmarchetti Oct 2, 2023
a0060f2
lint
mmarchetti Oct 2, 2023
a0d4d1c
move dev dependencies to pyproject.toml
mmarchetti Oct 2, 2023
9793d5e
wait for kernel if none yet
mmarchetti Oct 2, 2023
f0ff6c0
don't fail on empty notebooks
mmarchetti Oct 4, 2023
d4c880d
remove vscode settings and gitignore it
mmarchetti Oct 5, 2023
9790d06
provide a default title
mmarchetti Oct 5, 2023
fbc6871
title tests
mmarchetti Oct 5, 2023
f429c7d
remove manual release instructions
mmarchetti Oct 12, 2023
4236f6c
remove Playwright test setup
mmarchetti Oct 12, 2023
0a1331c
remove .copier-answers.yml
mmarchetti Oct 12, 2023
24076f5
remove playwright setup from example CI build.yml
mmarchetti Oct 12, 2023
5507140
remove unused example CI steps
mmarchetti Oct 12, 2023
66ad638
add README to example github workflow directory
mmarchetti Oct 12, 2023
e48d0e8
remove playwright from README
mmarchetti Oct 12, 2023
640f8e6
remove unused GET handler
mmarchetti Oct 12, 2023
6b71521
deploy to dogfood by default for now
mmarchetti Oct 12, 2023
de70a46
add note about putting agent on PATH
mmarchetti Oct 12, 2023
630e147
remove duplicate dependency on jupyterlab
mmarchetti Oct 12, 2023
84be05d
move kernel functions to a separate file
mmarchetti Oct 13, 2023
8e72c17
Merge branch 'main' into mm-jupyterlab
mmarchetti Oct 13, 2023
4af932b
install js deps first
mmarchetti Oct 13, 2023
2a643a7
Revert "install js deps first"
mmarchetti Oct 13, 2023
f08efeb
setup python and node
mmarchetti Oct 13, 2023
197f2d9
remove virtualenv
mmarchetti Oct 13, 2023
789d183
preinstall JupyterLab
mmarchetti Oct 13, 2023
950f0c8
no prettier for markdown files
mmarchetti Oct 13, 2023
b64107f
update test for default server
mmarchetti Oct 13, 2023
2a044c2
remove test for GET endpoint
mmarchetti Oct 13, 2023
15529de
verbose pip install for debugging
mmarchetti Oct 13, 2023
75ea48c
use jupyter example setup in CI
mmarchetti Oct 13, 2023
277cd90
add jlpm install
mmarchetti Oct 13, 2023
b5a40a0
jlpm install after jupyterlab installation
mmarchetti Oct 13, 2023
fd9ebe7
update yarn lockfile
mmarchetti Oct 13, 2023
85c48e6
install python deps
mmarchetti Oct 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/pull-request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -685,3 +685,35 @@ jobs:
# run: |
# just web/build && \
# just test/ui-client ${os}

validate-jupyterlab-plugin:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./extensions/jupyterlab/connect_jupyterlab
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Setup Just
uses: extractions/setup-just@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Create virtualenv
mmarchetti marked this conversation as resolved.
Show resolved Hide resolved
run: |
python -m venv ./env
. env/bin/activate
echo PATH=$PATH >> $GITHUB_ENV
-
name: Install JupyterLab Python extension
run: just install
-
name: Install JS dependencies
run: jlpm install
-
name: Validate extension code
run: just validate
-
name: Test extension code
run: just test
9 changes: 9 additions & 0 deletions cmd/connect-client/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ package main

import (
"os"
"runtime/pprof"

"github.com/alecthomas/kong"
"github.com/rstudio/connect-client/cmd/connect-client/commands"
Expand Down Expand Up @@ -63,6 +64,14 @@ func main() {
}
// Dispatch to the Run() method of the selected command.
args := kong.Parse(&cli, kong.Bind(ctx))
if cli.Profile != "" {
f, err := os.Create(cli.Profile)
if err != nil {
Fatal(log, "Error creating file for profile data", err)
}
pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()
}
if cli.Debug {
ctx.Logger = events.NewLogger(true)
}
Expand Down
19 changes: 19 additions & 0 deletions extensions/jupyterlab/connect_jupyterlab/.copier-answers.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Changes here will be overwritten by Copier; NEVER EDIT MANUALLY
mmarchetti marked this conversation as resolved.
Show resolved Hide resolved
_commit: v4.2.0
_src_path: https://github.com/jupyterlab/extension-template
author_email: [email protected]
author_name: Michael Marchetti
data_format: string
file_extension: ''
has_binder: false
has_settings: false
kind: server
labextension_name: connect_jupyterlab
mimetype: ''
mimetype_name: ''
project_short_description: A JupyterLab extension for publishing to Posit Connect.
python_name: connect_jupyterlab
repository: https://github.com/rstudio/publishing-client
test: true
viewer_name: ''

155 changes: 155 additions & 0 deletions extensions/jupyterlab/connect_jupyterlab/.github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
name: Build

on:
push:
branches: main
pull_request:
branches: '*'

jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Base Setup
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

- name: Install dependencies
run: python -m pip install -U "jupyterlab>=4.0.0,<5"

- name: Lint the extension
run: |
set -eux
jlpm
jlpm run lint:check

- name: Test the extension
run: |
set -eux
jlpm run test

- name: Build the extension
run: |
set -eux
python -m pip install .[test]

pytest -vv -r ap --cov connect_jupyterlab
jupyter server extension list
jupyter server extension list 2>&1 | grep -ie "connect_jupyterlab.*OK"

jupyter labextension list
jupyter labextension list 2>&1 | grep -ie "connect_jupyterlab.*OK"
python -m jupyterlab.browser_check

- name: Package the extension
run: |
set -eux

pip install build
python -m build
pip uninstall -y "connect_jupyterlab" jupyterlab

- name: Upload extension packages
uses: actions/upload-artifact@v3
with:
name: extension-artifacts
path: dist/connect_jupyterlab*
if-no-files-found: error

test_isolated:
needs: build
runs-on: ubuntu-latest

steps:
- name: Install Python
uses: actions/setup-python@v4
with:
python-version: '3.9'
architecture: 'x64'
- uses: actions/download-artifact@v3
with:
name: extension-artifacts
- name: Install and Test
run: |
set -eux
# Remove NodeJS, twice to take care of system and locally installed node versions.
sudo rm -rf $(which node)
sudo rm -rf $(which node)

pip install "jupyterlab>=4.0.0,<5" connect_jupyterlab*.whl


jupyter server extension list
jupyter server extension list 2>&1 | grep -ie "connect_jupyterlab.*OK"

jupyter labextension list
jupyter labextension list 2>&1 | grep -ie "connect_jupyterlab.*OK"
python -m jupyterlab.browser_check --no-browser-test

integration-tests:
mmarchetti marked this conversation as resolved.
Show resolved Hide resolved
name: Integration tests
needs: build
runs-on: ubuntu-latest

env:
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/pw-browsers

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Base Setup
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

- name: Download extension package
uses: actions/download-artifact@v3
with:
name: extension-artifacts

- name: Install the extension
run: |
set -eux
python -m pip install "jupyterlab>=4.0.0,<5" connect_jupyterlab*.whl

- name: Install dependencies
working-directory: ui-tests
env:
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
run: jlpm install

- name: Set up browser cache
uses: actions/cache@v3
with:
path: |
${{ github.workspace }}/pw-browsers
key: ${{ runner.os }}-${{ hashFiles('ui-tests/yarn.lock') }}

- name: Install browser
run: jlpm playwright install chromium
working-directory: ui-tests

- name: Execute integration tests
working-directory: ui-tests
run: |
jlpm playwright test

- name: Upload Playwright Test report
if: always()
uses: actions/upload-artifact@v3
with:
name: connect_jupyterlab-playwright-tests
path: |
ui-tests/test-results
ui-tests/playwright-report

check_links:
name: Check Links
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v3
- uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
- uses: jupyterlab/maintainer-tools/.github/actions/check-links@v1
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Check Release
mmarchetti marked this conversation as resolved.
Show resolved Hide resolved
on:
push:
branches: ["main"]
pull_request:
branches: ["*"]

jobs:
check_release:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Base Setup
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
- name: Check Release
uses: jupyter-server/jupyter_releaser/.github/actions/check-release@v2
with:

token: ${{ secrets.GITHUB_TOKEN }}

- name: Upload Distributions
uses: actions/upload-artifact@v3
with:
name: connect_jupyterlab-releaser-dist-${{ github.run_number }}
path: .jupyter_releaser_checkout/dist
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Enforce PR label
mmarchetti marked this conversation as resolved.
Show resolved Hide resolved

on:
pull_request:
types: [labeled, unlabeled, opened, edited, synchronize]
jobs:
enforce-label:
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: enforce-triage-label
uses: jupyterlab/maintainer-tools/.github/actions/enforce-label@v1
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: "Step 1: Prep Release"
on:
workflow_dispatch:
inputs:
version_spec:
description: "New Version Specifier"
default: "next"
required: false
branch:
description: "The branch to target"
required: false
post_version_spec:
description: "Post Version Specifier"
required: false
since:
description: "Use PRs with activity since this date or git reference"
required: false
since_last_stable:
description: "Use PRs with activity since the last stable git tag"
required: false
type: boolean
jobs:
prep_release:
runs-on: ubuntu-latest
steps:
- uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

- name: Prep Release
id: prep-release
uses: jupyter-server/jupyter_releaser/.github/actions/prep-release@v2
with:
token: ${{ secrets.ADMIN_GITHUB_TOKEN }}
version_spec: ${{ github.event.inputs.version_spec }}
post_version_spec: ${{ github.event.inputs.post_version_spec }}
branch: ${{ github.event.inputs.branch }}
since: ${{ github.event.inputs.since }}
since_last_stable: ${{ github.event.inputs.since_last_stable }}

- name: "** Next Step **"
run: |
echo "Optional): Review Draft Release: ${{ steps.prep-release.outputs.release_url }}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: "Step 2: Publish Release"
on:
workflow_dispatch:
inputs:
branch:
description: "The target branch"
required: false
release_url:
description: "The URL of the draft GitHub release"
required: false
steps_to_skip:
description: "Comma separated list of steps to skip"
required: false

jobs:
publish_release:
runs-on: ubuntu-latest
permissions:
# This is useful if you want to use PyPI trusted publisher
# and NPM provenance
id-token: write
steps:
- uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

- name: Populate Release
id: populate-release
uses: jupyter-server/jupyter_releaser/.github/actions/populate-release@v2
with:
token: ${{ secrets.ADMIN_GITHUB_TOKEN }}
branch: ${{ github.event.inputs.branch }}
release_url: ${{ github.event.inputs.release_url }}
steps_to_skip: ${{ github.event.inputs.steps_to_skip }}

- name: Finalize Release
id: finalize-release
env:
# The following are needed if you use legacy PyPI set up
# PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
# PYPI_TOKEN_MAP: ${{ secrets.PYPI_TOKEN_MAP }}
# TWINE_USERNAME: __token__
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
uses: jupyter-server/jupyter-releaser/.github/actions/finalize-release@v2
with:
token: ${{ secrets.ADMIN_GITHUB_TOKEN }}
release_url: ${{ steps.populate-release.outputs.release_url }}

- name: "** Next Step **"
if: ${{ success() }}
run: |
echo "Verify the final release"
echo ${{ steps.finalize-release.outputs.release_url }}

- name: "** Failure Message **"
if: ${{ failure() }}
run: |
echo "Failed to Publish the Draft Release Url:"
echo ${{ steps.populate-release.outputs.release_url }}
Loading