Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow to update JupyterLab dependencies automatically #7281

Merged
merged 14 commits into from
Dec 20, 2024

Conversation

itsmevichu
Copy link
Contributor

@itsmevichu itsmevichu commented Feb 27, 2024

Issue- #7271

Description:
This workflow can be used to update all the jupyterlab dependecies to the latest release. This can also be manually triggered, which takes the JupyterLab version as an input.

Things to re-visit:

  • Github secrets, user, email - to create branch and raise PR.
  • Executing "yarn install" post dependency updates results in a substantial amount of modifications, with over 15,000 additions and 15,000 deletions. - Done
  • Scheduling time for cron job.

The created PR looks like - Sample pull-request

Feel free to adapt the workflow based on your requirements.

Copy link
Contributor

Binder 👈 Launch a Binder on branch itsmevichu/notebook/workflow-update-jupyterlab-dependencies

@jtpio
Copy link
Member

jtpio commented Feb 27, 2024

Thanks @itsmevichu for looking into this!

@jtpio
Copy link
Member

jtpio commented Feb 27, 2024

Executing "yarn install" post dependency updates results in a substantial amount of modifications, with over 15,000 additions and 15,000 deletions.

Maybe it's because it's using the default yarn (version 1), while the repo here uses yarn 3 via jlpm. So we would probably need to install jupyterlab and use jlpm.

@jtpio
Copy link
Member

jtpio commented Mar 8, 2024

Thanks @itsmevichu.

Looks like this needs a lint pass.

@jtpio jtpio added this to the 7.1.x milestone Mar 8, 2024
@itsmevichu
Copy link
Contributor Author

@jtpio The lint check is failing because of the coverage for the scripts. Is it possible for us to exclude the "scripts" folder from coverage checks?

@itsmevichu
Copy link
Contributor Author

itsmevichu commented Mar 8, 2024

@jtpio I am fetching the latest versions of the lab dependencies from the below URL - https://raw.githubusercontent.com/jupyterlab/jupyterlab/v4.1.4/jupyterlab/staging/package.json.

The package "@jupyterlab/testutils" is not present in the above package.json. As a result, only that specific package won't get updated by this workflow.

@jtpio
Copy link
Member

jtpio commented Mar 11, 2024

@jtpio The lint check is failing because of the coverage for the scripts. Is it possible for us to exclude the "scripts" folder from coverage checks?

Yes I think so. An alternative could be to write the scripts in TypeScript and put them alongside other scripts here: https://github.com/jupyter/notebook/tree/main/buildutils/src

The package "@jupyterlab/testutils" is not present in the above package.json. As a result, only that specific package won't get updated by this workflow.

Right it's fine, we only need to update the packages used here in the notebook repo.

@itsmevichu
Copy link
Contributor Author

@jtpio I have converted the scripts to typescript and included them in the buildutils.

Pending tasks:

  1. To refer the appropriate secrets.
    GITHUB_USER: ${{ secrets.G_USER }}
    GITHUB_TOKEN: ${{ secrets.G_TOKEN }}
  2. Cron job timings.

@itsmevichu
Copy link
Contributor Author

@jtpio Are any changes needed in this PR? If not, we could simply utilize this workflow, as it would minimize the effort required to upgrade dependencies with each JupyterLab release.

@krassowski
Copy link
Member

It would be really handy to have this in!

@krassowski
Copy link
Member

It looks like it missed one package, "@jupyterlab/testutils": ~4.2.0

@itsmevichu
Copy link
Contributor Author

@jtpio I am fetching the latest versions of the lab dependencies from the below URL - https://raw.githubusercontent.com/jupyterlab/jupyterlab/v4.1.4/jupyterlab/staging/package.json.

The package "@jupyterlab/testutils" is not present in the above package.json. As a result, only that specific package won't get updated by this workflow.

I noticed the same, but it seems that the @jupyterlab/testutils information is not present in our source package.json.

@krassowski
Copy link
Member

I noticed the same, but it seems that the @jupyterlab/testutils information is not present in our source package.json.

I think it is under devDependencies.

@itsmevichu
Copy link
Contributor Author

I noticed the same, but it seems that the @jupyterlab/testutils information is not present in our source package.json.

I think it is under devDependencies.

I've looked through, it appears to be missing from devDependencies. Can I proceed adding it in the respective jupyterlab's package.json?

@krassowski
Copy link
Member

See this commit: f7f2ead

@itsmevichu
Copy link
Contributor Author

itsmevichu commented Aug 27, 2024

See this commit: f7f2ead

It seems there was a misunderstanding. The source from which we get the latest release version https://raw.githubusercontent.com/jupyterlab/jupyterlab/v4.2.0/jupyterlab/staging/package.json doesn't include the @jupyterlab/testutils package, which is why the script isn't updating it here. I think we need to add this as a devDependency in the source jupyterlab/jupyterlab/staging/package.json first.

@jtpio jtpio modified the milestones: 7.1.x, 7.3.x Oct 22, 2024
@jtpio
Copy link
Member

jtpio commented Dec 19, 2024

Thanks @itsmevichu and sorry for the late reply!

Looks like we could try this workflow for the next updates and iterate on it if needed 👍

The lint failure seems related, I'll push a fix right after.

The other CI failure don't seem related and should hopefully be fixed soon: #7539 (comment)

Copy link
Member

@jtpio jtpio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Let's try it!

@jtpio jtpio merged commit c0ddf01 into jupyter:main Dec 20, 2024
31 checks passed
@jtpio
Copy link
Member

jtpio commented Dec 20, 2024

Looks like it managed to create and push the branch, but failed to open the PR: https://github.com/jupyter/notebook/actions/runs/12435633780/job/34721856783

@jtpio
Copy link
Member

jtpio commented Dec 20, 2024

Opened #7545 to track follow-up improvements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants