Unify docs workflows #348

Czaki · 2024-02-01T11:16:32Z

Description

Closes: #284

This PR:

Unify build workflows from two to one.

Remove .doctree directory (save 400MB)

melissawm

Looks good, thank you!

psobolewskiPhD · 2024-02-14T21:33:14Z

Should delete the .doctrees from the uploaded artifact IMO. They aren't needed for anything related to the html as far as I understand.

Czaki · 2024-02-15T11:03:06Z

I do not know. Maybe they are usefull for some debug. I'm not such familiar with sphinx

psobolewskiPhD · 2024-02-15T16:34:54Z

here's the info:

Since Sphinx has to read and parse all source files before it can write an output file, the parsed source files are cached as “doctree pickles”. Normally, these files are put in a directory called .doctrees under the build directory; with this option you can select a different cache directory (the doctrees can be shared between all builders).

https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-d

They aren't needed for the website. I guess instead of delete, we could change the dir to not put them in the folder used for the artifact?

jni

maybe rename the workflow to build_and_deploy.yml?

jni · 2024-02-20T01:13:44Z

And I agree with @psobolewskiPhD that maybe we should just configure the .doctree dir to be in /tmp or something. And/or make a PR to sphinx to not put it in the build in the first place. 😂

jni · 2024-02-27T11:52:51Z

@Czaki

maybe rename the workflow to build_and_deploy.yml?

maybe we should just configure the .doctree dir to be in /tmp or something.

Do you want to implement either or both of these, or none?

psobolewskiPhD · 2024-02-28T21:28:43Z

I did some more digging around the sphinx docs, readthedocs docs, etc.
The defaults these days is to use -M instead of -b which puts the doctrees separate from the html. See:
https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-M
(html is the default builder)
So the you'd get:

docs/_build/html
docs/_build/doctrees

See for example:
https://www.sphinx-doc.org/en/master/tutorial/deploying.html#id5
where only the html is deployed to Github Pages.

psobolewskiPhD · 2024-02-28T21:49:15Z

I second the motion to change the name to build_and_deploy
I made a PR suggesting what I think the the proper way to handle the doctrees question.

Looking at the circleCI output of that PR, the artifact is ~126MB
https://app.circleci.com/pipelines/github/psobolewskiPhD/napari-docs/4/workflows/8173ddf0-6858-4024-af40-38e2abd91cb8/jobs/4/steps?invite=true#step-107-138189_28

While the circleCI output of this current branch has the artifact ~700 Mb:
https://app.circleci.com/pipelines/github/Czaki/napari-docs/73/workflows/a3651b6d-4eab-4eec-b947-b875e80696c8/jobs/73?invite=true#step-107-169391_28

psobolewskiPhD

I second the motion to rename the new workflow to build_and_deploy.yml

jni · 2024-02-29T06:05:52Z

I made a PR suggesting what I think the the proper way to handle the doctrees question.

Fantastic detective work @psobolewskiPhD! 🕵️ 🤓

This reverts commit a3e7e58.

.github/workflows/build_and_deploy.yml

.circleci/config.yml

.github/workflows/build_and_deploy.yml

psobolewskiPhD · 2024-03-01T02:08:05Z

Per docs here https://github.com/larsoner/circleci-artifacts-redirector-action
The circleCI redirector is expected to not work properly until this is merged.

psobolewskiPhD

I think everything is good now.
@Czaki Can you sanity check the tags bit?

Czaki · 2024-03-02T14:32:47Z

I think everything is good now.
@Czaki Can you sanity check the tags bit?

What tags?

psobolewskiPhD · 2024-03-02T14:55:46Z

This fix:
928551d
to this error:
https://github.com/napari/docs/actions/runs/8103613500

Czaki · 2024-03-02T15:41:47Z

This fix: 928551d to this error: napari/docs/actions/runs/8103613500

You suggest change in getting value from tags and point skipped action (caused by file rename I think).

psobolewskiPhD · 2024-03-02T15:54:30Z

Sorry if I wasn't being clear.
This action was failing with:

The workflow is not valid. .github/workflows/build_and_deploy.yml (Line: 92, Col: 14): Unexpected symbol: 'ref/refs\/tags\//'. Located at position 8 within expression: github.ref/refs\/tags\//

Here's that code section before my fix:

docs/.github/workflows/build_and_deploy.yml

Lines 92 to 97 in 8b695ca

    
                   run: | 
        
                     if [[ "${{ github.ref }}" == "refs/tags/"* ]]; then 
        
                       echo "branch_name=${{ github.ref/refs\/tags\// }}" >> $GITHUB_ENV  
        
                     else 
        
                       echo "branch_name=dev" >> $GITHUB_ENV  
        
                     fi

As far As I can tell, in github.ref/refs\/tags\// /refs\/tags\// is trying to get the name of the tag, so my fix is to use it directly, based on https://docs.github.com/en/actions/learn-github-actions/contexts

psobolewskiPhD · 2024-03-02T15:55:27Z

Because this is for deployment, it's not easy to check, so hence asking you to sanity check before we merge this and find out docs don't deploy right!

melissawm · 2024-03-21T20:21:54Z

It looks like this is good to go, pending @psobolewskiPhD 's comment above? If so, I just want to point out that if the deploy fails it's an easy fix and doesn't break the website (it will only not show the latest version until we fix it). So I'd say this is pretty safe.

psobolewskiPhD · 2024-03-24T19:45:24Z

I pushed the changes to the workflow from the video conversion PR.

melissawm · 2024-03-25T19:59:40Z

I am marking this as ready as I believe it's done - @Czaki last chance to take a look 😄

Czaki · 2024-03-25T21:27:14Z

I think it is ready. Still do not know what with deploy trigger.

# References and relevant issues After merged, docs deployment is failing, see: https://github.com/napari/napari/actions/runs/8569580333/job/23485774671 This is due to the napari/docs workflow change: napari/docs#348 # Description In this PR I change the name of the triggered workflow to the correct new name.

* Update `docs_deployment.md` now that we unified workflows in #348. * Change workflow name to be Build & Deploy PR Docs, since #348 * Add file paths to the workflow to aid other working on it --------- Co-authored-by: Peter Sobolewski <[email protected]>

#6814) # References and relevant issues I noticed CircleCI docs redirector is not working since napari/docs#348 See, e.g. https://output.circle-artifacts.com/output/job/f7b7b3bf-0e97-4ca0-a6dd-2f30998c625d/artifacts/0/docs/docs/_build/index.html?pr=6807 The docs build correctly, but the structure is different because of the change to the Makefile to put the html in `/html` # Description Updates config.yml to match the one in napari/docs https://github.com/napari/docs/blob/main/.circleci/config.yml and also fixes the redirector workflow to point to the right location.

unify docs workflows

07f9789

Czaki added the maintenance CI, dependencies, and other maintenance label Feb 1, 2024

Czaki added this to the 0.5.0 milestone Feb 1, 2024

Czaki requested a review from melissawm February 1, 2024 11:16

github-actions bot assigned Czaki Feb 1, 2024

github-actions bot added the task label Feb 1, 2024

melissawm approved these changes Feb 1, 2024

View reviewed changes

add missed enter

ff3d45e

jni approved these changes Feb 20, 2024

View reviewed changes

psobolewskiPhD added 2 commits February 28, 2024 16:33

use -M for html instead of -b

5eb3225

Only upload/use /_build/html to avoid doctrees

858d053

psobolewskiPhD mentioned this pull request Feb 28, 2024

Use -M sphinx-build flag to split location of doctrees and html Czaki/napari-docs#3

Merged

psobolewskiPhD requested changes Feb 28, 2024

View reviewed changes

rename workflow file

a3e7e58

Czaki added 3 commits February 29, 2024 22:41

Revert "rename workflow file"

b4baf2c

This reverts commit a3e7e58.

Merge pull request #3 from psobolewskiPhD/fix_docs_build

e6c76f6

rename workflow file

8b695ca

psobolewskiPhD reviewed Mar 1, 2024

View reviewed changes

.github/workflows/build_and_deploy.yml Outdated Show resolved Hide resolved

psobolewskiPhD reviewed Mar 1, 2024

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

psobolewskiPhD reviewed Mar 1, 2024

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

psobolewskiPhD added 2 commits February 29, 2024 20:38

fix circleCI action

f362f46

fix .github/workflows/build_and_deploy.yml

1e538d9

psobolewskiPhD reviewed Mar 1, 2024

View reviewed changes

.github/workflows/build_and_deploy.yml Outdated Show resolved Hide resolved

fix tags dir name for real?

928551d

psobolewskiPhD approved these changes Mar 1, 2024

View reviewed changes

bump actions

cd76a8e

psobolewskiPhD and others added 2 commits March 24, 2024 12:38

fix setup.cfg to pyproject.toml

5996c43

Add fall-back videos action

4193e61

Merge branch 'main' into fix_docs_build

07b43cf

melissawm added the ready to merge label Mar 25, 2024

psobolewskiPhD merged commit a77d2eb into napari:main Mar 27, 2024
7 checks passed

psobolewskiPhD removed the ready to merge label Mar 27, 2024

Czaki deleted the fix_docs_build branch March 27, 2024 20:41

This was referenced Mar 28, 2024

[Maint] Update build_napari.yml to drop doctrees napari/napari-sphinx-theme#160

Merged

[Maint] fix the download and deploy part of workflow #380

Merged

This was referenced Apr 5, 2024

Latest build_deploy workflow deleted dev on deploy to gh-pages #383

Closed

[maint] Fix deploy_docs.yml for change in napari/docs workflow napari/napari#6809

Merged

lucyleeow mentioned this pull request Apr 6, 2024

Update docs_deployment.md and add paths to workflow #390

Merged

psobolewskiPhD mentioned this pull request Apr 6, 2024

[Maint] Update CircleCI config.yml for changes in napari/docs Makefile napari/napari#6814

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unify docs workflows #348

Unify docs workflows #348

Czaki commented Feb 1, 2024 •

edited by psobolewskiPhD

Loading

melissawm left a comment

psobolewskiPhD commented Feb 14, 2024

Czaki commented Feb 15, 2024

psobolewskiPhD commented Feb 15, 2024

jni left a comment

jni commented Feb 20, 2024

jni commented Feb 27, 2024

psobolewskiPhD commented Feb 28, 2024

psobolewskiPhD commented Feb 28, 2024 •

edited

Loading

psobolewskiPhD left a comment

jni commented Feb 29, 2024

psobolewskiPhD commented Mar 1, 2024

psobolewskiPhD left a comment

Czaki commented Mar 2, 2024

psobolewskiPhD commented Mar 2, 2024

Czaki commented Mar 2, 2024

psobolewskiPhD commented Mar 2, 2024

psobolewskiPhD commented Mar 2, 2024

melissawm commented Mar 21, 2024

psobolewskiPhD commented Mar 24, 2024

melissawm commented Mar 25, 2024

Czaki commented Mar 25, 2024

Unify docs workflows #348

Unify docs workflows #348

Conversation

Czaki commented Feb 1, 2024 • edited by psobolewskiPhD Loading

Description

melissawm left a comment

Choose a reason for hiding this comment

psobolewskiPhD commented Feb 14, 2024

Czaki commented Feb 15, 2024

psobolewskiPhD commented Feb 15, 2024

jni left a comment

Choose a reason for hiding this comment

jni commented Feb 20, 2024

jni commented Feb 27, 2024

psobolewskiPhD commented Feb 28, 2024

psobolewskiPhD commented Feb 28, 2024 • edited Loading

psobolewskiPhD left a comment

Choose a reason for hiding this comment

jni commented Feb 29, 2024

psobolewskiPhD commented Mar 1, 2024

psobolewskiPhD left a comment

Choose a reason for hiding this comment

Czaki commented Mar 2, 2024

psobolewskiPhD commented Mar 2, 2024

Czaki commented Mar 2, 2024

psobolewskiPhD commented Mar 2, 2024

psobolewskiPhD commented Mar 2, 2024

melissawm commented Mar 21, 2024

psobolewskiPhD commented Mar 24, 2024

melissawm commented Mar 25, 2024

Czaki commented Mar 25, 2024

Czaki commented Feb 1, 2024 •

edited by psobolewskiPhD

Loading

psobolewskiPhD commented Feb 28, 2024 •

edited

Loading