Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Build with linkcheck=true #856

Merged
merged 8 commits into from
Sep 6, 2024
Merged

[docs] Build with linkcheck=true #856

merged 8 commits into from
Sep 6, 2024

Conversation

giordano
Copy link
Member

@giordano giordano commented Aug 6, 2024

No description provided.

@giordano
Copy link
Member Author

giordano commented Aug 6, 2024

Sadly, this doesn't seem to be viable, because there are loads of MPI functions which appear to be undocumented in MPICH and/or OpenMPI websites (https://github.com/JuliaParallel/MPI.jl/actions/runs/10270632053/job/28418784550):

┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Init_thread.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Finalize.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
[...]
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Accumulate_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Accumulate_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allgather_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Allgather_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allgather_init.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allgather_init_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Allgather_init_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allgatherv_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Allgatherv_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allgatherv_init.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allgatherv_init_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Allgatherv_init_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allreduce_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Allreduce_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allreduce_init.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Allreduce_init_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Allreduce_init_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Alltoall_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Alltoall_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Alltoall_init.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Alltoall_init_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Alltoall_init_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Alltoallv_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/latest/www3/MPI_Alltoallv_c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Alltoallv_init.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.open-mpi.org/doc/current/man3/MPI_Alltoallv_init_c.3.php' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
[...]

For example, documentation for MPI_Init_thread in MPICH seems to have been removed between v4.0 and v4.1 (see also https://www.mpich.org/static/docs/v4.0/ vs https://www.mpich.org/static/docs/v4.1/). Edit: missing MPICH manpages will be fixed by pmodels/mpich#7107. See also pmodels/mpich#7089

@giordano giordano closed this Aug 6, 2024
@giordano giordano deleted the mg/docs-linkcheck branch August 6, 2024 17:08
@giordano giordano mentioned this pull request Aug 6, 2024
@giordano giordano restored the mg/docs-linkcheck branch August 6, 2024 20:26
@giordano
Copy link
Member Author

giordano commented Aug 6, 2024

Actually... I think OpenMPI website moved to https://docs.open-mpi.org/en/main/man-openmpi/man3/$fname.3.html

@giordano giordano reopened this Aug 6, 2024
@giordano giordano force-pushed the mg/docs-linkcheck branch 4 times, most recently from 125f170 to 004a178 Compare August 6, 2024 20:51
@giordano
Copy link
Member Author

giordano commented Aug 6, 2024

With latest changes, missing docstrings reduced to

 ┌ Error: linkcheck 'https://www.mpich.org/static/docs/v4.0/www3/MPI_Status_c2f.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/v4.0/www3/MPI_Status_f2c.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/v4.0/www3/MPI_Type_create_f90_complex.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/v4.0/www3/MPI_Type_create_f90_integer.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44
┌ Error: linkcheck 'https://www.mpich.org/static/docs/v4.0/www3/MPI_Type_create_f90_real.html' status: 404.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44

Edit: all autogenerated links have been fixed by #859.

@giordano giordano force-pushed the mg/docs-linkcheck branch from 004a178 to 84f8a3c Compare August 6, 2024 21:02
@giordano giordano force-pushed the mg/docs-linkcheck branch 2 times, most recently from 449d417 to add526c Compare August 7, 2024 21:36
@giordano
Copy link
Member Author

giordano commented Aug 7, 2024

Last error, after #854:

┌ Error: linkcheck 'https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html' status: 403.
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/utilities/utilities.jl:44

but I'm confused, locally I get:

$ curl -I 'https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html'
HTTP/2 200
content-type: text/html;charset=utf-8
server: Apache
x-dispatcher: dispatcher2uswest1-b80
x-vhost: publish
last-modified: Wed, 07 Aug 2024 13:09:27 GMT
etag: W/"30f33-61f179eb85c9b-gzip"
expires: Wed, 07 Aug 2024 21:40:56 GMT
cache-control: max-age=0, no-cache
pragma: no-cache
date: Wed, 07 Aug 2024 21:40:56 GMT
set-cookie: detected_bandwidth=LOW; path=/; domain=.intel.com; secure; HttpOnly
set-cookie: src_countrycode=gb; path=/; domain=.intel.com; secure; HttpOnly
alt-svc: h3=":443"; ma=93600
content-security-policy: frame-ancestors 'self' https://premiersupport.intel.com https://c0.avaamo.com *.intel.com; object-src 'self';
x-grn: 0.8d3e1202.1723066856.1047ccba
x-frame-options: SAMEORIGIN
access-control-allow-origin: *
x-xss-protection: 1; mode=block
strict-transport-security: max-age=31536000 ; preload
x-content-type-options: nosniff

I hope they aren't blocking connections from GitHub-hosted runners...

@giordano
Copy link
Member Author

giordano commented Aug 7, 2024

Ah, the problem is the user agent used by Documenter:

$ curl --user-agent "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36" -sI 'https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html'
HTTP/2 403
server: AkamaiGHost
mime-version: 1.0
content-type: text/html
content-length: 460
expires: Wed, 07 Aug 2024 21:46:56 GMT
cache-control: max-age=0, no-cache
pragma: no-cache
date: Wed, 07 Aug 2024 21:46:56 GMT
set-cookie: detected_bandwidth=LOW; path=/; domain=.intel.com; secure; HttpOnly
set-cookie: src_countrycode=gb; path=/; domain=.intel.com; secure; HttpOnly
alt-svc: h3=":443"; ma=93600
content-security-policy: frame-ancestors 'self' https://premiersupport.intel.com https://c0.avaamo.com *.intel.com; object-src 'self';
x-grn: 0.8d3e1202.1723067216.1050300f
x-frame-options: SAMEORIGIN
access-control-allow-origin: *
x-xss-protection: 1; mode=block
strict-transport-security: max-age=31536000 ; preload
x-content-type-options: nosniff

Edit: issue reported at JuliaDocs/Documenter.jl#2557

@giordano giordano marked this pull request as ready for review September 6, 2024 09:42
@giordano giordano merged commit 80e84d3 into master Sep 6, 2024
2 of 3 checks passed
@giordano giordano deleted the mg/docs-linkcheck branch September 6, 2024 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant