Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review of scripts and env. variables provided by members of CRAN #17

Open
statnmap opened this issue Sep 19, 2022 · 6 comments
Open

Review of scripts and env. variables provided by members of CRAN #17

statnmap opened this issue Sep 19, 2022 · 6 comments

Comments

@statnmap
Copy link
Collaborator

statnmap commented Sep 19, 2022

Context

In August 2022, we received some links pointing to resources run by the CRAN team to (1) check incoming tar.gz packages, (2) regularly check all packages on CRAN.
The answers are there: https://github.com/RConsortium/r-repositories-wg/blob/main/Documents/Proposal%20to%20CRAN.md

In this issue, we can discuss the test of the scripts received:

  • What do we learn ?
  • What is needed to make them work ?
  • How are they usable on different OS flavours ?

Scripts

Let's start with the incoming check as there are the first step to pass to go to CRAN.
Then, we may explore regular checks.

Incoming checks

  • Incoming checks are run each time there is a new packages send to CRAN

=> It seems that each member of CRAN runs a different set of test, depending on their OS

Scripts are:

=> It seems that they do not use the --as-cran tag to run the check.

✔️ Tests on Ubuntu 22.04 LTS

Extra steps needed on local computer:

mkdir ~/tmp/
mkdir ~/tmp/scratch
mkdir ~/tmp/CRAN
  • System function getIncoming is used in the R file but does not exist on my system.

=> Need to comment the getIncoming part (L173-182)
=> Put the tar.gz of your package inside the check_dir, which by default is "~/tmp/CRAN"
=> Run the R code
=> The output looks like what we receive by email

Depends:
Package: fusen
  Depends: R (>= 3.5.0)
  Imports: attachment, cli, desc, devtools, glue, here (>= 1.0.0),
    magrittr, parsermd (>= 0.1.0), roxygen2, stats, stringi,
    tibble, tidyr, tools, usethis (>= 2.0.0), utils

Timings:
      utilisateur système  écoulé
fusen     107.585   9.573 143.396

Results:
Check status summary:
                  ERROR
  Source packages     1

Check results summary:
fusen ... ERROR
* checking CRAN incoming feasibility ... NOTE
* checking tests ... ERROR
* checking PDF version of manual ... WARNING

The full check directory is stored in the check_dir, which by default is "~/tmp/CRAN"
image

✔️ it works on Ubuntu 20.04 LTS as is

Regular checks

Scripts are:

=> This builds the R-devel version

=> This builds all R base packages

Tests on Ubuntu 22.04 LTS

  • Extra steps needed on local computer:
mkdir ~/tmp/
mkdir ~/tmp/R.check
  • Define these two directories inside file "check-R-ng"

  • R scripts directory :

R_scripts_dir=~/lib/R/Scripts
  • Shell scripts directory.
sh_scripts_dir=~/lib/bash
  • Error to be explored....
creating NEWS.2.pdf
/bin/bash: line 1: html: command not found
make: [Makefile:120: R-FAQ.epub] Error 127 (ignored)
/usr/bin/sed: can't read R-FAQ.epub.tmp: No such file or directory
make: *** [Makefile:121: R-FAQ.epub] Error 2
Fatal error: cannot open file '/home/srochett/lib/R/Scripts/check_CRAN_regular.R': No such file or directory

=> Need to be tested inside a Docker container

statnmap added a commit that referenced this issue Sep 19, 2022
Why ?

- Received link to CRAN check scripts
- Need an exploration and tests on different flavours

What?

- Scripts and env. variables for package incoming checks by Kurt
- Scripts and env. variables for CRAN regular checks of all packages

issue

issue #17
@statnmap
Copy link
Collaborator Author

I added what I could in {checkhelper} for tests

Experimental: Check as CRAN with CRAN global variables

Use the exploration of CRAN scripts by the RConsortium to check a package as CRAN does it with their env. variables. See #17 for more details.

install.packages('checkhelper', repos = 'https://thinkr-open.r-universe.dev')

check_dir <- tempfile("example")
# Check the current directory
check_as_cran(check_dir = check_dir)
# Open directory with all outputs
utils::browseURL(check_dir)

@hadley
Copy link
Member

hadley commented Sep 23, 2022

Thanks @statnmap! Would you mind tweaking your links to use https://github.com/r-devel/r-dev-web? That's a bit easier to navigate interactively.

@hadley
Copy link
Member

hadley commented Sep 23, 2022

I started putting together a brief big picture overview at https://docs.google.com/document/d/1H41Jjuj6T7o7zdhe4mh2rPyIi76YkpnY4Jd48tBBy_Q/edit?usp=sharing

@statnmap
Copy link
Collaborator Author

Also, I was wondering.
I think that Gabor already explored all these files to build {rhub} containers like CRAN.
Maybe we do not want to do the job a second time, and better have a look at what changed between the last time Gabor explored CRAN files and now ?

@llrs
Copy link
Collaborator

llrs commented Oct 10, 2022

As already mention I don't find clear when are the regular checks run:

I know the incoming tests are not performed all at the same time: windows checks are done later after a package is already accepted to CRAN. That's why sometimes maintainers get a message accepting their package and shortly after a notice to update their package to keep them in CRAN.

However, how does this happens for packages that are already in CRAN? How/when are the test run? Presumably there is some cron jobs (or similar) that would start checks or the scripts will decide which packages to test for instance based on change in dependencies, or updates in r-devel or machine changes.
However, this doesn't seem to be happening as often as I expected (daily in all setups), and I'm not sure where in the script this decisions are made.

Let me give an example (Perhaps it should be also added to #20 ) : lifecycle 1.0.3 has recently (2022-10-07) updated the messages output. The package is already in CRAN so presumably the reverse check dependencies were successfully. rtweet (which I maintain) uses it and has snapshots tests with the old output, so now checks should fail as I found out this via github checks. However, after 3 days none of the checks display any ERROR about said change (the webpage says it was updated today 2022-10-10 08:49:29 CEST).

Looking at the individual check pages of rtweet, at the moment of posting this says:

Flavor First line
r-devel-linux-x86_64-debian-clang 2022-10-04 r83023
r-devel-linux-x86_64-debian-gcc 2022-10-08 r83047
r-devel-linux-x86_64-fedora-clang 2022-10-07 r83044
r-devel-linux-x86_64-fedora-gcc 2022-10-07 r83044
r-devel-windows-x86_64 2022-10-07 r83044 ucrt
r-patched-linux-x86_64 2022-10-03 r83013
r-release-linux-x86_64 4.2.1 (2022-06-23)
r-release-macos-arm64 4.2.0 (2022-04-22)
r-release-macos-x86_64 4.2.0 (2022-04-22)
r-release-windows-x86_64 4.2.1 (2022-06-23 ucrt)
r-oldrel-macos-arm64 4.1.1 Patched (2021-09-05 r80862)
r-oldrel-macos-x86_64 4.1.2 (2021-11-01)
r-oldrel-windows-ix86+x86_64 4.1.3 (2022-03-10)

It is not clear to me when test were started for r-release, r-oldrel and r-patched version, and even with r-devel there are checks that haven't been run in a couple of days.

@hadley
Copy link
Member

hadley commented Oct 10, 2022

@llrs snapshot tests aren't run on CRAN

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants