Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revise the review policy #33

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

Revise the review policy #33

Changes from 11 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
d528045
Edit review policy
wlandau-lilly Nov 17, 2024
b666666
Remove automatic rejection (what a PR updates the org list?)
wlandau Nov 18, 2024
fc8e265
link format
wlandau Nov 18, 2024
95dd323
Update review.md
wlandau Nov 18, 2024
6650d55
Update review.md
wlandau Nov 18, 2024
a77dd36
Update review.md
wlandau Nov 18, 2024
73d511d
Update review.md
wlandau Nov 18, 2024
3610974
Update review.md
wlandau Nov 18, 2024
41806f6
Update review.md
wlandau Nov 18, 2024
98cbbb6
Update review.md
wlandau Nov 18, 2024
6d529d2
Update review.md
wlandau Nov 18, 2024
bb1febc
Update review.md
wlandau Nov 19, 2024
a846711
Update review.md
wlandau Nov 19, 2024
059a536
Update review.md
wlandau Nov 19, 2024
f6dc363
Update review.md
wlandau Nov 19, 2024
275a247
Require a valid open-source license
wlandau Nov 21, 2024
c996196
Say "owners" because sometimes owners and authors are not the same
wlandau Nov 21, 2024
5ad944f
Update review.md
wlandau Nov 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 69 additions & 45 deletions review.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,48 +2,72 @@
title: "Review Policy"
---

As the [contributors](contributors.md) page explains, updates to the R-multiverse package listings come from pull requests to <https://github.com/r-multiverse/contributions> from members of the R community. In the vast majority of cases, a GitHub app automatically merges the pull request. However, some pull requests need to be manually reviewed by an R-multiverse moderator. This document describes this manual review process. The goals are to:

1. Ensure that all pull requests are reviewed using a consistent set of standards and principles that do not vary according to moderator.
2. Ensure these standards and principles are clear and transparent for the R community.

## The automated review process

The [`review.yaml`](https://github.com/r-multiverse/community/blob/main/.github/workflows/review.yaml) [GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions) workflow runs periodically and reviews each new open pull request using [`multiverse.internals::review_pull_requests()`](https://github.com/r-multiverse/multiverse.internals/blob/main/R/review_pull_requests.R). Depending on the results of the automated checks, the bot will automatically merge the pull request, close it, or flag it for manual review. The bot will post an informative comment explaining the decision, and it may add a GitHub label to the pull request for triage purposes.

The pull request is automatically closed if:

1. It attempts to add/modify/delete a file outside the [`packages`](https://github.com/r-multiverse/contributions/tree/main/packages) folder.
1. It attempts to add/modify/delete a file in a subdirectory of the [`packages`](https://github.com/r-multiverse/contributions/tree/main/packages) folder.

The pull request is automatically flagged for manual review if:

1. The latest commit was not [created by the GitHub web interface](https://r-multiverse.org/contributors.html).
1. It attempts to modify or delete any existing file in [`packages`](https://github.com/r-multiverse/contributions/tree/main/packages).
1. A contributed text file is not a single-line file.
1. The text file looks like a custom JSON entry (for packages in a subdirectory of a GitHub repository).
1. The name of the the text file is not a valid R package name.
1. The URL in the text file cannot be parsed.
1. The URL scheme is anything other than HTTPS.
1. The domain of the URL is anything other than github.com or gitlab.com.
1. The URL points to an organization or user such as <https://github.com/r-lib> rather than a repository such as <https://github.com/r-lib/gh>.
1. The URL does not exist or is not online at the time it is checked (HTTP error trying to access it).
1. A [release](https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases) could not be found at the repo in the URL.
1. The version-controlled repository name in the URL is different from the name of the file. (For example, if the file is named `gh` as the package, then the URL <https://github.com/r-lib/gh-package> would be flagged for manual review, but <https://github.com/r-lib/gh> would not.)
1. The repository is part of the CRAN mirror at <https://github.com/cran>.
1. The package is also on CRAN, and the URL in the pull request cannot be found in the `DESCRIPTION` file of the latest CRAN release.

## The manual review process

If a pull request is flagged for manual review, an R-multiverse moderator will read the pull request and ask questions if necessary. Although the moderator may make optional suggestions on a case-by-case basis, package reviews must be consistent, reliable, and inclusive whenever possible. The decision to close or merge the pull request must be based exclusively on the following pre-defined list of requirements:

1. Each contribution must comply with the [code of conduct](conduct.md). Examples of prohibited content include profanity, malicious behavior, security risks, copyright violations, and other conduct which could reasonably be considered inappropriate in a professional setting. All this applies to the package, the URL, any other metadata in the contribution, and the contents of the package itself.
1. The package name, URL, and all other metadata must be complete and correct.
1. Each text file must apply to only one package.
1. The text file name must be the name of the package.
1. For JSON listings, the `"branch"` field must be `"*release"` (except in specific predetermined cases such as packages in <https://github.com/cran>), the `"subdirectory"` field must be supplied and exist, the `"url"` field must exist and be correct, and the `"package"` field must agree with the name of the text file.
1. Each contributed URL must point to an existing GitHub or GitLab repository.
1. The URL must be the true/official location of the source code or a faithful mirror of the true location. The package maintainers have the authority to choose the URL. Unofficial or unsupported forks should not be included. The moderator must use discretion on a case-by-case basis because sometimes a fork becomes the true version (e.g. if the original maintainer abandons a package and becomes unreachable indefinitely).
1. The URL must have a release on GitHub or GitLab so R-universe can process the package without error. As a last resort, if the maintainer does not provide their own releases, a repository from the CRAN mirror at <https://github.com/cran> may be registered.
1. If the listing of an existing package is modified, then the moderator must verify that the new information is complete and correct. In many cases, it may be necessary to obtain permission from the package maintainer.
1. The reasons for deleting a package listing may vary on a case-by-case basis. The moderator must carefully consider the impact that deletion would have on the community and on reverse dependencies. In many cases, it may be necessary to obtain permission from the package maintainer.
This policy dictates the process by which R-multiverse reviews and accepts contributed R packages.
All contributions must comply with R-multiverse policies, including but not limited to [Acceptable Use](aup.md), [Terms of Use](terms.md), and [Code of Conduct](conduct.md).

## How review works

When it reviews a [new pull request](https://github.com/r-multiverse/contributions/pulls), the bot makes one of three choices:

1. Merge the pull request to accept the contribution.
2. Flag the pull request for manual review by a [moderator](governance.md#moderator).

## Automatic acceptance

The bot automatically accepts the contribution if the author of the [pull request](https://github.com/r-multiverse/contributions/pulls):

1. Is a [public member](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-your-membership-in-organizations/publicizing-or-hiding-organization-membership) of one of the trusted [GitHub organizations](https://docs.github.com/en/organizations/collaborating-with-groups-in-organizations/about-organizations) listed at <https://github.com/r-multiverse/contributions/blob/main/organizations>.

and if the [pull request](https://github.com/r-multiverse/contributions/pulls) itself:

1. Was [created by the GitHub web interface](contributors.md).
1. Adds new contributed listings to the [`packages` folder](https://github.com/r-multiverse/contributions/tree/main/packages).
1. Does not add, modify, or delete any other files in <https://github.com/r-multiverse/contributions>.

and if each new [contributed listing](https://github.com/r-multiverse/contributions/tree/main/packages):

1. Is a single line of text with a valid HTTPS URL.
1. Points to an existing public GitHub/GitLab repository.

and if the contributed GitHub/GitLab repository:

1. Includes a GitHub/GitLab [release](https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases).
1. Includes an R package at the top level whose package name is the same as the repository name.
1. Is listed in the `URL` field of the corresponding CRAN page (if a package with the same name is on CRAN).

and if the R package:

1. Includes a license from the "Recommended licenses" section at the end of this policy.
1. Does not have an advisory in the [R Consortium Advisory Database](https://github.com/RConsortium/r-advisory-database).
1. Is not part of the CRAN mirror at <https://github.com/cran>.

## Manual review

R-multiverse [moderators](governance.md#moderator) review [pull requests](https://github.com/r-multiverse/contributions/pulls) that the bot flags for manual review.
The [moderator](governance.md#moderator) inspects the package for compliance with R-multiverse policies, including but not limited to [Acceptable Use](aup.md), [Terms of Use](terms.md), and [Code of Conduct](conduct.md).
The moderator accepts the contribution if and only if it complies with all policies.

## Removal

R-multiverse may remove a package from its own repositories at any time if the package violates R-multiverse policies.

## Modification

No R-multiverse staff member (administrator, moderator, or otherwise) may modify a package in R-multiverse without the explicit consent of the owners declared in the license of the package.
wlandau marked this conversation as resolved.
Show resolved Hide resolved

## Recommended licenses

The [Acceptable Use Policy](aup.md) prohibits packages that "violate any applicable laws, regulations, or third-party rights, including intellectual property rights".
In practice, this means each package must have a valid open-source license.
wlandau marked this conversation as resolved.
Show resolved Hide resolved
The following is a list of valid open-source licenses that the bot automatically accepts during reviews.
If the package includes one of the license below, it implies that the authors consent to distribute the package in R-multiverse.
wlandau marked this conversation as resolved.
Show resolved Hide resolved

* [Artistic 2.0](https://opensource.org/license/artistic-2-0)
wlandau marked this conversation as resolved.
Show resolved Hide resolved
* [BSD 2-Clause](https://opensource.org/license/bsd-2-clause)
* [BSD 3-Clause](https://opensource.org/license/bsd-3-clause)
* [GPL-2](https://opensource.org/license/gpl-2-0)
* [GPL-3](https://opensource.org/license/gpl-3-0)
* [LGPL-2](https://opensource.org/license/lgpl-2-0)
* [LGPL-2.1](https://opensource.org/license/lgpl-2-1)
* [LGPL-3](https://opensource.org/license/lgpl-3-0)
* [MIT](https://opensource.org/license/mit)