Add capability to add a github package to a miniCRAN repo

[andrie]

This is a feature that gets requested often

Related to:

• new function to add a local package #23 new function to add a local package
• Add capability to read package data from local file #9 Add capability to read package data from local file
• Add capability to read package data from github #10 Add capability to read package data from github

[achubaty]

see devtools::install_remote() to see how the package source bundle is downloaded. from there, #23's addLocalPackage can take over

[stephlocke]

This would be handy

[msteijaert]

Here's an example based on achubaty's comment

addGithubPackage <- function(githubPath,...){
    packageName <- basename(githubPath)
    exDir <- file.path(tempdir(),packageName)
    if(file.exists(exDir)) unlink(exDir, recursive=TRUE)
    zipFile <- file.path(tempdir(),paste0(packageName,".zip"))
    download.file(paste0(githubPath,"/","zipball/master"),zipFile)
    unzip(zipFile, exdir = exDir)
    file.rename(list.files(exDir,full.names=TRUE)[1],file.path(exDir,packageName))
    addLocalPackage(packageName,exDir,...)
}
addGithubPackage(githubPath="https://github.com/js229/Vennerable/",path=reposPath,build=TRUE)

[seasmith]

@msteijaert
I resolved a file path error by setting mode = "wb" in download.file(). Thank you, for the function.

OS: Windows 7
R: 3.4.4

[StaffanBetner]

@msteijaert's code doesn't work anymore. I get the error Error: No files found in path with extension '.tar.gz'

[achubaty]

@msteijaert thank you for taking a first crack at this. I've implemented a version of your function on the 50-addPackageGitHub branch, though it needs work. See the commit message there.

@msteijaert's code doesn't work anymore. I get the error Error: No files found in path with extension '.tar.gz'

Can you post the code you used? I can't reporduce the error.

[jonlachmann]

Hi! I needed this functionality and took the code from the branch 50-addPackageGitHub and extended it to support private repos etc by calling the appropriate functions in the remotes package.

The end result is below. Feel free to add it to the package or let me know if you would like a proper pull request.

addPackageGitHub <- function(repo, username = NULL, ref = "master", subdir = NULL, auth_token = remotes:::github_pat(quiet), ...) {
  packageName <- basename(repo)
  exDir <- file.path(tempdir(), packageName)
  if (dir.exists(exDir)) unlink(exDir, recursive = TRUE)

  remote <- remotes:::github_remote(repo = repo, username = username, ref = ref, subdir = subdir, auth_token = auth_token)
  tarFile <- remotes:::remote_download(remote)

  untar(tarFile, exdir = exDir)
  file.rename(list.files(exDir, full.names = TRUE)[1], file.path(exDir, packageName))

  ## TODO: addLocalPackage() does not add the dependencies of the locally built package!
  addLocalPackage(packageName, exDir, ..., build = TRUE)
}

addPackageGitHub("user/repo", auth_token=github_token, path=miniCRAN_dir)

[daattali]

Are there any concrete plans on adding this feature, or is it more of a "long term wish list"?

[indenkun]

It seems that the task has not been in progress for somewhat long. Are there any plans to add this feature? I believe this would be a very useful feature if implemented.

I need this feature myself and created it.
The following is an example of how I implemented using the remotes function, such as repository information in the description file when installing a package with the remotes package.

This code is implemented in a forked repository and added document in reference to other functions, so I can pull-request it if you like this functionality. Please note that I am not referring to much of the previous discussion.

Please consider it.

addGithubPackage <- function(repo = NULL, path = NULL,
                             type = "source", Rversion = R.version,
                             writePACKAGES = TRUE, deps = FALSE,
                             quiet = FALSE, ref = "HEAD", subdir = NULL,
                             auth_token = github_pat(quiet),
                             host = "api.github.com") {
  if (is.null(path) || is.null(repo)) {
    stop("path and repo must be specified.")
  }
  
  if(!requireNamespace("devtools", quietly = TRUE) || !requireNamespace("remotes", quietly = TRUE)){
    stop("you must first install the 'devtools' and 'remotes' package.")
  }
  
  # Download the source from GitHub, describe the repository information, etc., 
  # and rebuild the source package.
  remote <- remotes::github_remote(repo = repo, ref = ref, subdir = subdir, 
                                   auth_token = auth_token, host = host)
  pkgs <- remotes::remote_package_name(remote)
  temp_source <- remotes::remote_download(remote)
  utils::untar(temp_source, 
               exdir = tools::file_path_sans_ext(temp_source, compression = TRUE))
  file.rename(file.path(tools::file_path_sans_ext(temp_source, compression = TRUE), 
                        list.files(tools::file_path_sans_ext(temp_source, compression = TRUE))),
              sourcePath <- file.path(tools::file_path_sans_ext(temp_source, compression = TRUE),
                                      pkgs))
  pkgPath <- tools::file_path_sans_ext(temp_source, compression = TRUE)
  remote_sha <- remotes::remote_sha(remote)
  remotes::add_metadata(pkg_path = sourcePath, 
                        remotes::remote_metadata(x = remote, source = sourcePath, sha =remote_sha))

  addLocalPackage(pkgs = pkgs, pkgPath = pkgPath, path = path, type = type,
                  Rversion = Rversion, writePACKAGES = writePACKAGES, deps = deps,
                  quiet = quiet, build = TRUE)
}

#'
github_pat <- utils::getFromNamespace("github_pat", "remotes")

https://github.com/indenkun/miniCRAN/blob/main/R/addPackages.R#L401