diff --git a/.github/workflows/check-standard.yaml b/.github/workflows/check-standard.yaml index 3f5a748..a484448 100644 --- a/.github/workflows/check-standard.yaml +++ b/.github/workflows/check-standard.yaml @@ -20,9 +20,7 @@ jobs: config: - {os: macos-latest, r: 'release'} - {os: windows-latest, r: 'release'} - - {os: ubuntu-latest, r: 'release'} - # - {os: ubuntu-latest, r: 'devel', http-user-agent: 'release'} - # - {os: ubuntu-latest, r: 'oldrel-1'} + - {os: ubuntu-latest, r: 'release'} env: GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/pkgdown.yaml b/.github/workflows/pkgdown.yaml index 087f0b0..9deab3f 100644 --- a/.github/workflows/pkgdown.yaml +++ b/.github/workflows/pkgdown.yaml @@ -22,6 +22,11 @@ jobs: steps: - uses: actions/checkout@v3 + - uses: actions/setup-java@v3 + with: + distribution: temurin + java-version: 17 + - uses: r-lib/actions/setup-pandoc@v2 - uses: r-lib/actions/setup-r@v2 diff --git a/.github/workflows/test-coverage.yaml b/.github/workflows/test-coverage.yaml index ae03aef..92512c1 100644 --- a/.github/workflows/test-coverage.yaml +++ b/.github/workflows/test-coverage.yaml @@ -10,7 +10,7 @@ name: test-coverage jobs: test-coverage: - runs-on: macos-latest + runs-on: ubuntu-latest env: GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} diff --git a/.gitignore b/.gitignore index 234f028..1c9de78 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,13 @@ +## General ---- +*.swp +*.swo +*.html +*.pdf +.DS_Store +.httr-oauth +vignettes/*.log +vignettes/figure/ + .Rproj.user .Rhistory .RData diff --git a/DESCRIPTION b/DESCRIPTION index 6924e9c..261f1cf 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -1,6 +1,6 @@ Package: rPHG2 Title: R interface to PHGv2 -Version: 0.3 +Version: 0.3.1 Authors@R: person( given = "Brandon", @@ -20,6 +20,9 @@ Description: from unphased species characterized by low genetic diversity and high-density genotyping. License: Apache License (>= 2) | file LICENSE +URL: + https://github.com/maize-genetics/rPHG2, + https://rphg2.maizegenetics.net/ Encoding: UTF-8 Roxygen: list(markdown = TRUE) RoxygenNote: 7.3.1 @@ -35,6 +38,9 @@ Imports: methods, rJava, tibble -Suggests: +VignetteBuilder: knitr +Suggests: + knitr, + rmarkdown, testthat (>= 3.0.0) Config/testthat/edition: 3 diff --git a/NEWS b/NEWS deleted file mode 100644 index b197b54..0000000 --- a/NEWS +++ /dev/null @@ -1,33 +0,0 @@ -## CHANGES IN 0.3 -* Added new class, `PHGDataSet` - + In memory representation of hVCF data sets -* Added new function, `filterRefRanges()` - + Filters relevant rPHG2 data objects by reference range - + Requires a `GRanges` object as input -* Added new function, `filterSamples()` - + Filters relevant rPHG2 data objects by sample ID - + Input is a basic `character` vector of sample identifiers - - -## CHANGES IN 0.2 -* Added JVM connections via `rJava` interface -* Added new class, `HaplotypeGraph` - + Wrapper for PHGv2 API Java graph object -* Added new class, `PHGLocalCon` - + Interface to local hVCF database and file connections -* Added new function `initPHG()` - + Initializes JVM and adds PHGv2 Java JARs to classpath -* Added new function `readHapIds()` - + Reads haplotype IDs as a `character` matrix from a connection object -* Added new function `readHapIdMetaData()` - + Reads ALT header metadata for each haplotype ID from a connection object -* Added new function `readHapIdPosMetaData()` - + Reads ALT header positional metadata for each haplotype ID from a - connection object -* Added new function `readSamples()` - + Reads sample IDs from a connection object -* Added new function `readRefRanges()` - + Read reference range positional information as a `GRanges` object from a - connection object - - diff --git a/NEWS.md b/NEWS.md index 1005a78..2dae214 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,10 @@ +## rPHG2 0.4 +* Added vignettes and README updates + + `vignette("rPHG2")` +* Created build system for website generation + + [rphg2.maizegenetics.net](https://rphg2.maizegenetics.net) + + ## rPHG2 0.3 * Added new class, `PHGDataSet` + In memory representation of hVCF data sets diff --git a/README.md b/README.md index 4bb2bc4..f866a70 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ [![Life Cycle Status](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://www.tidyverse.org/lifecycle/#experimental) +[![R-CMD-check](https://github.com/maize-genetics/rPHG2/actions/workflows/check-standard.yaml/badge.svg)](https://github.com/maize-genetics/rPHG2/actions/workflows/check-standard.yaml) +[![codecov](https://codecov.io/gh/maize-genetics/rPHG2/graph/badge.svg?token=BEEI8KZ96H)](https://codecov.io/gh/maize-genetics/rPHG2) `rPHG2` is a system to interact with and retrieve information from **version 2** @@ -15,19 +17,69 @@ framework for genotype inference. This is accomplished by leveraging the You can install the development version of rphg2 from [GitHub](https://github.com/) with: ``` r -# install.packages("devtools") -devtools::install_github("maize-genetics/rPHG2") +# install.packages("pak") +pak::pak("maize-genetics/rPHG2") ``` -## Example -This is a basic example which shows you how to solve a common problem: +## Usage +### Initialize with PHGv2 JARs ``` r -library(rphg2) +initPhg("my/phg/jar/path") +``` + +### Create a connection +``` r +locCon <- list.files("my/hvcf/dir") |> + PHGLocalCon() +``` + +### Build a graph +``` r +graph <- locCon |> buildHaplotypeGraph() +``` + +### Read data into R +``` r +phgDs <- graph |> readPhgDataSet() +``` + +### Identify areas of interest +``` r +gr <- GRanges( + seqnames = c("1", "2"), + ranges = IRanges( + c(100, 800), + c(400, 900) + ) +) + +phgDs |> + filterSamples(c("B73", "Ky21", "Mo17")) |> + filterRefRanges(gr) +``` + + +## Lifecycle +[![Life Cycle Status](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://www.tidyverse.org/lifecycle/#experimental) -phgCon <- PHGServerCon("localhost", 8080, "http") +`rPHG2` is a relatively new R package and may be subject to changes in +workflow and function definitions. If you would like to see anything added +or run into problems, please feel free to contact the team using the +[issues page](https://github.com/maize-genetics/rPHG2/issues). -phgCon |> readSamples() + +## Learning rPHG2 +If installed locally, you can read the "Introduction to rPHG2" vignette by +using: + +``` r +vignette("rPHG2") ``` +Alternatively, the package's [website](https://rphg2.maizegenetics.net) is +another great place to read up on function usage, code changes, and +workflows. + + diff --git a/_pkgdown.yml b/_pkgdown.yml deleted file mode 100644 index badfedb..0000000 --- a/_pkgdown.yml +++ /dev/null @@ -1,14 +0,0 @@ -url: ~ -template: - bootstrap: 5 - -footer: - structure: - left: acknowledgment - components: - acknowledgment: rPHG2 is an R package developed by The Buckler Lab - -home: - links: - - text: Report a bug - href: https://github.com/maize-genetics/rPHG2/issues diff --git a/pkgdown/_pkgdown.yml b/pkgdown/_pkgdown.yml new file mode 100644 index 0000000..53eb2a4 --- /dev/null +++ b/pkgdown/_pkgdown.yml @@ -0,0 +1,33 @@ +url: https://rphg2.maizegenetics.net + +template: + theme: breeze-light + bootstrap: 5 + bslib: + base_font: {google: "Barlow"} + heading_font: {google: "Oswald"} + code_font: {google: "Roboto Mono"} + extra_css: pkgdown/extra.css + +navbar: + structure: + left: [intro, articles, reference, news] + right: [search, github, phg] + components: + phg: + icon: fa-dna + href: https://github.com/maize-genetics/phg_v2 + aria-label: PHG Source + +footer: + structure: + left: acknowledgment + components: + acknowledgment: rPHG2 is an R package developed by The Buckler Lab + +home: + links: + - text: Report a bug + href: https://github.com/maize-genetics/rPHG2/issues + + diff --git a/pkgdown/extra.css b/pkgdown/extra.css new file mode 100644 index 0000000..2bcd3fc --- /dev/null +++ b/pkgdown/extra.css @@ -0,0 +1,6 @@ +.navbar-brand { + font-family: 'Oswald', sans-serif; /* Change font family */ + font-size: 20px; /* Change font size */ + font-weight: bold; /* Make font bold */ + color: #363636; /* Change font color to red */ +} diff --git a/rphg2.Rproj b/rphg2.Rproj index fa0d77b..2ce7861 100644 --- a/rphg2.Rproj +++ b/rphg2.Rproj @@ -19,4 +19,4 @@ LineEndingConversion: Posix BuildType: Package PackageUseDevtools: Yes PackageInstallArgs: --no-multiarch --with-keep.source -PackageRoxygenize: rd,collate,namespace +PackageRoxygenize: rd,collate,namespace,vignette diff --git a/tests/testthat/setup.R b/tests/testthat/setup.R index 6f980a2..453ba4b 100644 --- a/tests/testthat/setup.R +++ b/tests/testthat/setup.R @@ -18,6 +18,12 @@ getLatestReleaseUrl <- function(repo) { response <- httr::GET(apiUrl) + message(" * API total rate...............: ", response$headers$`x-ratelimit-limit`) + message(" * API rate limit (remaining)...: ", response$headers$`x-ratelimit-remaining`) + message(" * API rate limit (used)........: ", response$headers$`x-ratelimit-used`) + message(" * UTC Epoch time to reset......: ", response$headers$`x-ratelimit-reset`) + + # Check if the request was successful if (httr::http_status(response)$category != "Success") { stop( diff --git a/vignettes/img/filtering/mat_01.svg b/vignettes/img/filtering/mat_01.svg new file mode 100644 index 0000000..6bd975b --- /dev/null +++ b/vignettes/img/filtering/mat_01.svg @@ -0,0 +1,335 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/vignettes/img/filtering/mat_02.svg b/vignettes/img/filtering/mat_02.svg new file mode 100644 index 0000000..aed290a --- /dev/null +++ b/vignettes/img/filtering/mat_02.svg @@ -0,0 +1,334 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/vignettes/img/filtering/mat_03.svg b/vignettes/img/filtering/mat_03.svg new file mode 100644 index 0000000..480d99f --- /dev/null +++ b/vignettes/img/filtering/mat_03.svg @@ -0,0 +1,334 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/vignettes/img/filtering/mat_04.svg b/vignettes/img/filtering/mat_04.svg new file mode 100644 index 0000000..5268e57 --- /dev/null +++ b/vignettes/img/filtering/mat_04.svg @@ -0,0 +1,334 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/vignettes/rPHG2.R b/vignettes/rPHG2.R new file mode 100644 index 0000000..cf69f4b --- /dev/null +++ b/vignettes/rPHG2.R @@ -0,0 +1,67 @@ +## ----setup, include=FALSE----------------------------------------------------- +knitr::opts_chunk$set( + fig.align = "center", + external = TRUE, + echo = TRUE, + warning = FALSE +) + +library(rPHG2) +source("../tests/testthat/setup.R") + +## ----eval=FALSE, echo=TRUE---------------------------------------------------- +# PHGLocalCon() + +## ----eval=FALSE, echo=TRUE---------------------------------------------------- +# system.file("extdata", "LineA.h.vcf", package = "rPHG2") + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +hVcfFiles <- system.file( + "extdata", + c("LineA.h.vcf", "LineB.h.vcf"), + package = "rPHG2" +) + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +localCon <- hVcfFiles |> PHGLocalCon() + +localCon + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +srvCon <- "phg.maizegdb.org" |> PHGServerCon() + +srvCon + +## ----eval=FALSE, echo=TRUE---------------------------------------------------- +# "www.my-unsecure-phg.org" |> +# PHGServerCon( +# port = 5300, +# protocol = "http" +# ) + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +graph <- localCon |> buildHaplotypeGraph() + +graph + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +graph |> javaRefObj() + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +graph |> readSamples() + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +graph |> readRefRanges() + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +m <- graph |> readHapIds() + +# Show only first 3 columns +m[, 1:3] + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +graph |> readHapIdMetaData() + +## ----eval=TRUE, echo=TRUE----------------------------------------------------- +graph |> readHapIdPosMetaData() + diff --git a/vignettes/rPHG2.Rmd b/vignettes/rPHG2.Rmd new file mode 100644 index 0000000..bee226c --- /dev/null +++ b/vignettes/rPHG2.Rmd @@ -0,0 +1,334 @@ +--- +title: "Introduction to rPHG2" +output: rmarkdown::html_vignette +description: > + Learn how to get started with the basics of rPHG2. +vignette: > + %\VignetteIndexEntry{Introduction to rPHG2} + %\VignetteEngine{knitr::rmarkdown} + %\VignetteEncoding{UTF-8} +editor_options: + markdown: + wrap: 70 +--- + +```{r setup, include=FALSE} +knitr::opts_chunk$set( + fig.align = "center", + external = TRUE, + echo = TRUE, + warning = FALSE +) + +library(rPHG2) + +source("../tests/testthat/setup.R") +``` + +The basic workflow of the `rPHG2` package is as follows: + +1. Create a connection object +2. Read data into the R environment +3. Analyze and visualize data retrieval + +This document introduces you to `rPHG2`'s methods and grammar, and +shows you how to apply them to the previously mentioned workflow. + + + +## Creating connection objects +rPHG objects can be created through two primary sources: +* local data +* server connections + +Creating initial "connection" objects helps unify downstream reading and +evaluation steps for PHGv2 data. In the next couple of sections, we will show +you how to create either local or server connnection objects. + + +### Local data +_Local_ connections are for local TileDB instances or direct locations of +[hVCF](https://github.com/maize-genetics/phg_v2/blob/main/docs/hvcf_specifications.md) +files on a local disk. To create a local connection, we will create a +`PHGLocalCon` object using the following constructor function: + +```{r, eval=FALSE, echo=TRUE} +PHGLocalCon() +``` + +For the above example, we will create a connection to some example local hVCF +files provided with the `rPHG2` package: + +* `LineA.h.vcf` +* `LineB.h.vcf` + +Since the full paths to these files will differ between each user, we can +use the `system.file()` function to get the full path: + +```{r, eval=FALSE, echo=TRUE} +system.file("extdata", "LineA.h.vcf", package = "rPHG2") +``` + +This will get the full path from the `extdata` directory for the file, +`LineA.h.vcf`, found in the `rPHG2` source code. + +We can further build on this to create a collection of full file paths to our +hVCF data: + +```{r, eval=TRUE, echo=TRUE} +hVcfFiles <- system.file( + "extdata", + c("LineA.h.vcf", "LineB.h.vcf"), + package = "rPHG2" +) +``` + + +Now that we have a collection of hVCF files, we can use the `PHGLocalCon()` +constructor function to create a `PHGLocalCon` object: + +```{r, eval=TRUE, echo=TRUE} +localCon <- hVcfFiles |> PHGLocalCon() + +localCon +``` + +From here, we can move to the [next section](#creating-jvm-objects) to create a `HaplotypeGraph` +object interface with the JVM. + + +### Server connections +Conversely, _server_ locations are for databases served on publicly available +web services leveraging the [Breeding API (BrAPI) endpoints](https://brapi.org/). +Since this is a connection to a server, a URL path instead of a local file +path will be needed. We will use the `PHGServerCon()` constructor function +to create a `PHGServerCon` object: + +```{r, eval=TRUE, echo=TRUE} +srvCon <- "phg.maizegdb.org" |> PHGServerCon() + +srvCon +``` + +In the above example, we have made the assumptions that this URL: + +1. Uses a secure transfer protocol ([`https`](https://en.wikipedia.org/wiki/HTTPS)) +2. Uses default ports for data serving +3. Has BrAPI specified endpoints + +For points **1** and **2**, if the URL uses non-secure protocols ("http") +and/or has a modified port number, you will need to specify these with the +`protocol` and `port` parameters in the constructor function. For example: + +```{r, eval=FALSE, echo=TRUE} +"www.my-unsecure-phg.org" |> + PHGServerCon( + port = 5300, + protocol = "http" + ) +``` + +For point **3**, if the constructor cannot resolve +[mandatory endpoints](https://app.swaggerhub.com/apis/PlantBreedingAPI/BrAPI-Core/2.1#/Server%20Info/get_serverinfo), +an exception will occur. + + + +## Creating JVM objects + +Now that we have either a local or server-based connection object, we can +convert the raw hVCF data into a +[`HaplotypeGraph`](https://github.com/maize-genetics/phg_v2/blob/main/src/main/kotlin/net/maizegenetics/phgv2/api/HaplotypeGraph.kt) +JVM object and bridge the Java reference pointer to R using the function, +`buildHaplotypeGraph()`: + + +```{r, eval=TRUE, echo=TRUE} +graph <- localCon |> buildHaplotypeGraph() + +graph +``` + +In the above example, we took the local connection object and passed it into +a `HaplotypeGraph` constructor. Here, we have a basic class that contains a +pointer object where we can direct data from Java to R: + +```{r, eval=TRUE, echo=TRUE} +graph |> javaRefObj() +``` + + + +## Reading data +Now that we have created a `HaplotypeGraph` object, we can begin **read**ing +data using the `read*` family of `rPHG2` functions. + + +### Sample IDs +To return a vector of sample IDs from the graph object, we can use the +`readSamples()` function: + +```{r, eval=TRUE, echo=TRUE} +graph |> readSamples() +``` + + +### Reference ranges +To return information about all reference ranges found within the graph object, +we can use the `readRefRanges()` function. This will return a `GRanges` object +which is a common data class in the +[`GenomicRanges`](https://bioconductor.org/packages/release/bioc/html/GenomicRanges.html) +package: + +```{r, eval=TRUE, echo=TRUE} +graph |> readRefRanges() +``` + + +### Haplotype IDs +To return all haplotype IDs as a "sample $\times$ reference range" `matrix` +object, we can use the `readHapIds()` function: + + +```{r, eval=TRUE, echo=TRUE} +m <- graph |> readHapIds() + +# Show only first 3 columns +m[, 1:3] +``` + + + +### Haplotype ID metadata +To return metadata for each haplotype ID as a `tibble` object, we can use +the `readHapIdMetaData()` function: + +```{r, eval=TRUE, echo=TRUE} +graph |> readHapIdMetaData() +``` + + +### Haplotype ID metadata (positions) +To return positional information for each haplotype ID as another `tibble` +object, we can use the `readHapIdPosMetaData()` function: + +```{r, eval=TRUE, echo=TRUE} +graph |> readHapIdPosMetaData() +``` + + +### All hVCF data +In a majority of cases, we may need more than one piece of hVCF data. We can +read all of the prior data simultaneously as `PHGDataSet` object which is +an in-R-memory representation of all hVCF data: + +```{r, eval=TRUE, echo=TRUE} +graph |> readPhgDataSet() +``` + +If this object is created, we can use the prior `read*` methods to +instantaneously pull out the previously mentioned R data objects: + +* `readSamples()` +* `readRefRanges()` +* `readHapIds()` +* `readHapIdMetaData()` +* `readHapIdPosMetaData()` + + + +## Filter data +In some cases, you may want to query information and focus on one +specific reference range and/or sample(s). We can **filter** our `PHGDataSet` +objects using the `filter*` family of functions. + +For the following examples, let's picture our primary data as a 2-dimensional +`matrix` of: + +* samples (rows) +* reference ranges (columns) +* hap ID data (elements) + +If we were to represent this as an object in R called `phgDs`, it would look +like the following diagram: + +```{r, eval=FALSE, echo=TRUE} +phgDs +``` + +![](img/filtering/mat_01.svg){fig.align="center"} + + +### Filter by sample +If we want to filter based on sample ID, we can use the `filterSamples()` +function. Simply add the sample ID or collection of sample IDs as a `character` +string or a vector of `character` strings, respectively. Using the prior +diagram as a reference, I will filter out anything that is not the following: + +* `B73` +* `Ky21` +* `Mo17` + +```{r, eval=FALSE, echo=TRUE} +phgDs |> filterSamples(c("B73", "Ky21", "Mo17")) +``` + +![](img/filtering/mat_02.svg){fig.align="center"} + +**Note**: If samples are added to the filter collection, but are not +present in the data, they will be discarded. If **no** samples are found, an +exception will be thrown stating that no samples could be found. + + +### Filter by reference range +If we want to filter based on sample ID, we can use the `filterRefRanges()` +function. Currently, this takes a `GRanges` object where we can specify +integer-based ranges located in genomic regions by chromosome (i.e., +`seqnames`). For example, if I want to return all ranges that intersect with +the following genomic regions: + +| Chromosome | Start (bp) | End (bp) | +|------------|------------|----------| +| `"1"` | `100` | `400` | +| `"2"` | `400` | `900` | + +I can create the following `GRanges` object and pass that to the filter +method: + +```{r, eval=FALSE, echo=TRUE} +gr <- GRanges( + seqnames = c("1", "2"), + ranges = IRanges( + c(100, 800), + c(400, 900) + ) +) + + +phgDs |> filterRefRanges(gr) +``` + +![](img/filtering/mat_03.svg) + + +### Chaining methods +Since we have two dimensions, we can filter simultaneously by "piping" or +combining filter methods in one pass: + +```{r, eval=FALSE, echo=TRUE} +gr <- GRanges( + seqnames = c("1", "2"), + ranges = IRanges( + c(100, 800), + c(400, 900) + ) +) + +phgDs |> + filterSamples(c("B73", "Ky21", "Mo17")) |> + filterRefRanges(gr) +``` + +![](img/filtering/mat_04.svg) + +