Skip to content

Commit

Permalink
update vignette
Browse files Browse the repository at this point in the history
  • Loading branch information
pweigmann committed Aug 29, 2024
1 parent c2d98b1 commit 37b99fd
Show file tree
Hide file tree
Showing 5 changed files with 69 additions and 52 deletions.
2 changes: 1 addition & 1 deletion .buildlibrary
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ValidationKey: '658779'
ValidationKey: '678776'
AutocreateReadme: yes
AcceptedWarnings:
- 'Warning: package ''.*'' was built under R version'
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ cff-version: 1.2.0
message: If you use this software, please cite it using the metadata from this file.
type: software
title: 'piamValidation: Validation Tools for PIK-PIAM'
version: 0.3.3
date-released: '2024-08-28'
version: 0.3.4
date-released: '2024-08-29'
abstract: The piamValidation package provides validation tools for the Potsdam Integrated
Assessment Modelling environment.
authors:
Expand Down
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Type: Package
Package: piamValidation
Title: Validation Tools for PIK-PIAM
Version: 0.3.3
Date: 2024-08-28
Version: 0.3.4
Date: 2024-08-29
Authors@R:
c(person("Pascal", "Weigmann",, "[email protected]", role = c("aut", "cre")),
person("Oliver", "Richters",, role = "aut"))
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Validation Tools for PIK-PIAM

R package **piamValidation**, version **0.3.3**
R package **piamValidation**, version **0.3.4**

[![CRAN status](https://www.r-pkg.org/badges/version/piamValidation)](https://cran.r-project.org/package=piamValidation) [![R build status](https://github.com/pik-piam/piamValidation/workflows/check/badge.svg)](https://github.com/pik-piam/piamValidation/actions) [![codecov](https://codecov.io/gh/pik-piam/piamValidation/branch/master/graph/badge.svg)](https://app.codecov.io/gh/pik-piam/piamValidation) [![r-universe](https://pik-piam.r-universe.dev/badges/piamValidation)](https://pik-piam.r-universe.dev/builds)

Expand Down Expand Up @@ -46,7 +46,7 @@ In case of questions / problems please contact Pascal Weigmann <pascal.weigmann@

To cite package **piamValidation** in publications use:

Weigmann P, Richters O (2024). _piamValidation: Validation Tools for PIK-PIAM_. R package version 0.3.3, <https://github.com/pik-piam/piamValidation>.
Weigmann P, Richters O (2024). _piamValidation: Validation Tools for PIK-PIAM_. R package version 0.3.4, <https://github.com/pik-piam/piamValidation>.

A BibTeX entry for LaTeX users is

Expand All @@ -55,7 +55,7 @@ A BibTeX entry for LaTeX users is
title = {piamValidation: Validation Tools for PIK-PIAM},
author = {Pascal Weigmann and Oliver Richters},
year = {2024},
note = {R package version 0.3.3},
note = {R package version 0.3.4},
url = {https://github.com/pik-piam/piamValidation},
}
```
105 changes: 61 additions & 44 deletions vignettes/validateScenarios.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,63 @@ For more detailed information please refer to the

# Usage

## REMIND output script

When working directly with REMIND, a script comes shipped with the code which
will make it easy to perform a basic scenario validation. Go to the REMIND
directory ``remind/`` and call ``Rscript output.R``. Then choose "Comparison
across runs" and "validateScenarios". Select the runs of interest and choose
one of the ``validationConfigs`` that are available via ``piamValidation``.
If you are not sure which config to choose, "default" is a good starting point.

## Scenario Validation

More generally, the function ``piamValidation::validateScenarios()`` performs
all necessary steps of the
validation process. It takes the config file and iterates through each row,
assembling the required scenario and reference data and checking the thresholds.

The output is a data.frame, which combines scenario and reference data with
the threshold that is applied to each respective data point and the result of
the validation checks.
These results can be found in the columns ``check_value`` and ``check``, with
the former containing the value that is directly compared to the thresholds (
e.g. the calculated growth rate when doing a growth rate check) and the latter
being the result in form of a traffic-light color.

![Traffic-light threshold check results.](./thresholds.png){width=80%}

Optionally, you can save the resulting data.frame to a .csv file by providing
an ``outputFile``.

```{r run validation, eval = FALSE}
df <- validateScenarios(c(scenPath, histPath), config, outputFile = NULL)
```

The function argument ``config`` can be either the name of a config file found
in ``piamValidation`` (find the name via
``inst/config/validationConfig_<name>.csv``) or a full path to a config file.

## Validation Report

To perform the validation and create an output document in one go, the function
``piamValidation::validationReport()`` can be used. It calls ``validateScenarios``
and additionally renders an .html file which features heat maps for all
variables described in the config file.

By default, the .Rmd ``validation_default`` is used, but alternatives can be
used or created according to individual needs in ``inst/markdown``.

The report is saved in a folder called ``output`` in the current working
directory.

-> Be careful when using this function on big data sets and configs with many
variables as it might create very large html files.

```{r create report, eval = FALSE}
validationReport(c(scenPath, histPath), config, report = "default")
```

## Scenario Data

You can pass IAMC-style .mif, .csv or .xlsx files or vectors of paths to the
Expand All @@ -51,7 +108,8 @@ function. More precisely, any data file which can be read by

Reference data should follow the same format guidelines as scenario data with
the exception that the ``scenario`` column needs to read ``historical``. It is
passed to the validation function as part of the ``dataPath`` argument, e.g.:
passed to the validation function together with the scenario data as part of the
``dataPath`` argument, e.g.:

``validateScenarios(dataPath = c("<path_to_scenario_data>", "<path_to_reference_data>"),
config = ...)``
Expand All @@ -69,9 +127,9 @@ performed, different columns can or need to be filled.
**General Rules**

- when using reference data, "historical" is required in the column "scenario"
- one set of thresholds per row
- one set of thresholds per row is allowed
- empty rows and rows without "variable" are ignored and can be used to structure the config
- later rows overwrite earlier rows if thresholds overlap
- later rows overwrite earlier rows if there is a data overlap
- only one reference period/model/scenario per data-slice (e.g. don't compare the same data against ref_model1 and ref_model2)
- define multiple variables with "\*" (one sub-level) or "\*\*" (all sub-levels)
- ``relative`` thresholds can be given as percentage (``20%``) or decimal (``0.2``).
Expand Down Expand Up @@ -203,44 +261,3 @@ Example:
|----------|----------|--------------|-------|-------|----------|--------|------|---------|---------|---------|---------|-----------|--------------|------------|
| absolute | yes | Cap\|Electricity\|Wind | GW | | | USA | | | | | 50 | | |

## Scenario Validation

The function ``validateScenarios()`` performs all necessary steps of the
validation process. It takes the config file and iterates through each row,
assembling the required scenario and reference data and checking the thresholds.

The output is a data.frame, which combines scenario and reference data with
the threshold that is applied to each respective data point and the result of
the validation checks.
These results can be found in the columns ``check_value`` and ``check``, with
the former containing the value that is directly compared to the thresholds (
e.g. the calculated growth rate when doing a growth rate check) and the latter
being the result in form of a traffic-light color.

![Traffic-light threshold check results.](./thresholds.png){width=80%}

Optionally, you can save the resulting data.frame to a .csv file by providing
an ``outputFile``.

```{r run validation, eval = FALSE}
df <- validateScenarios(c(scenPath, histPath), config, outputFile = NULL)
```

## Creating a Validation Report

To perform the validation and create an output document in one go, the function
``validationReport()`` can be used. It renders an .html file which features heat
maps for all variables described in the config file.

By default, the .Rmd ``validation_default`` is used, but alternatives can be
used or created according to individual needs in ``inst/markdown``.

The report is saved in a folder called ``output`` in the current working
directory.

-> Be careful when using this function on big data sets and configs with many
variables as it might create very large html files.

```{r create report, eval = FALSE}
validationReport(c(scenPath, histPath), config, report = "default")
```

0 comments on commit 37b99fd

Please sign in to comment.