Merge pull request #34 from IMMM-SFA/api

Merge Request for API branch

README.md

@@ -1,69 +1,32 @@
 [![build](https://github.com/IMMM-SFA/statemodify/actions/workflows/build.yml/badge.svg)](https://github.com/IMMM-SFA/statemodify/actions/workflows/build.yml)
 [![codecov](https://codecov.io/gh/IMMM-SFA/statemodify/branch/main/graph/badge.svg?token=csQBZMRSdp)](https://codecov.io/gh/IMMM-SFA/statemodify)
+[![Static Badge](https://img.shields.io/badge/Powered%20by-MSDLIVE-blue?label=Powered%20by&color=blue)](https://statemodify.msdlive.org)
 
-# statemodify
-A package to modify StateMod's input and output files for exploratory modeling
-
-## In development
-While in development, install `statemodify` using the following steps:
-- Clone the repo using `git clone https://github.com/IMMM-SFA/statemodify.git`
-- Navigate to the directory in which `statemodify` was cloned and run `python setup.py develop`.  Ensure your `python` points to a Python 3.8 and up instance.
-- Test the install by entering into a Python prompt and running:  `import statemodify`, if no errors, you are all good.
-
-## Functionality
-### Generate a set of .ddm files using a LHS
-```python
-import statemodify as stm
-
-# a dictionary to describe what you want to modify and the bounds for the LHS
-setup_dict = {
-    "names": ["municipal", "standard"],
-    "ids": [["7200764", "7200813CH"], ["7200764_I", "7200818"]],
-    "bounds": [[-1.0, 1.0], [-1.0, 1.0]]
-}
-
-output_directory = "<your desired output directory>"
-scenario = "<your scenario name>"
 
-# the number of samples you wish to generate
-n_samples = 4
+# statemodify
+`statemodify` is an open-source Python package that provides users with a way to easily modify [StateMod](https://github.com/OpenCDSS/cdss-app-statemod-fortran)'s input and output files to enable exploratory modeling. StateMod is written in Fortran and conducting high-performance computing enabled ensemble exploratory modeling with it requires a systematic and automated approach. Due to the model’s complexity, there are also nontrivial computational challenges in comprehensively sampling the model’s input space and managing the outputs of interest, especially for large ensembles. These challenges limit its use among researchers and broader operational users. Thus, we developed `statemodify`, a Python package and framework that allows users to easily interact with StateMod using Python exclusively. The user can implement statemodify functions to manipulate StateMod’s input files to develop alternative demand, hydrology, infrastructure, and institutional scenarios for Colorado’s West Slope basins and run these scenarios through StateMod. We also create methods to compress and extract model output into easily readable data frames and provide guidance on analysis and visualization of output in a series of Jupyter notebooks that step through the functionality of the package.
 
-# seed value for reproducibility if so desired
-seed_value = None
+## Documentation
+Full documentation and tutorials are provided [here](https://immm-sfa.github.io/statemodify).
 
-# my template file.  If none passed into the `modify_ddm` function, the default file will be used.
-template_file = "<your ddm template file>"
+## Installation
+Install `statemodify` using pip:
+```bash
+pip install statemodify
+```
 
-# the field that you want to use to query and modify the data
-query_field = "id"
+## Online quickstarter tutorials
+Take a `statemodify` for a spin in one of our no-install Jupyter quickstarters [here](https://statemodify.msdlive.org):
 
-# generate a batch of files using generated LHS
-stm.modify_ddm(modify_dict=setup_dict,
-               output_dir=output_directory,
-               scenario=scenario,
-               n_samples=n_samples,
-               seed_value=seed_value,
-               query_field=query_field,
-               template_file=template_file)
-```
+- **Notebook 1**:  Getting Started and using the DDM and DDR modification functions in the San Juan Subbasin.  This notebook has more general intro information on `statemodify` and shows how changes to demand and water rights can lead to changes to user shortages in the San Juan Subbasin.
 
-### Convert output .xdd files to .parquet files
-Parquet files are efficient columnar data stores that easily interoperate with pandas dataframes.
-```python
-from statemodify.xdd import XddConverter
+- **Notebook 2**:  Using the EVA modification functions in the Gunnison Subbasin.  This notebook looks at how changes in evaporation in reservoirs in the Gunnison subbasin lead to changes to reservoir levels.
 
-# set up the converter
-converter = XddConverter(
-    output_path='./output',
-    allow_overwrite=False,
-    xdd_files='**/*.xdd',
-    id_subset=None,
-    parallel_jobs=4,
-)
+- **Notebook 3**:  Using the RES modification function in the Upper Colorado Subbasin.  This notebook looks at how changes in storage in reservoirs in the Upper Colorado subbasin lead to changes to user shortages.
 
-# convert the files
-converter.convert()
+- **Notebook 4**:  Using the XBM and IWR modification functions across all basins.  This notebook debuts the stationary Hidden Markov Model to generate alternative streamflow scenarios across the basins.
 
-# look for your parquet files in './output'!
+- **Notebook 5**:  Sampling multiple uncertainties.  This notebook demonstrates how to create a global Latin hypercube sample to consider multiple uncertainties in a basin.
 
-```
+## Contributing to `statemodify`
+Whether you find a typo in the documentation, find a bug, or want to develop functionality that you think will make statemodify more robust, you are welcome to contribute!  See the full contribution guidelines in our [online documentation](https://immm-sfa.github.io/statemodify/reference/contributing.html).

docs/source/conf.py

@@ -96,9 +96,9 @@
 
 # The name of an image file (within the static path) to use as favicon of the
 # docs. This file should be a Windows icon file (.ico), 16x16 or 32x32 pixels.
-html_favicon = '_static/favicon.ico'
+# html_favicon = '_static/favicon.ico'
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named 'default.css' will overwrite the builtin 'default.css'.
-html_static_path = ['_static']
+# html_static_path = ['_static']

docs/source/getting-started/quickstarter.rst

@@ -1,2 +1,56 @@
 Quickstarter
 ==============
+
+To demonstrate the functionality of **statemodify**, we have generated 5 notebooks that highlight how to use the functions in the package and show some very basic analyses that can be done with the output data. Each notebook demonstrates one to two **statemodify** functions for a specific West Slope basin in Colorado. The notebooks usually follow a specific form:
+
+1. Run a baseline simulation for the basin (about 4 minutes)
+
+2. Plot the baseline shortages for specific users or reservoir storage
+
+3. Introduce a **statemodify** function that samples a specific uncertainty (evaporation, demand, streamflow, water rights, etc.)
+
+4. Create a new set of input files to run through StateMod
+
+5. Run each StateMod simulation (usually about 2 simulations, each at 4 minutes)
+
+6. Plot the new shortages or reservoir levels with respect to the baseline values.
+
+
+The notebooks are hosted in MSD-LIVE containers and can be found at: `https:/statemodify.msdlive.org <https:/statemodify.msdlive.org>` 
+
+
+You will be taken to a JupyterLab homepage with "data" and "notebook" buckets on the side.  The "data" directory contains all the StateMod datasets and some other template files. Click on the "notebooks" directory and choose a notebook to start. Press "shift + enter" to execute the cells in the notebook.
+
+
+.. figure:: ../images/quickstarter_1.png
+  :alt: JupyterLab home screen
+
+  Figure 1: JupyterLab home screen
+
+IMPORTANT: When you finish a notebook and move on to another one, please restart the kernel and clear the outputs as shown below. 
+
+.. figure:: ../images/quickstarter_2.png
+  :alt: Restarting the Kernel 
+
+  Figure 1: Restarting the Kernel 
+
+
+The table below lists the  
+
+.. list-table:: Notebook Topics
+   :widths: 25 50 50 50 50 50
+   :header-rows: 1
+
+   * - Topic
+     - Notebook 1: Getting Started and using the DDM and DDR modification functions in the San Juan Subbasin
+     - Notebook 2: Using the EVA modification functions in the Gunnison Subbasin
+     - Notebook 3: Using the RES modification function in the Upper Colorado Subbasin
+     - Notebook 4: Using the XBM and IWR modification functions across all basins
+     - Notebook 5: Sampling multiple uncertainties
+
+   * - Description
+     - This notebook has more general intro information on **statemodify** and shows how changes to demand and water rights can lead to changes to user shortages in the San Juan Subbasin. 
+     - This notebook looks at how changes in evaporation in  reservoirs in the Gunnison subbasin lead to changes to reservoir levels
+     - This notebook looks at how changes in storage in reservoirs in the Upper Colorado subbasin lead to changes to user shortages 
+     - This notebook debuts the stationary Hidden Markov Model to generate alternative streamflow scenarios across the basins. 
+     - This notebook demonstrates how to create a global Latin hypercube sample to consider multiple uncertainties in a basin. 

docs/source/getting-started/useful_links.rst

@@ -0,0 +1,33 @@
+Useful Links
+--------------------------------
+
+Planning and Management Models
++++++++++++++++++++++++++++++++
+
+The Colorado Water Conservation Board (CWCB) and the Division of Water Resources (DWR) have jointly developed Colorado's Decision Support System (CDSS), a collection of databases, data viewing and management tools, and models to support water resources planning in Colorado's major water basins (Malers et al., 2001). The CDSS is made up of a central database with water resources data (HydroBase), a public website where the data can be accessed, a Geographic Information System (GIS) for viewing and analyzing the data, and a consumptive use model (StateCU) that estimates consumptive use by each irrigation unit in a basin. The outputs from StateCU are then input to the State of Colorado's Stream Simulation Model (StateMod), a generic network-based water system model for water accounting and allocation, and the final component of CDSS. StateMod was developed to support comprehensive assessments of water demand, allocation, and use, as well as reservoir operations. It represents all of the major sub-basins within the state of Colorado (i.e., Parsons & Bennett, 2006; White, Yampa, Upper Colorado, Gunnison, Dolores, San Juan, and San Miguel CWCB, 2012). StateMod replicates each basin's unique application and enforcement of the prior appropriation doctrine and accounts for all of the consumptive use within the basins. To do so, it relies on the detailed historic demand and operation records contained in HydroBase that include individual water right information for all consumptive use, data on water structures (wells, ditches, reservoirs, and tunnels), and streamflow data. Further, StateMod uses irrigation consumptive use data output from StateCU, which calculates water consumption based on soil moisture, crop type, irrigated acreage, and conveyance and application efficiencies for each individual irrigation unit in the region.
+
+**statemodify** complements CDSS tools such as TSTools and StateDMI (available on the `OpenCDSS <https://opencdss.state.co.us/opencdss/>` website) by focusing on providing tools that accommodate large ensemble exploratory modeling and a Linux-based workflow and to provide additional options to develop more targeted file adjustments and richer streamflow scenarios. 
+
+
+
+Exploratory Modeling
++++++++++++++++++++++++++++++++
+
+Many of the methods and techniques displayed in the **statemodify** package have been debuted in peer-reviewed literature. Please see the following publications for more information: 
+
+Hadjimichael, A., Quinn, J., Wilson, E., Reed, P., Basdekas, L., Yates, D., & Garrison, M. (2020). Defining robustness, vulnerabilities, and consequential scenarios for diverse stakeholder interests in institutionally complex river basins. Earth's Future, 8(7), e2020EF001503.
+
+Quinn, J. D., Hadjimichael, A., Reed, P. M., & Steinschneider, S. (2020). Can exploratory modeling of water scarcity vulnerabilities and robustness be scenario neutral?. Earth's Future, 8(11), e2020EF001650.
+
+Hadjimichael, A., Quinn, J., & Reed, P. (2020). Advancing diagnostic model evaluation to better understand water shortage mechanisms in institutionally complex river basins. Water Resources Research, 56(10), e2020WR028079.
+
+
+
+References
+---------------------------
+Malers, S. A., Bennett, R. R., & Nutting-Lane, C. (2001). Colorado's decision support systems: Data-centered water resources planning and administration. In Watershed Management and Operations Management 2000 (pp. 1-9).
+
+Parsons, R., & Bennett, R. (2006). Reservoir operations management using a water resources model. In Operating Reservoirs in Changing Conditions (pp. 304-311).
+
+
+