From c6f64a38cf2b1add37fb7359150b2da9d797962c Mon Sep 17 00:00:00 2001 From: Simon Pfreundschuh Date: Sun, 11 Feb 2024 17:39:09 -0700 Subject: [PATCH] Removed old documentation. --- doc/source/cli.rst | 158 ----------------------------- doc/source/conf.py | 57 ----------- doc/source/gprof_nn/data.rst | 5 - doc/source/gprof_nn/statistics.rst | 5 - doc/source/gprof_nn/validation.rst | 5 - doc/source/index.rst | 28 ----- doc/source/installation.rst | 113 --------------------- doc/source/python_package.rst | 11 -- 8 files changed, 382 deletions(-) delete mode 100644 doc/source/cli.rst delete mode 100644 doc/source/conf.py delete mode 100644 doc/source/gprof_nn/data.rst delete mode 100644 doc/source/gprof_nn/statistics.rst delete mode 100644 doc/source/gprof_nn/validation.rst delete mode 100644 doc/source/index.rst delete mode 100644 doc/source/installation.rst delete mode 100644 doc/source/python_package.rst diff --git a/doc/source/cli.rst b/doc/source/cli.rst deleted file mode 100644 index 030f52a..0000000 --- a/doc/source/cli.rst +++ /dev/null @@ -1,158 +0,0 @@ -Command line interface -====================== - -After successfully installing the gprof_nn package, executing the ``gprof_nn`` -command should produce a list of sub-commands provided by the application. -Detailed information on the options available for each sub-command can be -printed by issuing the following command. - -.. code-block:: console - - gprof_nn --help - - -Running the retrieval ---------------------- - -There are two ways of running ``gprof_nn`` retrievals. The first one uses the -publicly available, released models. The second one runs the retrieval using an -arbitrary retrieval model. - -Operational models -^^^^^^^^^^^^^^^^^^ - -``gprof_nn`` provides the ``1d`` and ``3d`` sub-commands for running the -operational gprof-nn 1d and 3d retrievals. These commands automatically download -the publicly released retrieval models and run them on a given input file. The -following command runs the GPROF-NN 1D retrieval in the ``ERA5`` configuration -on an input file ``input.pp`` in the GPROF preprocessor format. The results -are written to a file ``output.bin`` in GPM binary output format. - -.. code-block:: console - - gprof_nn 1d ERA5 input.pp -o output.bin - - -The ``gprof_nn`` command can also be used to write the output directly to NetCDF4 format by simply changing the suffix of the output file. - -.. code-block:: console - - gprof_nn 1d ERA5 input.pp -o output.nc - - -Experimental models -^^^^^^^^^^^^^^^^^^^ - -To run the GPROF-NN retrieval using an arbitrary ``gprof_nn`` neural-network -model, the ``gprof_nn retrieve`` command can be used as follow: - -.. code-block:: console - - gprof_nn retrieve model.pckl input.pp output.nc - -This command will process the input file ``input.pp`` using the model -``model.pckl`` and write the results in NetCDF4 format to the file -``output.nc``. - -Input files -^^^^^^^^^^^ - -The retrieval input can be a file in preprocessor format ending in ``.pp``, a -L1C file ending in ``.HDF5``, a NetCDF file in the same format as the training -data or a bin-file from the GPROF a priori database. If the input is a L1C file, -the preprocessor will be run automatically. - -If a directory is given as the input, all files with suffixes ``.pp``, ``.HDF5`` -and ``.nc`` are processed. The processing is parallelized and the number -of processes used can be customized using the ``--n_processes`` flag. - - -Output format -^^^^^^^^^^^^^ - -The output format can be set explicitly for all processed files -by setting the ``--format`` flag to ``GPROF_BINARY`` or ``NETCDF``. -Otherwise the output format will be inferred from the name of the -output file. If the output file ends in ``.bin`` the binary -GPROF format will be used. Otherwise the outputs will be stored -in NetCDF4 format. If no explicit output filename is give, the -output will be in GPROF binary format only if the input file -is a preprocessor file. - - -Generating training, validation and test data ---------------------------------------------- - -Separate training, validation and test data must be generated for each sensor -and preprocessor configuration. The command below generates training data for -the ``GMI`` sensor and the ``ERA5`` preprocessor configuration: - -.. code-block:: console - - gprof_nn extract_data GMI ERA5 training output_folder - -Training, validation and test data -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The data from the GPROF a priori database is split into training, validation and -test data using the day of the month of each sample. The first three days of each -month are used for the training data, days 4 up until 6 for the validation data -and the remaining samples for the training data. - -Validation and test data for a given sensor can be generated by exchangin the -``training`` argument of the ``gprof_nn extract_data`` command: - -.. code-block:: console - - gprof_nn extract_data GMI ERA5 validation output_folder - - -Subsetting the extracted data -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The ``gprof_nn extract_data`` provides additional arguments to restrict the -extracted training data to samples from specific total column water vapor -(TCWV) and two-meter temperature (T2m) values as well as surface types. - -The command below restricts the extracted samples to TCWV values in the interval -[10 mm, 11 mm], T2m values with [280 K, 281 K] and surface types 1 (Ocean) and 2 -(sea ice). - -.. code-block:: console - - gprof_nn extract_data GMI ERA5 validation output_folder --tcwv_bounds 10 11 --t2m_bounds 280 281 --surface_types 1 2 - -Excluding observations over snow and sea ice -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Since operational GPROF uses separate a priori databases for observations over snow -and sea ice, it can make sense to exclude samples over snow and sea ice from the training data. This can be done by adding the ``--no_mrms`` and ``--no_seaice`` flags -to the ``gprof_nn extract_data`` command, respectively. - - -Calculation of training data and observations statistics --------------------------------------------------------- - -The ``calculate_statistics`` sub-command provides functionality to calculate relevant statistics from the training data as well as L1 observations. The statistics are used to calculate corrrections for the - - -Training data statistics -^^^^^^^^^^^^^^^^^^^^^^^^ - -The following command calculates statistics of the training data for the GPROF-NN 1D retrieval -located in ``training_data_folder`` and writes NetCDF4 containing the results to the ``destination_folder``. - -.. code-block:: console - - gprof_nn calculate_statistics GMI training_1d training_data_folder destination_folder - -Observation statistics -^^^^^^^^^^^^^^^^^^^^^^ - -The following command calculates observation statistics for L1C observations -located in ``observation_data_folder`` and writes NetCDF4 containing the results to the ``destination_folder``. - - -.. code-block:: console - - gprof_nn calculate_statistics GMI observations observation_data_folder destination_folder diff --git a/doc/source/conf.py b/doc/source/conf.py deleted file mode 100644 index 823d0a1..0000000 --- a/doc/source/conf.py +++ /dev/null @@ -1,57 +0,0 @@ -# Configuration file for the Sphinx documentation builder. -# -# This file only contains a selection of the most common options. For a full -# list see the documentation: -# https://www.sphinx-doc.org/en/master/usage/configuration.html - -# -- Path setup -------------------------------------------------------------- - -# If extensions (or modules to document with autodoc) are in another directory, -# add these directories to sys.path here. If the directory is relative to the -# documentation root, use os.path.abspath to make it absolute, like shown here. -# -# import os -# import sys -# sys.path.insert(0, os.path.abspath('.')) - - -# -- Project information ----------------------------------------------------- - -project = 'gprof_nn' -copyright = '2022, Simon Pfreundschuh' -author = 'Simon Pfreundschuh' - -# The full version, including alpha/beta/rc tags -release = '0.1' - - -# -- General configuration --------------------------------------------------- - -# Add any Sphinx extension module names here, as strings. They can be -# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom -# ones. -extensions = [ - "sphinx.ext.autodoc", - "sphinx.ext.napoleon" -] - -# Add any paths that contain templates here, relative to this directory. -templates_path = ['_templates'] - -# List of patterns, relative to source directory, that match files and -# directories to ignore when looking for source files. -# This pattern also affects html_static_path and html_extra_path. -exclude_patterns = [] - - -# -- Options for HTML output ------------------------------------------------- - -# The theme to use for HTML and HTML Help pages. See the documentation for -# a list of builtin themes. -# -html_theme = 'alabaster' - -# Add any paths that contain custom static files (such as style sheets) here, -# relative to this directory. They are copied after the builtin static files, -# so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['_static'] diff --git a/doc/source/gprof_nn/data.rst b/doc/source/gprof_nn/data.rst deleted file mode 100644 index 3d66c69..0000000 --- a/doc/source/gprof_nn/data.rst +++ /dev/null @@ -1,5 +0,0 @@ -gprof_nn.data -============= - -.. automodule:: gprof_nn.data - :members: diff --git a/doc/source/gprof_nn/statistics.rst b/doc/source/gprof_nn/statistics.rst deleted file mode 100644 index fd7437e..0000000 --- a/doc/source/gprof_nn/statistics.rst +++ /dev/null @@ -1,5 +0,0 @@ -gprof_nn.statistics -=================== - -.. automodule:: gprof_nn.statistics - :members: diff --git a/doc/source/gprof_nn/validation.rst b/doc/source/gprof_nn/validation.rst deleted file mode 100644 index 0be3a57..0000000 --- a/doc/source/gprof_nn/validation.rst +++ /dev/null @@ -1,5 +0,0 @@ -gprof_nn.validation -=================== - -.. automodule:: gprof_nn.validation - :members: diff --git a/doc/source/index.rst b/doc/source/index.rst deleted file mode 100644 index 9ad3569..0000000 --- a/doc/source/index.rst +++ /dev/null @@ -1,28 +0,0 @@ -gprof_nn -======== - -This package implements the GPROF-NN 1D and GPROF-NN 3D retrievals for the passive microwave -observations from the Global Precipitation Measurement (GPM) mission. It consists of two -principal parts: - -1. The ``gprof_nn`` command line application, which provides data processing, training and - retrieval functionality. -2. The ``gprof_nn`` Python package, which provides functions for the processing GPM-related - data. - -.. toctree:: - :maxdepth: 2 - :caption: Contents: - - installation - cli - python_package - - - -Indices and tables -================== - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` diff --git a/doc/source/installation.rst b/doc/source/installation.rst deleted file mode 100644 index b9240ef..0000000 --- a/doc/source/installation.rst +++ /dev/null @@ -1,113 +0,0 @@ -Installation -============ - -There are two ways of installing ``gprof_nn``: - -1. Installing an official (pre-)release from `PyPI `_ using PIP -2. Installing the current development version from `GitHub `_. - -The first option is recommended for those who only want to run one of the -released ``gprof_nn`` retrievals. The second option is recommended for those who -want to develop new retrievals. - -PIP ---- - -To install the latest, official release of ``gprof_nn`` simply run - -.. code-block:: console - - pip install -U gprof_nn - - - -Development version -------------------- - -The development version of ``gprof_nn`` is required to train new GPROF retrievals. - -External dependencies -^^^^^^^^^^^^^^^^^^^^^ - -Unfortunately, not all dependencies for the development verions of ``gprof_nn`` -are available through PyPI and therefore are not installed automatically with -``pip``. The dependencies are listed in the ``conda_environment.yml`` file -in the root directory of the repository. - -Some of these packages are only available through the ``conda-forge`` -channel. If you haven't done so already, it may thus be necessary to add -the channel to conda. This can be done with the following two commands. - -.. code-block:: console - - conda config --add channels conda-forge - conda config --set channel_priority strict - - -You can then install these packages into a new conda environment called -``gprof_nn`` using the command below. After activating the environment, your -system will be set up with all external dependencies of ``gprof_nn``. - -.. code-block:: console - - conda env create -f conda_environment.yml - -.. note :: - Don't forget to activate the environment using ``conda activate gprof_nn`` after installing - it. - -Getting or updating the code -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The central repository containing the source code for ``gprof_nn`` is located -at `github.com/simonpf/gprof_nn `_. To -obtain a local copy of the code, run - -.. code-block:: console - - git clone https://github.com/simonpf/gprof_nn - -If you already have a local copy of the code and want to update it to the most -recent version, use the following command from the folder you have copied the -code to. - -.. code-block:: console - - git pull - -Installing ``gprof_nn`` -^^^^^^^^^^^^^^^^^^^^^^^ - -Finally, you can install ``gprof_nn`` by issuing the following command from the folder -you have cloned the code into: - -.. code-block:: console - - pip install -e .[development] - -.. note :: - Passing the ``-e`` flag to the ``pip`` command installs the package in editable mode. - This ensures that changes to the source code in this folder propagate to the ``gprof_nn`` - package installed in your Python environment. - - -Preprocessor binaries -^^^^^^^^^^^^^^^^^^^^^ - -Generating the ``gprof_nn`` training data requires the GPROF preprocessor -binaries to be available on the binary search path. The preprocessor binary for -the sensor ``SENSOR`` is expected to follow the naming convention - - -.. code-block:: console - - gprof2021pp__L1C - -That is, the binary for GMI should be called ``gprof2021pp_GMI_L1C``. - - -For sensors other than GMI an additional preprocessor binary must be available. -This should be a modified version of the GMI preprocessor, which loads the -surface type map for the sensor. For a given sensor ``SENSOR``, the executable -is expected to be named -``gprof2021pp_GMI__L1C``. diff --git a/doc/source/python_package.rst b/doc/source/python_package.rst deleted file mode 100644 index d844049..0000000 --- a/doc/source/python_package.rst +++ /dev/null @@ -1,11 +0,0 @@ -Python package -============== - - -.. toctree:: - :maxdepth: 2 - :caption: Sub-modules: - - gprof_nn/data - gprof_nn/validation - gprof_nn/statistics