Modify pipeline

.bumpversion.cfg

@@ -1,7 +1,7 @@
 [bumpversion]
 commit = True
 tag = True
-current_version = 1.0.0-dev
+current_version = 1.1.0-dev
 parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+))?
 serialize = 
 	{major}.{minor}.{patch}-{release}

README.md

@@ -1,53 +1,47 @@
 ![pytest](https://github.com/N3PDF/pycompressor/workflows/pytest/badge.svg)
 [![documentation](https://github.com/N3PDF/pycompressor/workflows/docs/badge.svg)](https://n3pdf.github.io/pycompressor/)
 
-### pycompressor
+## pycompressor
 
-Fast and efficient python implementation of PDF set **compressor** (https://arxiv.org/abs/1504.06469). 
+Fast and efficient python implementation of PDF **compression** (https://arxiv.org/abs/1504.06469). 
 
-#### New features
-
-Additional new features have been added to the following python package. The two main features are:
-- **Covariance Matrix Adaptation-Evlotion strategy (CMA-ES):** in addition to the Genetic 
-Algorithm (GA), there is now the possibility to choose as a minimizer the CMA. The choice
-of minimizer can be defined in the `runcard.yml` file.
-- **Generative Adversarial Strategy (GANs):** this is a standalone python [package](https://github.com/N3PDF/ganpdfs/tree/master)
-that can enhance the statistics of the prior PDF replicas before compression by generating 
-synthetic replicas. For more details, refer to the [documentation](https://n3pdf.github.io/ganpdfs/) 
-(still has to be done). In a similar way, in order to trigger the enhancement, one just has to set 
-the value of `enhance` in the runcard to be `True`. Setting this value to `False` will just run the 
-standard compression. The GANs also requires extra-parameters (as shown in the example 
-[runcard.yml](https://github.com/N3PDF/pycompressor/blob/master/runcard.yml)) that defines
-the structure of the networks.
-
-#### Installation
+### How to install
 
 To install `pyCompressor`, just type:
 ```bash
-python setup.py install
-```
-or if you are a developer:
-```bash
-python setup.py develop
+python setup.py install # or python setup.py develop (if you want development mode)
 ```
 
-#### How to use
+### How to use
+
+#### Standard compression
 
-The input parameters that define the compression is contained in a YAML file. To run
-the `pycompressor` code, just type the following: 
+The input parameters that define the compression are contained in a YAML file. To run the standard compression,
+use the reference [runcard](https://github.com/N3PDF/pycompressor/blob/master/runcards/runcard.yml) as it is by 
+just replacing the entry of the `pdf` key with the name of the PDF set to be compressed, then run the following:
 ```bash
 pycomp runcards/runcard.yml [--threads NUMB_THREADS]
 ```
-A detailed instruction on how to set the different parameters in the runcard can be found here.
 
-#### Generating compressed PDF set & post-analysis
+#### Using GAN and/or Compressing from an enhanced set
+
+Although it is advised to run the [ganpdfs](https://github.com/N3PDF/ganpdfs) independently, it is possible
+to generate enhanced PDF replicas within the `pycompressor`. To do so, just set the entry `enhance` in the
+runcard to `True` and specify the total number of replicas (prior+synthetics).
+
+Finally, in  order to perform a compression with an enhanced set, set the entry `existing_enhanced` to `True`.
+
+A detailed instruction on how to set the different parameters in the runcard can be found 
+[here](https://n3pdf.github.io/pycompressor/howto/howto.html).
+
+### Generating compressed PDF set & post-analysis
 
 The code will create a folder named after the prior PDF sets. To generate the
 compressed PDF grid, run the following command:
 ```bash
 get-grid -i <PDF_NAME>/compressed_<PDF_NAME>_<NB_COMPRESSED>_output.dat
 ```
-Note that if the compression is done from an enhanced set, the output folder will be append by `_enhanced`.
+Note that if the compression is done from an enhanced set, the output folder will be appended by `_enhanced`.
 
 Finally, in order to generate ERF plots, enter in the `erfs_output` directory and run the following:
 ```bash
@@ -56,7 +50,7 @@ validate --random erf_randomized.dat --reduced erf_reduced.dat
 This script can also plot the ERF validation from the old compressor code by adding the flag
 `--format ccomp`.
 
-#### Warning
+### Warning
 
 This package cannot be installed with python 3.9 yet due to the numba dependency. This will be resolved
 soon according to [#6579](https://github.com/numba/numba/pull/6579).

doc/source/howto/howto.rst

@@ -24,7 +24,7 @@ One of the keys for the ``gan`` entry is a ``runcard`` which gets passed to the
 For details on how to set the parameters for the GAN, have a look `here <https://n3pdf.github.io/ganpdfs/howto/howto.html>`_.
 
 
-An example of run card is shown below:
+An example of an input card is shown below:
 
 
 .. code-block:: yaml
@@ -75,7 +75,14 @@ An example of run card is shown below:
      total_replicas: 3000
 
 
-If ``enhance`` is set to `True`, the code will first enhance the statistic the prior using GANs.
+
+Running GANs within pyCompressor
+--------------------------------
+
+
+Although it is advised to run the `ganpdfs` code independently, it is possible to call it
+within the `pyCompressor` code by setting ``enhance`` to `True` in the runcard. In this
+scenario, the code will first enhance the statistic the prior using GANs.
 Once the generation of the extra-replicas is finished, the output grids are evolved using
 `evolven3fit <https://github.com/NNPDF/nnpdf/blob/master/n3fit/evolven3fit/evolven3fit.cc>`_. 
 Then, the :mod:`pyCompressor.postgans` module (in a similar fashion as postfit) creates a 
@@ -123,6 +130,18 @@ If ``enhance`` is instead set to `False`, the folder will just simply be:
 
 
 
+Adiabatic minimization
+----------------------
+
+
+Since compressing from an enhanced set could be difficult due to the limitation of the minimization
+algorithm, it is possible to perfrom an adiabatic minimization by setting ``existing_enhanced`` to
+`True` in the runcard. In this case, the minimization is perfromed in two steps: (1) a standard
+compression of the prior, (2) a compression using the enhanced set but using as a starting point
+the space in which the best from the standard compression was generated.
+
+
+
 PDF grid and Validation plot
 ============================
 

runcards/ganpdfs.yml

@@ -1,8 +1,3 @@
-#############################################################################################
-# Input PDF                                                                                 #
-#############################################################################################
-pdf: NNPDF40_nnlo_as_0118_1000
-
 #############################################################################################
 # PDF Grids:                                                                                #
 # ---------                                                                                 #
@@ -70,4 +65,3 @@ nd_steps   : 4                                       # Number of steps to train
 ng_steps   : 3                                       # Number of steps to train the Generator for one training run
 batch_size : 70                                      # Batch size per epoch in terms of percentage
 epochs     : 1000                                    # Number of epochs
-pdf: NNPDF40_nnlo_as_0118_1000

setup.py

@@ -34,7 +34,7 @@
 
 setup(
     name=PACKAGE,
-    version='1.0.0-dev',
+    version='1.1.0-dev',
     description="PDF Compression",
     long_description=LONG_DESCRIPTION,
     long_description_content_type="text/markdown",

src/pycompressor/__init__.py

@@ -1 +1 @@
-__version__ = "1.0.0-dev"
+__version__ = "1.1.0-dev"