Pipestat standardizes reporting of pipeline results. It provides 1) a standard specification for how pipeline outputs should be stored; and 2) an implementation to easily write results to that format from within Python or from the command line. A pipeline author defines all the outputs produced by a pipeline by writing a JSON-schema. The pipeline then uses pipestat to report pipeline outputs as the pipeline runs, either via the Python API or command line interface. The user configures results to be stored either in a YAML-formatted file or a PostgreSQL database.
See Pipestat documentation for complete details.
Note: to run the pytest suite locally, you will need to install the related requirements:
cd pipestat
pip install -r requirements/requirements-test.txt
Many of the tests require a postgres database to be set up otherwise many of the tests will skip.
We recommend using docker:
docker run --rm -it --name pipestat_test_db \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=pipestat-password \
-e POSTGRES_DB=pipestat-test \
-p 5432:5432 \
postgres