Initial High Level Docs

docs/README.md

@@ -22,38 +22,3 @@ make html
 _Please make sure the documentation can build correctly before merging changes to main!_
 
 The full doc files will be in `build` and can be displayed in the browser by opening `build/html/index.html`.
-
-## Setting up and Training your Model
-
-We provide our config system as well as our own engine and command line interface. This allows us to train models in a very flexible way. We also provide a number of pre-defined models and datasets that you can use to train your models.
-
-### Config System
-
-The config system is a simple way to configure your model. It heavily relies on the [ml_collections](https://github.com/google/ml_collections) library and is structured in a similar way.
-In order to configure your model and training pipeline, a python file with a `get_config` function is required. This function should return a `ConfigDict` object. Have a look at the example config files located at `vis4d/config/examples` for more information.
-
-Additionally, you can also define a `get_sweep` function that returns a `ConfigDict` object in the same file. This function is used to define a hyperparameter sweep.
-
-### Training
-
-To train a model, you can either use our engine or use the pytorch_lightning trainer directly. The CLI interfaces for both are the same, however, the main script is either `vis4d.engine.run` or `vis4d.pl.run`.
-
-Here are a few examples on how to train a model using the CLI interfaces:
-
-#### Using our engine
-
-```bash
-# Training faster_rcnn on the coco dataset and one gpu
-python -m vis4d.engine.run --config vis4d/config/example/faster_rcnn_coco.py --gpus 1
-# You can overwrite any config value by passing it to the cli. This updates the "num_epochs" value in the config dict.
-python -m vis4d.engine.run --config vis4d/config/example/faster_rcnn_coco.py --gpus 1 --config.num_epochs 100
-# You can also perform a hyperparameter sweep by passing a sweep config file to the cli.
-python -m vis4d.engine.run --config vis4d/config/example/faster_rcnn_coco.py --sweep vis4d/config/example/faster_rcnn_coco.py
-```
-
-#### Using Pytorch Lightning
-
-```bash
-# Training faster_rcnn on the coco dataset and one gpu
-python -m vis4d.pl.run --config vis4d/config/example/faster_rcnn_coco.py --gpus 1
-```

docs/requirements.txt

@@ -5,3 +5,4 @@ sphinx_autodoc_defaultargs
 sphinx_autodoc_typehints
 sphinx_copybutton
 sphinx_design
+sphinxcontrib-aafig

docs/source/conf.py

@@ -45,6 +45,7 @@
     "sphinx_copybutton",
     "sphinx.ext.githubpages",
     "sphinx_design",
+    "sphinxcontrib.aafig",
     "myst_nb",
 ]
 
@@ -101,13 +102,13 @@
 # -- auto doc settings -------------------------------------------------------
 autosummary_generate = True
 autodoc_member_order = "groupwise"
-autoclass_content = "class"
+autoclass_content = "both"
 add_module_names = False  # Remove namespaces from class/method signatures
 autodoc_default_options = {
     "members": True,
     "methods": True,
-    "special-members": "__call__,__init__",
-    "exclude-members": "_abc_impl",
+    "special-members": "__call__",
+    "exclude-members": "_abc_impl,__init__",
 }
 
 # -- Napoleon settings -------------------------------------------------------

docs/source/dev_guide/cli.rst

@@ -0,0 +1,92 @@
+###
+CLI
+###
+We provide a command line interface for training and evaluating your models.
+Assuming you have installed the package using pip, you can use the command `vis4d` to access the CLI.
+
+Alternatively, you can run the CLI using `python -m vis4d.engine.cli` or `python -m vis4d.pl.cli` if you want to use the PyTorch Lightning version.
+
+The CLI relies on a configuration file to specify each experiment. We use `ml_collections <https://github.com/google/ml_collections>`_ as underlying framework to define the configuration files.
+You can read up on our configuration files in the `Config System <configuration_files>`_ section.
+
+=============
+CLI Interface
+=============
+The provided examples assume that the experiment configuration file is located at `path_to_experiment_cfg.py`.
+You can read up on our configuration files in the `Config System <configuration_files>`_ section.
+
+We support both, our own training engine as well as `PyTorch Lightning <https://www.pytorchlightning.ai/>`_.
+
+------------
+CLI Commands
+------------
+.. code-block:: bash
+
+  vis4d [fit | test] --config path_to_experiment_cfg.py
+                     --ckpt: Checkpoint path
+                     --config: path to config file [.py |.yaml].
+                     --gpus: Number of GPUs to use. Default 0.
+                     --print-config: If set, prints the configuration to the console.
+                     --resume: Resume training from the provided checkpoint.
+                     --sweep: path to a parameter sweep configuration file [.py |.yaml].
+
+.. code-block:: bash
+
+  vis4d-pl [fit | test] --config path_to_experiment_cfg.py
+                        --ckpt: Checkpoint path
+                        --config: path to the config file [.py |.yaml].
+                        --gpus: Number of GPUs to use. Default 0.
+                        --print-config: If set, prints the configuration to the console.
+                        --resume: Resume training from the provided checkpoint.
+
+-----------
+Quick Start
+-----------
+
+**Train a model**
+
+.. code-block:: bash
+
+  vis4d fit --config path_to_experiment_cfg.py
+
+
+**Test a model**
+
+.. code-block:: bash
+
+   vis4d fit --config path_to_experiment_cfg.py
+
+**Overwrite Config Parameters**
+
+.. code-block:: bash
+
+   vis4d fit --config path_to_experiment_cfg.py --config.my_field=2 --config.my.nested.field="test"
+
+
+
+**Perform Parameter Sweeps**
+
+.. code-block:: bash
+
+   vis4d fit --config path_to_experiment_cfg --sweep path_to_sweep_cfg.py
+
+---------------------------
+Overwrite Config Parameters
+---------------------------
+
+We support overwriting config parameters via the CLI. Assuming you have a config parameter `params.lr` in your config file, you can overwrite it using the following command:
+
+.. code-block:: bash
+
+   vis4d fit --config path_to_experiment_cfg.py --config.params.lr=0.01
+
+Note that misstyping a config parameter
+
+.. code-block:: bash
+
+   vis4d fit --config path_to_experiment_cfg.py --config.params.lrs=0.01
+
+will result in the following error:
+.. code-block:: bash
+
+   AttributeError: Did you mean "lr" instead of "lrw"?'

docs/source/dev_guide/config.rst

@@ -0,0 +1,194 @@
+######
+Config
+######
+
+We provide a simple and flexible config system that allows you to easily define experiments as well as create new models, datasets, and other components.
+For this, we build up on `ml_collections <https://github.com/google/ml_collections>`_ to provide a simple and flexible config system.
+While it is possible to create configs using yaml files, we recommend using the provided python API to create configs.
+Using the python API allows you to use the IDE to autocomplete config fields and allows to utilize pythons built-in import system as well as type annotations.
+
+We use `FieldConfigDict <TODO>`_ as the base class for all configs. This class works similar to a python dictionary, but uses references instead of values to store the config values.
+
+=================
+Experiment Config
+=================
+Each experiment is defined by a config that inherits from `ExperimentConfig <TODO>`_.
+A valid experiment config must define the following fields:
+
+data
+    FieldConfigDict containining test (and training) data loader.
+output_dir
+    Output Directory for the experiment. Log files, checkpoints and reproducible configs will be stored here.
+train_data_connector
+    DataConnector for the training data. This defines how the training data should be fed to the model.
+test_data_connector
+    DataConnector for the test data. This defines how the test data should be fed to the model.
+model
+    ModelConfig defining the model. It is assumed to have a `forward_train` and `forward_test` method.
+optimizers
+    List of optimizers to use for training.
+loss
+    Loss config defining the loss function to use.
+callbacks
+    List of callbacks to use during training.
+params
+    Parameters for the experiment. This can be used to store arbitrary values which are often
+    modified during training. Allowing for easy access to these values using the CLI.
+
+===================
+Instantiate Configs
+===================
+A key feature of the config system is the ability to instantiate configs from FieldConfigDict.
+By defining the config in python code, we can use the IDE to autocomplete config fields and use pythons import system.
+This allows us to resolve the full class and function names without having to explicitly specify the full path.
+For example, we can define a model config as follows:
+
+.. code-block:: python
+
+    from vis4d.config import FieldConfigDict, instantiate_classes, class_config
+    from vis4d.model.detect.mask_rcnn import MaskRCNN
+
+    # Create an instantiable config that can be used to create a MaskRCNN instance
+    # With provvided kwargs
+    config = class_config(MaskRCNN, num_classes = 10)
+    model = instantiate_classes(config)
+    print(type(model))
+
+.. code-block:: bash
+
+    >> <class 'vis4d.model.detect.mask_rcnn.MaskRCNN'>
+
+Note that the class_config function will automatically resolve the full class or function.
+This means that we can use the class name directly without having to specify the full path.
+Alternatively, we can also use the full path to the class or function:
+
+.. code-block:: python
+
+    config = class_config("vis4d.model.detect.mask_rcnn.MaskRCNN", num_classes = 10)
+    model = instantiate_classes(config)
+
+Or directly define the config structure ourselves:
+
+.. code-block:: python
+
+    config = FieldConfigDict()
+    config.class_path = "vis4d.model.detect.mask_rcnn.MaskRCNN"
+    config.init_args = FieldConfigDict()
+    config.init_args.num_classes = 10
+    model = instantiate_classes(config)
+
+=========================
+Referencing Config Fields
+=========================
+A key functionality of the config system is the ability to reference other config fields.
+This allows to easily reuse configs and to create complex configs that are easy to modify.
+
+By default, all config fields will be treated as references. This means, that
+changing a field in one config will also change the field in all other configs that reference it.
+
+.. code-block:: python
+
+    from vis4d.config import FieldConfigDict
+    c1, c2 = FieldConfigDict(), FieldConfigDict()
+    c1.field = "test"
+    c2.field = c1.field
+    print(c1.field.get(), c2.field.get())
+    # >> test test
+    c1.field = "changed"
+    print(c1.field.get(), c2.field.get())
+    # >> changed changed
+
+This means, that the dot operator will always return a reference to the field.
+Once you are done building the config, you should call `confgi.value_mode()` to switch to value mode, which will return the actual value instead of a reference.
+
+.. code-block:: python
+
+    from vis4d.config import FieldConfigDict
+    c1 = FieldConfigDict()
+    c1.field = "test"
+    print(c1.field)
+
+.. code-block:: bash
+
+    >>  <ml_collections.config_dict.config_dict.FieldReference object at 0x7f17e7507d60>
+
+.. code-block:: python
+
+    # Changing config dict to value mode
+    c1.value_mode()
+    print(c1.field)
+
+.. code-block:: bash
+
+    >> "test"
+
+.. code-block:: python
+
+    # Change back to reference mode
+    c1.ref_mode()
+    print(c1.field)
+
+.. code-block:: bash
+
+    >>  <ml_collections.config_dict.config_dict.FieldReference object at 0x7f17e7507d60>
+
+===============================
+Callbacks and Trainer Arguments
+===============================
+We support custom Callbacks as well as Pytorch Lightning Trainer Arguments.
+
+--------------------
+Using the Python API
+--------------------
+While we provide a CLI for training and evaluating your models, you can also use the python API directly.
+
+-----------------------
+Using the Trainer class
+-----------------------
+The following example shows how to train a model using our own training engine.
+We provide a `Trainer` class that handles the training and evaluation loop for you.
+For more details, head over to the `Trainer <TODO>`_ class documentation.
+
+.. code-block:: python
+
+   from vis4d.engine.experiment import run_experiment
+   from vis4d.config import instantiate_classes
+   from vis4d.engine.optim import set_up_optimizers
+
+   # Load your Config here
+   # from your_config import get_config
+   config = get_config()
+   model = instantiate_classes(config.model)
+
+   # Callbacks
+   callbacks = [instantiate_classes(cb) for cb in config.callbacks]
+   mode = "fit|test" # Set to "fit" if you want to train a model, "test" if you want to evaluate a model
+
+    # Setup Dataloaders & seed
+    if mode == "fit":
+        train_dataloader = instantiate_classes(config.data.train_dataloader)
+        train_data_connector = instantiate_classes(config.train_data_connector)
+        optimizers, lr_schedulers = set_up_optimizers(config.optimizers, [model])
+        loss = instantiate_classes(config.loss)
+    else:
+        train_dataloader = None
+        train_data_connector = None
+
+    test_dataloader = instantiate_classes(config.data.test_dataloader)
+    test_data_connector = instantiate_classes(config.test_data_connector)
+
+    trainer = Trainer(
+        device=device,
+        output_dir=config.output_dir,
+        train_dataloader=train_dataloader,
+        test_dataloader=test_dataloader,
+        train_data_connector=train_data_connector,
+        test_data_connector=test_data_connector,
+        callbacks=callbacks,
+        num_epochs=config.params.get("num_epochs", -1),
+    )
+
+    if mode == "fit":
+        trainer.fit(model, optimizers, lr_schedulers, loss)
+    elif mode == "test":
+        trainer.test(model)

docs/source/dev_guide/data_connectors.rst

@@ -0,0 +1,2 @@
+Data Connectors
+===============

docs/source/extensions.rst

@@ -0,0 +1,3 @@
+**********
+Extensions
+**********