22.12

Main notes

The new version of DeepPhysX brings a lot of changes with the data storage and the data flow between components.
Instead of using Numpy tensors referenced as input and output of neural networks, data is now streamed on a SQL Database that can contain any data field.
This allows to easily design new data flows and to interface with more complex network architectures and learning algorithms.

The SQL Database features are implemented in an external Python library - SimulationSimpleDatabase - that must be installed to use this release.
--> quick install with pip install SimulationSimpleDatabase

Changes

Dataset

• the project now uses SimulationSimpleDatabase to store the Dataset;
• any data field can be added to the Database;
• you can convert your previouly generated datasets to match the new release using DeepPhysX.Core.Utils.converter.DatasetConverter tools;
• some API updates for DatabaseConfig (see examples/tutorial for further details).

  from DeepPhysX.Core.Utils.converter import DatabaseConverter
  
  # Considering that your dataset is in the working session 'my_session/dataset/<dataset_partitions>'
  converter = DatabaseConverter(session_path='my_session')
  converter.numpy_to_database(batch_size=16, max_file_size=3, normalize=True)

Environment

• the project now uses SimulationSimpleDatabase for the visualization tool, see the dedicated documentation to use the Factory;
• any data field can be added to the Database, they must be initialized in the new init_database method, using the set_training_data and set_additional_data methods;
• training data and additional data must be defined using the set_training_data and set_additional_data methods respectively, using the fields defined at initialization;
• the apply_prediction method now receives a dictionary that contains all the fields produced by the Network (these data fields are defined in the Network);
• some API updates for BaseEnvironment and BaseEnvironmentConfig (see examples/tutorial for further details).

  from DeepPhysX.Core.Environment.BaseEnvironment import BaseEnvironment
  
  class MyEnvironment(BaseEnvironment):
  
      def __init__(self, as_tcp_ip_client=True, instance_id=1, instance_nb=1, **kwargs):
          BaseEnvironment.__init__(self, as_tcp_ip_client=as_tcp_ip_client, instance_id=instance_id, instance_nb=instance_nb)
  
      def init_database(self):
          self.define_training_fields(fields=[('input', ndarray), ('ground_truth', ndarray)])
          self.define_additional_fields(fields=[('value', int)])
  
      def init_visualization(self):
          self.factory.add_mesh(positions=..., cells=...)
  
      def step(self):
          self.set_training_data(input=..., ground_truth=...)
          self.set_additional_data(value=...)
  
      def apply_prediction(self, prediction):
          network_output = prediction['prediction'] # Depends on the fields defined in the Network

Network

• the Network object now has three variables that must be set to define the data field names: net_fields for the input data fields (should match the data field names defined in the Environment), opt_fields for the data used for the optimization process, pred_fields for the data that will be returned to the Environment;
• some API updates for BaseNetwork and BaseNetworkConfig (see examples/tutorial for further details).

  from DeepPhysX.Core.Network.BaseNetwork import BaseNetwork
  
  class MyNetwork(BaseNetwork):
    
        def __init__(self, config):
            BaseNetwork.__init__(self, config)
            self.net_fields = ['input']
            self.opt_fields = ['ground_truth']
            self.pred_fields = ['prediction']