Merge pull request #428 from assume-framework/RL%Splitting-Workshop-DACH

Market zones and DRL workshop for the DACH conference

README.md

@@ -97,12 +97,15 @@ To ease your way into ASSUME we provided some examples and tutorials. The former
 The tutorials work completly detached from your own machine on google colab. They provide code snippets and task that show you, how you can work with the software package one your own. We have two tutorials prepared, one for introducing a new unit and one for getting reinforcement learning ready on ASSUME.
 
 How to configure a new unit in ASSUME?
-**Coming Soon**
+[![Open Learning Tutorial in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/assume-framework/assume/blob/main/examples/notebooks/03_custom_unit_example.ipynb)
 
-How to introduce reinforcement learning to ASSUME?
+How to use reinforcement learning for new market participants in ASSUME?
 
 [![Open Learning Tutorial in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/assume-framework/assume/blob/main/examples/notebooks/04_reinforcement_learning_example.ipynb)
 
+How to change and adapt reinforcement learning algorithms in ASSUME?
+
+[![Open Learning Tutorial in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/assume-framework/assume/blob/main/examples/notebooks/04_reinforcement_learning_algorithm_example.ipynb)
 
 
 ### The Examples

assume/reinforcement_learning/learning_unit_operator.py

@@ -252,3 +252,38 @@ async def write_to_learning_role(
                     "data": rl_agent_data,
                 },
             )
+
+    async def formulate_bids(
+        self, market: MarketConfig, products: list[tuple]
+    ) -> Orderbook:
+        """
+        Formulates the bid to the market according to the bidding strategy of the each unit individually.
+
+        Args:
+            market (MarketConfig): The market to formulate bids for.
+            products (list[tuple]): The products to formulate bids for.
+
+        Returns:
+            OrderBook: The orderbook that is submitted as a bid to the market.
+        """
+
+        orderbook: Orderbook = []
+
+        for unit_id, unit in self.units.items():
+            product_bids = unit.calculate_bids(
+                market,
+                product_tuples=products,
+            )
+            for i, order in enumerate(product_bids):
+                order["agent_id"] = (self.context.addr, self.context.aid)
+
+                if market.volume_tick:
+                    order["volume"] = round(order["volume"] / market.volume_tick)
+                if market.price_tick:
+                    order["price"] = round(order["price"] / market.price_tick)
+                if "bid_id" not in order.keys() or order["bid_id"] is None:
+                    order["bid_id"] = f"{unit_id}_{i+1}"
+                order["unit_id"] = unit_id
+                orderbook.append(order)
+
+        return orderbook

docs/source/examples_basic.rst

@@ -18,6 +18,7 @@ Here you can find several tutorials on how to use ASSUME framework to get you st
    examples/02_automated_run_example.nblink
    examples/03_custom_unit_example.nblink
    examples/04_reinforcement_learning_example.nblink
+   examples/04_reinforcement_learning_algorithm_example.nblink
    examples/05_market_comparison.nblink
    examples/06_advanced_orders_example.nblink
    examples/07_interoperability_example.nblink

docs/source/img/Assume_run_learning_loop.png.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: ASSUME Developers
+
+SPDX-License-Identifier: AGPL-3.0-or-later

docs/source/img/assume_logo.png copy.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: ASSUME Developers
+
+SPDX-License-Identifier: AGPL-3.0-or-later