Skip to content

v1.0.0
Compare
Choose a tag to compare
@ChrisMcCarthyDev ChrisMcCarthyDev released this 05 Dec 19:15
· 1368 commits to main since this release
v1.0.0
a66aa96
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

YAWNING-TITAN v1.0.0

🎉 What's New

OS-Agnostic

Yawning-Titan is now OS-agnostic('ish). There are two parts to this.

💻 v1.0.0 has been tested on Windows 10 (Pro build 19045.2251), Ubuntu (18.04 LTS, 20.04 LTS, 22.04 LTS), and MacOS (Catalina 10.15.7). While Yawning-Titan should work on most variations/distros and custom builds of the three operating systems listed above, it has not been possible to test this.

Dynamic Paths

Hard-coded file paths have been removed and replaced with dynamic path creation using combinations of pathlib.Path and/or os.path.join.

Ray 3.0.0 Beta Support for Windows

The previous version of Yawning-Titan used the latest release build of Ray, Ray version 2.., which does not support Windows. While it was possible to get this working with a number of tweaks in the backend, we felt this was very cumbersome. The standard pip install ray[rllib]==2.1.0 on Windows just doesn’t work for some:

(venv) PS C:\Users\sandbox\source\repos\methodsanalytics\YAWNING-TITAN> pip install ray[rllib]==2.1.0
ERROR: Could not find a version that satisfies the requirement ray[rllib]==2.1.0 (from versions: none)
ERROR: No matching distribution found for ray[rllib]==2.1.0

Ray are working towards windows support in version 3.., and thus have released Ray version 3.0.0.dev0 Beta version.

Ray currently supports Linux, MacOS and Windows. Ray on Windows is currently in beta.

Source: https://docs.ray.io/en/latest/ray-overview/installation.html

After extensive testing of Ray version 3.0.0.dev0 with Yawning-Titan, the decision was made to include it as a dependency in setup.py.

Packaged & Installable

Yawning-Titan is slowly progressing from a GitHub repo that is cloned and used from the directory it is cloned into, to a piece of packaged software that is installed on a machine and used without needing to clone the repo. For this to happen, anything that sat outside of the repo that was required for its use, needed to be brought in and installed as package data. The current list of package data is:

  • yawning_titan/config/_package_data/logging_config.yaml
  • yawning_titan/config/_package_data/game_modes/default_game_mode.yaml
  • yawning_titan/config/_package_data/game_modes/low_skill_red_with_random_infection_perfect_detection.yaml
  • yawning_titan/notebooks/_package_data/sb3/End to End Generic Env Example - Env Creation, Agent Train and Agent Rendering.ipynb
  • yawning_titan/notebooks/_package_data/sb3/Using an Evaluation Callback to monitor progress during training.ipynb
  • yawning_titan/notebooks/_package_data/Creating and playing as a Keyboard Agent.ipynb

App Directories

Yawning-Titan now comes with a set of application directories and user directories. These directories are used by Yawning-Titan to store default files, data, config etc., but also provide the user with a home to store their custom config, Jupyter notebooks, agents etc. The directories are created automatically when Yawning-Titan is installed. Below shows the Yawning-Titan directory tree for Windows, Linux, and Mac operating systems:

Windows

~/
├─ AppData/
│  │  ├─ yawning_titan/
│  │  ├─ app_images/
│  │  ├─ config/
│  │  ├─ db/
│  │  ├─ docs/
│  │  ├─ logs/
├─ yawning_titan/
│  ├─ agents/
│  ├─ game_modes/
│  ├─ images/
│  ├─ notebooks/

Linux

~/
├─ .cache/
│  ├─ yawning_titan/
│  │  ├─ log/
├─ .config/
│  ├─ yawning_titan/
├─ .local/
│  ├─ share/
│  │  ├─ yawning_titan/
│  │  │  ├─ app_images/
│  │  │  ├─ db/
│  │  │  ├─ docs/
├─ yawning_titan/
│  ├─ agents/
│  ├─ game_modes/
│  ├─ images/
│  ├─ notebooks/

MacOs

~/
├─ Library/
│  ├─ Application Support/
│  │  ├─ Logs/
│  │  │  ├─ yawning_titan/
│  │  │  │  ├─ log/
│  │  ├─ Preferences/
│  │  │  ├─ yawning_titan/
│  │  ├─ yawning_titan/
│  │  │  ├─ app_images/
│  │  │  ├─ db/
│  │  │  ├─ docs/
├─ yawning_titan/
│  ├─ agents/
│  ├─ game_modes/
│  ├─ images/
│  ├─ notebooks/

Platform Logging

Logging has been setup and is configured in Yawning-Titan’s top-level init.py. Logging is formatted using the YAWNING-TITAN.yawning_titan.config._package_data.logging_config.yaml. Logs are sent to two handlers:

  • debug_console_handler, a StreamHandler that send logs to stderr at DEBUG level.
  • info_rotating_file_handler, a RotatingFileHandler that send logs to LOG_DIR/yawning_titan.log at INFO level. The maximum log file size is 10MB and it will rotate 10 times, therefore no more than 110MB of logs will ever exist.

Loggers have a format: '%(asctime)s::%(levelname)s::%(name)s::%(funcName)s::%(lineno)s::%(message)s'

Config Classes

A structure of Python Data Classes has been built to:

  • Encapsulate the creation and validation of game mode and network configs.
  • To enable the creation of custom game modes and networks using a GUI (currently in development).
  • To decouple the game mode config from the network config.
  • To provide documentation of config key/value pairs as docstrings.

The classes utilise Python Data Classes for automatic __init__, __hash__, __eq__, __repr__, and __dict__, but still use traditional OOP getters and setters to enable docstrings to be used by Sphinx. An abstract base class, yawning_titan.config.game_config.config_abc.ConfigABC, has been defined that has a to_dict() method, and create(config_dict) and validate(config_dict) abstract class methods.

GameModeConfig

The yawning_titan.config.game_config.game_mode_config.GameModeConfig class now serves as the main config class for the game rules. GameModeConfig has a single instance of RedAgentConfig, BlueAgentConfig, GameRulesConfig, ObservationSpaceConfig, ResetConfig, RewardsConfig, and MiscellaneousConfig. The GameModeConfig class has the following methods:

  • create(config_dict) - This method is a factory method that takes a parsed .yaml config file as a Python dictionary and returns an instance of GameModeConfig.
  • GameModeConfig.create_from_yaml(config_path - This class method takes a .yaml config file path, reads and parses it into a Python dictionary, and passes it to create before returning the created instance of GameModeConfig. The config_path parameter has a default value of default_game_mode_path().
  • to_dict(key_upper) - The method serializes the GameModeConfig as a Python dict. If key_upper=True, the top-level keys (red, blue, game_rules observation_space, reset, rewards, miscellaneous) are uppercase to match the config .yaml files.
  • to_yaml(config_path) - Calls to_dict_key_upper=True) and then dumps the returned dict down to .yaml at the provided config path.

Below is a UML class diagram of the GameModeConfig.

game_mode_config_class_diagram

NetworkConfig

The yawning_titan.config.network_config.network_config.NetworkConfig class now serves as the main config class for the network. The NetworkConfig class has the following methods:

  • create - This method is a factory method that takes a parsed .yaml config file as a Python dictionary and returns an instance of NetworkConfig.
  • create_from_args - This method is a factory method that takes a matrix and positions, and optional entry_nodes, vulnerabilities, and high_value_nodes, and returns an instance of NetworkConfig.

Below is a UML class diagram of the NetworkConfig.

network_config_class_diagram

High Value Target Terminology Changes

For added clarity, the term “high value target” has been changed to “high value node”. This change reflects situations where a node in a network may be of high value but is not defined as a target node. This change affects all config .yaml files, with keys containing high_value_target now containing high_value_node.

Multiple High Value Nodes

A network can now contain multiple high value nodes as opposed to just one. By defining multiple high value nodes in a network, an agent has more chances to ‘instantly win’.

  • Can now specify the number of high value targets in the game via number_of_high_value_nodes in GameRulesConfig.
  • Can now specify the exact nodes that will be high value nodes by providing a list of strings to NetworkConfig.high_value_nodes.
  • choose_high_value_target_placement_at_random renamed to choose_high_value_nodes_placement_at_random.
  • choose_high_value_target_furthest_away_from_entry renamed to choose_high_value_nodes_furthest_away_from_entry.
  • choose_new_high_value_target_on_reset renamed to choose_new_high_value_nodes_on_reset.

Seeds and Randomisation

There are 3 primary sources of RNGs in the system; random, numpy.random and the user of sets – which although not designed to be random add random behaviour due to element retrieval being non deterministic.

This behaviour can now be overwritten through added specificity of deterministic=True parameters added to action loops and an additional random_seed setting in the miscellaneous section of the config file. This serves to propagate deterministic behaviour throughout the game and training episodes.

In order to correctly proporgate the seed to the agent training module the user must manually set the seed in the training algorithm class thusly.

agent = PPO(
      PPOMlp, env, verbose=1, seed=env.network_interface.random_seed
)

This manual requirement is counter intuitive and should hopefully be phased out in favour of a wrapper function or an alternative method of automatically passing of this parameter.

The repeatability of episodes has also been improved through introduction of appropriate reset procedures to reset the state of several of the agents values which had previously been persisted erroneously.

Jupyter Environment

A start_jupyter_session method in yawning_titan.notebooks.jupyter will spin-up a Jupyter notebook environment in the yawning_titan.NOTEBOOKS_DIR.

❕ See Known Issues for more information.

Sphinx Documentation & GitHub Pages

A huge effort has been made to bring the docs up to date and provide full API coverage. The docs structure has been rebuilt. Full recursive API coverage has been implemented to make use of Sphinx docstring in the codebase, with autodoc being extended using autosummary and automod alongside custom class and module .rst templates (credit to https://github.com/JamesALeedham/Sphinx-Autosummary-Recursion). Finally, module, class, function, and attribute mapping, for both the Yawning-Titan docs to code, and inter-sphinx docs to code (Python, Numpy, Matplotlib, Pandas, Stable Baselines 3 etc.) has begun. This is being tackled in a ‘fix it as we come across’ is fashion.

The Sphinx docs are now built automatically from GitHub actions upon merging into dev and are hosted on GitHub pages. Please see: https://dstl.github.io/YAWNING-TITAN/index.html.

🐛 Bug Fixes

#7 - high_value_target not working with lose_when_high_value_target_lost setting (i.e self.gr_loss_hvt is True)

This was an issue raised on the dstl/YAWNING-TITAN repo by a member of the community (https://github.com/john-cardiff). This fixed an issue whereby high value nodes supplied in a NetworkConfig were dropped by the NetworkInterface when game_mode.game_rules.lose_when_high_value_node_lost = True.

Issue: #7

#8 - Non reproducibility

This was an issue raised on the dstl/YAWNING-TITAN repo by a member of the community (https://github.com/john-cardiff). This fixed an issue that prevented runs of Yawning-Titan from being reproduceable/repeatable.

Issue: #8

#11 - Scanning action not useful as blue agent sees the true compromised state

This was an issue raised on the dstl/YAWNING-TITAN repo by a member of the community (https://github.com/john-cardiff). This fixed an issue where the blue agent was able to see the true compromised state of a network rather than blues view of the compromised state of the network.

Issue: #11

#13 - Fix typo in "Ridley 2017" reference

This was a pull request submitted on the DSTL repo by a member of the community (https://github.com/Cory-Watson ). This fixed a broken URL in the comments of notebooks/sb3/End to End Generic Env Example - Env Creation, Agent Train and Agent Rendering.ipynb.

PR #13

#17 - Supplied HVN is dropped by NetworkInterface if lose_when_high_value_node_lost is False

This was an issue raised on the dstl/YAWNING-TITAN repo by a member of the community(). This fixed an issue whereby high value nodes supplied in a NetworkConfig were dropped by the NetworkInterface when game_mode.game_rules.lose_when_high_value_node_lost = False. This is near enough the same issue as #7.

Issue: #17

☠️ Known Issues

  • The start_jupyter_session method in yawning_titan.notebooks.jupyter currently only works on Windows operating system. Follow these steps to get a session running from Linux or MacOS:
    • cd to the NOTEBOOKS_DIR with: cd ~/yawning_titan/notebooks.
    • Activate your env where Yawning-Titan is installed with: source <path to Python bin> activate.
    • Spin-up a Jupyter notebook session with: Jupyter notebook.

🛠 Engineering Notes

  • The save_network and load_network methods in the yawning_titan.envs.generic.helpers.network_creator module are being depreciated in the next release as network config is going to be stored in a lightweight document database (See: https://tinydb.readthedocs.io/en/latest/).

🏆 Community Notes

✨ How to Install & Run

Install Yawning-Titan

As this is the first versioned release of Yawning-Titan that has lots of API breaking changes, it is recommended that all users perform a new installation by following the code block below for your desired operating system, replacing the <path to downloaded yawningtitan-1.0.0.tar.gz> with the location of the downloaded yawningtitan-1.0.0.tar.gz file.

Windows

mkdir ~\yawning_titan
cd ~\yawning_titan
python3 -m venv .venv
attrib +h .venv /s /d # Hides the .venv directory
.\.venv\Scripts\activate
pip install <path to downloaded yawningtitan-1.0.0.tar.gz>

Unix

mkdir ~/yawning_titan
cd ~/yawning_titan
python3 -m venv .venv
source .venv/bin/activate
pip install <path to downloaded yawningtitan-1.0.0.tar.gz>

Run Jupyter from your Yawning-Titan Virtual Environment

Windows

cd ~\yawning_titan
.\.venv\Scripts\activate
cd notebooks
jupyter notebook

Unix

cd ~/yawning_titan
source .venv/bin/activate
cd notebooks
jupyter notebook

Install Yawning-Titan Dev Environment

The Yawning-Titan repo can be cloned and installed as a dev environment for user customisation. Follow the code block below for your desired operating system.

Windows

mkdir ~\yawning_titan
mkdir ~\yawning_titan\src
cd ~\yawning_titan\src
git clone https://github.com/dstl/YAWNING-TITAN.git
cd YAWNING-TITAN
python3 -m venv .venv
attrib +h .venv /s /d # Hides the .venv directory
.\.venv\Scripts\activate
pip install -e .[dev]

Unix

mkdir ~/yawning_titan
mkdir ~/yawning_titan/src
cd ~/yawning_titan/src
git clone https://github.com/dstl/YAWNING-TITAN.git
cd YAWNING-TITAN
python3 -m venv .venv
source .venv/bin/activate
pip install -e .[dev]

Contributors

Full Changelog: v0.1.0...v1.0.0

Contributors

ChrisMcCarthyDev, Cory-Watson, and 2 other contributors
Assets 3
Loading
1 Join discussion