Consequently, to install and use the ML-Agents Toolkit you will need to:
mlagents-envs
mlagents
Download and install Unity. We strongly recommend that you install Unity through the Unity Hub as it will enable you to manage multiple Unity versions.
On Windows, you'll have to install the PyTorch package separately prior to -installing ML-Agents. Activate your virtual environment and run from the command line:
pip3 install torch~=1.13.1 -f https://download.pytorch.org/whl/torch_stable.html +installing ML-Agents in order to make sure the cuda-enabled version is used, +rather than the CPU-only version. Activate your virtual environment and run from +the command line: +pip3 install torch~=2.2.1 --index-url https://download.pytorch.org/whl/cu121 Note that on Windows, you may also need Microsoft's Visual C++ Redistributable if you don't have it already. See the PyTorch installation guide for more installation options and versions. +(OS X) Installing GRPC libraries +On OS X, you may need to explicitly install the GRPC runtime libraries to avoid hitting errors when training like dlopen(/Users/alex.mccarthy/miniconda3/envs/mlagents/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_CFRelease'. +pip3 install grpcio + Installing mlagents To install the mlagents Python package, activate your virtual environment and run from the command line: @@ -1474,11 +1515,19 @@ Help + + + + + + +
pip3 install torch~=2.2.1 --index-url https://download.pytorch.org/whl/cu121
Note that on Windows, you may also need Microsoft's Visual C++ Redistributable if you don't have it already. See the PyTorch installation guide for more installation options and versions.
On OS X, you may need to explicitly install the GRPC runtime libraries to avoid hitting errors when training like dlopen(/Users/alex.mccarthy/miniconda3/envs/mlagents/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_CFRelease'.
dlopen(/Users/alex.mccarthy/miniconda3/envs/mlagents/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_CFRelease'
pip3 install grpcio +
To install the mlagents Python package, activate your virtual environment and run from the command line:
BufferSensor
The BufferSensorComponent Editor inspector has two arguments: - - Observation Size : This is how many floats each entities will be +
BufferSensorComponent
Observation Size
The BufferSensorComponent Editor inspector has two arguments:
Maximum Number of Entities
To add an entity's observations to a BufferSensorComponent, you need to call BufferSensorComponent.AppendObservation() in the Agent.CollectObservations() method @@ -2419,11 +2438,19 @@
BufferSensorComponent.AppendObservation()
ML-Agents provide an implementation of two reinforcement learning algorithms:
The default algorithm is PPO. This is a method that has been shown to be more @@ -2003,11 +2020,13 @@
The loss (the squared difference between the predicted and actual encoded observations) of the trained model is used as intrinsic reward. The more an Agent visits a state, the more accurate the predictions and the lower the rewards which encourages the Agent to @@ -2056,9 +2075,9 @@
ML-Agents provides the functionality to train both symmetric and asymmetric adversarial games with -Self-Play. A symmetric game is +Self-Play. A symmetric game is one in which opposing agents are equal in form, function and objective. Examples of symmetric games are our Tennis and Soccer example environments. In reinforcement learning, this means both agents have the same observation and @@ -2346,11 +2365,19 @@
gym-unity
ml-agents-envs
python + from gym_unity.unity_gym_env import UnityToGymWrapper
python + from mlagents_envs.envs.unity_gym_env import UnityToGymWrapper
from gym_unity.unity_gym_env import UnityToGymWrapper -
from mlagents_envs.envs.unity_gym_env import UnityToGymWrapper -
GetMaxBoardSize()
The sensor configuration has changed: -* The sensor implementation has been refactored and exsisting GridSensor created from extension package +* The sensor implementation has been refactored and existing GridSensor created from extension package will not work in newer version. Some errors might show up when loading the old sensor in the scene. You'll need to remove the old sensor and create a new GridSensor. * These parameters names have changed but still refer to the same concept in the sensor: GridNumSide -> GridSize, @@ -1146,8 +1167,8 @@
GridNumSide
GridSize
dataBuffer
IsDataNormalized()
GetObjectData()
The way that Sentis processes LSTM (recurrent neural networks) has changed. As a result, models trained with previous versions of ML-Agents will not be usable at inference if they were trained with a memory @@ -1162,7 +1183,7 @@
memory
VectorSensor.AddObservation(IEnumerable<float>)
VectorSensor.AddObservation(IList<float>)
ObservationWriter.AddRange()
ObservationWriter.AddList()
ActuatorComponent.CreateAcuator()
ActuatorComponent.CreateActuators
ActuatorComponent.CreateActuator()
InitializeAgent()
Initialize()
AgentAction()
OnActionReceived()
AgentReset()
OnEpsiodeBegin()
OnEpisodeBegin()
Done()
EndEpisode()
GiveModel()
SetModel()
IFloatProperties
You can find them at Edit > Project Settings... > ML-Agents. It lists out all the available settings and their default values.
Edit
Project Settings...
ML-Agents
In order to to use your own settings for your project, you'll need to create a settings asset.
You can do this by clicking the Create Settings Asset buttom or clicking the gear on the top right and select New Settings Asset.... +
Create Settings Asset
New Settings Asset...
You can do this by clicking the Create Settings Asset button or clicking the gear on the top right and select New Settings Asset.... The asset file can be placed anywhere in the Asset/ folder in your project. After Creating the settings asset, you'll be able to modify the settings for your project and your settings will be saved in the asset.
Asset/
You can create multiple settings assets in one project.
By clicking the gear on the top right you'll see all available settings listed in the drop-down menu to choose from.
This allows you to create different settings for different scenatios. For example, you can create two +
This allows you to create different settings for different scenarios. For example, you can create two separate settings for training and inference, and specify which one you want to use according to what you're currently running.
As part of the ML-Agents Tookit, we provide a lightweight profiling system, in +
As part of the ML-Agents Toolkit, we provide a lightweight profiling system, in order to identity hotspots in the training process and help spot regressions from changes.
Timers are hierarchical, meaning that the time tracked in a block of code can be @@ -1070,11 +1087,19 @@
Ml-agents
mlagents-learn
Model-free RL algorithms generally fall into two broad categories: on-policy and off-policy. On-policy algorithms perform updates based on data gathered from the current policy. Off-policy algorithms learn a Q function from a buffer of previous data, then use this Q function to make decisions. Off-policy algorithms have three key benefits in the context of ML-Agents: They tend to use fewer samples than on-policy as they can pull and re-use data from the buffer many times. They allow player demonstrations to be inserted in-line with RL data into the buffer, enabling new ways of doing imitation learning by streaming player data.
To add new custom trainers to ML-agents, you would need to create a new python package. @@ -1026,11 +1043,19 @@
UnityEnvironment
The gym wrapper is part of the mlgents_envs package. Please refer to the +
mlgents_envs
The gym wrapper is part of the mlagents_envs package. Please refer to the mlagents_envs installation instructions.
mlagents_envs
The gym interface is available from gym_unity.envs. To launch an environment @@ -1543,11 +1560,19 @@
gym_unity.envs
registry = UnityEnvRegistry() -entry = registry[<environment_identifyier>] +entry = registry[<environment_identifier>]
An entry has the following properties : * identifier : Uniquely identifies this environment @@ -1514,7 +1531,7 @@
identifier
description
To launch a Unity environment from a registry entry, use the make method:
make
registry = UnityEnvRegistry() -env = registry[<environment_identifyier>].make() +env = registry[<environment_identifier>].make()
Gathers the data of one lesson for one environment parameter including its name, -the condition that must be fullfiled for the lesson to be completed and a sampler +the condition that must be fulfilled for the lesson to be completed and a sampler for the environment parameter. If the completion_criteria is None, then this is the last lesson in the curriculum.
Arguments:
batch
next_obs
done
agent_id
With the increasing interest in multi-agent training with a gym-like API, we provide a -PettingZoo Wrapper around the Petting Zoo API. Our wrapper +PettingZoo Wrapper around the Petting Zoo API. Our wrapper provides interfaces on top of our UnityEnvironment class, which is the default way of interfacing with a Unity environment via Python.
The PettingZoo wrapper is part of the mlgents_envs package. Please refer to the +
The PettingZoo wrapper is part of the mlagents_envs package. Please refer to the mlagents_envs installation instructions.
[Colab] PettingZoo Wrapper Example
This colab notebook demonstrates the example usage of the wrapper, including installation, @@ -1068,7 +1085,7 @@
This wrapper is compatible with PettingZoo API. Please check out -PettingZoo API page for more details. +PettingZoo API page for more details. Here's an example of interacting with wrapped environment:
from mlagents_envs.environment import UnityEnvironment from mlagents_envs.envs import UnityToPettingZooWrapper @@ -1083,8 +1100,8 @@ API interface
env.step(action)
You can also check out our new web docs!
The table below lists all our releases, including our main branch which is -under active development and may be unstable. A few helpful guidelines: -- The Versioning page overviews how we manage our GitHub - releases and the versioning process for each of the ML-Agents components. -- The Releases page - contains details of the changes between releases. -- The Migration page contains details on how to upgrade - from earlier releases of the ML-Agents Toolkit. -- The Documentation links in the table below include installation and usage +under active development and may be unstable. A few helpful guidelines:
main
com.unity.ml-agents
The ML-Agents Toolkit is an open-source project and we encourage and welcome @@ -1219,11 +1238,19 @@