Offline Meta Reinforcement Learning with In-Distribution Online Adaptation

For Walker environments, MuJoCo131 is required. Simply install it the same way as MuJoCo200. To swtch between different MuJoCo versions:

export MUJOCO_PY_MJPRO_PATH=~/.mujoco/mjpro${VERSION_NUM}
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.mujoco/mjpro${VERSION_NUM}/bin

Data Generation

Example of training policies and generating trajectories on multiple tasks: For point-robot and cheetah-vel:

python policy_train.py ./configs/cpearl-sparse-point-robot.json   # actually dense reward is used. To run the sparse reward version, uncomment line 205 in ./Offline-MetaRL/rlkit/envs/point_robot.py.
python policy_train.py ./configs/cheetah-vel.json
python policy_train_mt1.py ./configs/cpearl-mt1.json

For Meta-World ML1 tasks (you can modify the task in ./configs/ml1.json):

python data_collection_ml1.py  ./configs/ml1.json

Generated data will be saved in ./data/

Offline RL Experiments

To reproduce an Meta-World ML1 experiment, run:

run_ml1.sh

To run different tasks, modify "env_name" in ./configs/cpearl-ml1.json as well as "datadirs" in run_ml1.sh.

Similarly, for point-robot and cheetah-vel:

run_point.sh
run_cheetah.sh