This repo is based on RL implementation of Atari games using Gym library. We have to aims in this project: (1) comparing different networks and their performance on these games and (2) comparing the parformance of different function approximtors like KNN, Ridge Regression and Tree based learner.
917845196 Ammar Haydari
917779990 Muhammed Sarmad Saeed
We suggest to read the files in this order.
1- Main project report is given in this file.
2- CartPole, Breaokut and Breakout LSTM folders have the code and results stored in text file.
Each folder has one single notebook file. Please refer those files for the detailed explanation of codes
We studied Q learning based RL algorithms on two atari games namely Cartpole and Breakout. In this project, we employed regression based function approximators (K nearest neighbors, Ridge, XGBoost and Light GBM based gradient boosted tree model) and nonlinear neural network approximators on Cartpole game and image formatted state input structure with CNN and RNN neural networks on breakout game. In the cartpole game, we see that for linear function approximators, batch replay from experience replay memory does not perform well, therefore we change the setting as training the learners with whole memory. However we used batch sampling method on neural network models and this gives us very good results. The results indicates that, although LightGBM may reach the closest performance to the neural network based model, it is highly time consumed model, Hence, the optimum learning model even in simpler problems is neural network base RL. In the breakout game, we compared two configurations with the baseline models (1) CNN with max pooling and Adam optimizer and (2) LSTM model. Our results shows that our first configuration performs the worst while take shortest training time. The second model which uses LSTM also does not overer come the baseline CNN model. Our LSTM model also takes more training time than CNN.