• Understand the trade-off between exploitation and exploration with ε-policy

• ε-policy
• Bellman equation
• Deep Q networks (2 hidden layers)
• Replay buffer

Use Google Colab (with GPU enabled) to train neural networks (CartPole.ipynb)

4 trained models (after over 10000 iterations) are located in the directory pretrained_models:

• 0.1-ckpt.pth: ε = 0.1
• 0.01-ckpt.pth: ε = 0.01
• 0.5-ckpt.pth: ε = 0.5
• 0.05-ckpt.pth: ε = 0.05

• Ubuntu 22.04 LTS
• Python 3.8.0
• Pytorch 1.11.0

Command:

python main.py

Average reward (moving average with window=500) for different epsilons after 10000 iterations

• Greedy test: Model trained with ε=0.01

• Greedy test: Model trained with ε=0.05

• Greedy test: Model trained with ε=0.1

• Greedy test: Model trained with ε=0.5

The code is based on https://github.com/seungeunrho/minimalRL/blob/master/dqn.py