Playing Atari with Deep RL

Playing Atari With Deep RL (Mnih et al., 2013)

Preprocessing Steps

1. Obtain raw pixels of size $$210 \times 160$$
2. Grayscale and downsample to $$110 \times 84$$
3. Crop representative $$84 \times 84$$ region
4. Stack the last 4 frames in history to form the $$84 \times 84 \times 4$$ input

DQN

1. Use of §experience_replay buffer
2. Separate target network stabilizes optimization targets:

$$\delta = r_t + \gamma \mathrm{max}_a Q(s_{t+1}, a ; \theta’) - Q(s_t, a_t; \theta)$$

The network parameterized with $$\theta ‘$$ is a snapshot of the network at some point in time, so the optimization target doesn’t change so rapidly.

1. Clip $$\delta$$ to $$\left[1, -1\right]$$

Bibliography

Icon by Laymik from The Noun Project. Website built with ♥ with Org-mode, Hugo, and Netlify.