Jethro's Braindump

Distributed Reinforcement Learning

Parallelizing Reinforcement Learning ⭐.

History of Distributed RL

  1. DQN (Mnih et al., 2013): §mnih2013_atari_deeprl
  2. GORILA (Nair et al., 2015)
  3. A3C (Mnih et al., 2016)
  4. IMPALA (Espeholt et al., 2018)
  5. Ape-X (Horgan et al., 2018)
  6. R2D3 (Paine et al., 2019)



Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M., Playing atari with deep reinforcement learning, arXiv preprint arXiv:1312.5602, (), (2013).

Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., Maria, A. D., Panneershelvam, V., …, Massively parallel methods for deep reinforcement learning, CoRR, (), (2015).

Mnih, V., Badia, Adri`a Puigdom`enech, Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., …, Asynchronous methods for deep reinforcement learning, CoRR, (), (2016).

Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., …, Impala: scalable distributed deep-rl with importance weighted actor-learner architectures, CoRR, (), (2018).

Horgan, D., Quan, J., Budden, D., Barth-Maron, G., Hessel, M., Hasselt, H. v., & Silver, D., Distributed Prioritized Experience Replay, CoRR, (), (2018).

Paine, T. L., Gulcehre, C., Shahriari, B., Denil, M., Hoffman, M., Soyer, H., Tanburn, R., …, Making efficient use of demonstrations to solve hard exploration problems, CoRR, (), (2019).

Icon by Laymik from The Noun Project. Website built with ♥ with Org-mode, Hugo, and Netlify.