Lessons learned reproducing a deep reinforcement learning paper

Abstract

Reinforcement learning (RL) is a rapidly developing, yet still challenging field of research in machine learning. This article shares lessons learned from a side project of reproducing and extending upon an RL paper. RL’s complexity requires thoughtful implementation and attention to detail, as well as proper mental strategies for when tackling long iteration times. Another important consideration is the amount of compute resources needed for RL projects due to repetitive training runs that take hours, and the unstable nature of RL can also necessitate running multiple iterations of the same task to obtain statistical significance. Additionally, it might be more efficient to focus on improving engineering skills rather than research skills for those interested in pursuing RL projects. – AI-generated abstract.