Diagnosing Non-Intermittent Anomalies in Reinforcement Learning Policy Executions (Short Paper)

Due to the safety risks and training sample inefficiency, it is often preferred to develop controllers in simulation. However, minor differences between the simulation and the real world can cause a significant sim-to-real gap. This gap can reduce the effectiveness of the developed controller. In this paper, we examine a case study of transferring an octorotor reinforcement learning controller from simulation to the real world. First, we quantify the effectiveness of the real-world transfer by examining safety metrics. We find that although there is a noticeable (around 100%) increase in deviation in real flights, this deviation may not be considered unsafe, as it will be within > 2m safety corridors. Then, we…

• NA
  National Aeronautics and Space Administration

  Awards: 80NSSC21M0087, 80NSSC21M0087-21-S06
• DF
  Deutsche Forschungsgemeinschaft

  Awards: 146371743, TRR 89
• NS
  Nuclear Safety and Security Commission
• IS
  Israel Science Foundation