Multi-Modal 3D Object Detection in Autonomous Driving: A Survey and Taxonomy

Autonomous vehicles require constant environmental perception to obtain the distribution of obstacles to achieve safe driving. Specifically, 3D object detection is a vital functional module as it can simultaneously predict surrounding objects' categories, locations, and sizes. Generally, autonomous vehicles are equipped with multiple sensors, including cameras and LiDARs. The fact that single-modal methods suffer from unsatisfactory detection performance motivates utilizing multiple modalities as inputs to compensate for single sensor faults. Although many multi-modal fusion detection algorithms exist, there is still a lack of comprehensive and in-depth analysis of these methods to clarify how to fuse…

• NN
  National Natural Science Foundation of China

  Awards: U1964203, 62273198, 62073101
• CP
  China Postdoctoral Science Foundation

  Award: 2021M691780