PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet

PointNet has revolutionized how we think about representing point clouds. For classification and segmentation tasks, the approach and its subsequent variants/extensions are considered state-of-the-art. To date, the successful application of PointNet to point cloud registration has remained elusive. In this paper we argue that PointNet itself can be thought of as a learnable "imaging" function. As a consequence, classical vision algorithms for image alignment can be brought to bear on the problem -- namely the Lucas & Kanade (LK) algorithm. Our central innovations stem from: (i) how to modify the LK algorithm to accommodate the PointNet imaging function, and (ii) unrolling PointNet and the LK algorithm into a…