PixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction
Massachusetts Institute of Technology · University of Toronto · +1 more institution
Abstract
We introduce pixelSplat, a feed-forward model that learns to reconstruct 3D radiance fields parameterized by 3D Gaussian primitives from pairs of images. Our model features real-time and memory-efficient rendering for scalable training as well as fast 3D reconstruction at inference time. To overcome local minima inherent to sparse and locally supported representations, we predict a dense probability distribution over 3D and sample Gaussian means from that probability distribution. We make this sampling operation differentiable via a reparameterization trick, allowing us to back-propagate gradients through the Gaussian splatting representation. We benchmark our method on wide-baseline novel view synthesis on…
Citation impact
- FWCI
- 103.27
- Percentile
- 100%
- References
- 70
Authors
4Topics & keywords
- Computer science
- Scalability
- Gaussian
- Iterative reconstruction
- Artificial intelligence
- Computer vision
- Image (mathematics)
- Gaussian process