LERF: Language Embedded Radiance Fields
University of California, Berkeley · Berkeley College
Abstract
Humans describe the physical world using natural language to refer to specific 3D locations based on a vast range of properties: visual appearance, semantics, abstract associations, or actionable affordances. In this work we propose Language Embedded Radiance Fields (LERFs), a method for grounding language embeddings from off-the-shelf models like CLIP into NeRF, which enable these types of open-ended language queries in 3D. LERF learns a dense, multi-scale language field inside NeRF by volume rendering CLIP embeddings along training rays, supervising these embeddings across training views to provide multi-view consistency and smooth the underlying language field. After optimization, LERF can extract 3D…
Citation impact
- FWCI
- 32.57
- Percentile
- 100%
- References
- 42
Authors
5- JKJustin KerrCorresponding
University of California, Berkeley, Berkeley College
- CMChung Min Kim
University of California, Berkeley, Berkeley College
- KGKen Goldberg
Berkeley College, University of California, Berkeley
- AKAngjoo Kanazawa
University of California, Berkeley, Berkeley College
- MTMatthew Tancik
University of California, Berkeley, Berkeley College
Topics & keywords
- Computer science
- Rendering (computer graphics)
- Radiance
- Artificial intelligence
- Vocabulary
- Volume rendering
- Language model
- Language understanding
- Quality Education