Robotics Thesis Proposal
Speaker
MING-FANG (ALLIE) CHANG
Ph.D. Student
Robotics Institute
Carnegie Mellon University
When
-
Where
Virtual Presentation - ET
Description
We address the problem of large-scale and long-term neural map representations. Maps, as our prior understanding toward the environment, provide valuable information for modern robotics applications such as autonomous driving and AR/VR. The size of maps largely affects the end task performance: usually a more detailed map can support better performance, but would cost a large system burden to store and process. Besides the map size, the real world is dynamic, adapting a pre-built map to fit the current environment is also very important. In this talk, we will l first introduce our previous work about compressing the maps image-to-SfM map registration, and then describe our plan about LiDAR-assisted neural map representation for large-scale and long-term environments. Recently, NeRF-inspired neural scene representations have shown very promising results in indoor and static scenes by compressing the rich scene information within a deep neural network. However, the existing approaches still struggle in large scenes due to high computation cost, dynamic objects, and lack of camera sample coverage. On the other hand, LiDAR data are widely available on city-scale and provide accurate 3D scene geometry. We aim to leverage the strength of LiDAR sensor to overcome the drawbacks in the current neural scene representation systems and to improve the performance of large-scale and long-term novel view synthesis. To achieve this, we will have to overcome multiple challenges, including the sparsity of LiDAR data, seasonal changes, temporal consistency, computational cost, and more. In short, given sparsely collected data, our goal is to interpolate/synthesize novel views not only spatially but also temporally with our LiDAR-assisted neural map representations.
Thesis Committee: Michael Kaess (Chair) Simon Lucey (Co-chair) Matthew Johnson-Roberson Ian Reid (The University of Adelaide) Zoom Participation. See announcement.