lidar render for novel view

[szhang963]

Hi, thanks for your excellent work.
I would like to know whether you will support lidar render for novel view, specifically to implement the function _generate_rays_from_coords for lidar modeling.
Best.

[carlinds]

Lidar rendering is already supported for novel views, it is the same for novel views as for training views. You just need to specify a lidar at a novel view and then run pipeline.model.get_outputs_for_lidar(lidar, batch=batch). You can get some inspiration by looking at our dataset render:
https://github.com/georghess/neurad-studio/blob/main/nerfstudio/scripts/render.py#L1157

[atonderski]

To clarify: the way it is implemented in the render relies on the dataparser having inferred the missing points to create a complete point cloud where missing rays are represented by dummy points. It would be valuable to implement the functionality you request, where you don't need a dummy point cloud but can instead generate rays directly from a lidar model, but we haven't had time to implement this properly. There is some semblance of what you are asking for in the viewer lidar rendering code:
https://github.com/georghess/neurad-studio/blob/5101feee192947b372cc2471364163fe327783ae/nerfstudio/viewer/render_state_machine.py#L361C7-L361C20

[atonderski]

This part could definitely use some love, but I don't foresee us cleaning this up anytime soon unfortunately :/ If you end up implementing something for _generate_rays_from_coords it would be very welcome of course!

[szhang963]

Thank you for your useful guidance, and I will try it.