Add CameraTrajectoryPanel to view/edit camera trajectories

[jkulhanek]

[jkulhanek]

Please let me know your thoughts on this.

[brentyi]

Cool! It seems like there are a lot of things here that could be used to improve the Nerfstudio / viewer beta render panel.

As an initial thought: existing GUI elements in viser all synchronize state with the server, so changes to the GUI persist if you refresh the page, and synchronize across multiple connected clients. It would be nice to make this panel consistent with that.

Related, my main concern is generalizability: one of the core focuses of viser (in contrast to the original Nerfstudio viewer) was to replace hardcoded frontend logic with more modular UI components, and then shift as much high-level logic to Python as possible. This makes the frontend easier to integrate with things implemented in Python, like the recent Perspective/Fisheye/Equirectangular preview support added in Nerfstudio: nerfstudio-project/nerfstudio#2667.

Are there disadvantages to breaking down this panel implementation into smaller+more general frontend components with Python APIs, which could then be stitched back together in Python to produce the same functionality?

• The multislider for keyframe timing + keyframe reordering UI look particularly nice!
• The camera path upload would require finishing Handlers and Messages for uploading files #121.
• The rest of the UI looks like it could already be put together via some permutations of .add_gui_button(), .add_gui_number(), .add_gui_vector2(), etc, although effort would be needed to get things as pretty as what you have. (icons for the number inputs, etc)

[jkulhanek]

I see. I will try to make it more modular. My original motivation was to make it fast enough to save the callback roundtrip eg when you move the slider. I am thinking about the following separation: (player+frustum it controls), (multislider), (camera list) + standard components.

[brentyi]

Thanks @jkulhanek!

Yeah, when the slider is moved what we currently have is something like:

• [client -> server] GUI update message (triggers GUI update callback)
• [server -> client] visualized frustum update
• [server -> client] camera pose update (does not trigger camera update callback; could be optimized!!)
• [client -> server] camera pose update (triggers camera update callback)
• [server -> client] new rendered image

...interspersed with logic for various locks, thread spawning, windowing, etc. Agree that it would be nice to optimize this whole thing. 🙃

That said, I'd still prefer to minimize compromising on the goals that have guided viser development so far. To me this reduces to:

• Prioritizing versatility/modularity/reusability over performance or application-specific UI/UX advantages
• Building something generally useful that happens to work for Nerfstudio (as opposed to something for Nerfstudio that happens to be generally useful)
• Keeping things focused + maintanable; being conservative about adding features!

For the render panel specifically, it would be nice if we can integrate the changes in this PR without regressing on some of the new features in the beta viewer, like being able to check the "Move Keyframes" box and change the position/orientation of keyframes, or being able to click on frustums for specific keyframes and adjust the FOV of just that keyframe.