OpenGL visualisation with Python

[aaronvark]

Don't merge yet, this is a wip-discussion PR...

Current issues:

• OpenGL context is not properly closed / cleaned up. I think this is because the call to event_loop.exit() or window.close() is never executed because pyglet schedules it for the "next tick", which doesn't happen because the actor gets destroyed. This means re-creating the actor fails to open a window.
• The actor falls behind no matter what timeout is set, so there appears to be some added unexpected latency happening.

In order for this to work, you need to pip install the relevant dependencies:

• pyglet

And I had to also install "requests". In windows this needs to be done from the embedded python instance in build/python by using "python -m pip install ...", and then run build_windows.bat so it gets copied to the executable folders.

[aaronvark]

Concrete questions:

• How do we create an "open/close" button on the actor from within the python code, for example (can also be other editable input variables)
• Do we make it a generic visualiser, or more specialized?

[sphaero]

let's keep it specialised for now as it will give us quite some interesting insights I think. Hence the latency and context closing.

Also it might be easier to do this from C instead of Python.

[sphaero]

I was just thinking that the vulkan API might be relevant as well as it already provides thread safe constructs

[aaronvark]

To do threaded rendering at all, we'd need to move to a "command buffer" approach, which we'd need to implement ourselves: https://gpfault.net/posts/opengl-command-buffers.txt.html

Or we'd need to move to more modern contexts that do support it (vulkan/metal), which for now isn't realistic.

I have confirmed the pyglet window gets closed, but still doesn't re-open properly, so not really sure what's happening there (keeping a second python instance running doesn't change the result either).

[aaronvark]

I've added a PyOpenGL variant, which is more stable but a little harder to install. The main python dependencies are PyOpenGL & PyOpenGL_accellerate, but those are recommended to be installed using a Python Wheel:

• https://pypi.org/project/PyOpenGL/#files
• https://pypi.org/project/PyOpenGL-accelerate/3.1.7/#files

On top of this, freeglut needs to be included: https://freeglut.sourceforge.net/index.php#download

Opening multiple instances is still problematic. As well as copying the actor while the window is open.

The base code of the actor currently looks for a sequence of floats at the end of a Socket OSC message, and then attempts to interpret this as positions and/or velocities. The most common conflict is when we have Euler angles and positions in 6 floats, because it might assume these are two sets of positions. Some work needs to be done to guesstimate the exact content, or to expose settings to allow users to control the guess.

[aaronvark]

Two further options to try:

• Using alterations in ImGui to render a threaded window with its own event loop, from a C++ actor
• Receiving some kind of draw-list (e.g. ImGui) and rendering it from the ActorContainer (taking all the rendering out of actor code)