Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support remote rendering #3323

Draft
wants to merge 42 commits into
base: master
Choose a base branch
from
Draft

Conversation

linguini11
Copy link

@linguini11 linguini11 commented Sep 8, 2023

Support the ability to render egui remotely using eframe, inspired by these imgui projects:

Addresses the feature request discussed here: #3292

An example is included in examples/remote_rendering of how to set this up. This example uses the tungstenite crate to create websockets for the client and server, but other client/server set ups should work too.
As a basic overview, the server sends the serialised FullOutput to the client to render it, and the client sends RawInput back to the server.

serde-diff is being included as a feature flag (though this won't currently run, I need to fix some errors). Performance is greatly improved when using it, but it requires a couple of pull requests to be merged in order for it to work (amethyst/serde-diff#36 and amethyst/serde-diff#45), and currently the crate is unmaintained. While a way forward is being looked into for that, the remote_rendering works without but is a bit slow.

Copy link
Owner

@emilk emilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very interesting! Using serde-diff is a nice idea that i missed in eterm.

I only took a brief look: are you serializing Shapes directly then? Does that work when adding new glyphs, e.g. by pasting in some emojis in a TextEdit? I had problems with that in eterm.

Is it running synchronously, waiting for the next frame before returning?

examples/remote_rendering/src/server.rs Outdated Show resolved Hide resolved
let full_output_diff = received_message.into_text();

if let Ok(diff) = full_output_diff {
let deserialized: egui::FullOutput = serde_json::from_str(&diff).unwrap();
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using json for this will be very slow and wasteful - I suggest using bincode or MsgPack instead

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my earlier tests I experimented with those formats, and found no visible difference in performance. I ended up sticking with json because it was (surprisingly) the quickest at serializing, and server-side speed was my priority. This may have been specific to my use case though. Given that json also causes the INF and NAN issues it may be best to change it regardless.

.default_pos(egui::pos2(100.0, 0.0))
.show(&ctx, |ui| {
ui.label(format!("Count: {:?}", count));
});
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use egui_demo_lib instead - that would give a more convincing example, and a better benchmark for performance!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thanks

crates/epaint/Cargo.toml Outdated Show resolved Hide resolved
where
S: serde::Serializer,
{
// Workaround for the fact that INFINITY and NEG_INFINITY serialize as null and then fail to deserialize
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only in JSON. Most other formats support INF and NAN.

@linguini11
Copy link
Author

It serializes FullOutput directly by implementing Serialize and Deserialize for FullOutput and all of its fields. Shapes are done in this way, with the exception that Callback does not work (serde(skip) is used to ignore it).
I had not tried with Glyphs. Quickly testing it out now, the thumbs up emoji has got a blank box beside it when rendered. Is that similar to the problems you were getting?

image

The examples are running synchronously yes, as this was sufficient for my purposes. I don't think switching to asynchronous would be too much of a problem if that is preferable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants