Version 0.4 Full Release Tracking Issue

[ryanisaacg]

During Alpha

Features that have yet to be (re-)implemented, and are planned:

• Render-to-texture / Surface ([Question] [2D] Documentation for off-screen rendering? #537, Initial implementation of render-to-texture #561)
• Text rendering (Add font support #581)
• Resize handling (Handle resizes #566)
• Blending support (Add support for setting the blend mode #562)
• Helpers for planning when to update / draw (Add implementation for timers #587)
• Texture filtering and tiling (Initial implementation of render-to-texture #561)
• Set browser tab titles blinds#5
• Make events model more forward-compatible blinds#14
• Re-export crates in that appear in the public API, like blinds and mint (Re-export libraries that appear in the public API #568)
• Expose mipmapping?
• Expose setting the log level to the user (Create the 'input' module to replace 'lifecycle' #604)
• Wrap blinds types (Create the 'input' module to replace 'lifecycle' #604)

Features that are yet to be (re-)implemented, and are a maybe:

• Input state caches (like Keyboard, Mouse, Modifiers) (Add a module to cache event inputs blinds#22)
• Direct access to the golem graphics API (Add a method to convert Graphics to Context #588)

Documentation:

• Overhaul the workflow docs (like the issue templates, PR template, CONTRIBUTING, etc.)
• Indicate which libraries/tutorials/games are 0.3.x and which are 0.4.x
• Examples:
  • Window resize example (Actually implement a resize API #616)
  • Reading keyboard and mouse states (Create the 'input' module to replace 'lifecycle' #604)
  • Responding to events (Create the 'input' module to replace 'lifecycle' #604)
  • Font rendering (Add font support #581)

Feedback required:

• Should the Scalar API be kept? (e.g. keep paying the compile-time process for generics to be able to write terse things like Rectangle::new((x, y), (width, height)) or require explicitness)
• Should geometry be split into a separate crate to allow use outside Quicksilver?
• Should image formats other than PNG be supported out-of-the-box? They're not widely used, can cause issues (Loading a jpg image causes a panic in WASM #508), and cause binary bloat.
• How should key repeats be handled? (See Key presses get repeated #523)

Bugs / tickets:

• Problem with rendering to texture #601 (Don't allow images to drop while they're in-flight #643)
• How to properly do resizing #602 / The ResizeHandler mentioned by changelog doesn't really exist yet #594 (Actually implement a resize API #616)
• Alpha channel for color behaves inconsistently between web and desktop #606
• Font rendering fails to draw characters on screen #607
• An easy way to get the cursor location that takes into account resize strategy #620 (Remove the manual viewport API #641)
• Incorrect coordinate system scale when rendering on a surface #623 (Remove the manual viewport API #641)
• Rendering to surface is upside down #631 (Remove the manual viewport API #641)
• Text extents says y size is 0 #633 / Font rendering fails to draw characters on screen #607 (Fix text extents not reflecting line height #642)
• Crash when attempting to load font in the web + wasm_bindgen context #636
• Settings icon_path doesn't seem to do anything on Web or MacOS? #639

Investigate:

• Canvas high-dpi
• Are all the re-export paths sensible or do some require referencing external crates?

API reworks from alpha experience:

• Text does not work great with Transform::scale #617
• Take away control of viewport, and automatically set it when you retarget the window (maybe additionally have an automatic orthographic projection, to address Incorrect coordinate system scale when rendering on a surface #623?) (Remove the manual viewport API #641)

When Alpha ends:

• Update the website (Missing the example code #596)

During Beta

• Migration guide from 0.3 to 0.4
• Note how to use mint to convert Quicksilver types into other geometry library types
• Migrate to web-sys (Deprecate stdweb and migrate to web-sys #653)

Future Minor Release

• A Lyon helper (Line width and end-cap options for draw_mesh. #638)
• Investigate RPi / GL ES 3.0 Support golem#28
• End-to-end game example
• Document the async workflow for loading assets / using other event sources during the event loop

Future Major Release

• Audio support
• Migrate default / recommended web backend to web-sys

[lenscas]

Should image formats other than PNG be supported out-of-the-box? They're not widely used, can cause issues (#508), and cause binary bloat.

Personally I can see someone writing code that on the web it first loads jpg versions of (some of) the textures to get the loading done quicker and after that load the better png versions on the background. Especially combined with a service worker to get the png images in cache for next time.

However, according to the linked issue this isn't an option anyway as jpg's crash on the web. Add to this that I see no benefit for them on native and my vote is a clear remove. At least for the time being until it can work again on the web.

As for svg's (Which I assume is the other format being talked about). I have no problem with their removal as I don't use them myself. I don't think they are great for textures leaving their use just for UI elements. I also don't think most other engines have support for svg anyway (at the very least unity doesn't) so not having support for svg's isn't even that surprising.

So, my vote on svg support: It is nice to have but I won't care if it gets removed.

[ryanisaacg]

Svgs aren't actually the other formats I was mentioning (though they are vaguely supported through lyon, once that is re-implemented). The image crate includes tiff, gif, jpeg, bmp, ico, and png. Of those, I think png is used the most, but every crate that pulls in Quicksilver takes a compile time and binary size hit to support the others.

[MaulingMonkey]

Huzzah, progress! Some minor notes from an upgrade attempt, since I'm not sure if these are just tracking missing API bits, or if they include default implementation stuff:

Features

• Blending support

The default blending mode in 0.4.0-alpha0 appears to ignore the image alpha channel now on both desktop (transparent areas are black) and web (transparent areas are white!).

• Resize handling

Settings already exposes resize, but the GL viewport is neither automatically set, nor settable through quicksilver, making it impossible to render to the expanded window size.

Feedback

• Should the Scalar API be kept? (e.g. keep paying the compile-time process for generics to be able to write terse things like Rectangle::new((x, y), (width, height)) or require explicitness)

No strong preference either way. The case where I get more annoyed is when encountering things like Window::size() or Image::size() returning f32 vectors that I want to convert back to integers, triggering paranoia about fractional pixels, negative dimensions, and NaNs.

• Should geometry be split into a separate crate to allow use outside Quicksilver?

Might be better to try and extend an existing math crate like nalgebra to contain the types?

• Should image formats other than PNG be supported out-of-the-box? They're not widely used, can cause issues (Loading a jpg image causes a panic in WASM #508), and cause binary bloat.

I think it's fine for the default feature set to not support PNGs. Just have a decent error message for devs when you try to load .jpeg s anyways.

• How should key repeats be handled? (See Key presses get repeated #523)

This is a whole rabbit hole, be aware that fully solving the problem perfectly is nontrivial to impossible. Having a basic "best effort" BTreeSet<Key> somewhere that's set on keydown - and cleared on keyup and window blur - would be reasonable and nice. Forwarding any OS-provided "repeat" bits might also be useful (maybe as a Modifiers bit?)

Some rabbit hole details:

OS event queues may conflate left/right modifiers like control, shift, alt, etc. in a way that makes perfect key state tracking annoying, impossible, or expensive.

XB1 chatpads like https://user-images.githubusercontent.com/33192525/63055071-f323e380-be99-11e9-81c2-9fe66b51310c.jpg send keyup despite holding buttons, and then re-spam keydown / keyup pairs on repeats.

IE11 also doesn't set the input event repeat field on repeat events, but IE11 doesn't natively support WASM so we probably don't care, even if some crazy person might try to get it working via polyfill.

[lenscas]

Svgs aren't actually the other formats I was mentioning (though they are vaguely supported through lyon, once that is re-implemented). The image crate includes tiff, gif, jpeg, bmp, ico, and png. Of those, I think png is used the most, but every crate that pulls in Quicksilver takes a compile time and binary size hit to support the others.

bmp is (if I remember correctly) uncompressed and thus a bad choice for anything web related. That alone is in my opinion enough reason not to support it.

I don't think anyone will try tiff and ico files, because at least to me they seem like weird choices for sprites. So, I think they are save to go.

I care little about what happens to gif support.

Png's should stay

Obviously, nice error messages along with documentation on what is supported is always appreciated :)

[elsassph]

JPEG would have been nice for web target.

[ryanisaacg]

It seems like jpeg support is desired, so it stays! See #558. As a nice side effect, rayon is no longer included in the build.

@MaulingMonkey

The default blending mode in 0.4.0-alpha0 appears to ignore the image alpha channel now on both desktop (transparent areas are black) and web (transparent areas are white!).

That was a bug, my bad. Fixed in #560

Settings already exposes resize, but the GL viewport is neither automatically set, nor settable through quicksilver, making it impossible to render to the expanded window size.

Yup, that's part of what remains to be done.

No strong preference either way. The case where I get more annoyed is when encountering things like Window::size() or Image::size() returning f32 vectors that I want to convert back to integers, triggering paranoia about fractional pixels, negative dimensions, and NaNs.

Good point; things that can be ints, probably should be.

Might be better to try and extend an existing math crate like nalgebra to contain the types?

With the exception of maybe vek, most math crates have some minor or major problem that puts me off using them.

keyboard event rabbit hole

Thanks for that info, which is a good look into how horrifying platform APIs are. I think I agree, that a simple best-effort set of currently-down keys (and maybe a separate set for mouse buttons, gamepad buttons, etc

[elsassph]

What's the story for framerate control? Will there be a helper in the lib?

[ryanisaacg]

Yup, the checkmark for that is "Helpers for planning when to update / draw"

[paulirotta]

Would you be open to having a "canvas size matches screen/web-window size" feature for full screen?

Something along the lines of:

canvas.set_width(canvas.offset_width() as u32);
canvas.set_height(canvas.offset_height() as u32);

( https://github.com/koute/stdweb/blob/master/examples/canvas/src/main.rs )

[elsassph]

Related to @paulirotta 's question: I noticed the canvas isn't high dpi.

[ryanisaacg]

@paulirotta Hm, why match the canvas to the size of the window rather than native fullscreen?

@elsassph If the canvas isn't high-dpi, that could be a winit bug. I'll add investigating that as a task for this issue.

[paulirotta]

@ryanisaacg Native full screen sounds great!

[lenscas]

Personally, I wouldn't mind to also have an option to set the width and height of the canvas to the size of the browser window. I can defiantly see people (Even myself) wanting to play a game in a screen as big as possible without hiding the rest of the browser.

On native this option should just be the same as maximizing a program.

[lenscas]

I have a question regarding font/text rendering.

I have a sudden urge to try and get a text input to work in Mergui. This will probably be done by rendering every character inside of it separately, instead of making 1 big image out of the text.

Do you foresee many changes between the current system and the new one? And if so, are there things I can already plan for?

Thanks in advance and also, thanks for making quicksilver :)

[ryanisaacg]

@lenscas I've been working on font support locally, which I've just pushed up as #581. The new font system essentially bears no resemblance to the old one, so instead of trying to explain the differences I figured showing them would be better.

[lenscas]

Thanks for the previous answer, it gave me everything I needed to know to start porting mergui :)

However, a few more questions

I saw that there is no function to draw with an Z order. Is that function coming back/am I blind? I am also guessing that this means that draw order now works correctly? ( 🎉 )

If it is coming back, how would a draw with a z value interact with one that doesn't? Specifically in the case where the draw without a z value comes after the one with one set? Its fine if you don't know this one, it is more to manage my own time :)

Thanks in advance for answering :)

[ryanisaacg]

Z ordering is dead, and draw order Just Works correctly.

[lenscas]

In the case where you draw to a texture after having made some draw calls that are supposed to go to the window. Is it required to flush first before calling Graphics::fit_to_surface, or is there another way to get the desired result?

//these 2 are supposed to be rendered to the window
gfx.clear(Color::BLACK);
gfx.fill_rect(
    &Rectangle::new(Vector::new(350.0, 100.0), Vector::new(100.0, 100.0)),
    Color::BLUE,
);

gfx.flush(None)?; //This one seems to be needed

//the calls from this point on need to go to the surface
let mut surface = Surface::new(
    &gfx,
    Image::from_raw(&gfx, None, 512, 512, PixelFormat::RGBA)?,
)?;
// Set the render area to the surface size
gfx.fit_to_surface(&surface)?;
gfx.clear(Color::WHITE);

[ryanisaacg]

This is a failure of the API to communicate what's actually happening. gfx.fit_to_surface doesn't change the render target; it changes what the viewport is. You need to call gfx.flush(None)? to send the previous draw commands to the window, and presumably later you'll call gfx.flush(Some(&surface))? to send some draw commands to that surface.

[lenscas]

Ok, then I'm not doing anything wrong :)

Maybe gfx.fit_to_surface should do the call to gfx.flush for you? And if there is a significant cost to flushing and a method that doesn't do it? Or perhaps an extra enum argument to tell it to either flush or not?

I can probably make a pull request for this this weekend.

[lenscas]

I have a weird problem when rendering to a texture. The generated image seems to always be black. However,if I copy the code to to another project it works as expected. The rest of the window gets rendered correctly. So, I'm not even sure if I have other drawing code that causes the problem :(

The code in question

gfx.flush(None)?;
let mut surface = Surface::new(
    gfx,
    Image::from_raw(gfx, None, 512, 512, PixelFormat::RGBA)?,
)?;
gfx.fit_to_surface(&surface)?;
gfx.clear(Color::BLUE);
gfx.fill_rect(&Rectangle::new((0, 0), (10, 10)), Color::GREEN);
gfx.flush(Some(&surface))?;
//gfx.clear(Color::RED);
gfx.fit_to_window(&window);
let image = surface
    .detach()
    .ok_or(quicksilver::QuicksilverError::SurfaceImageError)?;
gfx.draw_image(&image, Rectangle::new((10, 10), (512, 512)));

In case it helps, I commited how far I got, the code can be viewed at :
https://github.com/lenscas/mergui/blob/21ea53807a9097fdd1638ec01c265475a29692ca/src/widgets/input.rs#L140

I use the all example to test it

Sorry in advance of the state of that file. I was busy in redoing how it worked to make use of rendering to texture and I tried many things after that to try and discover why it didn't work.

[ryanisaacg]

It looks like everything you're doing is aboveboard, can you open this as an issue? I think it's probably a Quicksilver bug.

[lenscas]

I have a question about Graphics::into_raw_context(self) and what it means for libraries that need Graphics to draw things.

Right now, a library that needs to draw things can not be used by people who want to use custom shaders because there is no way to get the Graphics object back. A way to reduce this problem is to introduce a new trait that is implemented by Graphics and contains the same or a subset of the draw calls currently implemented by Graphics.

This means that libraries can be generic over this trait if the maintainer wants to support users with custom shaders, and developers who write custom shaders have a single trait to implement to be able to use these libraries.

My questions are:

• Would a trait like this make sense to exist at all.
• Would a trait like this make sense to be included inside quicksilver? Or do you think every library that needs to draw using quicksilver is better off making their own trait(s) for this purpose instead?

Thanks in advance for answering these 2 questions.

[ryanisaacg]

@lenscas When using custom shaders, it's hard to maintain the soundness of the Graphics API. golem relies on the user upholding the contract of the prepare_draw. The reason you lose access to the Graphics API is to ensure the safety of flush.

It may be better, rather than trying to replicate the entire Graphics interface through a trait, to have a trait that just incorporates what you need.

[lenscas]

Ok, this makes sense. Thanks for the answer :)

[paulirotta]

Rasperry Pi 4 raspbian, example 02_image throws a (kudos) clear whine:

ERROR [golem::shader] Failed to compile vertex shader: 0:1(10): error: GLSL 1.50 is not supported. Supported versions are: 1.10, 1.20, 1.00 ES, and 3.00 ES

[ryanisaacg]

@paulirotta I opened an issue at ryanisaacg/golem#28 to investigate / fix this.

[lenscas]

I decided to make a simple animation system that can either be its own library or be merged with Quicksilver (Whatever you prefer)

However when writing it I noticed that there doesn't seem to be a way to rotate a single image. I saw Graphics::set_transform() however that doesn't seem to work for this as simply setting the rotation causes my image to rotate around a certain point, rather than it spinning at its center.

I only needed it for an example so it isn't a huge deal right now as I can just change the color or something for the time being but it did kind of surprise me.

Note: it may very well just be me overlooking something. In case it is, the code I used is equivalent to

let step_size: f32 = 5.;
let amount_of_steps = (360. / step_size).ceil() as usize;
let step = tick % amount_of_steps;
let rotation = step as f32 * step_size;
let rotation = dbg!(rotation);
gfx.set_transform(Transform::rotate(rotation));
gfx.draw_image(state, location);
gfx.set_transform(Transform::IDENTITY);

[ryanisaacg]

@lenscas Graphics::set_transform sets a transformation matrix; if you want to spin an image at its center, you have to offset the transform to the center of the image, like:

let origin = location.center();
gfx.set_transform(Transform::translate(origin) * Transform::rotate(rotation) * Transform::translate(-origin));
gfx.draw_image(state, location);
gfx.set_transform(Transform::IDENTITY);

[nzsg]

I’ve been following this with interest and just wanted to say thanks for making and improving Quicksilver!