Web touch event #2188

ryo33 · 2022-02-08T01:59:44Z

Tested on all platforms changed
Added an entry to CHANGELOG.md if knowledge of this change could be valuable to users
Updated documentation to reflect any user-facing changes, including notes of platform-specific behavior
Created or updated an example program if it would help users understand this functionality
Updated feature matrix, if new features were added or implemented

Closes #1673

This PR is inherited from inactive PR #1945.

The differences are as follows:

Correctly calculates the physical position by treating with scale factor ~~and flipping the Y-axis~~.
Support touch pressure.
add touch-action: none to a canvas to prevent the browser from canceling pointer events.
Working demo (pressure will be reflected in the cursor size).

Source code of the demo is here

src/platform_impl/web/web_sys/canvas/pointer_handler.rs

CHANGELOG.md

src/platform_impl/web/event_loop/window_target.rs

src/platform_impl/web/web_sys/canvas/pointer_handler.rs

ryo33 · 2022-05-26T07:54:21Z

I resolved the conflicts.

ryo33 · 2022-05-26T08:05:08Z

Also, I update the demo with the latest code

Liamolucko

LGTM

ryo33 · 2022-05-26T13:50:04Z

Thanks for reviewing!

ryo33 · 2022-05-30T07:17:15Z

@ryanisaacg Could you review this and suggest another reviewer for this?

ryanisaacg · 2022-06-01T12:28:23Z

I haven't been involved in any winit development for a few years at least; I'm not a good reviewer any longer. (Besides, it does look like you have an approving review?)

gilescope · 2022-11-12T17:13:17Z

One weird thing, the y axis seems inverted - needs to be height - y. Top left should be 0,0. egui touch handling and my brain expects this, but it seems that bottom left is currently 0,0.
I'm assuming that oddness is coming from this PR.

ryo33 · 2022-11-13T06:35:57Z

Oh, I may misunderstand the treatment of the y-axis in winit.

https://docs.rs/winit/latest/winit/event/enum.WindowEvent.html#variant.CursorMoved

(x,y) coords in pixels relative to the top-left corner of the window. Because the range of this data is limited by the display area and it may have been transformed by the OS to implement effects such as cursor acceleration, it should not be used to implement non-cursor-like interactions such as 3D camera control.

So we don't need y-axis flipping?

gilescope · 2022-11-13T09:22:24Z

Personally I don’t think so. Have you got a second flip that offsets it in the web demo? Just wondering why that one works.

…

On Sun, 13 Nov 2022 at 06:36, Ryo Hirayama ***@***.***> wrote: Oh, I may misunderstand the treatment of the y-axis in winit. https://docs.rs/winit/latest/winit/event/enum.WindowEvent.html#variant.CursorMoved (x,y) coords in pixels relative to the top-left corner of the window. Because the range of this data is limited by the display area and it may have been transformed by the OS to implement effects such as cursor acceleration, it should not be used to implement non-cursor-like interactions such as 3D camera control. So we don't need y-axis flipping? — Reply to this email directly, view it on GitHub <#2188 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGEJCEFF7OIY435XLVMONTWICD5RANCNFSM5NZDJQ4A> . You are receiving this because you commented.Message ID: ***@***.***>

mockersf · 2022-11-15T01:53:49Z

So we don't need y-axis flipping?

I'm trying this PR in Bevy, and I had to flip back the Y axis in Bevy for it to work. ~~So I would say that it isn't needed.~~
Nevermind I think it's expected in Bevy to flip it for now.

donmai-me · 2022-11-22T08:20:45Z

I've tried the demo on my iPad and it works great! However, there are two problems I've encountered. First is that touch events using Apple Pencil don't register. Second is that canceled touch events, such as when switching windows when using four-finger swipes, aren't removed.

The last one may be because the demo's source code doesn't handle canceled touch events.

ryo33 · 2022-11-22T12:15:15Z

Apple Pencil emits pointer events with pointer_type = "pen".
Does anyone know how winit handles Apple Pencil on iOS target? If it handles it as touches, we need to handle the "pen" pointer type.

ryo33 · 2022-11-26T08:57:25Z

I'm finally confident that we don't need the y-flip, and it's ready to merge.
I don't think we should support pen events for now because other platforms do not support it yet, as #99.

Demo improvement (now it's really simple, comprehensive, and easy to understand)

See demo and its source.

Changes are:

Use UI instead of Sprite because bevy's UI and winit share the same coordinate system (so no complex position transformation).
Handle the touch cancel event (thanks to @donmai-me for reporting).

adsick · 2022-11-30T13:47:48Z

tried to run the demo on my android phone using chrome browser and don't see anything going on except for some gray area. (I'm pushing the screen here and there)

Screenshot

maroider · 2022-12-01T01:28:00Z

I can reproduce the issue with the demo on Chrome for Android, but it works fine everywhere else I could test it (Desktop Chromium, Firefox on Desktop and Android). Our web example also works fine on Chrome for Android, so I wouldn't consider this to be a blocking issue, as this is likely an issue with how the demo is set up.

gilescope · 2022-12-01T06:28:38Z

I note the same on android but I too suspect it is not due to this pr. neet to run an emulator or steal someone’s phone to check out what’s happening.

…

On Thu, 1 Dec 2022 at 01:28, Markus Røyset ***@***.***> wrote: I can reproduce the issue with the demo on Chrome for Android, but it works fine *literally* everywhere else. Our web example also works fine on Chrome for Android, so I wouldn't consider this to be a blocking issue. — Reply to this email directly, view it on GitHub <#2188 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGEJCDYKICKF62OUX6KGY3WK75KXANCNFSM5NZDJQ4A> . You are receiving this because you commented.Message ID: ***@***.***>

peterhirn · 2022-12-18T18:25:28Z

I was curious and fixed the android issue: demo, repo. Everything seems to work as expected. Nice work.

gilescope · 2022-12-19T06:48:34Z

(My android issue was unrelated - was using a texture > 4096 high.)

…

On Sun, 18 Dec 2022 at 18:25, peterhirn ***@***.***> wrote: I was curious and fixed the android issue: demo <https://peterhirn.github.io/winit-web-touch-event-test/>, repo <https://github.com/peterhirn/winit-web-touch-event-test>. Everything seems to work as expected. Nice work. — Reply to this email directly, view it on GitHub <#2188 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGEJCCOJS3CHYPMZYDX4KTWN5JKHANCNFSM5NZDJQ4A> . You are receiving this because you commented.Message ID: ***@***.***>

madsmtm

I'm sorry, I haven't really been following this issue, and am a bit confused - @ryo33, do you feel this is ready now?

I'd be fine with merging it as-is (once CI passes), it has gone for long enough and while it may not be perfect it's clearly an improvement :)

src/event.rs

Co-authored-by: Mads Marquart <[email protected]>

ryo33 · 2022-12-23T03:46:34Z

@madsmtm Yes, I think it’s ready to merge.

madsmtm · 2022-12-23T05:55:58Z

Cool, thanks for your contribution, and sorry again it took so long (it's difficult when we don't have a maintainer for web)!

ryo33 · 2022-12-23T09:02:46Z

I haven’t figured winit out yet, but I hope I would be a maintainer for web in the future maybe. I’m creating a huge product that depends on winit, and it should work well on web. I may work on keyboard issues when my hands are free.

Liamolucko reviewed Feb 12, 2022

View reviewed changes

src/platform_impl/web/web_sys/canvas/pointer_handler.rs Outdated Show resolved Hide resolved

maroider added DS - web C - waiting on maintainer A maintainer must review this code labels Feb 20, 2022

This was referenced May 25, 2022

Weird touch behavior on mobile device vladbat00/bevy_egui#104

Closed

Implement touch handling on web target(s) #1673

Closed

Liamolucko reviewed May 25, 2022

View reviewed changes

dsluijk and others added 15 commits May 26, 2022 16:50

feat: add pointer events to web

52fe7c7

feat: remove PointerType for touch events

93422c1

Remove duplicate

08cdb4a

Changelog and features

a09201c

Remove PointerType

950a7e5

feat: renamed events, added touch type guard

6822fc8

Rename

e47fe8d

Flip the y axis

2c7eab9

Fix physical position and add force

0094ec2

Update comment

4f38d82

Update features

6b22593

Use normalized force

5ef9679

Remove unnecessary todos

84f9c66

Update comment

3c5635f

Refactor add touch_handler

1911832

ryo33 force-pushed the web-touch-event branch from 907ebfb to 1911832 Compare May 26, 2022 07:53

ryo33 added 2 commits May 26, 2022 19:11

Rephrase by Liamolucko

a977817

Update CHANGELOG.md

2162420

Liamolucko approved these changes May 26, 2022

View reviewed changes

Remove y-axis flip

79209f9

Merge branch 'master' into web-touch-event

0ddacfb

ryo33 requested review from maroider and removed request for madsmtm November 26, 2022 09:03

madsmtm approved these changes Dec 23, 2022

View reviewed changes

src/event.rs Outdated Show resolved Hide resolved

Update src/event.rs

8db6ac4

Co-authored-by: Mads Marquart <[email protected]>

madsmtm added 3 commits December 23, 2022 06:38

Merge branch 'master' into web-touch-event

9e25a4b

Fix platform specific comment

49890f1

Fix extra argument to touch_position function

1e9dca1

madsmtm merged commit f43ce2a into rust-windowing:master Dec 23, 2022

madsmtm mentioned this pull request Dec 23, 2022

Add support for event::Touch on web targets #1945

Closed

8 tasks

ryo33 deleted the web-touch-event branch December 23, 2022 08:53

madsmtm mentioned this pull request Mar 4, 2023

web: Fix position of touch events to be relative to the canvas #2704

Merged

5 tasks

This was referenced Jun 2, 2023

wasm: Touch events not received on Android browsers #1286

Closed

Web: Pointer Cleanup #2847

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Web touch event #2188

Web touch event #2188

ryo33 commented Feb 8, 2022 •

edited

Loading

ryo33 commented May 26, 2022

ryo33 commented May 26, 2022 •

edited

Loading

Liamolucko left a comment

ryo33 commented May 26, 2022

ryo33 commented May 30, 2022

ryanisaacg commented Jun 1, 2022

gilescope commented Nov 12, 2022

ryo33 commented Nov 13, 2022

gilescope commented Nov 13, 2022 via email

mockersf commented Nov 15, 2022 •

edited

Loading

donmai-me commented Nov 22, 2022

ryo33 commented Nov 22, 2022 •

edited

Loading

ryo33 commented Nov 26, 2022

adsick commented Nov 30, 2022 •

edited by maroider

Loading

maroider commented Dec 1, 2022 •

edited

Loading

gilescope commented Dec 1, 2022 via email

peterhirn commented Dec 18, 2022

gilescope commented Dec 19, 2022 via email

madsmtm left a comment

ryo33 commented Dec 23, 2022

madsmtm commented Dec 23, 2022 •

edited

Loading

ryo33 commented Dec 23, 2022

Web touch event #2188

Web touch event #2188

Conversation

ryo33 commented Feb 8, 2022 • edited Loading

ryo33 commented May 26, 2022

ryo33 commented May 26, 2022 • edited Loading

Liamolucko left a comment

Choose a reason for hiding this comment

ryo33 commented May 26, 2022

ryo33 commented May 30, 2022

ryanisaacg commented Jun 1, 2022

gilescope commented Nov 12, 2022

ryo33 commented Nov 13, 2022

gilescope commented Nov 13, 2022 via email

mockersf commented Nov 15, 2022 • edited Loading

donmai-me commented Nov 22, 2022

ryo33 commented Nov 22, 2022 • edited Loading

ryo33 commented Nov 26, 2022

Demo improvement (now it's really simple, comprehensive, and easy to understand)

adsick commented Nov 30, 2022 • edited by maroider Loading

maroider commented Dec 1, 2022 • edited Loading

gilescope commented Dec 1, 2022 via email

peterhirn commented Dec 18, 2022

gilescope commented Dec 19, 2022 via email

madsmtm left a comment

Choose a reason for hiding this comment

ryo33 commented Dec 23, 2022

madsmtm commented Dec 23, 2022 • edited Loading

ryo33 commented Dec 23, 2022

ryo33 commented Feb 8, 2022 •

edited

Loading

ryo33 commented May 26, 2022 •

edited

Loading

mockersf commented Nov 15, 2022 •

edited

Loading

ryo33 commented Nov 22, 2022 •

edited

Loading

adsick commented Nov 30, 2022 •

edited by maroider

Loading

maroider commented Dec 1, 2022 •

edited

Loading

madsmtm commented Dec 23, 2022 •

edited

Loading