SHIFT combined keypresses sent from "software/macro keyboard" are not working

[noREAVER]

OS: Windows 10 x64
Browser: Chrome
noVNC 1.4.0-4
noVNC 1.3.0 is working though

I noticed this issue while trying to use Streamdeck to type some text to noVNC consoles in Proxmox.
noVNC is 1.4.0-0 and has the big problem of not registering anything that's supposed to be a combination with SHIFT key
Anything that's not directly sending hardware keycodes fails.
For example typing with InputStick and xKeys works fine in hardware mode.
But if I type/send text with xKeys/MacroWorks in software mode or Streamdeck, then it fails to type correctly.
BUT noVNC 1.3.0 (i have one installed on a ondemand instance) doesn't have this issue at all.

Is strange that the first "<" character is typed correctly.
Also the text appears very fast, almost instantly typed on 1.3.0 but much slower and wrong on 1.4.0-4

Originally posted by @noREAVER in #1799 (comment)

[CendioOssman]

Not sure what we changed with regard to keyboards between those two releases.

Could you enable debug logging in noVNC and see what the browser console says when you press the problematic keys. Please test both 1.3.0 and 1.4.0.

[noREAVER]

Here are screenshots of both versions side by side (1.3.0<>1.4.0-4) when
using keyboard (manual) and streamdeck (auto)
What is strange is that when i opened first time the console and tried, i
only got/saw once the keycode for SHIFT key in the beginning of the
sequence, and never again after.
Is strange that is not showing SHIFT in either versions debug messages even
if one version works and the other doesn't
Ideas on how to debug this deeper?

Both have the same server protocol version

Server choose sub-protocol: binary
app.js?ver=1.5.0-1:12205 Server ProtocolVersion: 003.008
app.js?ver=1.5.0-1:12794 Screen: 1600x1200, bpp: 32, depth: 24, bigEndian: 0, trueColor: 1, redMax: 255, greenMax: 255, blueMax: 255, redShift: 16, greenShift: 8, blueShift: 0

This is a test without+with keyboard pressed shift while streamdeck is typing.

As you see the keycodes do correspond to the right characters (a97 A65) but is not sent further as it should

[noREAVER]

I think it might be related to this "noVNC and QEMU RFB keyboard extension" which is another difference. Proxmox seems to NOT use this extension.

• https://danielhb.github.io/article/2019/05/06/noVNC-QEMU-RFB.html
• Use the QEMU Extended Key Event message to (optionally) transmit keycodes instead of keysyms TurboVNC/turbovnc#108

[CendioOssman]

Debug logging does not seem to be properly enabled in those dumps, unfortunately. There should be lines with "onkeyevent" in there.

Check that you don't have any filters enabled in your browser. noVNC might be logging properly, but the browser might be configured to hide those things.

[noREAVER]

app.js?ver=1.5.0-1:11553 [Violation] Added non-passive event listener to a scroll-blocking 'touchstart' event. Consider marking event handler as 'passive' to make the page more responsive. See https://www.chromestatus.com/feature/5745543795965952
_connect @ app.js?ver=1.5.0-1:11553
_updateConnectionState @ app.js?ver=1.5.0-1:11791
RFB @ app.js?ver=1.5.0-1:11289
connect @ app.js?ver=1.5.0-1:15021
(anonymous) @ app.js?ver=1.5.0-1:14346
success @ app.js?ver=1.5.0-1:14202
API2Request.xhr.onload @ app.js?ver=1.5.0-1:13976
app.js?ver=1.5.0-1:11559 [Violation] Added non-passive event listener to a scroll-blocking 'wheel' event. Consider marking event handler as 'passive' to make the page more responsive. See https://www.chromestatus.com/feature/5745543795965952
_connect @ app.js?ver=1.5.0-1:11559
_updateConnectionState @ app.js?ver=1.5.0-1:11791
RFB @ app.js?ver=1.5.0-1:11289
connect @ app.js?ver=1.5.0-1:15021
(anonymous) @ app.js?ver=1.5.0-1:14346
success @ app.js?ver=1.5.0-1:14202
API2Request.xhr.onload @ app.js?ver=1.5.0-1:13976
app.js?ver=1.5.0-1:11246 >> RFB.constructor
app.js?ver=1.5.0-1:3527 >> Display.constructor
app.js?ver=1.5.0-1:3548 User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
app.js?ver=1.5.0-1:3549 << Display.constructor
app.js?ver=1.5.0-1:11782 New state 'connecting', was ''.
app.js?ver=1.5.0-1:11533 >> RFB.connect
app.js?ver=1.5.0-1:11535 connecting to wss://192.168.10.39:8006/api2/json/nodes/pve-39/qemu/39302/vncwebsocket?port=5900&vncticket=PVEVNC%3A674089D4%3A%3AgCui346ZtQXK8SwTVznthvZNTTsh9VnTeRP0GvkkhNyQsbtfgwint9BSnuD%2FDUGJ1Jr7T2Xvrtvjq9VFMWwhsMvjKfIYwtxnZ5aVxjkvTG5yBZOCrmWISyDN2UaOtAy4lDXKjzxkRyrbz7pOoCW1a%2FUYNGujAB2h2AM50yYgEUIOZHbruUZi%2FRMJcmKLWVilTS8ck9COERwuX7uXgc7U3KpOwnrGOOH0gaGTEAKzojl30XYLgBTLvZNWbQQ69eU4abReKD1PjV7TsAgHQ%2Bz2LJ8TIU3pVlYM4wGXU%2FXtCL2%2BPu4ZJkkNQMZ5c8otcywsfcz4TMfqCn4zVBz6xojpxw%3D%3D
app.js?ver=1.5.0-1:11563 << RFB.connect
app.js?ver=1.5.0-1:11290 << RFB.constructor
app.js?ver=1.5.0-1:8172 >> WebSock.onopen
app.js?ver=1.5.0-1:8174 Server choose sub-protocol: binary
app.js?ver=1.5.0-1:11598 Starting VNC handshake
app.js?ver=1.5.0-1:8177 << WebSock.onopen
app.js?ver=1.5.0-1:12205 Server ProtocolVersion: 003.008
app.js?ver=1.5.0-1:12243 Sent ProtocolVersion: 003.008
app.js?ver=1.5.0-1:12273 Server security types: 2
app.js?ver=1.5.0-1:12299 Authenticating using scheme: 2
app.js?ver=1.5.0-1:12738 Authentication OK
app.js?ver=1.5.0-1:12794 Screen: 1600x1200, bpp: 32, depth: 24, bigEndian: 0, trueColor: 1, redMax: 255, greenMax: 255, blueMax: 255, redShift: 16, greenShift: 8, blueShift: 0
app.js?ver=1.5.0-1:3609 Setting viewport to full display region
app.js?ver=1.5.0-1:11782 New state 'connected', was 'connecting'.
app.js?ver=1.5.0-1:12882 ServerCutText
app.js?ver=1.5.0-1:3609 Setting viewport to full display region
app.js?ver=1.5.0-1:3229 >> Keyboard.allKeysUp
app.js?ver=1.5.0-1:3233 << Keyboard.allKeysUp
app.js?ver=1.5.0-1:3229 >> Keyboard.allKeysUp
app.js?ver=1.5.0-1:3233 << Keyboard.allKeysUp
app.js?ver=1.5.0-1:3229 >> Keyboard.allKeysUp
app.js?ver=1.5.0-1:3233 << Keyboard.allKeysUp
app.js?ver=1.5.0-1:3229 >> Keyboard.allKeysUp
app.js?ver=1.5.0-1:3233 << Keyboard.allKeysUp
app.js?ver=1.5.0-1:3229 >> Keyboard.allKeysUp
app.js?ver=1.5.0-1:3233 << Keyboard.allKeysUp
app.js?ver=1.5.0-1:3229 >> Keyboard.allKeysUp
app.js?ver=1.5.0-1:3233 << Keyboard.allKeysUp
app.js?ver=1.5.0-1:3091 onkeyevent down, keysym: 126 , code: Platform231, numlock: true, capslock: false
app.js?ver=1.5.0-1:11486 Sending keysym (down): 126
app.js?ver=1.5.0-1:3091 onkeyevent up, keysym: 126 , code: Platform231, numlock: null, capslock: null
app.js?ver=1.5.0-1:11486 Sending keysym (up): 126
app.js?ver=1.5.0-1:3091 onkeyevent down, keysym: 60 , code: Platform231, numlock: true, capslock: false
app.js?ver=1.5.0-1:11486 Sending keysym (down): 60
app.js?ver=1.5.0-1:3091 onkeyevent up, keysym: 60 , code: Platform231, numlock: null, capslock: null
app.js?ver=1.5.0-1:11486 Sending keysym (up): 60
app.js?ver=1.5.0-1:3091 onkeyevent down, keysym: 62 , code: Platform231, numlock: true, capslock: false
app.js?ver=1.5.0-1:11486 Sending keysym (down): 62
app.js?ver=1.5.0-1:3091 onkeyevent up, keysym: 62 , code: Platform231, numlock: null, capslock: null
app.js?ver=1.5.0-1:11486 Sending keysym (up): 62
app.js?ver=1.5.0-1:3091 onkeyevent down, keysym: 63 , code: Platform231, numlock: true, capslock: false
app.js?ver=1.5.0-1:11486 Sending keysym (down): 63
app.js?ver=1.5.0-1:3091 onkeyevent up, keysym: 63 , code: Platform231, numlock: null, capslock: null
app.js?ver=1.5.0-1:11486 Sending keysym (up): 63
app.js?ver=1.5.0-1:3091 onkeyevent down, keysym: 65 , code: Platform231, numlock: true, capslock: false
app.js?ver=1.5.0-1:11486 Sending keysym (down): 65
app.js?ver=1.5.0-1:3091 onkeyevent up, keysym: 65 , code: Platform231, numlock: null, capslock: null
app.js?ver=1.5.0-1:11486 Sending keysym (up): 65
app.js?ver=1.5.0-1:3091 onkeyevent down, keysym: 66 , code: Platform231, numlock: true, capslock: false
app.js?ver=1.5.0-1:11486 Sending keysym (down): 66
app.js?ver=1.5.0-1:3091 onkeyevent up, keysym: 66 , code: Platform231, numlock: null, capslock: null
app.js?ver=1.5.0-1:11486 Sending keysym (up): 66
app.js?ver=1.5.0-1:3091 onkeyevent down, keysym: 67 , code: Platform231, numlock: true, capslock: false
app.js?ver=1.5.0-1:11486 Sending keysym (down): 67
app.js?ver=1.5.0-1:3091 onkeyevent up, keysym: 67 , code: Platform231, numlock: null, capslock: null
app.js?ver=1.5.0-1:11486 Sending keysym (up): 67
app.js?ver=1.5.0-1:3229 >> Keyboard.allKeysUp
app.js?ver=1.5.0-1:3233 << Keyboard.allKeysUp
app.js?ver=1.5.0-1:3229 >> Keyboard.allKeysUp
app.js?ver=1.5.0-1:3233 << Keyboard.allKeysUp

[noREAVER]

And this is with noVNC 1.3.0 and RFB on open on demand

>> RFB.constructor
display.js:26 >> Display.constructor
display.js:56 User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
display.js:58 << Display.constructor
rfb.js:780 New state 'connecting', was ''.
rfb.js:468 >> RFB.connect
rfb.js:471 connecting to wss://...
rfb.js:522 << RFB.connect
rfb.js:250 << RFB.constructor
vnc.html:1 [DOM] Input elements should have autocomplete attributes (suggested: "current-password"): (More info: https://goo.gl/9p2vKq) <input id=​"noVNC_password_input" type=​"password">​
websock.js:251 >> WebSock.onopen
rfb.js:562 Starting VNC handshake
websock.js:257 << WebSock.onopen
rfb.js:1240 Server ProtocolVersion: 003.008
rfb.js:1280 Sent ProtocolVersion: 003.008
vnc.html:1 <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes">Understand this warningAI
rfb.js:1299 Server security types: 19,2,16
rfb.js:1331 Authenticating using scheme: 16
ui.js:64 Fetch finished loading: GET "https://.../pun/sys/dashboard/noVNC-1.3.0/package.json".
start @ ui.js:64
(anonymous) @ ui.js:47
Promise.then
prime @ ui.js:45
(anonymous) @ ui.js:1755
rfb.js:1598 Server Tight authentication types: VENCVENCRYPT,STDVVNCAUTH_,TGHTULGNAUTH
rfb.js:1603 Selected authentication type: STDVVNCAUTH_
rfb.js:1663 Authentication OK
rfb.js:1737 Screen: 1688x1144, bpp: 32, depth: 24, bigEndian: 0, trueColor: 1, redMax: 255, greenMax: 255, blueMax: 255, redShift: 16, greenShift: 8, blueShift: 0
display.js:144 Setting viewport to full display region
rfb.js:780 New state 'connected', was 'connecting'.
ui.js:1017 >> UI.clipboardSend: code: Platform231...
ui.js:1019 << UI.clipboardSend
rfb.js:1851 ServerCutText
rfb.js:2121 Enabling continuous updates.
rfb.js:694 Requested new desktop size: 1688.1817626953125x1144.54541015625
2display.js:144 Setting viewport to full display region
display.js:144 Setting viewport to full display region
vnc.html:1 The resource https://.../pun/sys/dashboard/noVNC-1.3.0/app/images/error.svg was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.Understand this warningAI
vnc.html:1 The resource https://.../pun/sys/dashboard/noVNC-1.3.0/app/images/warning.svg was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.Understand this warningAI
keyboard.js:50 onkeyevent down, keysym: 126 , code: Platform231
rfb.js:432 Sending keysym (down): 126
keyboard.js:50 onkeyevent up, keysym: 126 , code: Platform231
rfb.js:432 Sending keysym (up): 126
keyboard.js:50 onkeyevent down, keysym: 60 , code: Platform231
rfb.js:432 Sending keysym (down): 60
keyboard.js:50 onkeyevent up, keysym: 60 , code: Platform231
rfb.js:432 Sending keysym (up): 60
keyboard.js:50 onkeyevent down, keysym: 62 , code: Platform231
rfb.js:432 Sending keysym (down): 62
keyboard.js:50 onkeyevent up, keysym: 62 , code: Platform231
rfb.js:432 Sending keysym (up): 62
keyboard.js:50 onkeyevent down, keysym: 63 , code: Platform231
rfb.js:432 Sending keysym (down): 63
keyboard.js:50 onkeyevent up, keysym: 63 , code: Platform231
rfb.js:432 Sending keysym (up): 63
keyboard.js:50 onkeyevent down, keysym: 65 , code: Platform231
rfb.js:432 Sending keysym (down): 65
keyboard.js:50 onkeyevent up, keysym: 65 , code: Platform231
rfb.js:432 Sending keysym (up): 65
keyboard.js:50 onkeyevent down, keysym: 66 , code: Platform231
rfb.js:432 Sending keysym (down): 66
keyboard.js:50 onkeyevent up, keysym: 66 , code: Platform231
rfb.js:432 Sending keysym (up): 66
keyboard.js:50 onkeyevent down, keysym: 67 , code: Platform231
rfb.js:432 Sending keysym (down): 67
keyboard.js:50 onkeyevent up, keysym: 67 , code: Platform231
rfb.js:432 Sending keysym (up): 67
vnc.html:1 The resource https://..../pun/sys/dashboard/noVNC-1.3.0/app/images/error.svg was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.Understand this warningAI
vnc.html:1 The resource https://..../pun/sys/dashboard/noVNC-1.3.0/app/images/warning.svg was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.Understand this warningAI
keyboard.js:240 >> Keyboard.allKeysUp
keyboard.js:244 << Keyboard.allKeysUp
keyboard.js:240 >> Keyboard.allKeysUp
keyboard.js:244 << Keyboard.allKeysUp

[noREAVER]

So it seems that this is a "limitation" of noVNC that is fixed by using RFB