Windows -> Wayland (Sway) Modifier Keys not registering

[Vassili-Dev]

When running Windows + Sway:

Windows -> Sway: Modifier+Key only seems to register Key (Can't use Sway Shortcuts, Can't do ^C, etc)
Sway -> Windows: Works as expected

Windows

• Windows Version: Windows 11 23H2
• Emulation Backend: windows
• Capture Backend: windows
• Frontend: gtk (Also daemonized)

Sway

• Sway Version: 1.9 (Arch 6.10.10-arch1-1, x86_64)
• Emulation Backend: wlroots
• Capture Backend: layer-shell
• Frontend: gtk (Also daemonized)

Logs:
LanMouseLog_Sway.txt
LanMouseLog_Windows.txt

[Vassili-Dev]

I won't pretend to know how this works, so please take this with a bucket of salt, but I suspect it has to do with to_key_event not generating KeyboardEvent::Modifiers.

lan-mouse/input-capture/src/windows.rs

Lines 184 to 225 in b071201

	unsafe fn to_key_event(wparam: WPARAM, lparam: LPARAM) -> Option<KeyboardEvent> {
	let kybrdllhookstruct: KBDLLHOOKSTRUCT = *(lparam.0 as *const KBDLLHOOKSTRUCT);
	let mut scan_code = kybrdllhookstruct.scanCode;
	log::trace!("scan_code: {scan_code}");
	if kybrdllhookstruct.flags.contains(LLKHF_EXTENDED) {
	scan_code |= 0xE000;
	}
	let Ok(win_scan_code) = scancode::Windows::try_from(scan_code) else {
	log::warn!("failed to translate to windows scancode: {scan_code}");
	return None;
	};
	log::trace!("windows_scan: {win_scan_code:?}");
	let Ok(linux_scan_code): Result<Linux, ()> = win_scan_code.try_into() else {
	log::warn!("failed to translate into linux scancode: {win_scan_code:?}");
	return None;
	};
	log::trace!("windows_scan: {linux_scan_code:?}");
	let scan_code = linux_scan_code as u32;
	match wparam {
	WPARAM(p) if p == WM_KEYDOWN as usize => Some(KeyboardEvent::Key {
	time: 0,
	key: scan_code,
	state: 1,
	}),
	WPARAM(p) if p == WM_KEYUP as usize => Some(KeyboardEvent::Key {
	time: 0,
	key: scan_code,
	state: 0,
	}),
	WPARAM(p) if p == WM_SYSKEYDOWN as usize => Some(KeyboardEvent::Key {
	time: 0,
	key: scan_code,
	state: 1,
	}),
	WPARAM(p) if p == WM_SYSKEYUP as usize => Some(KeyboardEvent::Key {
	time: 0,
	key: scan_code,
	state: 1,
	}),
	_ => None,
	}
	}

[feschber]

Yeah, I hinted at this in the README: "Gnome -> Sway only partially works (modifier events are not handled correctly)"
This is a bit of a lie, as it does not only apply to GNOME but technically anything but Sway at this point, as you have realized.

You are totally right about the to_key_event function, however I think it's ultimately better to generate the Modifiers event on the client side rather than on every capture backend (this would need to be added to the input-capture-portal backend as well for example). The wlroots backend is the only backend that actually needs modifier events as far as I remember.

Unfortunately I have not found the time to implement this yet but its definitely a priority once the encryption stuff is done :)