Mozc switches to direct input (English) haphazardly

[niwazukihon]

Description

I want mozc to always be in the mode I specified (hiragana), however, even without typing any keys that would switch it to direct input (English), mozc often switches to English by itself, which is very annoying. In fact, Google Japanese Input and Microsoft Japanese IME have similar issue (but don't seem to be exactly the same).

The behavior is kind of inconsistent. On most apps, it always starts as direct input but seems to stay the same if I change it to hiragana for once. However, if I exit and reopen them, it will be in direct input again. In some cases, it will always switch to direct input whenever the app window is switched to be in focus.

This issue was mentioned in #907.

Steps to reproduce

Steps to reproduce the behavior:

1. Go to an app.
2. The input mode of mozc switches to direct input (English).

Expected behavior

Mozc doesn't change input mode unless I tell it to do so.

Actual behavior

Mozc changes input mode to direct input unexpectedly.

Start as direct input after app starts: PDF-XChange, Windows Explorer, Windows settings search box, VS Code, MATLAB, Directory Opus, Obsidian, ...,
even, Windows desktop (the input mode when no app window is in focus, it changes to direct input after Windows restarts).

Always switches to direct input: Windows Terminal.

Version or commit-id

Mozc-2.30.5618.100+24.11.oss

Environment

• OS: Windows 11.

[yukawa]

The short answer is that currently there is no way to control the behavior as you expected. Sorry about that.

The long answer is that this is due to Windows API contracts between applications and IMEs. On Windows, there are several public APIs that can be called by applications to specify what kind of characters are expected in the current editing context. This means when an application calls a certain Windows API, Mozc does change its mode as requested by the application. I'm mostly certain that this is what you have been observing.

Windows Terminal you mentioned in indeed a good example.

Always switches to direct input: Windows Terminal.

The behavior to change the IME mode to the direct input upon startup is indeed explicitly implemented in the Windows Terminal per the following request.

• Feature request: Set IME input mode to English when a Windows Terminal window is opened microsoft/terminal#12731

Then they had also received several complaints about such a behavior.

• Has the automatic switching input method function been canceled? microsoft/terminal#17816 (comment)

And finally the behavior is now configurable.

• Add input scope startup setting microsoft/terminal#17953

Expected behavior

Mozc doesn't change input mode unless I tell it to do so.

One idea is to implement a new optional mode to completely ignore such Windows APIs. I think it's technically doable, but unfortunately I most likely don't have time to design and implement it in the near time future.

[niwazukihon]

Thank you for the reply. I found #381 which seems to be the same issue, but was solved. I guess it's not easy to do it on Windows then...

I turned off that option in Windows Terminal and it now has the same behavior as other apps --- always switches to direct input after app starts but can stay in hiragana while the app is running.

This even happens when the display language, region, and system locale are all set to Japanese. I don't understand why the apps or Windows decide to have such behavior..

[niwazukihon]

Come to think of it, maybe it's possible to use the Windows API that the script I wrote in the other comment uses to do this? Like the LoadKeyboardLayout in winuser.h, etc. So instead of ignoring those Windows API, just actively change it back.

[yukawa]

Come to think of it, maybe it's possible to use the Windows API that the script I wrote in the other comment uses to do this?

I'm not familiar with AutoHotKey, so just give some general history of the IME APIs on Windows.

As of Windows 11, basically two sets of IME APIs are widely used.

A. Input Method Manager (IMM32)
B. Text Services Framework (TSF)

IMM32 is roughly 30-year old API, while TSF is roughly 20-year old API.

If you are an IME developer (e.g. a developer of Mozc), you basically have no choice. TSF API is the only practical choice for your IME to work well on Windows 8 and later versions.

If you are an applicaiton developer (e.g. a developer of text editor, web browser, and so on), then you can choose which API to use to develop your application. When an app is built on top of TSF, basically all the API calls will be directly dispatched to the IME. When an app still relies on IMM32, there is an emulation layer called "Cicero unaware application support" (CUAS), which translates IMM32 API calls into TSF API calls. Microsoft used to publish its technical detail as IMM32互換性情報.doc 18 years ago for application developers in Japan, but it seems that the document is no longer available online.

So regarding API approaches, you basically have two choices: 1) use IMM32 APIs or 2) use TSF APIs.

Probably the option 1 would be easier as you can send Win32 messages such as WM_IME_CONTROL from your own processes instead of injecting your own code into the target application to call certain APIs. Check out Internet articles about how to use IMC_SETOPENSTATUS to enable/disable IME on the target process.

• https://qiita.com/kenichiro_ayaki/items/d55005df2787da725c6f

Under the hood, CUAS translates WM_IME_CONTROL with IMC_SETOPENSTATUS into the TSF counterpart GUID_COMPARTMENT_KEYBOARD_OPENCLOSE, which is checked by Mozc in the following location.

mozc/src/win32/tip/tip_status.cc

Lines 45 to 56 in bc546b2

	bool TipStatus::IsOpen(ITfThreadMgr *thread_mgr) {
	// Retrieve the compartment manager from the thread manager, which contains
	// the configuration of the owner thread.
	HResultOr<wil::unique_variant> var =
	TipCompartmentUtil::Get(thread_mgr, GUID_COMPARTMENT_KEYBOARD_OPENCLOSE);
	if (!var.has_value()) {
	return false;
	}
	// Open/Close compartment should be Int32 (I4).
	return var->vt == VT_I4 && var->lVal != FALSE;
	}

[niwazukihon]

Thanks for the knowledge. The ahk library you linked looks much more complete. I was thinking about making it built-in to Mozc. But I guess one can probably, for example, change the IME mode automatically on new window creation by an ahk script, although I'm not familiar enough with autohotkey to do this atm..

[niwazukihon]

I put together an ahk (v2) script to partially deal with this issue.

GetKeyboard() {
    hwnd := WinGetID("A")
    threadID := DllCall("GetWindowThreadProcessId", "UInt", hwnd, "UInt", 0)
    langCode := DllCall("GetKeyboardLayout", "UInt", threadID, "UInt")
    return langCode
}

SetKeyboard(LocaleID) {
    Static
    SPI_SETDEFAULTINPUTLANG := 0x005A
    SPIF_SENDWININICHANGE := 2
    Lan := DllCall("LoadKeyboardLayout", "Str", Format("{:08x}", LocaleID), "Int", 0)
    binaryLocaleID := Buffer(16)
    NumPut("UInt", LocaleID, binaryLocaleID)
    DllCall("SystemParametersInfo", "UInt", SPI_SETDEFAULTINPUTLANG, "UInt", 0, "Ptr", binaryLocaleID, "UInt", SPIF_SENDWININICHANGE)
    id := WinGetID("A")
    PostMessage 0x50, 0, Lan, , "ahk_id " id
}

RShift:: {
    cil := GetKeyboard()
    if cil = 68224017 { ; 日本語
        SetKeyboard(0x0409) ; English (US)
    }
    else {
        SetKeyboard(0x0411) ; 日本語
        Send "{vk16sc1F2}"
    }
}

When any keyboard other than the Japanese one is selected, pressing right shift will switch it to Japanese input and explicitly set it to hiragana input. When the Japanese keyboard is selected, it will change the keyboard to English. It can be customized into other behaviors. The script used code in [1, 2, 3].

It sets the Japanese keyboard to hiragana mode by sending the vk16sc1F2 key code. I'm actually not sure what this key code means but it seems to do the job. I think it can also be done by sending the key codes I do know such as the 全角/半角キー, 英数キー, etc [4], but corresponding keymap setup in moze is probably needed.

[yukawa]

It sets the Japanese keyboard to hiragana mode by sending the vk16sc1F2 key code. I'm actually not sure what this key code means but it seems to do the job.

I guess vk16 corresponds to VK_IME_ON, which has been supported by Mozc since version 2.26.4409.x with the following commit.

• 8e8e24e

VK_IME_ON and VK_IME_OFF are relatively new virtual key codes introduced in Windows 10 Version 1903.

• https://learn.microsoft.com/en-us/windows-hardware/design/component-guidelines/keyboard-japan-ime