Frequent High CPU usage when starting Firefox

[foenyx]

Extension version : 1.14.1
Browser: firefox beta 115
OS: linux
Noteworthy: browser.privatebrowsing.autostart is set to true (with a "-private" flag on command line option), which may triggers weird bugs like "chrome.extension.getBackgroundPage doesn't work in Private Mode" ( https://bugzilla.mozilla.org/show_bug.cgi?id=1329304 )

Description: Recently, when I start the browser, often (maybe once everyday) a CPU core get near 100% usage for several minutes (it triggers the cpu fan).

So I opened "about:processes" and started profiling on an "extension" process which had a high CPU usage.

After starting the 5s long profiling in firefox, the stack samples count on that process showed roughly this proportions :

96% promise callback
        79% AsyncFunctionNext
                40% doUpdate               (moz-extension://.../10ten-ja-jpdict.js)
                39% updateSeries           (moz-extension://.../10ten-ja-jpdict.js)
         8% AsyncGeneratorNext
                 8% getEvents              (moz-extension://.../10ten-ja-jpdict.js)

Seeing it seemed related to update in 10ten extension, I looked in browser console log.

The browser console log, shows that the updates take nearly 5 minutes :

23:22:10,872 Requesting download for 'kanji' series with current version undefined 10ten-ja-jpdict.js:2348:39
23:22:15,600 Requesting download for 'radicals' series with current version undefined 10ten-ja-jpdict.js:2348:39
23:22:15,731 Successfully completed update. 10ten-ja-jpdict.js:3724:39
23:22:15,731 Requesting download for 'names' series with current version undefined 10ten-ja-jpdict.js:2348:39
23:25:33,233 Successfully completed update. 10ten-ja-jpdict.js:3724:39
23:25:33,234 Requesting download for 'words' series with current version undefined 10ten-ja-jpdict.js:2348:39
23:27:06,836 Successfully completed update. 10ten-ja-jpdict.js:3724:39

Is this behavior expected? if yes, could the dictionaries updates be set on a less frequent basis or on manual update?

(I'm hoping this report is somehow useful, thank you for reading it)

[birtles]

Wow, thank you for the very thorough report! That's very clear and helpful. I think I understand what is happening.

10ten ships with a snapshot of the dictionary data, but only for the word data and only for English glosses. After installing, it downloads the latest data for all the dictionaries (words, kanji, names, and radicals) and stores them in IndexedDB. From there it periodically checks for updates but these are typically very small. For browsers where IndexedDB is not available, it simply uses the snapshot.

I suspect what is happening in this case is because you have private browsing configured, the data in the IndexedDB database is deleted each time you close the browser so it re-fetches it each time you start Firefox.

You mentioned that this only happens recently and you are on 115 beta. I suspect this didn't happen before then because IndexedDB was disabled for private browsing (see https://bugzilla.mozilla.org/show_bug.cgi?id=1831058 and https://bugzilla.mozilla.org/show_bug.cgi?id=1639542#c15).

Now, in all browsers, writing to IndexedDB is quite slow (Chrome is even slower than Firefox), especially given the volume of records we're writing. We've optimized how we write to IndexedDB so I'm not sure there's much more we can do to make that faster and less resource intensive.

Instead, I've been toying with using SQLite since all browsers now support that. I've been wondering if it's worth the effort, however, since it complicates the migration to MV3 in Chrome. Given this issue, however, that's an extra reason to give it a try. At this rate, I suspect it will be at least September before I get to working on it, however. Perhaps we could try to add an option to turn off database updates altogether until then leaving you with just the words snapshot? Other than that, I'm not sure what we can do.

[foenyx]

Thank you for your quick reply!

Perhaps we could try to add an option to turn off database updates altogether until then leaving you with just the words snapshot?

That could be an option. And even for regular users, being able to choose to avoid an automatic update could be useful, as there are use-cases where you may want to save as much data and battery as possible, such as when traveling frequently.
I think that adding a way to start a manual update (like a "check for update" button) could be a nice addition in the case where the automatic update is disable.

Maybe another option could be to just disable the update when in private/incognito mode for now? If we know the data will be deleted at the end of the session, it might not be necessary to start an automatic update.

[birtles]

Likewise, thank you for your response! I think the incremental updates are so small as to be insignificant with regards to battery life of data usage. For example today's word update was 4.86kb and the names update was 1.04kb, and these updates only occur twice a week. So I think this option would only make sense for users with always-on private browsing.

I'm a little hesitant about forcing it to be true for all always-on private browsing users, however. When we don't run the initial update users are left with an old snapshot or the words data, with only English glosses, and with no name or kanji data so the extension is less useful.

There is already a button to force a manual update (see below) but perhaps if we exposed the manual update button as part of the pop-up in this mode it would be ok?

It's starting to sound like a lot of work, however, and I wonder if it might be better to just try and make the initial update more performant since that would benefit everyone.

[neo1973]

I have the same or a similar issue. Interestingly in about:processes it's not an extension process that's at 100% but the main Firefox one. In htop I see that it's always one of the IndexedDB threads that's going crazy.

I use Firefox in regular mode not private browsing. I'm also 99% certain that 10ten is the trigger, at the moment I have the problem always when the addon is enabled but never when it's disabled.

[birtles]

I think if you have some setting that clears user data on shutdown or startup it would produce similar symptoms. (I suppose a similar situation could occur if you were short on disk space and the browser automatically purged data?)

Did this start recently? e.g. with Firefox 115? If so it's likely related to the private browsing changes.

Unfortunately updating IndexedDB is just really slow. To fix this we either need to transition away from IndexedDB (which I think we will do in the coming months anyway) or somehow try to detect this case and throttle the update then (but not for a new install).