Interactive widgets not loading

[mshallow]

I recently set up the keypoint-moseq environment on a new mac with the M3 Pro chip, and since then I have had limited to no success on getting the interactive widgets in both the analysis and modeling notebooks. When the widgets did load, they were extremely buggy and didn't really allow for the full functionality, and after a google chrome update, they now no longer load at all. This could be an issue with a chrome update/ settings on the computer, but I was also curious if it could be something to do with the build of the environment since the new M3 Pro chip has GPU cores and the Mac environment is only setup for CPUs.
I also tried building an environment from scratch with the GPU requirements and got the following error.
Could the issues with the interactive widgets be an environment issue, or should I track down computer/ program specs that could be causing the problem.

[calebweinreb]

Hi,

With regard to the second issue, I don't think JAX supports apple silicon GPUs, so mac is still CPU-only for now.

Regarding the interactive widgets, hmm I'm not sure the best way to debug since I don't have access to an M3 mac. But it seems like this might be a general issue with interactive jupyter widgets? I'd be curious if a minimal example from e.g. holoviz works (https://holoviews.org/reference/containers/bokeh/DynamicMap.html)

[mshallow]

Following up on this issue:
I tried to look into the examples from what you suggested above and that example notebook works completely fine when run in my base environment and in the keypoint_moseq environment. I have been working with IT from our university since this is a university managed machine and they think that there might be some issue with the keypoint environment not specifying a version of bokeh or jupyter. In particular, when the widgets do load, the ones that have repeatedly caused issues are the two in the analysis.ipynb notebook. These will load (most of the time) but I cannot type into the cells if I have to type a letter that sometimes can be used as a keyboard shortcut in jupyter. For example, labeling of some of my conditions and syllables requires me to type the letters C or R. I can type until I reach these letters and then with R for example, it assumes that I am trying to use the keyboard shortcut to mark a cell as raw and does that. This does not happen when I type these into normal jupyter cells, just in the interactive widgets. I was just wondering if you had insight into the version of jupyter lab and bokeh that your environment uses, or if it just uses the base version on whichever computer it is installed on. Thanks!

[calebweinreb]

Hi Molly,

This is really helpful detail! We'll look into this and get back to you asap.

[versey-sherry]

Hi Molly, thank you! This is very useful information. I think the easiest solution is to open the csv files with Excel (index.csv and syll_info.csv) and edit the tables there directly. The saved information would be directly loaded to the relevant part. Some JS components are disabled on school managed machines so this is a simple workaround for this problem. Please let me know if this works!

[mshallow]

Hi Sherry, Thanks for getting back to me. I have tried the work around directly editing the csv, and it works, but it is incredibly time consuming since I have to open the videos separately and play them and then go into the csv and edit it to have the corresponding label. This is also how I was doing it previously before I got the environment working correctly on my previous computer. Do you have any suggestions as to how to fix the issue with JS components being disabled as I am trying to streamline and decrease the amount of time these steps take as much as possible. Best, Molly From: Sherry ***@***.***> Date: Wednesday, April 24, 2024 at 2:37 PM To: dattalab/keypoint-moseq ***@***.***> Cc: Molly Shallow ***@***.***>, Author ***@***.***> Subject: Re: [dattalab/keypoint-moseq] Interactive widgets not loading (Issue #142) Hi Molly, thank you! This is very useful information. I think the easiest solution is to open the csv files with Excel (index.csv and syll_info.csv) and edit the tables there directly. The saved information would be directly loaded to the relevant part. Some JS components are disabled on school managed machines so this is a simple workaround for this problem. Please let me know if this works! — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/dattalab/keypoint-moseq/issues/142*issuecomment-2075889891__;Iw!!C5qS4YX3!Bm_QDC8NWe1qcabmSTXOQR8RSuQ-wGpeQG4DAC2eAdmzcjXjjFIXyW7tAIfmXqKXI9PlhBp4X-yzYymeIDbYe6N5n3Q$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AJ6GK2RW7AF7TXMFC3JOZBTY7AQW5AVCNFSM6AAAAABFX7SMVWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZVHA4DSOBZGE__;!!C5qS4YX3!Bm_QDC8NWe1qcabmSTXOQR8RSuQ-wGpeQG4DAC2eAdmzcjXjjFIXyW7tAIfmXqKXI9PlhBp4X-yzYymeIDbYLrY5_Zk$>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[versey-sherry]

Hi Molly, in that case, unfortunately you'd need to go to your IT for this. If you use VLC or video players that would allow you to queue up multiple videos, that may make the labelling less timeconsuming.

[mshallow]

Hi Sherry,
Thanks this is helpful information. Have you heard any other instances of this happening on the Mac silicon processors yet?

[calebweinreb]

It doesn't happen on M1 or M2 as far as we know but never tried with M3

[staze]

Hey @calebweinreb I'm actually attempting to help @mshallow with this (I work for her campus IT). On a fresh environment (that you assisted with in slack) I can confirm this same behavior happens on both M3 and M1. If you attempt to edit the "Assign Groups" list that's generated after running the line
kpms.interactive_group_setting(project_dir, model_name)

Some typing works (like, I can delete the letter "l" (lowercase L) and retype "l". but if you hit "a" for example, it acts like a hotkey and generates a new cell above the "kpms.interactive_group_setting" line.

I do assume this is some jax/bokehJS/etc thing...

We're not configuring any settings in Chrome, Safari, etc regarding JS, etc.

How can we best troubleshoot this issue? Are you all able to confirm you do/don't see this behavior?

[staze]

Looks like it might be this? https://discourse.jupyter.org/t/keyboard-input-in-panel-bokeh-widget-results-in-keyboard-shortcut/24713/3

Also here: jupyterlab/jupyterlab#16066

[mshallow]

Following up here after @staze linked the potentially issue with bokeh. I've now tried this on an unmanaged Mac M3, and it does seem to be some sort of interaction between the newer versions of jupyter lab and bokeh that is causing this issue. I did however spend some time trying to downgrade jupyter lab to older versions as that was suggested as a potential fix, and this didn't seem to change anything. There seems to have been a pull request submitted to jupyter a little while ago about these problems (linked below) but it doesn't seem to have many things that we can change from the user experience level to get around this.
jupyterlab/jupyterlab#14115
It seems to maybe have something to do with the shift to trying to make jupyter accessible to use in a keyboard only configuration, but I'm not entirely sure. https://jupyterlab.readthedocs.io/en/latest/extension/extension_migration.html#change-of-notebook-focus-handling-impacting-command-mode-shortcut-selectors
I don't really know where to go from here as it seems that something on the new backend for both of these packages is not interacting correctly with each other so might require a larger top down change. It might be worth submitting this as an issue to either jupyter or bokeh/ holovis as it seems that a change may have to come from them to fully resolve this issue. Suggestions from other people encountering similar issues interact poorly with other keypoint_moseq dependencies leading to the package to not import correctly.
Do you have any insight into the versions of these packages that were being used on the M1 and M2 that the tests were successful on? I'm working on tracking down the specific version changes that may have caused this, as it seems that it might not specifically be an issue with the chips themselves but the timing of the install/ environment creation.

[staze]

@mshallow Thanks! Not sure if you tried, but I did notice it seems like you have to rebuild your env after upgrading/downgrading jupyter. Just not sure how far you'd have to downgrade it.

I did inquire with jupyter devs to see when the fix is targeted for jupyter, and what version.

@calebweinreb given the links, and I admit, I'm no jupyter/python expert, is there anything @mshallow can set in the notebook that might resolve the issue? The last post on this thread: https://discourse.jupyter.org/t/keyboard-input-in-panel-bokeh-widget-results-in-keyboard-shortcut/24713/3 indicates there MIGHT be some way to fix in the widget?