Unable to use Widget() inside Shiny

[cpsievert]

I just took a quick look into rendering quak.Widget() in Shiny, and I get the following JS error:

import polars as pl
import shiny.express
from shinywidgets import render_widget
import quak

df = pl.read_parquet("https://github.com/uwdata/mosaic/raw/main/data/athletes.parquet")

@render_widget
def _():
    return quak.Widget(df)

What's interesting here is that ./lib/widget.ts's import of apache-arrow appears to work for stuff under ipc/*, but not for this line of io/stream.ts. Any idea why that might be?

[manzt]

Sorry not an avid shiny user... can you please provide a minimum reproducible example for how to setup an environment and run this code?

EDIT: I can reproduce with the following:

python3 -m venv .venv
source .venv/bin/activate
pip install shiny shinywidgets quak polars
shiny run app.py

[cpsievert]

Oh interesting, I also get a similar error when running a hello mosaic-widget example:

import pandas as pd
from mosaic_widget import MosaicWidget
import shiny.express
from shinywidgets import render_widget
import yaml

weather = pd.read_csv(
    "https://uwdata.github.io/mosaic-datasets/data/seattle-weather.csv",
    parse_dates=["date"],
)

# Load weather spec, remove data key to ensure load from Pandas
with open("weather.yaml") as f:
    spec = yaml.safe_load(f)
    _ = spec.pop("data")

@render_widget
def _():
    return MosaicWidget(spec, data={"weather": weather})

[manzt]

This sniffs of some runtime global inspection by arrow (or one of this dependencies). I'm guessing Shiny includes a dep that sets a global (e.g., define from requirejs) which changes the behavior between the front end environments.

[cpsievert]

Ohh, yea, that seems likely. shiny/shinywidgets puts define and require on the window so that it can do a run-time request path configuration (e.g., window.require.config({"paths": {"anywidget": "lib/anywidget-0.0/index"}}))

[manzt]

Just speculation. Don't know if that's it alone. I know that require and define are also in class Jupyter notebooks, but not JupyterLab.

[manzt]

Ah, wait.... I'm being silly. I meant to bring this up later. shinywidgets does not support sending binary data to/from the front end... so arrow.tableFromIPC is just getting undefined from shiny.

This is essential for applications that use apache arrow. I tried to look into how the custom Shiny comm impl could satisfy this requirement... but I didn't see any methods on Shiny global for dispatching that kind of data.

Many of the data-intensive anywidgets (Mosaic, lonboard, jupyter-scatter) make use of Jupyter's Widgets support for sending binary data. Otherwise all data needs to be JSON serializable....which has consequences for performance.

EDIT: shinywidgets does support binary data (although through base64 encoding), but the decoded binary blobs are ArrayBuffers not DataViews, which is the API expected by widgets:

• Ensure binary data is a DataView posit-dev/py-shinywidgets#152

[manzt]

I'm going to close. Feel free to re-open if there is an issue when/if shinywidgets adds support for binary data! once posit-dev/py-shinywidgets#152 is merged!

[cpsievert]

Ahh, nice catch, thank you!

[kylebarron]

Hmm I've gotten lonboard to work in shiny widgets I think

[manzt]

@kylebarron You are right. I dug in and shinywidgets comm impl base64 encodes the binary stuff. However, I’m guessing your handling of buffers is more flexible than arrow.

something like new Uint8Array(x) would cast the data.

see posit-dev/py-shinywidgets#152