This repository contains an experimental utility to monitor the visual output of cells from Jupyter notebooks.
To install, check out this repository and:
pip install -e .
Python 3.10 or later is supported (Python 3.12 or later on Windows).
If this is the first time using playwright, you will also need to run:
playwright install firefox
First, write one or more blocks of code you want to benchmark each in a cell. In addition, as early as possible in the notebook, make sure you set the border color on any ipywidget layout you want to record:
widget.layout.border = '1px solid rgb(143, 56, 3)'
The R and G values should be kept as (143, 56), and the B color should be unique for each widget and be a value between 0 and 255 (inclusive).
Then, to run the notebook and monitor the changes in widget output, run:
jupyter-output-monitor monitor --notebook mynotebook.ipynb
Where mynotebook.ipynb is the name of your notebook. By default, this will
open a window showing you what is happening, but you can also pass --headless
to run in headless mode.
If you want to test this on an existing Jupyter Lab instance, including
remote ones, you can use --url instead of --notebook:
jupyter-output-monitor monitor --url http://localhost:8987/lab/tree/notebook.ipynb?token=7bb9a...
Note that the URL should include the path to the notebook, and will likely require the token too.
You should make sure that all output cells in the notebook have been cleared before running the above command, and that the widget border color has been set as mention in the Quick start guide above.
If you make use of the jupyter-collaboration plugin on the Jupyter Lab server, you will be able to
more easily e.g. clear the output between runs and edit the notebook in
between runs of jupyter-output-monitor.
The general approach here is to use playwright to open a notebook, and run the cells one by one, and the script will then watch for any output cells that have a special border and take screenshots of them and any changes.
This border is identified by having a very specific set of 256 colors which is
(143, 56, *). If a cell output contains a frame with a border that has this
color, then we start recording screenshots for this cells, where the blue color
gives the index of the set of screenshots. For instance, if * is 3, the
script will save a set of screenshots that looks like:
output-003-2024-11-06T11:13:05.657891.png
output-003-2024-11-06T11:13:06.468521.png
output-003-2024-11-06T11:13:06.733932.png
output-003-2024-11-06T11:13:06.982627.png
output-003-2024-11-06T11:13:07.238872.png
output-003-2024-11-06T11:13:08.075732.png
Screenshots are only saved if the output has changed. By default, the bytes of the screenshot have to match the previous one exactly in order to not be saved, though we could make it have some amount of tolerance.
In addition to screenshots, an event log event_log.csv is written out in csv
format, and looks like:
time,event,index,screenshot
2024-11-06T23:47:10.156918,execute-input,0,output-2024-11-06T23:46:59.265044/input-000-2024-11-06T23:47:10.156918.png
2024-11-06T23:47:10.938298,output-changed,201,output-2024-11-06T23:46:59.265044/output-201-2024-11-06T23:47:10.938298.png
2024-11-06T23:47:11.456103,output-changed,201,output-2024-11-06T23:46:59.265044/output-201-2024-11-06T23:47:11.456103.png
2024-11-06T23:47:20.848153,execute-input,1,output-2024-11-06T23:46:59.265044/input-001-2024-11-06T23:47:20.848153.png
2024-11-06T23:47:22.643143,output-changed,201,output-2024-11-06T23:46:59.265044/output-201-2024-11-06T23:47:22.643143.png
2024-11-06T23:47:31.346982,execute-input,2,output-2024-11-06T23:46:59.265044/input-002-2024-11-06T23:47:31.346982.png
2024-11-06T23:47:41.713318,execute-input,3,output-2024-11-06T23:46:59.265044/input-003-2024-11-06T23:47:41.713318.png
2024-11-06T23:47:42.525010,output-changed,201,output-2024-11-06T23:46:59.265044/output-201-2024-11-06T23:47:42.525010.png
2024-11-06T23:47:42.973950,output-changed,201,output-2024-11-06T23:46:59.265044/output-201-2024-11-06T23:47:42.973950.png
This shows when each input was executed, as well as any associated screenshot.
The index column gives the index of the input cell in the notebook for the
execute-input events, though note that this may not always line up with
Jupyter's numbering, so to avoid any confusion, a matching screenshot of the
input cell is taken. For output-changed events, the index is that given by
the border color as described above.
We now look at how to set the frame color and trigger the recording. In order to
start recording a cell output, the top level of that cell output has to be an
ipywidget object. The .layout on that object can then be set to add a border
color. For example, if using glue-jupyter, one can do:
scatter = app.scatter2d()
scatter.layout.layout.border = '2px solid rgb(143, 56, 3)'
and if using jdaviz:
imviz.app.layout.border = '2px solid rgb(143, 56, 3)'
To stop recording output for a given cell, you can set the border attribute to
''.
To run in headless mode, include --headless
Since the monitoring script has no way of knowing when a cell has finished fully
executing, including any UI updates which might happen after the Python code has
finished running, we use a simpler approach - we execute each cell a fixed time
after the previous one. This is 10s by default but can be customized with
--wait-after-execute=20 for example. You should set this value so that the
cell that takes the longest to fully execute will be expected to take less than
this time.
You can generate a copy of the input notebook with output screenshots and profiling results inserted by using e.g.:
jupyter-output-monitor report --notebook mynotebook.ipynb --results-dir=output
Where --results-dir is the output directory generated with the monitor
command. BY default, this will write a report.ipynb notebook, but you can
overwrite the filename with --output-report-name.