Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[draft] Add Weight Visualization #1492

Draft
wants to merge 11 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/VisualTensorView.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Visual Tensor View

## Table of Contents
- [About](#about)
- [How to use](#how-to-use)

## About

Visual Tensor View is a tool that alows to visualize tensor data as a 2D-heatmap.

<img src="./images/VisualTensorViewOverview.jpg"/>

## How to use

Currently Visual Tensor View is integrated into property sidebar of Circle Graph View.
The visualizer is shown for all node properties that are tensors of 2 or more dimentions:

<img src="./images/VisualTensorViewExample.jpg"/>

In order to visualize tensor data:
- we need a *.circle model that contains at least one node with tensor properties with 2 or more dimentions like 'weights', 'filter', etc.
- open a folder that contains such model.
- click on the model file in Explorer.
- click on such node (Conv2D, for example).
<img src="./images/VisualTensorViewNode.jpg"/>

- in 'NODE PROPERTIES' sidebar find 'INPUTS' containing the tensor property ('filter' in case of Conv2D).
- expand it with '+' button.
<img src="./images/VisualTensorViewExpand.jpg"/>

- the visualizer should appear.
<img src="./images/VisualTensorViewExpanded.jpg"/>

- you can modify the heatmap scale or swap 'x' and 'y' using respective UI elements.
<img src="./images/VisualTensorViewScale.jpg"/>

- the heatmap colors correspond to respective tensor values, the higher wavelength, the higher value:
* 'blue' - the lowest value in the current heatmap
* 'green' - the value is about (min + max) / 2
* 'red' - the highest value in the current heatmap

- hover the mouse over the heatmap pixel to see the actual value in the tooltip.
<img src="./images/VisualTensorViewHeatmap.jpg"/>

- if the tensor is 2D all tensor data is shown as a 2D-heatmap at once.

- if the tensor is 3D, 4D...:
* exactly two axes should be selected with the checkboxes, the selected axes correspond to 'x' and 'y' of the heatmap.
<img src="./images/VisualTensorViewAxes.jpg"/>

* for all other axes fixed values should be set.
<img src="./images/VisualTensorViewValues.jpg"/>

* in this example the heatmap point (x, y) corresponds to tensor[5][y][2][x].

Binary file added docs/images/VisualTensorViewAxes.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/VisualTensorViewExample.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/VisualTensorViewExpand.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/VisualTensorViewExpanded.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/VisualTensorViewHeatmap.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/VisualTensorViewNode.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/VisualTensorViewOverview.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/VisualTensorViewScale.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/VisualTensorViewValues.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions media/CircleGraph/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@
<script type="text/javascript" nonce="%nonce%">
// TODO find better way to set initial view mode
var __viewMode = '%viewMode%';
// TODO find better way to enable VisualTensorView
var __enableVisualTensorView = '%enableVisualTensorView%';
</script>
<style>
html { touch-action: none; overflow: hidden; width: 100%; height: 100%; -ms-text-size-adjust: 100%; -webkit-text-size-adjust: 100%; text-rendering: optimizeLegibility; -webkit-text-rendering: optimizeLegibility; -moz-text-rendering: optimizeLegibility; -ms-text-rendering: optimizeLegibility; -o-text-rendering: optimizeLegibility; -webkit-font-smoothing: antialiased; -moz-font-smoothing: antialiased; -ms-font-smoothing: antialiased; -o-font-smoothing: antialiased; }
Expand Down
2 changes: 2 additions & 0 deletions media/CircleGraph/view-sidebar.css
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ https://github.com/lutzroeder/netron/blob/ae449ff55642636e6a1eef092eda34ffcba1c6
.sidebar-view-item-value-line-content { white-space: pre; word-wrap: normal; overflow: auto; display: block; }
.sidebar-view-item-value-expander { font-family: 'SFMono-Regular', Consolas, 'Liberation Mono', Menlo, Courier, monospace; float: right; color: #aaa; cursor: pointer; user-select: none; -webkit-user-select: none; -moz-user-select: none; padding: 4px 6px 4px 6px; }
.sidebar-view-item-value-expander:hover { color: #000; }
.sidebar-view-item-value-number { font-family: 'SFMono-Regular', Consolas, 'Liberation Mono', Menlo, Courier, monospace; text-align: center; float: left; font-size: 11px !important; width: 14px !important; height: 11px !important; }
.sidebar-view-item-value-number::-webkit-outer-spin-button, .sidebar-view-item-value-number::-webkit-inner-spin-button { -webkit-appearance: none; margin: 0; }
.sidebar-view-item-select {
font-family: inherit; font-size: 12px;
background-color: #fcfcfc; border: #fcfcfc; color: #333;
Expand Down
Loading