You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#2186 added AD to the interpreter. For reverse-mode, this is done by associating each value with a computation graph (really, DAG) that represents how the value was constructed. This is equivalent to the "Wengert tape" notion of AD. To compute the derivative, this graph is then traversed. Currently, the computation graph is represented as a tree, meaning that we redundantly recompute parts of it, which results in potentially exponential overhead. To fix this, we must exploit sharing inherent in the graph structure. The easiest solution may be to associate every node in the tree with a unique number, which during the traversal would let us recognise that we are re-visiting a part of the graph. Since this number must be globally unique, it must be maintained in the interpreter state.
I am also open to other ways to represent the graph, but this is the simplest one I could come up with.
The text was updated successfully, but these errors were encountered:
#2186 added AD to the interpreter. For reverse-mode, this is done by associating each value with a computation graph (really, DAG) that represents how the value was constructed. This is equivalent to the "Wengert tape" notion of AD. To compute the derivative, this graph is then traversed. Currently, the computation graph is represented as a tree, meaning that we redundantly recompute parts of it, which results in potentially exponential overhead. To fix this, we must exploit sharing inherent in the graph structure. The easiest solution may be to associate every node in the tree with a unique number, which during the traversal would let us recognise that we are re-visiting a part of the graph. Since this number must be globally unique, it must be maintained in the interpreter state.
I am also open to other ways to represent the graph, but this is the simplest one I could come up with.
The text was updated successfully, but these errors were encountered: