Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: handle zero_tangent from cyclic data structures v2 Via premapping #655

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

oxinabox
Copy link
Member

This is an alternative to #654
I think it is more promising and should minimize the type instability outside of cyclic locations

Basically here is the idea:

  1. Identify all objects that have multiple references to them
  2. Go through and construct tangents for everything;
  3. save the tangents for things with >1 reference into an IdDict keyed by primal
  4. when constructing tangent fields for things that have >1 reference to them, initially treat them as per undef elements, so set the tangent value as also undef or to a ZeroTangent
  5. go back through the tangent object and set the values for things with multiple references

This will also handle aliases outside of loops correctly.

@oxinabox oxinabox mentioned this pull request Jan 25, 2024
4 tasks
@oxinabox oxinabox force-pushed the ox/mutation_cyclic_premap branch from f4621e7 to ddd1647 Compare January 25, 2024 09:55
@oxinabox
Copy link
Member Author

no this plan won't work right as as.
You can't always mute in the references after the fact.
Because a immutable type might have fields that are references.

I think we are going to need to topo sort this or something.
I am going to draw some pictures

Base automatically changed from ox/mutabletangent to main January 25, 2024 11:05
@oxinabox
Copy link
Member Author

ok, so cycles are only possible with mutation.
So the kinda extreme version of what we can do is

  1. do cycle detection + reference counting
  2. break the graph at (at least one) mutable struct field/array element; creating a Tangent putting in a ZeroTangent for that value, but otherwise constructing as per step 3 below; and adding to a shared primal indexed Dict of of lists of pending work a function that will mutate the tangent to attach a tangent computed later.
  3. Construct tangent recursively, For any field with multiple references and that is not isbits (these two checks are an optimization and could be skipped) check in a primal keyed dict of already constructed tangents and if it is found use that, otherwise add it to that.
  4. run all the pending work from step 2 to relink up the graph.

@oxinabox oxinabox force-pushed the ox/mutation_cyclic_premap branch from ddd1647 to 9df22e9 Compare January 25, 2024 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant