Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate asynchronous NN model update #286

Open
jardinetsouffleton opened this issue Nov 8, 2022 · 0 comments
Open

Investigate asynchronous NN model update #286

jardinetsouffleton opened this issue Nov 8, 2022 · 0 comments

Comments

@jardinetsouffleton
Copy link
Member

The goal of this issue would be to investigate the possibility of de-coupling the model used to make desicions (value assignment) from the model that is being updated. As of now, the backpropagation occurs at every update_freq time steps (after update_freq decisions). When the model is being updated by the backpropagation, the trajectory generation must stop. However, as the trajectory generation is done one sample at a time, there is a lot of space remaining on the GPU to compute the backprop asynchronously. Doing this would allow us to generate more trajectories and possibly speed up the resolution of problems

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant