Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autoeviction for Task Inputs #136

Open
WardLT opened this issue Apr 14, 2024 · 1 comment
Open

Autoeviction for Task Inputs #136

WardLT opened this issue Apr 14, 2024 · 1 comment

Comments

@WardLT
Copy link
Collaborator

WardLT commented Apr 14, 2024

We don't auto-evict data from proxystore for inputs from completed tasks, which can lead to memory/disk use issues.

We could...

  • Make autoeviction on read for the inputs from the compute worker, which will break task restarting
  • Make autoeviction after the tasks complete on the worker
  • ...
@gpauloski
Copy link
Collaborator

Some other ideas:

  • Add a callback to Result.__del__ which evicts all the associated proxies. Not my favourite idea because one could pickle a Result and send it to another process causing the value to be GC collected on the original process.
  • Add a callback to the Future of the task result.
    • Would address the task restarting problem.
    • The decision to evict the inputs could be tied to the keep_inputs flag. This would mean that inputs eviction is done on the task server, and value eviction would be done when access by the Thinker.
    • We would need to make it clear in some way that if keep_inputs=True then managing input proxies is up to the user (and potentially add a convenience callback like Result.cleanup() or something to clean up those associated proxies).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants