You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A common use case we have is a fire a forget mode where one launches only one container that executes the real workload (allowing for example to do hyper parameter tuning with many of those containers in parallel)
In this case it is useful to have an additional layer on top of service, applicationspec to only have one entry point. Also in interactive/jupyter mode it is sometimes useful to directly use a function and use cloudpickle to ship this function.
A common use case we have is a fire a forget mode where one launches only one container that executes the real workload (allowing for example to do hyper parameter tuning with many of those containers in parallel)
In this case it is useful to have an additional layer on top of service, applicationspec to only have one entry point. Also in interactive/jupyter mode it is sometimes useful to directly use a function and use cloudpickle to ship this function.
We have added this code to one of our projects but I think the best place would be to host it somewhere in this repo.
https://github.com/criteo/cluster-pack/blob/master/cluster_pack/skein/skein_launcher.py
There is no real risk on maintenance I think. In the worst case it is unused. Not much complexity.
WDYT ? I can do that PR and add some tests if you think it makes sens to add it here.
The text was updated successfully, but these errors were encountered: