Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to split EKO computation for a dataset #201

Open
felixhekhorn opened this issue Oct 28, 2024 · 1 comment
Open

Allow to split EKO computation for a dataset #201

felixhekhorn opened this issue Oct 28, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@felixhekhorn
Copy link
Contributor

While the computation of a single 4D EKO is reasonably fast, some datasets have many scales which makes them slow again. This is becoming a bottle neck on clusters, where a time limit of 24h seems to have risen.

Starting from NNPDF/eko#296 EKO provides some support to split the computation into several pieces. Note NNPDF/eko#399

@scarlehoff is wishing for

Pineko-wise it would be enough being able to continue an eko where we left it
So I get an operator card and an eko.tar file and instead of refusing to run or rerunning it looks inside and continues

this should be doable

@alecandido
Copy link
Member

It is, but you can even be more ambitious (if needed) since the recipes mechanism was made on purpose for that. So, the recipes generation is instantaneous, after that, you can decide which ones (or simply how many) you want to compute right now, or somewhere. Then, you will only consume the recipes you have computed (though marking them as done is a more or less manual operation).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants