setting up a parameter space search with athena

[teuben]

Is your feature request related to a problem? Please describe.
I'm wondering about a piece of infrastructure (maybe just one script) that probably a lot of us have written, and that is how to orchestrate a series of athena runs to run over parameter space. in addition, analysis scripts can then loop over these parameters and extract information and meta-data.

Describe the solution you'd like
Perhaps a tool that essential would do something like this:
run_athena template_file run_directory -- a=1 b=2 c=3
this would create a set of run directories, where the three parameters that need to be present in the template_file (an athinput formatted file), replace them, and run athena.

Additional bonus points if this can work with slurm.

It would also help if in a verbose way a manual example would be described in the documentation. What I usually do it configure and compile athena, but copy bin/athena (since the name is not uniq) to ~/bin/athena_SOMENAME which I then use in my run_athena.

Describe alternatives you've considered
Well, in unix there are probably many different solutions to this, I'm looking for an athena specific one, as given in my example above. So one simple script, as opposed to a hack using grep/awk/sed/....

Additional context
I have some tools in this direction in NEMO, but if there is interest from the athena community, I'd like to expand my proposal and implement it. But I'd like more input from the different workflows that people have established.

[felker]

I once had my own tool setup using https://www.pytables.org/ and a binary and directory naming hierarchy like you suggested. But it was burdensome to maintain.

I doubt that anyone in the Athena++ community has anything more sophisticated than their own Bash scripts for doing this. I would recommend using third-party general purpose Python tools that are geared towards HPC, like https://github.com/argonne-lcf/balsam, https://docs.signac.io/en/latest/. I have contributed to an older version of Balsam; if there is general interest, we could add an Athena++ Balsam integration script to the repository. And/or a well-written and minimal Bash script for simple parameter sweeps could be put under version control.

[teuben]

In discussions with a colleagues, I was given this link, of somebody who used github to manage their experiments (in MESA) : https://github.com/adamjermyn/remote_experiments

[alwinm]

If I recall correctly, I believe the athena binary takes command line arguments, so a hack using grep/awk/sed to change input files is not necessary.

The -d flag sets an output directory and other parameters can be set in command line. See https://github.com/PrincetonUniversity/athena/wiki/Running-the-Code

Regression tests provide great examples for running Athena using python scripts and using command line overrides, and python can certainly help map from 1D slurm array index -> multidimensional indices -> each slurm task runs a python script calling athena

https://github.com/PrincetonUniversity/athena/blob/master/tst/regression/scripts/tests/

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.