Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add minimal case cfgs #608

Merged
merged 1 commit into from
Jul 19, 2024
Merged

Add minimal case cfgs #608

merged 1 commit into from
Jul 19, 2024

Conversation

forsyth2
Copy link
Collaborator

Add minimal case cfgs. Addresses 'Automated testing of the "combinatorial explosion" of parameter options. Often we need to know how zppy behaves when different combination of parameters are applied. Setting up automatic cfg generation and testing of these cases would be immensely helpful.' in #572.

This pull request aims to add some important "minimal cases" -- i.e., cfgs with as few parameters as possible to test specific cases.

@forsyth2 forsyth2 self-assigned this Jun 19, 2024
@forsyth2
Copy link
Collaborator Author

I've broken up the complete_run test into components or "minimal cases" as I call them here.

I've also moved the input over from v2 to v3, in some sense resolving #552. To fully resolve #552, we'd need to have an actual test call (as #604 would do with the complete_run test) rather than just producing output (as these cfgs do).

My plan going forward is to use these minimal case cfgs to test future pull requests -- only running the relevant cfgs. E.g., no sense running ILAMB if I'm changing global_time_series code.

As for testing the main branch on some recurring basis, we need to either A) automate running every single one of these minimal cases and somehow checking their results, or B) convert the complete_run tests from v2 to v3 so we have some method of testing main. (A) is more comprehensive, (B) is more practical.

Now, two caveats:

  1. I haven't actually run these minimal cases because there are quite a few of them. My plan is to modify them as needed when actually using them to debug/test future pull requests.
  2. The v3 input is currently hard coded to a simulation on Chrysalis. We'll need to copy over the input data to Perlmutter and Compy (or be content running these minimal cases on Chrysalis only).

@forsyth2 forsyth2 marked this pull request as ready for review July 19, 2024 00:38
@forsyth2 forsyth2 merged commit 4c4c58b into main Jul 19, 2024
4 checks passed
@forsyth2 forsyth2 deleted the issue-572-cfg-options branch July 19, 2024 17:59
This was referenced Jul 31, 2024
@forsyth2 forsyth2 mentioned this pull request Oct 16, 2024
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant