Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve zppy developer experience #572

Open
7 of 11 tasks
forsyth2 opened this issue Apr 11, 2024 · 0 comments
Open
7 of 11 tasks

Improve zppy developer experience #572

forsyth2 opened this issue Apr 11, 2024 · 0 comments

Comments

@forsyth2
Copy link
Collaborator

forsyth2 commented Apr 11, 2024

Request criteria

  • I searched the zppy GitHub Discussions to find a similar question and didn't find it.
  • I searched the zppy documentation.
  • This issue does not match the other templates (i.e., it is not a bug report, documentation request, feature request, or a question.)

Issue description

zppy has many complex inter-connected pieces. It can therefore be a challenging package to develop and debug (and debugging zppy could well mean debugging a package it calls).

There are some things we could do to improve the developer experience:

  • Have zppy print out the environment_commands for each task on the submitting... line. This information is in the generated bash files, but this change would make this information hard to miss. This is important because a very common problem in zppy development is not having the environments for each task configured properly. (Fixes and test/doc improvements #576)
  • Have zppy tests print out exact environments being used as well. Notably the setting for generate_cfgs(unified_testing=False) should be printed. (Fixes and test/doc improvements #576)
  • Create a debugging guide for common challenges in debugging zppy (zppy debugging guide for developers #573)
  • Replacing the expected images tests with a metric checker test (Replace image diff tests with metrics tests #536). The latter is less prone to noise.
  • Automated testing of the "combinatorial explosion" of parameter options. Often we need to know how zppy behaves when different combination of parameters are applied. Setting up automatic cfg generation and testing of these cases would be immensely helpful. (Add minimal case cfgs #608, Testing update #604)
  • More thorough testing in general, so more bugs are caught automatically.
  • Nightly testing using cron or something like it. This will be necessary if the above points on more thorough testing are implemented, since zppy testing is already a lengthy process.
  • Epic: improve testing framework #553
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant