Draft: unit testing

[brryan]

It'd be great to have a harness for unit testing. One way to do this would be to have a separate unit tests executable that can be build optionally, and calls a bunch of individual test functions. These test functions could call some utility libraries to build up a basic driver and mesh, so we can call any of our existing functions without much trouble, and then just compare results of individual functions with known results (e.g. x1v in different geometries at different points in space.

[pdmullen]

What if we added a (few) simple pgen(s) that checked a lot of these things (e.g., x1v), and then just immediately returned thereafter (i.e., no actual evolution)?

In this way, we could use the existing testing infrastructure, just throwing a unit suite that interfaces with said pgen(s). I might prefer that kinda approach, because then we would ensure that all the package initialize functions are called and the mesh is constructed, avoiding a common issue I hit with parthenon's unit testing infrastructure, where not everything I needed was constructed.

[brryan]

What if we added a (few) simple pgen(s) that checked a lot of these things (e.g., x1v), and then just immediately returned thereafter (i.e., no actual evolution)?

In this way, we could use the existing testing infrastructure, just throwing a unit suite that interfaces with said pgen(s). I might prefer that kinda approach, because then we would ensure that all the package initialize functions are called and the mesh is constructed, avoiding a common issue I hit with parthenon's unit testing infrastructure, where not everything I needed was constructed.

My thought was that this approach would do something quite similar to that, except with our own quick custom no-timestepping driver (like the calculate_pi example or something, which might give us some more flexibility but would definitely isolate our unit tests from the pgen folder which on the whole I would prefer.

I absolutely agree we want to have all the package initialize functions called for all the tests so we don't waste time trying to set up a boutique environment for every test -- that's the basic goal of what I'm trying to do here.

[pdmullen]

What if we added a (few) simple pgen(s) that checked a lot of these things (e.g., x1v), and then just immediately returned thereafter (i.e., no actual evolution)?
In this way, we could use the existing testing infrastructure, just throwing a unit suite that interfaces with said pgen(s). I might prefer that kinda approach, because then we would ensure that all the package initialize functions are called and the mesh is constructed, avoiding a common issue I hit with parthenon's unit testing infrastructure, where not everything I needed was constructed.

My thought was that this approach would do something quite similar to that, except with our own quick custom no-timestepping driver (like the calculate_pi example or something, which might give us some more flexibility but would definitely isolate our unit tests from the pgen folder which on the whole I would prefer.

I absolutely agree we want to have all the package initialize functions called for all the tests so we don't waste time trying to set up a boutique environment for every test -- that's the basic goal of what I'm trying to do here.

Sounds great :) Sounds like we are on the same page. Wonder if we can keep everything as you have it here, but make it callable by the run_tests.py environment, e.g., python3 run_tests.py unit.