CI: Add coverage for all address family combinations.

[dceara]

Originally discussed in PR #183 (#183 (comment)), a possible solution was suggested in ac3cba3 but that has downsides.

Instead we should probably have two layers of caching in CI:
a. a global cache to allow re-using runtimes across jobs
b. a per job cache to allow re-using the runtime within a job (for different address family + test combinations)

This would also allow us to reduce the number of CPUs when running tests.

CC: @igsilya