You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Complementing the example directory, a set of unit tests and a framework for testing needs to be created for all Kokkos Tools libraries. This includes in-house tooling libraries, e.g., kernel-logger, vendor connectors, e.g., roctx-connector, and utilities, e.g., Kokkos-sampler, and sophisticated tpls, apex Kokkos Tool library.
In the near-term and of priority: the focus will be on new tools integrated being into the Kokkos Tools repo, in particular the sampler utility and fixes in the nvtx-connnector, which is expected to have heavy use. (The rocprof-connector and vtune-connector should also have similar tests).
Longer-term: more tests, perhaps in part community-driven, will be incorporated in to the set of tests.
This may eventually involve a testing framework like Google Test and the testing would then happen within the new Kokkos Tools make build system and nightly CI builds of Kokkos Tools
The text was updated successfully, but these errors were encountered:
I propose a process for developing these tests that is directly motivated by Kokkos applications using Kokkos Tools.
For now, let's define a unit as callback of a particular tool. Each unit test should verify (is the answer correct given the function's implementation), validate (is the answer correct based on requirements of function defined), and assess performance. These three aspects give the Kokkos Tools users an assessment of quality of the implementation.
The process for establishing a set of unit tests for the Kokkos Tools would be something like:
1st phase: Obtain simple example codes from applications and benchmarks, and ensure they work. An example of a test of the sampler utility's parallel_for tools callback applied for the simple kernel timer is shown in the file. outputForSampler with Atomic Benchmark
2nd phase: By stripping out functions and code from applications: Create tests of output of tool, with one test per unit, via assertions in code.
3rd phase: Use testing framework like ctest (and could be subsequently using Google Test framework) and integrate the tests under the framework into the Kokkos Tools cmake build system.
The 1st and 2nd phase would be stored in a separate repo and be not part of the Kokkos Tools repo. The code developed in the 3rd phase would be the official tests of the repo, which would be community-built and agreed upon via a community of Kokkos users.
This process could be followed or follow that of tests in Kokkos-core.
vlkale
added
the
testing
Testing for Kokkos Tools functionality (unit tests and integration tests)
label
May 5, 2023
vlkale
changed the title
Create unit tests for all Kokkos Tools connectors and utilities
Create unit tests for all Kokkos Tools libraries and utilities
Jun 4, 2024
Complementing the example directory, a set of unit tests and a framework for testing needs to be created for all Kokkos Tools libraries. This includes in-house tooling libraries, e.g., kernel-logger, vendor connectors, e.g., roctx-connector, and utilities, e.g., Kokkos-sampler, and sophisticated tpls, apex Kokkos Tool library.
In the near-term and of priority: the focus will be on new tools integrated being into the Kokkos Tools repo, in particular the sampler utility and fixes in the nvtx-connnector, which is expected to have heavy use. (The rocprof-connector and vtune-connector should also have similar tests).
Longer-term: more tests, perhaps in part community-driven, will be incorporated in to the set of tests.
This may eventually involve a testing framework like Google Test and the testing would then happen within the new Kokkos Tools make build system and nightly CI builds of Kokkos Tools
The text was updated successfully, but these errors were encountered: