Initialization, finalization & management for kokkos-comm

[dssgabriel]

As discussed during meetings, the project may think about providing dedicated initialization and finalization functions:

auto KokkosComm::initialize(int* argc, char*** argv) -> void;
auto KokkosComm::finalize(void) -> void;

The return value of these functions should also be thought about: std::expected? int? etc...

Some things we may want to think about regarding initialization:

• Use of MPI Sessions for handling our own communicators?
  • All kokkos-comm communications happen within that session?;
  • Multiple communicators in our (aka kokkos-comm's) MPI Session: one per node? one per NIC/node? one per GPU/node? This should go hand in hand with Kokkos' upcoming multi-GPU support (already available with CUDA);
  • Use of hints through MPI_Info: e.g. prevent MPI from checking pointer provenance when Kokkos tells us the view is in a device memory space;
• We handle all the tags? Prevents using "unlawful" values (> MPI_TAG_UB). This will depend on whether our target applications have a specific use of tags;
• We handle the communicator(s)?? This could enable some optimization: preventing the use of MPI_ANY_SOURCE means we can better leverage multiple NICs as we can "guarantee" on which one the communication will arrive. However, the same limitation as for automatic tag handling (with even tighter constraints?): target applications should not rely on specific handling of their communicators to facilitate kokkos-comm adoption...

Please share your ideas and thoughts on this topic!

[cedricchevalier19]

We should add more user stories to illustrate more precisely the needs.