[CI] add wheel tests in CI

[jameslamb]

Description

This project currently produces wheels but doesn't have any tests on those wheels in CI.

Some should be added.

Benefits of this work

• improves release confidence in wheels
  • (esp. for changes like Update RAPIDS Python packages to use rapids-build-backend build-planning#31 that alter how the wheels are built)
• reduces the effort required to fix wheel-specific issues
  • (via finding them here, at the source, instead of via bug reports from downstream projects or users)

Acceptance Criteria

• dask-cuda wheels are tested in CI, for at least one combination of ({python}, {cuda}, {linux}) from the set of things supported by the rest of RAPIDS (and ideally more)

Approach

Use the existing wheel-testing workflows from shared-workflows, e.g. https://github.com/rapidsai/shared-workflows/blob/branch-24.08/.github/workflows/wheels-test.yaml.

For a specific example testing another pure-Python package, see cuxfilter:

• https://github.com/rapidsai/cuxfilter/blob/7e63c04fc121b347040bb465845748983a79bc3b/.github/workflows/pr.yaml#L69
• https://github.com/rapidsai/cuxfilter/blob/7e63c04fc121b347040bb465845748983a79bc3b/.github/workflows/test.yaml#L25
• https://github.com/rapidsai/cuxfilter/blob/7e63c04fc121b347040bb465845748983a79bc3b/ci/test_wheel.sh

Notes

As of this writing, some tests rely on kvikio but kvikio does not yet have wheels (rapidsai/kvikio#369). If there are still not kvikio wheels as of when this is picked up, it could be installed in the test environment using conda.

doesn't have any tests on those wheels in CI.

There are other RAPIDS projects that depend on dask-cuda, install its wheels, and use the package installed via those wheels in CI. For example, cuml: https://github.com/rapidsai/cuml/blob/1560886d30cfe7d4472f334d487e5b9c136a270d/python/pyproject.toml#L91. That does provide some indirect testing.

This issue is just about adding such testing here in this repo, where the wheels are produced.

[vyasr]

I'd rather wait to test kvikio until we have kvikio wheels. Aside from that, agree with everything here.

[jakirkham]

Wheels were added in the since merged PR: rapidsai/kvikio#369

Also with the completion of the RAPIDS 24.8 release, PyPI now has kvikio-cu11 and kvikio-cu12

So think this is now actionable

[jakirkham]

Should add I saw this comment in the code that was added in PR: #1343

dask-cuda/dependencies.yaml

Lines 185 to 188 in 49ebabc

	- output_types: [requirements, pyproject]
	matrices:
	# kvikio should be added to the CUDA-version-specific matrices once there are wheels available
	# ref: https://github.com/rapidsai/kvikio/pull/369

Is this also part of this task or is there a separate issue tracking this item?

[jameslamb]

Is this also part of this task?

I think yes, because that section only updates the [test] extra for dask-cuda wheels, which wouldn't be used until we're running wheel tests. Some context for that comment: https://github.com/rapidsai/dask-cuda/pull/1343/files#r1624950506