Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

decompress pickled messages #8216

Merged
merged 3 commits into from
Sep 29, 2023
Merged

Conversation

madsbk
Copy link
Contributor

@madsbk madsbk commented Sep 27, 2023

Closes rapidsai/dask-cuda#1246

Make sure to decompress pickled messages when communicating. Currently, we don't decompress out-of-band buffers when using pickle!

  • Tests added / passed
  • Passes pre-commit run --all-files

@madsbk madsbk added the bug Something is broken label Sep 27, 2023
@pentschev
Copy link
Member

rerun tests

@madsbk madsbk marked this pull request as ready for review September 27, 2023 12:08
@madsbk madsbk requested a review from fjetter as a code owner September 27, 2023 12:08
@github-actions
Copy link
Contributor

github-actions bot commented Sep 27, 2023

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       21 files  ±  0         21 suites  ±0   10h 18m 58s ⏱️ - 10m 31s
  3 815 tests ±  0    3 706 ✔️ +  3     107 💤 ±0  2  - 3 
36 875 runs  +12  35 072 ✔️ +20  1 799 💤  - 4  4  - 4 

For more details on these failures, see this check.

Results for commit a051bdd. ± Comparison against base commit a74f7cf.

♻️ This comment has been updated with latest results.

Copy link
Member

@pentschev pentschev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @madsbk !

wence- added a commit to wence-/dask-cuda that referenced this pull request Sep 27, 2023
In versions of distributed after dask/distributed#8067 but before
dask/distributed#8216, we must patch protocol.loads to include the
same decompression fix.
rapids-bot bot pushed a commit to rapidsai/dask-cuda that referenced this pull request Sep 27, 2023
In versions of distributed after dask/distributed#8067 but before dask/distributed#8216, we must patch protocol.loads to include the same decompression fix.

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Peter Andreas Entschev (https://github.com/pentschev)

URL: #1247
@quasiben quasiben mentioned this pull request Sep 29, 2023
4 tasks
@crusaderky crusaderky merged commit b0e08c0 into dask:main Sep 29, 2023
21 of 26 checks passed
@jrbourbeau
Copy link
Member

Thanks @madsbk @crusaderky

pentschev added a commit to pentschev/dask-cuda that referenced this pull request Oct 4, 2023
This change was introduced in
rapidsai#1247 to include the fix from
dask/distributed#8216 for the Dask-CUDA 23.10
which pinned to Distributed 2023.9.2. Starting from Distributed
2023.9.3, the fix is merged in Distributed and this isn't required
anymore.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is broken
Projects
None yet
Development

Successfully merging this pull request may close these issues.

UCX serialization errors after dumps_task removal from Distributed
4 participants