Reduce P2P transfer task overhead #8912
Conversation
Unit Test ResultsSee test report for an extended history of previous test failures. This is useful for diagnosing flaky tests. 25 files ±0 25 suites ±0 10h 16m 34s ⏱️ + 2m 50s For more details on these failures, see this check. Results for commit 9dba740. ± Comparison against base commit 2953090. ♻️ This comment has been updated with latest results. |
distributed/shuffle/_core.py
Outdated
| def __getstate__(self): | ||
| state = super().__getstate__() | ||
| state["spec"] = self.spec | ||
| return state | ||
|
|
||
| def __setstate__(self, state): | ||
| super().__setstate__(state) | ||
| self.spec = state["spec"] |
There was a problem hiding this comment.
I'm OK with this solution. Just to be clear, though, when we talked the other day I was hoping to be able to implement a more robust version of this in the base class that doens't require the child classes to overwrite. If that's not easily possible, that's fine.
There was a problem hiding this comment.
Noted, but I'd like to isolate changes in the base class from this PR. I'm already juggling two repos here and would like to avoid adding a third into the mix.
This reverts commit c0d6c63.
Closes #xxxx
pre-commit run --all-files