Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge projection selects too many columns #283

Merged
merged 3 commits into from
Aug 30, 2023
Merged

Conversation

phofl
Copy link
Collaborator

@phofl phofl commented Aug 30, 2023

No description provided.

Copy link
Member

@rjzamora rjzamora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @phofl ! Let's use _convert_to_list (assuming that works)

Comment on lines 206 to 213
if isinstance(self.left_on, list):
left_on = self.left_on
else:
left_on = [self.left_on] if self.left_on is not None else []
if isinstance(self.right_on, list):
right_on = self.right_on
else:
right_on = [self.right_on] if self.right_on is not None else []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if isinstance(self.left_on, list):
left_on = self.left_on
else:
left_on = [self.left_on] if self.left_on is not None else []
if isinstance(self.right_on, list):
right_on = self.right_on
else:
right_on = [self.right_on] if self.right_on is not None else []
left_on = _convert_to_list(self.left_on)
right_on = _convert_to_list(self.right_on)

This seems more readable to me, but you will need to add the following import _convert_to_list from dask_expr._util at the top.

Comment on lines +208 to +209
if left_on is None:
left_on = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why you need this? Tests seem to pass fine without it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is necessary, then we probably want to clean up the dead logic below that assumes left_on and right_on could be None.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah sorry that got a bit too entangled, that's needed for #284 (which also cleans up the dead logic)

@rjzamora rjzamora merged commit ec30959 into dask:main Aug 30, 2023
4 checks passed
@phofl phofl deleted the merge_projection branch August 30, 2023 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants