Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Parquet encoding statistics to column chunk metadata #15452

Merged
merged 20 commits into from
Apr 26, 2024

Conversation

etseidl
Copy link
Contributor

@etseidl etseidl commented Apr 3, 2024

Description

Closes #15313

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@etseidl etseidl requested a review from a team as a code owner April 3, 2024 15:00
@etseidl etseidl requested review from robertmaynard and ttnghia April 3, 2024 15:00
Copy link

copy-pr-bot bot commented Apr 3, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label Apr 3, 2024
@GregoryKimball GregoryKimball added the cuIO cuIO issue label Apr 3, 2024
@@ -16,6 +16,9 @@

#include "compact_protocol_reader.hpp"

#include "parquet.hpp"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the empty line above 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, but clang-format insists on that line being there 😅

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's weird. Definitely clang-format bug 😓

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Maybe a special group for header with the same root as the compilation unit 🤷

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harrism Would you mind looking into this please?

@ttnghia ttnghia added feature request New feature or request 3 - Ready for Review Ready for review by team non-breaking Non-breaking change labels Apr 8, 2024
@ttnghia
Copy link
Contributor

ttnghia commented Apr 8, 2024

/ok to test

@ttnghia
Copy link
Contributor

ttnghia commented Apr 9, 2024

/ok to test

@mhaseeb123
Copy link
Member

Thanks for the effort Ed, looks good to me!

@mhaseeb123
Copy link
Member

/ok to test

@ttnghia
Copy link
Contributor

ttnghia commented Apr 26, 2024

/ok to test

@ttnghia
Copy link
Contributor

ttnghia commented Apr 26, 2024

/merge

@ttnghia
Copy link
Contributor

ttnghia commented Apr 26, 2024

/ok to test

@rapids-bot rapids-bot bot merged commit d91a4ad into rapidsai:branch-24.06 Apr 26, 2024
69 checks passed
@etseidl etseidl deleted the enc_stats branch April 26, 2024 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Add encoding_stats footer metadata to the parquet writer
4 participants