Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve AggFormula performance by re-using GroupByChunkedOperator rather than re-creating new operators. #6363

Open
lbooker42 opened this issue Nov 11, 2024 · 0 comments
Assignees
Labels
core Core development tasks feature request New feature or request query engine
Milestone

Comments

@lbooker42
Copy link
Contributor

lbooker42 commented Nov 11, 2024

Initial provision is made to re-use a shared GroupByChunkedOperator in FormulaMultiColumnChunkedOperator and FormulaChunkedOperator but more work is needed before this can be fully used.

  1. We need to optimize the operator order in AggregationOptimizer to look for GroupByChunkedOperator instances from group aggs or that may be created for other formulas.
  2. For FormulaMultiColumnChunkedOperator, we need to update GroupByChunkedOperator to keep two sets of output columns: public columns for requested GroupBy aggs and "hidden" ones only exposed to internal operators.
@lbooker42 lbooker42 added feature request New feature or request triage labels Nov 11, 2024
@lbooker42 lbooker42 changed the title Improve AggFormula by re-using GroupByChunkedOperator rather than re-creating new operators. Improve AggFormula performance by re-using GroupByChunkedOperator rather than re-creating new operators. Nov 11, 2024
@lbooker42 lbooker42 added this to the Backlog milestone Nov 11, 2024
@rcaudy rcaudy added query engine core Core development tasks and removed triage labels Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core development tasks feature request New feature or request query engine
Projects
None yet
Development

No branches or pull requests

2 participants