Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimizing batch sizing based on cost #1416

Closed
wants to merge 1 commit into from
Closed

optimizing batch sizing based on cost #1416

wants to merge 1 commit into from

Conversation

shikhar1729
Copy link

@shikhar1729 shikhar1729 commented Nov 25, 2023

Optimize EvaDB

Changes Overview:

  • evadb/executor/storage_executor.py : Adjusted the execution of reading data for TableType.NATIVE_DATA by incorporating the specified batch memory size. This ensures efficient resource utilization during data retrieval.

  • evadb/optimizer/cost_model.py : Implemented a zero_cost property within the CostModel class, providing a baseline cost entry for comparison. Revised the calculate_cost method to accept a list of children costs, enabling a more accurate representation of overall costs. Enhanced the cost calculation for various plan types, taking into account child costs and adjusting accordingly.

  • evadb/optimizer/group_expression.py : Improved the representation of GroupExpression instances for better clarity and debugging.

  • evadb/optimizer/optimizer_tasks.py : Modified the execution logic within the OptimizeInputs task to consider child costs when calculating the overall cost. This adjustment ensures a more accurate representation of the optimization process.

  • evadb/optimizer/rules/rules.py : Refined the application of rules related to LogicalGet instances, specifically addressing batch memory size configuration within the SeqScanPlan. Experimentation with heuristics for optimal batch memory size is also noted for future consideration.

  • evadb/plan_nodes/vector_index_scan_plan.py : Improved the __str__ method for better readability and understanding of VectorIndexScan instances.

@shikhar1729 shikhar1729 closed this by deleting the head repository Jan 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant