Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): Min-max versions of histogram quantile and fraction for exp histograms #1891

Conversation

vishramachandran
Copy link
Member

@vishramachandran vishramachandran commented Nov 15, 2024

Pull Request checklist

  • The commit(s) message(s) follows the contribution guidelines ?
  • Tests for the changes have been added (for bug fixes / features) ?
  • Docs have been added / updated (for bug fixes / features) ?

Use min and max when available to calculate better quantiles and fractions.

Updated perf test results for quantiles with exponential histograms are shown below. There is a proportional degradation of performance as the number of buckets increase. Thus, exponential histograms will benefit from pre-aggregation when a large number of time series is involved. Note these results supersede the results in earlier exponential histogram PR.

32 buckets without profiling
[info] Benchmark                                                          Mode  Cnt    Score   Error  Units
[info] Base2ExponentialHistogramQueryBenchmark.histQuantileRangeQueries  thrpt    2  404.993          ops/s

64 buckets without profiling
[info] Benchmark                                                          Mode  Cnt    Score   Error  Units
[info] Base2ExponentialHistogramQueryBenchmark.histQuantileRangeQueries  thrpt    2  252.478          ops/s

128 buckets without profiling
[info] Benchmark                                                          Mode  Cnt    Score   Error  Units
[info] Base2ExponentialHistogramQueryBenchmark.histQuantileRangeQueries  thrpt    2  234.001          ops/s

160 buckets without profiling
[info] Benchmark                                                          Mode  Cnt    Score   Error  Units
[info] Base2ExponentialHistogramQueryBenchmark.histQuantileRangeQueries  thrpt    2  219.758          ops/s

200 buckets without profiling
[info] Benchmark                                                          Mode  Cnt    Score   Error  Units
[info] Base2ExponentialHistogramQueryBenchmark.histQuantileRangeQueries  thrpt    2  137.804          ops/s

256 buckets without profiling
[info] Benchmark                                                          Mode  Cnt    Score   Error  Units
[info] Base2ExponentialHistogramQueryBenchmark.histQuantileRangeQueries  thrpt    2  119.017          ops/s

Fitted Linear Regression:
image

JMH Benchamark CPU Profile
image

@vishramachandran vishramachandran marked this pull request as draft November 15, 2024 00:29
@vishramachandran vishramachandran force-pushed the min-max-functions-exp-histograms branch from a3861f4 to e0a8788 Compare November 21, 2024 22:49
@vishramachandran vishramachandran marked this pull request as ready for review November 23, 2024 00:46
@vishramachandran vishramachandran force-pushed the min-max-functions-exp-histograms branch 3 times, most recently from 4f233d1 to a736903 Compare November 25, 2024 13:28
@vishramachandran vishramachandran force-pushed the min-max-functions-exp-histograms branch from 9eb1a01 to 82a40ca Compare December 2, 2024 18:08
@sandeep6189
Copy link
Contributor

thank you for adding unit test @vishramachandran

@vishramachandran vishramachandran merged commit 4ec845a into filodb:develop Dec 3, 2024
1 check passed
@vishramachandran vishramachandran deleted the min-max-functions-exp-histograms branch December 3, 2024 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants