Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix incorrect rounding in tdigest_compute_quantiles(_of)
The functions used int64 counter to track the number of values in the centroids, but right before approximating the two centroids we either add or subtract (prev->count / 2.0); and assign the value back to the counter, forcing rounding. That may do nothing (if there's just a single value, as in the centroids on the tails) or add/subtract a bit more than expected. Both cases may produce bogus results. Reported by Matt Watson (sporty81) on tdigest_percentile(), but AFAIC the inverse has roughly the same problem, so fix that too.
- Loading branch information