Prometheus Native Histogram support phase2

[krajorama]

Continued from #3478 and continued in #7229

Fully support Prometheus Native Histograms

Add recording rules and query sharding support.

Additional goals

Tasks

Beta Give feedback

1. ruler: Support native histograms in recording rules #4155
   +0
   
2. Enable query sharding when tenant accepts native histograms #3992
   +0
   
3. Let mimir-continuous-test generate native histograms #4044
   +0
   
4. vendor out of order support from prometheus #9421
   +0
   
5. Implement TimeseriesToMatrix for native histograms #4378
   +0
   
6. Rewrite mockQuerier to be able to return histograms #4379
   +0
   
7. Have per-tenant limit on the number of buckets per native histogram #4542
   +0
   
8. Direct cast remote Histogram spans to tsdb spans and verify compatibility #4742
   +0
   
9. Fix encoding samples in ChunkSeries prometheus/prometheus#12185
   +0
10. Handle more arithmetic operators for native histograms prometheus/prometheus#12262
    +0
11. Implement bucket limit for native histograms prometheus/prometheus#12254
    +0
12. Update native histograms documentation with bucket limit, protobuf #5019
    +0
13. Active series should track native histograms and buckets #5365
    +0
    
14. Tenants dashboard should show tracking of native histograms #5542
    +0
15. native histograms: Reduce resolution instead of dropping when bucket limit is hit #5367
    2 of 4
    +0
16. histogram_stdvar and histogram_stddev query sharding support #5929
    +0
    

Optimizations

• investigate turning promql results into JSON in query frontend. Is it optimal? Is it making some memory copies it doesn't need to ?
• investigate not converting integer histogram to float histogram on merged iterator level (which is above mimir's chunk iterator wrapper on top of tsdb chunk iterator), instead let the tsdb chunk convert the data directly

Stretch goals

Stretch goal tasks

Beta Give feedback

1. Start using native histograms for cortex_request_duration_seconds #4175
   +0
2. querier chunkIterator may return values after Seek beyond end of chunk #4415
   +0
   
3. Convert remote write sample histogram directly to float histogram where possible #4488
   +0
   
4. Cache converted integer to float histogram in chunk_iterator #4489
   +0
   
5. Restore mimirpb.FromPointsToSamples performance #3738
   +0
   

PRs outside mimir about the ecosystem

Ecosystem issues

Beta Give feedback

1. Histogram: Should handle both classic and native histograms correctly for Prometheus datasource grafana#69790
   area/dataplane area/panel/histogram effort/medium internal prio/medium type/feature-request +6
   
2. Support new PromQL native histogram functions grafana#76111
   datasource/Prometheus needs investigation type/feature-request +3
   
3. Improve native histogram visualization in explore grafana#76130
   prio/medium type/feature-request +2
   
4. Improve native histogram tooltip visualization in heatmap grafana#76131
   area/panel/heatmap area/tooltip type/feature-request +3
   
5. Native histograms: error on ingesting out-of-order exemplars prometheus/prometheus#12971
   component/scraping kind/bug priority/P2 +3
6. promql: numerical comparison of label values prometheus/prometheus#12934
   component/promql kind/feature priority/Pmaybe +3
7. Improve the readability of y axis #6482
   +0
8. scrape: make le and quantile normalization configurable prometheus/prometheus#12984
   component/scraping kind/feature priority/Pmaybe +3
9. Heatmap panel: Prometheus exemplars not shown for native histograms grafana#77277
   area/panel/heatmap +1
   
10. tsdb/head: wlog exemplars after samples prometheus/prometheus#13113
    +0
11. Fix wrong order of exemplars vs samples agent#5750
    frozen-due-to-age +1
12. Add min time to histogram chunks prometheus/prometheus#13062
    +0
13. storage: Fix mixed samples handling in sampleRing prometheus/prometheus#13055
    +0
14. Instrumentation: Enable native histograms for HTTP requests grafana#75731
    area/backend area/frontend area/instrumentation no-backport no-changelog type/docs +6
15. histograms: Support exemplars in pure Native Histograms prometheus/client_golang#1126
    keep-open +1
    
16. Support out of order ingestion for native histograms prometheus/prometheus#11220
    component/tsdb priority/P2 +2