Add metrics for input data considered and taken for compression

[arpad-m]

If compression is enabled, we currently try compressing each image larger than a specific size and if the compressed version is smaller, we write that one, otherwise we use the uncompressed image. However, this might sometimes be a wasteful process, if there is a substantial amount of images that don't compress well.

The compression metrics added in #8420 pageserver_compression_image_in_bytes_total and pageserver_compression_image_out_bytes_total are well designed for answering the question how space efficient the total compression process is end-to-end, which helps one to decide whether to enable it or not.

To answer the question of how much waste there is in terms of trial compression, so CPU time, we add two metrics:

• one about the images that have been trial-compressed (considered), and
• one about the images where the compressed image has actually been written (chosen).

There is different ways of weighting them, like for example one could look at the count, or the compressed data. But the main contributor to compression CPU usage is amount of data processed, so we weight the images by their uncompressed size. In other words, the two metrics are:

• pageserver_compression_image_in_bytes_considered
• pageserver_compression_image_in_bytes_chosen

Part of #5431

[github-actions]

3138 tests run: 3017 passed, 0 failed, 121 skipped (full report)

---

Flaky tests (1)

Postgres 14

• test_subscriber_restart: release

Code coverage* (full report)

• functions: 32.8% (7022 of 21383 functions)
• lines: 50.0% (55881 of 111692 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
0e7d493 at 2024-07-30T05:42:01.765Z :recycle:}