fix(pageserver): reduce aux compaction threshold

[skyzh]

Problem

ref #9441
The metrics from LR publisher testing project: ~300KB aux key deltas per 256MB files. Therefore, I think we can do compaction more aggressively as these deltas are small and compaction can reduce layer download latency. We also have a read path perf fix #9631 but I'd still combine the read path fix with the reduce of the compaction threshold.

Summary of changes

• reduce metadata compaction threshold
• use num of L1 delta layers as an indicator for metadata compaction
• dump more logs

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

5328 tests run: 5106 passed, 0 failed, 222 skipped (full report)

---

Code coverage* (full report)

• functions: 31.6% (7799 of 24712 functions)
• lines: 49.0% (61148 of 124775 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
b0ef4e7 at 2024-11-05T21:56:44.504Z :recycle:}