refactor(pageserver): make image layer creation atomic

[skyzh]

Problem

If image layer creation failed in the middle, it will leave some temporary files on the disk. Part of #8362, also need to fix L0 compaction path in the future.

Summary of changes

Use the newly-added batch layer writer to ensure either all of them are created successfully or none of them are created successfully.

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

5328 tests run: 5106 passed, 0 failed, 222 skipped (full report)

---

Flaky tests (1)

Postgres 17

• test_pg_regress[4]: debug-x86-64

Code coverage* (full report)

• functions: 31.5% (7778 of 24698 functions)
• lines: 49.0% (61045 of 124704 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
8ee5588 at 2024-10-31T17:37:55.027Z :recycle:}