Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(pageserver): refactor split writers using batch layer writer #9493

Merged
merged 2 commits into from
Oct 24, 2024

Conversation

skyzh
Copy link
Member

@skyzh skyzh commented Oct 23, 2024

Problem

part of #9114, #8836, #8362

The split layer writer code can be used in a more general way: the caller puts unfinished writers into the batch layer writer and let batch layer writer to ensure the atomicity of the layer produces.

Summary of changes

  • Add batch layer writer, which atomically finishes the layers. BatchLayerWriter::finish is simply a copy-paste from previous split layer writers.
  • Refactor split writers to use the batch layer writer.
  • The current split writer tests cover all code path of batch layer writer.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@skyzh skyzh requested a review from problame October 23, 2024 18:44
@skyzh skyzh requested a review from a team as a code owner October 23, 2024 18:44
Signed-off-by: Alex Chi Z <[email protected]>
Copy link

5238 tests run: 5024 passed, 0 failed, 214 skipped (full report)


Code coverage* (full report)

  • functions: 31.4% (7676 of 24436 functions)
  • lines: 48.9% (60309 of 123413 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
c5ff298 at 2024-10-23T19:55:45.027Z :recycle:

@skyzh skyzh requested a review from arpad-m October 23, 2024 19:57
@skyzh skyzh merged commit fb0406e into main Oct 24, 2024
80 checks passed
@skyzh skyzh deleted the skyzh/batch-writer branch October 24, 2024 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants