Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pageserver): support partial gc-compaction for lowest retain lsn #9134

Merged
merged 9 commits into from
Oct 29, 2024

Conversation

skyzh
Copy link
Member

@skyzh skyzh commented Sep 24, 2024

Problem

part of #8921, #9114

Summary of changes

We start the partial compaction implementation with the image layer partial generation. The partial compaction API now takes a key range. We will only generate images for that key range for now, and remove layers fully included in the key range after compaction.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@skyzh skyzh requested a review from a team as a code owner September 24, 2024 20:27
@skyzh skyzh requested a review from problame September 24, 2024 20:27
Copy link

github-actions bot commented Sep 24, 2024

5292 tests run: 5070 passed, 0 failed, 222 skipped (full report)


Code coverage* (full report)

  • functions: 31.4% (7748 of 24639 functions)
  • lines: 49.0% (60989 of 124526 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
4aa8ec6 at 2024-10-29T17:30:44.815Z :recycle:

@skyzh skyzh force-pushed the skyzh/partial-compaction branch from 12ae882 to 21cbd41 Compare September 24, 2024 22:26
@skyzh skyzh force-pushed the skyzh/partial-compaction branch 2 times, most recently from ba7fa81 to 959a425 Compare October 20, 2024 18:55
@skyzh skyzh force-pushed the skyzh/partial-compaction branch from 959a425 to 6e89a2e Compare October 20, 2024 18:57
@skyzh skyzh requested a review from arpad-m October 21, 2024 16:43
@skyzh skyzh changed the title feat(pageserver): support partial gc-compaction feat(pageserver): support partial gc-compaction for lowest retain lsn Oct 21, 2024
@skyzh skyzh mentioned this pull request Oct 21, 2024
28 tasks
pageserver/src/tenant/timeline/compaction.rs Outdated Show resolved Hide resolved
pageserver/src/tenant/timeline/compaction.rs Outdated Show resolved Hide resolved
pageserver/src/tenant.rs Outdated Show resolved Hide resolved
pageserver/src/tenant.rs Show resolved Hide resolved
@skyzh skyzh requested a review from problame October 28, 2024 14:35
@skyzh skyzh enabled auto-merge (squash) October 29, 2024 14:59
Signed-off-by: Alex Chi Z <[email protected]>
@skyzh skyzh merged commit 88ff8a7 into main Oct 29, 2024
80 checks passed
@skyzh skyzh deleted the skyzh/partial-compaction branch October 29, 2024 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants