Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pageserver): drop disposable keys during gc-compaction #9765

Merged
merged 3 commits into from
Nov 18, 2024

Conversation

skyzh
Copy link
Member

@skyzh skyzh commented Nov 14, 2024

Problem

close #9552, close #8920, part of #9114

Summary of changes

  • Drop keys not belonging to this shard during gc-compaction to avoid constructing history that might have been truncated during shard compaction.
  • Run gc-compaction at the end of shard compaction test.

@skyzh skyzh requested a review from problame November 14, 2024 18:13
@skyzh skyzh requested a review from a team as a code owner November 14, 2024 18:13
@skyzh skyzh force-pushed the skyzh/partial-compaction-shard branch from 0062acf to a593cde Compare November 14, 2024 19:00
Copy link

github-actions bot commented Nov 14, 2024

5525 tests run: 5298 passed, 1 failed, 226 skipped (full report)


Failures on Postgres 17

# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_sharding_split_failures[debug-pg17-failure20]"

Code coverage* (full report)

  • functions: 31.5% (7930 of 25166 functions)
  • lines: 49.6% (62842 of 126581 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
eba488a at 2024-11-18T19:37:36.702Z :recycle:

@skyzh skyzh requested a review from arpad-m November 14, 2024 20:04
test_runner/regress/test_compaction.py Outdated Show resolved Hide resolved
@skyzh skyzh mentioned this pull request Nov 18, 2024
28 tasks
@skyzh skyzh enabled auto-merge (squash) November 18, 2024 17:30
@skyzh skyzh merged commit e5c89f3 into main Nov 18, 2024
78 checks passed
@skyzh skyzh deleted the skyzh/partial-compaction-shard branch November 18, 2024 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pageserver: failed gc-compaction runs in staging gc-compaction: filter keys not in shard / not in keyspace
2 participants