Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling in-flight requersts in prefetch buffer resize #9968

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

knizhnik
Copy link
Contributor

@knizhnik knizhnik commented Dec 2, 2024

Problem

See #9961
Current implementation of prefetch buffer resize doesn't correctly handle in-flight requests

Summary of changes

  1. Fix index of entry we should wait for if new prefetch buffer size is smaller than number of in-flight requests.
  2. Correctly set flush position

@knizhnik knizhnik requested review from a team as code owners December 2, 2024 15:27
@knizhnik knizhnik requested review from kelvich and yliang412 December 2, 2024 15:27
Copy link

github-actions bot commented Dec 2, 2024

7018 tests run: 6710 passed, 0 failed, 308 skipped (full report)


Flaky tests (2)

Postgres 16

Postgres 14

Code coverage* (full report)

  • functions: 30.4% (8270 of 27227 functions)
  • lines: 47.7% (65193 of 136583 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
cf8809e at 2024-12-02T16:41:15.197Z :recycle:

@hlinnaka
Copy link
Contributor

hlinnaka commented Dec 9, 2024

Approved as this fixes the flakiness. I'm still pretty scared of the readahead_buffer_resize() function though. It gets executed rarely, and if the compaction logic is wrong, you get subtle, silent corruption.

Calling prefetch_wait_for() in the assign hook is pretty scary too. Most assign hooks are short functions that just do some calculation or transformation and assign the new value to a global variable. But that function does network I/O. I'd suggest just remembering the new value somewhere and applying it later at a convenient time.

@knizhnik knizhnik added this pull request to the merge queue Dec 10, 2024
Merged via the queue into main with commit 311ee79 Dec 10, 2024
82 checks passed
@knizhnik knizhnik deleted the prefetch_buffer_resize_fix_v2 branch December 10, 2024 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants