Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harden publishing #4555

Merged
merged 3 commits into from
Nov 23, 2024
Merged

Harden publishing #4555

merged 3 commits into from
Nov 23, 2024

Conversation

marta-lokhova
Copy link
Contributor

@marta-lokhova marta-lokhova commented Nov 22, 2024

This PR contains two changes:

  • Properly fsync HAS files that act as a "publish queue". Note that the checkpoints themselves are using XDR streams with fsync turned on already, so this change impacts HAS files only.
  • Gracefully handle situations where nodes enable publish mid-checkpoint. Core will wait until the next checkpoint to begin publishing. Without this change, nodes would need to rebuild state before enabling publishing (although it's unclear how often partial archives occur in practice).

Additionally, I added a few more thing to aid debugging, such as printing checkpoint files with dump-xdr and an offline command to dump all checkpoints scheduled to publish.

graydon
graydon previously approved these changes Nov 22, 2024
graydon
graydon previously approved these changes Nov 22, 2024
graydon
graydon previously approved these changes Nov 22, 2024
SirTyson
SirTyson previously approved these changes Nov 22, 2024
Copy link
Contributor

@SirTyson SirTyson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after the MODE_DISABLES_FSYNC change.

@marta-lokhova marta-lokhova added this pull request to the merge queue Nov 22, 2024
Merged via the queue into stellar:master with commit 0bda87b Nov 23, 2024
13 checks passed
@marta-lokhova marta-lokhova deleted the PublishFsync branch November 23, 2024 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants