Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the compaction warning more tolerant #5024

Merged
merged 1 commit into from
Aug 18, 2023
Merged

Conversation

arpad-m
Copy link
Member

@arpad-m arpad-m commented Aug 17, 2023

Problem

The performance benchmark in test_runner/performance/test_layer_map.py is currently failing due to the warning added in #4888.

Summary of changes

The test mentioned has a compaction_target_size of 8192, which is just one page size. This is an unattainable goal, as we generate at least three pages: one for the header, one for the b-tree (minimally sized ones have just the root node in a single page), one for the data.

Therefore, we add two pages to the warning limit. The warning text becomes a bit less accurate but I think this is okay.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@github-actions
Copy link

github-actions bot commented Aug 17, 2023

1608 tests run: 1533 passed, 0 failed, 75 skipped (full report)


The comment gets automatically updated with the latest test results
415f49d at 2023-08-18T13:59:19.783Z :recycle:

@arpad-m arpad-m marked this pull request as ready for review August 17, 2023 14:33
@arpad-m arpad-m requested review from a team as code owners August 17, 2023 14:33
@arpad-m arpad-m requested review from knizhnik and koivunej and removed request for a team August 17, 2023 14:33
Copy link
Member

@koivunej koivunej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this is any better. Could we just allowed_error for the one test?

(2+2)*8192 for the test_layer_map.rs here: https://neon-github-public-dev.s3.amazonaws.com/reports/main/5889544346/index.html#suites/b8bb9797b578168beaf6275d4a5cca96/a842de44855768cd

@arpad-m
Copy link
Member Author

arpad-m commented Aug 17, 2023

I don't really like allowing tests that use bad config parameters. Instead, I prefer either disallowing those bad config parameters via an error, or silently fixing them (like what my PR proposes). Could we talk about the details in a google call? I've sent you an e-mail.

@arpad-m arpad-m force-pushed the arpad/fix_compaction_warning branch from 132e3e7 to 415f49d Compare August 18, 2023 13:35
@arpad-m arpad-m requested a review from jcsp August 18, 2023 13:35
@arpad-m arpad-m changed the title Make the compaction warning more tolerant and reduce cut limit Make the compaction warning more tolerant Aug 18, 2023
@arpad-m
Copy link
Member Author

arpad-m commented Aug 18, 2023

Removed the part where I reduced the potential overshoot. Personally I still think that's a good idea, but it isn't as pressing as fixing the warning.

@arpad-m arpad-m merged commit f4da010 into main Aug 18, 2023
@arpad-m arpad-m deleted the arpad/fix_compaction_warning branch August 18, 2023 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants