[NO-TICKET] Fix flaky profiler spec by adjusting wait condition #4072
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**What does this PR do?** This PR fixes a profiler flaky spec from https://app.circleci.com/pipelines/github/DataDog/dd-trace-rb/17279/workflows/bafa8b55-7fee-414f-8875-c7a8ef8b56dd/jobs/615562 : ``` 1) Datadog::Profiling::Collectors::CpuAndWallTimeWorker#start Process::Waiter crash regression tests can sample an instance of Process::Waiter without crashing Failure/Error: expect(sample.locations.first.path).to eq "In native code" NoMethodError: undefined method `locations' for nil ``` The fix in this case is to make sure that we keep running the profiler until we observe a sample for the thread we expect, not just for any thread. **Motivation:** We aim to always have zero flaky specs in the profiler. **Additional Notes:** If for some reason we're not able to sample the target thread, the test will still correctly fail (with the `try_wait_until` failing). Fixes DataDog/ruby-guild#182 **How to test the change?** Validate that CI is still green :)
ivoanjo
added
profiling
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #4072 +/- ##
==========================================
- Coverage 97.72% 97.72% -0.01%
==========================================
Files 1338 1338
Lines 80248 80248
Branches 4016 4016
==========================================
- Hits 78426 78421 -5
- Misses 1822 1827 +5 ☔ View full report in Codecov by Sentry. |
BenchmarksBenchmark execution time: 2024-11-06 09:01:57 Comparing candidate commit 5d0390d in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 28 metrics, 2 unstable metrics. scenario:profiler - stack collector
|
anmarchenko
approved these changes
Nov 6, 2024
AlexJF
approved these changes
Nov 6, 2024
ivoanjo
added a commit
that referenced
this pull request
Nov 6, 2024
**What does this PR do?** This PR relaxes the GitHub `CODEOWNERS` file rules for things related to profiling. Specifically, it adds `@DataDog/ruby-guild` as a second owner on top of just `@DataDog/profiling-rb`. **Motivation:** Now that we've enableed the "Require review from Code Owners" feature on GitHub, since there's not a lot of us working on profiling, it hasn't worked very well. Specifically, for very small PRs (#4072 is a good example where we slightly tweaked a test to avoid flakiness), it's annoying to have to wait for a very small reviewer pool, when most folks that are on the Ruby guild can help out review such changes. **Additional Notes:** N/A **How to test the change?** Check that future PRs affecting profiling can be merged with a review from anyone on the ruby-guild team.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR fixes a profiler flaky spec from
https://app.circleci.com/pipelines/github/DataDog/dd-trace-rb/17279/workflows/bafa8b55-7fee-414f-8875-c7a8ef8b56dd/jobs/615562 :
The fix in this case is to make sure that we keep running the profiler until we observe a sample for the thread we expect, not just for any thread.
Motivation:
We aim to always have zero flaky specs in the profiler.
Change log entry
(No entry needed for this one)
Additional Notes:
If for some reason we're not able to sample the target thread, the test will still correctly fail (with the
try_wait_untilfailing).Fixes https://github.com/datadog/ruby-guild/issues/182
How to test the change?
Validate that CI is still green :)