ci: increase timeout for k8s intg tests #9929
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for determined-ui ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9929 +/- ##
==========================================
- Coverage 59.18% 54.52% -4.67%
==========================================
Files 751 1252 +501
Lines 104462 156550 +52088
Branches 3598 3599 +1
==========================================
+ Hits 61824 85354 +23530
- Misses 42506 71064 +28558
Partials 132 132
Flags with carried forward coverage won't be shown. Click here to find out more. |
amandavialva01
approved these changes
Sep 12, 2024
rb-determined-ai
force-pushed
the
rb/new-k8s
branch
from
910c942 to
e746c06
Compare
rb-determined-ai
changed the title
rb-determined-ai
force-pushed
the
rb/new-k8s
branch
from
e746c06 to
f3fd024
Compare
kkunapuli
approved these changes
Sep 12, 2024
The latest versions of k8s have a new Job condition called JobFailureTarget which is a signal to the jobs controller to kill off the pods of the job. Our jobUpdatedCallback() function waits for the JobFailure condition, which now comes after the pod is fully terminated, which takes a lot longer. Probably we need to make sure our k8s logic is still valid, and deal with the additional time it takes a job to reach JobFailed, if it affects anything other than this test. Until then, let's unblock CI for the whole team by just increasing the test time for TestExternalPodDelete and TestNodeWorkflows.
rb-determined-ai
force-pushed
the
rb/new-k8s
branch
from
f3fd024 to
282d891
Compare
salonig23
pushed a commit
that referenced
this pull request
Sep 17, 2024
The latest versions of k8s have a new Job condition called JobFailureTarget which is a signal to the jobs controller to kill off the pods of the job. Our jobUpdatedCallback() function waits for the JobFailure condition, which now comes after the pod is fully terminated, which takes a lot longer. Probably we need to make sure our k8s logic is still valid, and deal with the additional time it takes a job to reach JobFailed, if it affects anything other than this test. Until then, let's unblock CI for the whole team by just increasing the test time for TestExternalPodDelete and TestNodeWorkflows.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The latest versions of k8s have a new Job condition called
JobFailureTarget which is a signal to the jobs controller to kill off
the pods of the job.
Our jobUpdatedCallback() function waits for the JobFailure condition,
which now comes after the pod is fully terminated, which takes a lot
longer.
Probably we need to make sure our k8s logic is still valid, and deal
with the additional time it takes a job to reach JobFailed, if it
affects anything other than this test.
Until then, let's unblock CI for the whole team by just increasing the
test time for TestExternalPodDelete and TestNodeWorkflows.