Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: conditionally run unit tests #15347

Merged
merged 9 commits into from
Dec 9, 2024
Merged

Conversation

erikburt
Copy link
Collaborator

@erikburt erikburt commented Nov 20, 2024

Changes

  • Introduce composite action setup-ci-core-tests, which consolidates the some of setup for the tests
    • Not all steps can be consolidated because nesting reusable action calls has some weird side effects
  • Introduce new workflow ci-core-partial which uses the node action from action: go conditional tests .github#709 to only run test binaries with changes
    • This will be run alongside CI-Core for testing purposes until adequately validated and a transition can be made
    • Fuzz and Race tests are included in this, but only on a schedule for now. Once we replace/consolidate this with CI-Core workflow we will run them on PR as well.

Testing

Workflow running all tests, and updating index: https://github.com/smartcontractkit/chainlink/actions/runs/12187711071/job/33999115005?pr=15347
- This workflow ran all tests because the index had been previously updated with hashes from binaries built with coverage flags. Note: this won't happen during normal execution because coverage being enabled actually causes the workflow to skip updating the hash.

Workflow running no tests: https://github.com/smartcontractkit/chainlink/actions/runs/12188016602/job/34000123940?pr=15347

  • The previous run updated the index and then no changes were made, so the hashes of the test binaries shouldn't be changed, and nothing should run.

Test package concurrency of 16 seemed more stable than 32.

Rollout

Run this alongside existing validation for at least a week. Only after we've confirmed that the new validation mechanism is working as expected should we replace the existing validation.


RE-3168

@erikburt erikburt self-assigned this Nov 20, 2024
@erikburt erikburt force-pushed the feat/go-test-binary-comparison branch 10 times, most recently from 0b75b37 to 3ebea69 Compare November 27, 2024 19:30
@smartcontractkit smartcontractkit deleted a comment from github-actions bot Nov 27, 2024
@erikburt erikburt force-pushed the feat/go-test-binary-comparison branch 6 times, most recently from 13976e4 to d501ceb Compare December 4, 2024 17:46
@erikburt erikburt force-pushed the feat/go-test-binary-comparison branch from 74713e7 to a03ce66 Compare December 5, 2024 20:41
@erikburt erikburt force-pushed the feat/go-test-binary-comparison branch from a03ce66 to 6467b19 Compare December 5, 2024 21:03
@smartcontractkit smartcontractkit deleted a comment from github-actions bot Dec 5, 2024
@erikburt erikburt marked this pull request as ready for review December 5, 2024 22:18
@erikburt erikburt requested a review from a team as a code owner December 5, 2024 22:18
@erikburt erikburt requested review from a team as code owners December 5, 2024 22:18
chainchad
chainchad previously approved these changes Dec 6, 2024
Did some testing trying to see if I could speed up test binary building
through caching. It seems like the cache saving doesn't work when the
composite action is nested.
@erikburt erikburt added this pull request to the merge queue Dec 9, 2024
Merged via the queue into develop with commit 45898fc Dec 9, 2024
102 of 103 checks passed
@erikburt erikburt deleted the feat/go-test-binary-comparison branch December 9, 2024 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants