Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QBFT soak test run nightly, with status badge #7986

Open
jflo opened this issue Dec 4, 2024 · 2 comments
Open

QBFT soak test run nightly, with status badge #7986

jflo opened this issue Dec 4, 2024 · 2 comments
Labels
devops DevOps related task QBFT QBFT Consensus re;ated testing

Comments

@jflo
Copy link
Contributor

jflo commented Dec 4, 2024

There is a long-running integration test covering QBFT networks, implemented via the acceptanceTestBftSoak target in gradle.

This is a useful, but long (2hrs?) test, and so running it nightly should be sufficient to alert us to any introduced regressions.

@jflo
Copy link
Contributor Author

jflo commented Dec 4, 2024

  • what should be the consequences of job failure?
  • how do we expect to track success/failure?
  • failure should be a blocker to release

@jflo jflo added testing QBFT QBFT Consensus re;ated labels Dec 4, 2024
@jflo jflo added this to the Besu 25.1 milestone Dec 4, 2024
@matthew1001
Copy link
Contributor

Thanks for looking into this @jflo Some thoughts on the above.

  • Test failure
    • Maybe to start with we could look at raising an alert in the besu-github discord channel to indicate the failure. I don't mind being tagged in the alert at least to begin with
    • Can other GHA's be dependent on the most recent scheduled run being green? I'm thinking of release-related GHAs particularly.
  • Tracking & release
    • Perhaps part of the release process is to manually check the recent scheduled runs to ensure that the most recent one is green
    • If they have clearly gone pass, pass, pass, fail, fail, fail... then whoever is doing the release process might need to investigate more closely
    • If they are intermittent then maybe it's a sign that the test's reliability needs looking into

@jflo jflo removed this from the Besu 25.1 milestone Dec 5, 2024
@jflo jflo added the devops DevOps related task label Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops DevOps related task QBFT QBFT Consensus re;ated testing
Projects
None yet
Development

No branches or pull requests

2 participants