Skip to content

Commit

Permalink
CI(report-benchmarks-failures): report benchmarks failures to slack (#…
Browse files Browse the repository at this point in the history
…7678)

## Problem

`benchmarks` job that we run on the main doesn't block anything, so it's
easy to miss its failure.

Ref neondatabase/cloud#13087

## Summary of changes
- Add `report-benchmarks-failures` job that report failures of
`benchmarks` job to a Slack channel
  • Loading branch information
bayandin authored May 13, 2024
1 parent 6ff7429 commit 55ba885
Showing 1 changed file with 18 additions and 0 deletions.
18 changes: 18 additions & 0 deletions .github/workflows/build_and_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -546,9 +546,27 @@ jobs:
# XXX: no coverage data handling here, since benchmarks are run on release builds,
# while coverage is currently collected for the debug ones

report-benchmarks-failures:
needs: [ benchmarks, create-test-report ]
if: github.ref_name == 'main' && failure()
runs-on: ubuntu-latest

steps:
- uses: slackapi/slack-github-action@v1
with:
channel-id: C060CNA47S9 # on-call-staging-storage-stream
slack-message: |
Benchmarks failed on main: ${{ github.event.head_commit.url }}
Allure report: ${{ needs.create-test-report.outputs.report-url }}
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}

create-test-report:
needs: [ check-permissions, regress-tests, coverage-report, benchmarks, build-build-tools-image ]
if: ${{ !cancelled() && contains(fromJSON('["skipped", "success"]'), needs.check-permissions.result) }}
outputs:
report-url: ${{ steps.create-allure-report.outputs.report-url }}

runs-on: [ self-hosted, gen3, small ]
container:
Expand Down

1 comment on commit 55ba885

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3141 tests run: 2995 passed, 0 failed, 146 skipped (full report)


Flaky tests (2)

Postgres 15

  • test_vm_bit_clear_on_heap_lock: debug

Postgres 14

  • test_compute_pageserver_connection_stress: debug

Code coverage* (full report)

  • functions: 31.4% (6329 of 20161 functions)
  • lines: 47.3% (47811 of 101037 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
55ba885 at 2024-05-13T14:47:34.001Z :recycle:

Please sign in to comment.