Skip to content

Commit

Permalink
build(ci): improve test workflow (#3)
Browse files Browse the repository at this point in the history 
## Description

This PR improves the test workflow.

---

### Author Checklist

*All items are required. Please add a note to the item if the item is
not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type
prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json)
in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR
Targeting](https://github.com/desmos-labs/desmos/blob/master/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building
modules](https://docs.cosmos.network/v0.44/building-modules/intro.html)
- [ ] included the necessary unit and integration
[tests](https://github.com/desmos-labs/desmos/blob/master/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go
code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable
and please add
your handle next to the items reviewed if you only reviewed selected
items.*

I have...

- [ ] confirmed the correct [type
prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json)
in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
  • Loading branch information
@RiccardoM
RiccardoM authored May 10, 2024
1 parent a0af10d commit abb683a
Show file tree
Hide file tree
Showing 2 changed files with 344 additions and 65 deletions.
349 changes: 302 additions & 47 deletions .github/workflows/test.yml
View file
Original file line number Diff line number Diff line change
@@ -1,67 +1,322 @@
name: Build & Test
on:
pull_request:
paths:
- "**.go"
- "**.mv"
- "**.move"
push:
branches:
- main
- "release/*"
paths:
- "**.go"
- "**.mv"
- "**.move"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
GOPRIVATE: github.com/milkyway-labs
GH_ACCESS_TOKEN: ${{ secrets.GOPRIVATE_ACCESS_TOKEN }}

jobs:
test-coverage-upload:
name: Run test and upload codecov
env:
# for private repo access
GOPRIVATE: github.com/initia-labs/*
GITHUB_ACCESS_TOKEN: ${{ secrets.GH_READ_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
Cleanup-runs:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-go@v3
- name: Cleanup 🧹
uses: rokroskar/workflow-run-cleanup-action@master
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
if: "!startsWith(github.ref, 'refs/tags/') && github.ref != 'refs/heads/master'"

Install-tparse:
runs-on: ubuntu-latest
steps:
- name: Setup Go 🧰
uses: actions/setup-go@v5
with:
go-version: '1.22'

- name: Display go version 🛂
run: go version

- name: Setup GOPRIVATE 🛡️
run: git config --global url.https://[email protected]/.insteadOf https://github.com/

- name: Install tparse 🔍
run: |
export GO111MODULE="on" && go install github.com/mfridman/[email protected]
- name: Cache 💾
uses: actions/cache@v4
with:
path: ~/go/bin
key: ${{ runner.os }}-go-tparse-binary

Build:
runs-on: ubuntu-latest
strategy:
matrix:
go-arch: [ "amd64" ]
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4

- name: Setup Go 🧰
uses: actions/setup-go@v5
with:
go-version: 1.22
- name: Install openssl
run: sudo apt-get install libssl-dev
- uses: actions/checkout@v3
- uses: technote-space/get-diff-action@v4
go-version: '1.22'

- name: Setup GOPRIVATE 🛡️
run: git config --global url.https://[email protected]/.insteadOf https://github.com/

- name: Compute diff 📜
uses: technote-space/[email protected]
id: git_diff
with:
PATTERNS: |
**/**.go
go.mod
go.sum
# for private repo access
- run: git config --global url.https://${GITHUB_ACCESS_TOKEN}:[email protected]/.insteadOf https://github.com/
- name: build
- name: Build 🔨
run: GOARCH=${{ matrix.go-arch }} LEDGER_ENABLED=false make build

Split-test-files:
runs-on: ubuntu-latest
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4

- name: Setup Go 🧰
uses: actions/setup-go@v5
with:
go-version: '1.22'

- name: Setup GOPRIVATE 🛡️
run: git config --global url.https://[email protected]/.insteadOf https://github.com/

- name: Create a file with all the pkgs 📜
run: go mod tidy && go list ./... > pkgs.txt

- name: Split pkgs into 4 files ✂️
run: split -d -n l/4 pkgs.txt pkgs.txt.part.

- name: Upload part 00 📤
uses: actions/upload-artifact@v4
with:
name: "${{ github.sha }}-00"
path: ./pkgs.txt.part.00

- name: Upload part 01 📤
uses: actions/upload-artifact@v4
with:
name: "${{ github.sha }}-01"
path: ./pkgs.txt.part.01

- name: Upload part 02 📤
uses: actions/upload-artifact@v4
with:
name: "${{ github.sha }}-02"
path: ./pkgs.txt.part.02

- name: Upload part 03 📤
uses: actions/upload-artifact@v4
with:
name: "${{ github.sha }}-03"
path: ./pkgs.txt.part.03

Tests:
runs-on: ubuntu-latest
needs: split-test-files
strategy:
fail-fast: false
matrix:
part: [ "00", "01", "02", "03" ]
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4

- name: Setup Go 🧰
uses: actions/setup-go@v5
with:
go-version: '1.22'

- name: Setup GOPRIVATE 🛡️
run: git config --global url.https://[email protected]/.insteadOf https://github.com/

- name: Compute diff 📜
uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Download packages file 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-${{ matrix.part }}"
if: env.GIT_DIFF

- name: Test & coverage report creation 🧪
run: |
make build
- name: test & coverage report creation
cat pkgs.txt.part.${{ matrix.part }} | xargs go test -mod=readonly -timeout 30m -coverprofile=${{ matrix.part }}profile.out -covermode=atomic -tags='norace ledger test_ledger_mock'
if: env.GIT_DIFF

- name: Upload coverage 📤
uses: actions/upload-artifact@v4
with:
name: "${{ github.sha }}-${{ matrix.part }}-coverage"
path: ./${{ matrix.part }}profile.out

Upload-coverage-report:
runs-on: ubuntu-latest
needs: tests
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4

- name: Compute diff 📜
uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Download coverage 00 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-00-coverage"
if: env.GIT_DIFF

- name: Download coverage 01 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-01-coverage"
if: env.GIT_DIFF

- name: Download coverage 02 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-02-coverage"
if: env.GIT_DIFF

- name: Download coverage 03 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-03-coverage"
if: env.GIT_DIFF

- name: Join coverage 📋
run: |
cat ./*profile.out | grep -v "mode: atomic" >> coverage.txt
if: env.GIT_DIFF

- name: Filter out DONTCOVER 🔍
run: |
module=$(cat go.mod | grep module)
module=$(echo $module | sed "s/module //g")
excludelist="$(find ./ -type f -name '*.go' | xargs grep -l 'DONTCOVER')"
excludelist+=" $(find ./ -type f -name '*.pb.go')"
excludelist+=" $(find ./ -type f -name '*.pb.gw.go')"
excludelist+=" $(find ./ -type f -path './tests/mocks/*.go')"
for filename in ${excludelist}; do
filename=$(echo $filename | sed "s|^.|$module|g")
echo "Excluding ${filename} from coverage report..."
sed -i.bak "/$(echo $filename | sed 's/\//\\\//g')/d" coverage.txt
done
if: env.GIT_DIFF

- name: Upload coverage to Codecov 📤
uses: codecov/codecov-action@v4
with:
file: ./coverage.txt
if: env.GIT_DIFF

Test-race:
runs-on: ubuntu-latest
needs: split-test-files
strategy:
fail-fast: false
matrix:
part: [ "00", "01", "02", "03" ]
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4

- name: Setup Go 🧰
uses: actions/setup-go@v5
with:
go-version: '1.22'

- name: Setup GOPRIVATE 🛡️
run: git config --global url.https://[email protected]/.insteadOf https://github.com/

- name: Compute diff 📜
uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Download packages file 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-${{ matrix.part }}"
if: env.GIT_DIFF

- name: Test & coverage report creation 🧪
run: |
go test ./... -mod=readonly -timeout 12m -race -coverprofile=coverage.txt -covermode=atomic -tags='ledger test_ledger_mock'
if: env.GIT_DIFF
# - name: filter out DONTCOVER
# run: |
# excludelist="$(find ./ -type f -name '*.go' | xargs grep -l 'DONTCOVER')"
# excludelist+=" $(find ./ -type f -name '*.pb.go')"
# for filename in ${excludelist}; do
# filename=$(echo $filename | sed 's/^./github.com\/initia-labs\/initia/g')
# echo "Excluding ${filename} from coverage report..."
# sed -i.bak "/$(echo $filename | sed 's/\//\\\//g')/d" coverage.txt
# done
# if: env.GIT_DIFF
- uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.txt
fail_ci_if_error: true
xargs --arg-file=pkgs.txt.part.${{ matrix.part }} go test -mod=readonly -json -timeout 30m -race -tags='cgo ledger test_ledger_mock' | tee ${{ matrix.part }}-race-output.txt
if: env.GIT_DIFF

- name: Upload coverage 📤
uses: actions/upload-artifact@v4
with:
name: "${{ github.sha }}-${{ matrix.part }}-race-output"
path: ./${{ matrix.part }}-race-output.txt

Race-detector-report:
runs-on: ubuntu-latest
needs: [ test-race, install-tparse ]
timeout-minutes: 5
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4

- name: Compute diff 📜
uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Download coverage 00 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-00-race-output"
if: env.GIT_DIFF

- name: Download coverage 01 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-01-race-output"
if: env.GIT_DIFF

- name: Download coverage 02 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-02-race-output"
if: env.GIT_DIFF

- name: Download coverage 03 📥
uses: actions/download-artifact@v4
with:
name: "${{ github.sha }}-03-race-output"
if: env.GIT_DIFF

- name: Save cache 💾
uses: actions/cache@v4
with:
path: ~/go/bin
key: ${{ runner.os }}-go-tparse-binary
if: env.GIT_DIFF

- name: Generate test report (go test -race) 📜
run: cat ./*-race-output.txt | ~/go/bin/tparse
if: env.GIT_DIFF
Loading

0 comments on commit abb683a

Please sign in to comment.