Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A0-4268: chain-bootstrapper CI #1720

Merged
merged 21 commits into from
May 20, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .github/actions/run-e2e-test/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ inputs:
description: 'Whether to deploy the adder sample contract to the node.'
required: false
default: 'false'
image-path:
description: 'Custom path to docker image for aleph-node'
aleph-node-image-artifact-name:
description: 'Name of aleph-node image stored in GH Artifacts'
required: false
default: aleph-test-docker
node-image:
description: 'Custom name of aleph-node image'
default: aleph-node-image-test
aleph-node-image-tag:
description: 'Name of aleph-node docker image tag'
required: false
default: aleph-node:latest
compose-file:
Expand Down Expand Up @@ -61,7 +61,7 @@ runs:
- name: Download node docker image
uses: actions/download-artifact@v4
with:
name: ${{ inputs.image-path }}
name: ${{ inputs.aleph-node-image-artifact-name }}

- name: Load node docker image
shell: bash
Expand All @@ -79,7 +79,7 @@ runs:
- name: Run consensus party
shell: bash
run: |
NODE_IMAGE='${{ inputs.node-image }}' \
NODE_IMAGE='${{ inputs.aleph-node-image-tag }}' \
DOCKER_COMPOSE='${{ inputs.compose-file }}' \
./.github/scripts/run_consensus.sh -n '${{ inputs.node-count }}'

Expand Down
37 changes: 17 additions & 20 deletions .github/actions/store-node-and-runtime/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,15 @@ name: Store node and runtime
description: This workflow stores test or release version of aleph-node and aleph-runtime in S3, in
a given bucket
inputs:
aleph-node-artifact-name:
required: true
description: 'Name of artifact aleph-node binary'
aleph-runtime-artifact-name:
required: true
description: 'Name of artifact aleph-runtime'
profile:
required: true
description: test or production
description: 'test or production'
aws-access-key-id:
required: true
description: 'AWS Access Key ID to be used when storing artifacts'
Expand All @@ -27,33 +33,24 @@ runs:
exit 1
fi

- name: Get local envs
id: get-local-envs
shell: bash
env:
CARGO_PROFILE_DIRECTORY: ${{ inputs.profile == 'test' && 'release' || 'production' }}
run: |
cargo_profile_local=${{ env.CARGO_PROFILE_DIRECTORY }}
echo "cargo_profile=$cargo_profile_local" >> $GITHUB_OUTPUT

- name: Checkout aleph-node source code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Call action get-ref-properties
id: get-ref-properties
uses: Cardinal-Cryptography/github-actions/get-ref-properties@v1
uses: Cardinal-Cryptography/github-actions/get-ref-properties@v6

- name: Download binary from GH artifacts
uses: actions/download-artifact@v4
with:
name: aleph-${{ inputs.profile }}-node
path: target/${{ steps.get-local-envs.outputs.cargo_profile }}
name: ${{ inputs.aleph-node-artifact-name }}
path: target

- name: Download runtime from GH artifacts
uses: actions/download-artifact@v4
with:
name: aleph-${{ inputs.profile }}-runtime
path: target/${{ steps.get-local-envs.outputs.cargo_profile }}/wbuild/aleph-runtime
name: ${{ inputs.aleph-runtime-artifact-name }}
path: target

- name: Configure AWS credentials for S3 AWS
uses: aws-actions/configure-aws-credentials@v4
Expand All @@ -68,10 +65,10 @@ runs:
aws-secret-access-key: ${{ inputs.aws-secret-access-key }}
aws-region: ${{ env.AWS_REGION }}

- name: Copy binary to S3 AWS bucket
uses: Cardinal-Cryptography/github-actions/copy-file-to-s3@v1
- name: Copy aleph-node binary to S3 AWS bucket
uses: Cardinal-Cryptography/github-actions/copy-file-to-s3@v6
with:
source-path: target/${{ steps.get-local-envs.outputs.cargo_profile }}
source-path: target
source-filename: aleph-node
s3-bucket-path:
# yamllint disable-line rule:line-length
Expand All @@ -83,7 +80,7 @@ runs:
- name: Copy runtime to S3 AWS bucket
uses: Cardinal-Cryptography/github-actions/copy-file-to-s3@v1
with:
source-path: target/${{ steps.get-local-envs.outputs.cargo_profile }}/wbuild/aleph-runtime
source-path: target
source-filename: aleph_runtime.compact.compressed.wasm
s3-bucket-path:
# yamllint disable-line rule:line-length
Expand Down
91 changes: 91 additions & 0 deletions .github/workflows/_build-aleph-node.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
# This workflow builds aleph-node binary. It outputs name of artifacts uploaded to GH Artifacts.
name: Build aleph-node
on:
workflow_call:
inputs:
ref:
description: 'git ref: hash, branch, tag to build aleph-node binary from'
type: string
required: true
production:
description: 'Set to true to build production binary, otherwise set to false'
type: boolean
required: true
outputs:
artifact-name-binary:
description: 'Name of artifact aleph-node binary'
value: ${{ jobs.main.outputs.artifact-name-binary }}
artifact-name-image:
description: 'Name of artifact aleph-node image'
value: ${{ jobs.main.outputs.artifact-name-image }}
jobs:
main:
name: Build aleph-node (production=${{ inputs.production }})
runs-on: [self-hosted, Linux, X64, large]
env:
RUST_BACKTRACE: full
RUSTC_WRAPPER: sccache
CARGO_FOLDER: ${{ inputs.production == true && 'production' || 'release' }}
ARTIFACT_NAME_SUFFIX: ${{ inputs.production == true && 'production' || 'test' }}
outputs:
artifact-name-binary: ${{ steps.get-artifact-name-binary.outputs.name }}
artifact-name-image: ${{ steps.get-artifact-name-image.outputs.name }}
steps:
- name: Checkout aleph-node source code
uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}
fetch-depth: 0

- name: Call action get-ref-properties
id: get-ref-properties
uses: Cardinal-Cryptography/github-actions/get-ref-properties@v6

- name: Install Rust toolchain
uses: Cardinal-Cryptography/github-actions/install-rust-toolchain@v6

- name: Build test aleph-node
if: ${{ inputs.production != true }}
run: |
cargo build --release -p aleph-node --features only_legacy

- name: Build production aleph-node
if: ${{ inputs.production == true }}
run: cargo build --profile production -p aleph-node

- name: Get binary artifact name
id: get-artifact-name-binary
run: |
echo "name=aleph-node-${{ env.ARTIFACT_NAME_SUFFIX }}" >> $GITHUB_OUTPUT

- name: Upload binary to GH Artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ steps.get-artifact-name-binary.outputs.name }}
path: target/${{ env.CARGO_FOLDER }}/aleph-node
if-no-files-found: error
retention-days: 7

- name: Build aleph-node docker image
run: |
chmod +x target/${{ env.CARGO_FOLDER }}/aleph-node
if [[ ${{ inputs.production }} == true ]]; then
mkdir -p target/release
mv target/production/aleph-node target/release/
fi
docker build --tag aleph-node:latest -f ./docker/Dockerfile .
docker save -o aleph-node.tar aleph-node:latest

- name: Get image artifact name
id: get-artifact-name-image
run: |
echo "name=aleph-node-image-${{ env.ARTIFACT_NAME_SUFFIX }}" >> $GITHUB_OUTPUT

- name: Upload docker to GH Artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ steps.get-artifact-name-image.outputs.name }}
path: aleph-node.tar
if-no-files-found: error
retention-days: 7
76 changes: 0 additions & 76 deletions .github/workflows/_build-and-push-featurenet-node-image.yml

This file was deleted.

27 changes: 23 additions & 4 deletions .github/workflows/_build-chain-bootstrapper.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,13 @@ on:
description: 'set to true to use production profile'
type: boolean
required: true

outputs:
artifact-name-binary:
description: 'Name of artifact chain-bootstrapper binary'
value: ${{ jobs.main.outputs.artifact-name-binary }}
artifact-name-image:
description: 'Name of artifact chain-bootstrapper image'
value: ${{ jobs.main.outputs.artifact-name-image }}
jobs:
main:
name: Build chain-bootstrapper
Expand All @@ -23,6 +29,9 @@ jobs:
RUSTC_WRAPPER: sccache
CARGO_FOLDER: ${{ inputs.production == true && 'production' || 'release' }}
ARTIFACT_NAME_SUFFIX: ${{ inputs.production == true && 'release' || 'test' }}
outputs:
artifact-name-binary: ${{ steps.get-artifact-name-binary.outputs.name }}
artifact-name-image: ${{ steps.get-artifact-name-image.outputs.name }}
steps:
- name: Checkout aleph-node source code
uses: actions/checkout@v4
Expand Down Expand Up @@ -50,29 +59,39 @@ jobs:
run: |
cargo build --profile production -p chain-bootstrapper

- name: Get binary artifact name
id: get-artifact-name-binary
run: |
echo "name=chain-bootstrapper-${{ env.ARTIFACT_NAME_SUFFIX }}" >> $GITHUB_OUTPUT

- name: Upload binary to GH Artifacts
uses: actions/upload-artifact@v4
with:
name: chain-bootstrapper-${{ env.ARTIFACT_NAME_SUFFIX }}
name: ${{ steps.get-artifact-name-binary.outputs.name }}
path: target/${{ env.CARGO_FOLDER }}/chain-bootstrapper
if-no-files-found: error
retention-days: 7

- name: Build docker
id: build-image
run: |
chmod +x target/${{ env.CARGO_FOLDER }}/chain-bootstrapper
chmod +x target/${{ env.CARGO_FOLDER }}/chain-bootstrapper
if [[ ${{ inputs.production }} == true ]]; then
mkdir -p target/release
mv target/production/chain-bootstrapper target/release/
fi
docker build --tag chain-bootstrapper:latest -f ./bin/chain-bootstrapper/Dockerfile .
docker save -o chain-bootstrapper.tar chain-bootstrapper:latest

- name: Get image artifact name
id: get-artifact-name-image
run: |
echo "name=chain-bootstrapper-image-${{ env.ARTIFACT_NAME_SUFFIX }}" >> $GITHUB_OUTPUT

- name: Upload docker image to GH Artifacts
uses: actions/upload-artifact@v4
with:
name: chain-bootstrapper-image-${{ env.ARTIFACT_NAME_SUFFIX }}
name: ${{ steps.get-artifact-name-image.outputs.name }}
path: chain-bootstrapper.tar
if-no-files-found: error
retention-days: 7
Loading
Loading