Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

re-enable AMI deploys #446

Draft
wants to merge 11 commits into
base: master
Choose a base branch
from
Draft

re-enable AMI deploys #446

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
d990b05
re-enable AMI deploys
rcooke-warwick Nov 6, 2024
ad6d012
propogate yocto-scripts-version env var to AMI step
rcooke-warwick Nov 11, 2024
dd51fea
propogate aws credentials to AMI step
rcooke-warwick Nov 12, 2024
e1de9d3
assume AWS role if deploy-ami is enabled
rcooke-warwick Nov 12, 2024
6806280
check device type arch correctly
rcooke-warwick Nov 12, 2024
36d3a48
add deploy path to AMI publish image volume
rcooke-warwick Nov 12, 2024
52fb278
fix reference to config app
rcooke-warwick Nov 13, 2024
05a99bd
try using aws session token for auth
rcooke-warwick Nov 13, 2024
ab9f49c
add os version to ami name
rcooke-warwick Nov 15, 2024
2972875
replace "+" chars in AMI names
rcooke-warwick Nov 15, 2024
509d395
DEBUG: try using generic-amd64
rcooke-warwick Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/genericx86-64-ext.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# to the inputs and the defaults.

# TODO: We need a system to keep these inputs aligned across all device repos
name: Generic x86_64 (legacy MBR)
name: Generic x86_64 (GPT)

on:
# With these triggers the Yocto jobs will run
Expand Down Expand Up @@ -46,11 +46,12 @@ jobs:
if: (github.event.pull_request.head.repo.full_name == github.repository) == (github.event_name == 'pull_request')
secrets: inherit
with:
machine: genericx86-64-ext
machine: generic-amd64
deploy-environment: balena-staging.com
# device-repo and device-repo-ref inputs should not be provided on device repos
device-repo: balena-os/balena-intel
device-repo: balena-os/balena-generic
device-repo-ref: master
deploy-ami: true
# Use qemu workers for testing
test_matrix: >
{
Expand Down
169 changes: 91 additions & 78 deletions .github/workflows/yocto-build-deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -637,13 +637,17 @@
# # https://github.com/aws-actions/configure-aws-credentials
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
if: steps.should-deploy.outputs.deploy
if: steps.should-deploy.outputs.deploy || inputs.deploy-ami
id: aws-creds
with:
role-to-assume: ${{ vars.AWS_IAM_ROLE }}
role-session-name: github-${{ github.job }}-${{ github.run_id }}-${{ github.run_attempt }}
aws-region: ${{ vars.AWS_REGION || 'us-east-1' }}
# https://github.com/orgs/community/discussions/26636#discussioncomment-3252664
mask-aws-account-id: false
# For use with AMI publishing step - where we must propogate the credentials to the container
# https://github.com/aws-actions/configure-aws-credentials/tree/e3dd6a429d7300a6a4c196c26e071d42e0343502/?tab=readme-ov-file#retrieving-credentials-from-step-output-assumerole-with-temporary-credentials
output-credentials: true

# "If no keys are provided, but an IAM role is associated with the EC2 instance, it will be used transparently".
# https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/rm.html
Expand Down Expand Up @@ -954,87 +958,96 @@
# AMI Deploy
##############################

# - name: Set AMI arch
# id: ami-arch
# if: inputs.deploy-ami == true
# run: |
# if [ "${dt_arch}" = "amd64" ]; then
# echo "string=x86_64" >>"${GITHUB_OUTPUT}"
# elif [ "${dt_arch}" = "aarch64" ]; then
# echo "string=arm64" >>"${GITHUB_OUTPUT}"
# fi
- name: Set AMI arch
id: ami-arch
if: inputs.deploy-ami == true
env:
DT_ARCH: ${{ steps.balena-lib.outputs.dt_arch }}
run: |
if [ "${DT_ARCH}" = "amd64" ]; then
echo "string=x86_64" >>"${GITHUB_OUTPUT}"

Check failure

Code scanning / octoscan

Write to "$GITHUB_OUTPUT" in a bash script. Error

Write to "$GITHUB_OUTPUT" in a bash script.
elif [ "${DT_ARCH}" = "aarch64" ]; then
echo "string=arm64" >>"${GITHUB_OUTPUT}"

Check failure

Code scanning / octoscan

Write to "$GITHUB_OUTPUT" in a bash script. Error

Write to "$GITHUB_OUTPUT" in a bash script.
fi

# # AMI name format: balenaOS(-installer?)(-secureboot?)-VERSION-DEVICE_TYPE
# - name: Set AMI name
# id: ami-name
# if: inputs.deploy-ami == true
# run: |
# if [ "${{ inputs.sign-image }}" = "true" ]; then
# echo "string=balenaOS-secureboot-${VERSION}-${MACHINE}" >>"${GITHUB_OUTPUT}"
# else
# echo "string=balenaOS-${VERSION}-${MACHINE}" >>"${GITHUB_OUTPUT}"
# fi

# - name: Pull helper image
# id: ami-helper-image
# if: inputs.deploy-ami == true
# env:
# HELPER_IMAGE_REPO: ghcr.io/balena-os/balena-yocto-scripts
# YOCTO_SCRIPTS_VERSION: ${{ steps.balena-lib.outputs.yocto_scripts_version }}
# YOCTO_SCRIPTS_REF: ${{ steps.balena-lib.outputs.yocto_scripts_ref }}
# HELPER_IMAGE_VARIANT: yocto-build-env
# run: |
# image_tag="${HELPER_IMAGE_REPO}:${YOCTO_SCRIPTS_VERSION}-${HELPER_IMAGE_VARIANT}"
# if ! docker pull "${image_tag}"; then
# image_tag="${HELPER_IMAGE_REPO}:${YOCTO_SCRIPTS_REF}-${HELPER_IMAGE_VARIANT}"
# docker pull "${image_tag}"
# fi
- name: Set AMI name
id: ami-name
if: inputs.deploy-ami == true
env:
VERSION: "${{ steps.balena-lib.outputs.os_version }}"
run: |
if [ "${{ inputs.sign-image }}" = "true" ]; then
echo "string=balenaOS-secureboot-${VERSION}-${MACHINE}" | sed 's/+/-/g' >>"${GITHUB_OUTPUT}"

Check failure

Code scanning / octoscan

Write to "$GITHUB_OUTPUT" in a bash script. Error

Write to "$GITHUB_OUTPUT" in a bash script.
else
echo "string=balenaOS-${VERSION}-${MACHINE}" | sed 's/+/-/g' >>"${GITHUB_OUTPUT}"

Check failure

Code scanning / octoscan

Write to "$GITHUB_OUTPUT" in a bash script. Error

Write to "$GITHUB_OUTPUT" in a bash script.
fi

- name: Pull helper image
id: ami-helper-image
if: inputs.deploy-ami == true
env:
HELPER_IMAGE_REPO: ghcr.io/balena-os/balena-yocto-scripts
YOCTO_SCRIPTS_VERSION: ${{ steps.balena-lib.outputs.yocto_scripts_version }}
YOCTO_SCRIPTS_REF: ${{ steps.balena-lib.outputs.yocto_scripts_ref }}
HELPER_IMAGE_VARIANT: yocto-build-env
run: |
image_tag="${HELPER_IMAGE_REPO}:${YOCTO_SCRIPTS_VERSION}-${HELPER_IMAGE_VARIANT}"
if ! docker pull "${image_tag}"; then
image_tag="${HELPER_IMAGE_REPO}:${YOCTO_SCRIPTS_REF}-${HELPER_IMAGE_VARIANT}"
docker pull "${image_tag}"
fi

# image_id="$(docker images --format "{{.ID}}" "${image_tag}")"
# echo "id=${image_id}" >>"${GITHUB_OUTPUT}"
image_id="$(docker images --format "{{.ID}}" "${image_tag}")"
echo "id=${image_id}" >>"${GITHUB_OUTPUT}"

Check failure

Code scanning / octoscan

Write to "$GITHUB_OUTPUT" in a bash script. Error

Write to "$GITHUB_OUTPUT" in a bash script.

# - name: Deploy AMI
# if: inputs.deploy-ami == true
# env:
# AWS_DEFAULT_REGION: "${{ vars.AWS_REGION || 'us-east-1' }}"
# S3_BUCKET: "${{ vars.AWS_S3_BUCKET || vars.S3_BUCKET }}"
# AWS_SESSION_TOKEN: "" # only required if MFA is enabled
# AWS_SUBNET_ID: ${{ vars.AWS_SUBNET || 'subnet-02d18a08ea4058574' }}
# AWS_SECURITY_GROUP_ID: ${{ vars.AWS_SECURITY_GROUP || 'sg-057937f4d89d9d51c' }}
# BALENACLI_TOKEN: ${{ secrets.BALENA_API_DEPLOY_KEY }}
# HOSTOS_VERSION: "${{ steps.balena-lib.outputs.os_version }}"
# AMI_NAME: "${{ steps.ami-name.outputs.string }}"
# AMI_ARCHITECTURE: "${{ steps.ami-arch.outputs.string }}"
# AMI_SECUREBOOT: "${{ inputs.sign-image }}"
# BALENA_PRELOAD_APP: "balena_os/cloud-config-${{ steps.ami-arch.outputs.string }}"
# BALENA_PRELOAD_COMMIT: current
# IMAGE: ${{ runner.temp }}/deploy/image/balena.img
# run: |
# docker run --rm -t \
# --privileged \
# --network host \
# -v "${WORKSPACE}:${WORKSPACE}" \
# -v /var/run/docker.sock:/var/run/docker.sock \
# -e VERBOSE \
# -e AWS_ACCESS_KEY_ID \
# -e AWS_SECRET_ACCESS_KEY \
# -e AWS_DEFAULT_REGION \
# -e AWS_SESSION_TOKEN \
# -e AMI_NAME \
# -e AMI_ARCHITECTURE \
# -e AMI_SECUREBOOT \
# -e S3_BUCKET \
# -e BALENA_PRELOAD_APP \
# -e BALENARC_BALENA_URL \
# -e BALENACLI_TOKEN \
# -e BALENA_PRELOAD_COMMIT \
# -e IMAGE \
# -e MACHINE \
# -e HOSTOS_VERSION \
# -e AWS_SUBNET_ID \
# -e AWS_SECURITY_GROUP_ID \
# -w "${WORKSPACE}" \
# "${{ steps.ami-helper-image.outputs.id }}" /balena-generate-ami.sh
- name: Deploy AMI
if: inputs.deploy-ami == true
env:
AWS_DEFAULT_REGION: "${{ vars.AWS_REGION || 'us-east-1' }}"
S3_BUCKET: "${{ vars.AWS_S3_BUCKET || vars.S3_BUCKET }}"
AWS_SESSION_TOKEN: ${{ steps.aws-creds.outputs.aws-session-token }}
AWS_SUBNET_ID: ${{ vars.AWS_SUBNET || 'subnet-02d18a08ea4058574' }}
AWS_SECURITY_GROUP_ID: ${{ vars.AWS_SECURITY_GROUP || 'sg-057937f4d89d9d51c' }}
BALENACLI_TOKEN: ${{ secrets.BALENA_API_DEPLOY_KEY }}
HOSTOS_VERSION: "${{ steps.balena-lib.outputs.os_version }}"
AMI_NAME: "${{ steps.ami-name.outputs.string }}"
AMI_ARCHITECTURE: "${{ steps.ami-arch.outputs.string }}"
AMI_SECUREBOOT: "${{ inputs.sign-image }}"
BALENA_PRELOAD_APP: "balena_os/cloud-config-${{ steps.balena-lib.outputs.dt_arch }}"
BALENA_PRELOAD_COMMIT: current
BALENA_YOCTO_SCRIPTS_VERSION: ${{ steps.balena-lib.outputs.yocto_scripts_version }}
IMAGE: ${{ env.DEPLOY_PATH }}/image/balena.img
AWS_ACCESS_KEY_ID: ${{ steps.aws-creds.outputs.aws-access-key-id }}
AWS_SECRET_ACCESS_KEY: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
run: |
docker run --rm -t \

Check failure

Code scanning / octoscan

Expression injection, "steps..outputs." is potentially untrusted. Error

Expression injection, "steps.**.outputs.**" is potentially untrusted.
--privileged \
--network host \
-v "${WORKSPACE}:${WORKSPACE}" \
-v "${DEPLOY_PATH}:${DEPLOY_PATH}" \
-v /var/run/docker.sock:/var/run/docker.sock \
-e VERBOSE \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-e AWS_DEFAULT_REGION \
-e AWS_SESSION_TOKEN \
-e AMI_NAME \
-e AMI_ARCHITECTURE \
-e AMI_SECUREBOOT \
-e S3_BUCKET \
-e BALENA_PRELOAD_APP \
-e BALENARC_BALENA_URL \
-e BALENACLI_TOKEN \
-e BALENA_PRELOAD_COMMIT \
-e IMAGE \
-e MACHINE \
-e HOSTOS_VERSION \
-e AWS_SUBNET_ID \
-e AWS_SECURITY_GROUP_ID \
-e BALENA_YOCTO_SCRIPTS_VERSION \
-w "${WORKSPACE}" \
"${{ steps.ami-helper-image.outputs.id }}" /balena-generate-ami.sh

##############################
# Leviathan Test
Expand Down
Loading