Skip to content

Commit

Permalink
Currency jenkins to public travis migration and related script changes (
Browse files Browse the repository at this point in the history
ppc64le#3449)

* Travis currency poc 2 (ppc64le#3439)

* Travis currency poc (ppc64le#3314)

* currency build on travis POC code changes ppc64le#3320)

- Reformatted code
- Added code ( python + shell) to match passed version with keys in build_info.json
- Fixed issues about build logging
- Parallel jobs  for package and docker build
- Renamed some variables in standard format
- updated read_buildinfo.sh to process and generate additional variables

* Created a new folder travis-currency-ymls and added travis currency yml files into it (ppc64le#3330)

* created a directory for currency yml files

* updated travis config files name

* updated .travis.yml file

* updated file name

* updated travis file

* modified file name

* Update currency-build.yml

* Update validate_builds_currency.py

* Update build_package.sh

* Update build_package.sh

* Update validate_builds_currency.py

* Update validate_builds_currency.py

* Update currency-build.yml

* Create trivy_code_scan.sh

* Travis currency poc (ppc64le#3373)

* Update currency-build.yml

* Update validate_builds_currency.py

* Update build_package.sh

* Update build_package.sh

* Update validate_builds_currency.py

* Update validate_builds_currency.py

* Update currency-build.yml

* Create trivy_code_scan.sh

* Travis pre process and remaining scanner stages(syft and grype) (#1)

* added pre-process stage

* Added pre_process.sh file and also updated the code in currency-build.yml

* Update currency-build.yml

* Update pre_process.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Create syft_code_scan.sh

* Create grype_code_scan

* Update syft_code_scan.sh

* Update currency-build.yml

* Update pre_process.sh

* Rename grype_code_scan to grype_code_scan.sh

* Update grype_code_scan.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update currency-build.yml

* Update pre_process.sh

* Travis currency poc (ppc64le#3380)

* Update currency-build.yml

* Update validate_builds_currency.py

* Update build_package.sh

* Update build_package.sh

* Update validate_builds_currency.py

* Update validate_builds_currency.py

* Update currency-build.yml

* Create trivy_code_scan.sh

* Travis pre process and remaining scanner stages(syft and grype) (#1)

* added pre-process stage

* Added pre_process.sh file and also updated the code in currency-build.yml

* Update currency-build.yml

* Update pre_process.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Create syft_code_scan.sh

* Create grype_code_scan

* Update syft_code_scan.sh

* Update currency-build.yml

* Update pre_process.sh

* Rename grype_code_scan to grype_code_scan.sh

* Update grype_code_scan.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update currency-build.yml

* Update pre_process.sh

* Update validate_builds_currency.py

* Update read_buildinfo.sh

* resolved merge conflicts

* currency build on travis POC code changes ppc64le#3320)

- Reformatted code
- Added code ( python + shell) to match passed version with keys in build_info.json
- Fixed issues about build logging
- Parallel jobs  for package and docker build
- Renamed some variables in standard format
- updated read_buildinfo.sh to process and generate additional variables

* Created a new folder travis-currency-ymls and added travis currency yml files into it (ppc64le#3330)

* created a directory for currency yml files

* updated travis config files name

* updated .travis.yml file

* updated file name

* updated travis file

* modified file name

* Update currency-build.yml

* Update validate_builds_currency.py

* Update build_package.sh

* Update build_package.sh

* Update validate_builds_currency.py

* Update validate_builds_currency.py

* Update currency-build.yml

* Create trivy_code_scan.sh

* Travis pre process and remaining scanner stages(syft and grype) (#1)

* added pre-process stage

* Added pre_process.sh file and also updated the code in currency-build.yml

* Update currency-build.yml

* Update pre_process.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Create syft_code_scan.sh

* Create grype_code_scan

* Update syft_code_scan.sh

* Update currency-build.yml

* Update pre_process.sh

* Rename grype_code_scan to grype_code_scan.sh

* Update grype_code_scan.sh

* resolved conflicts

* Update currency-build.yml

* Update trivy_code_scan.sh

* Update grype_code_scan.sh

* Update syft_code_scan.sh

* Create upload_build_log.sh

* Create dockerfile_non_root

* Update read_buildinfo.sh

* Update build_package.sh

* Update validate_builds_currency.py

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

---------

Co-authored-by: nikhil-kalbande <[email protected]>

* Currency travis migration (ppc64le#3447)

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update read_buildinfo.sh

* Update currency-build.yml

* Update currency-build.yml

* Create upload_docker_image.sh

* Update currency-build.yml

* Update currency-build.yml

* Update read_buildinfo.sh

* Update build_docker.sh

* Update build_package.sh

* Update grype_code_scan.sh

* Update grype_scan.sh

* Update pre_process.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update upload_docker_image.sh

* Update currency-build.yml

* Update syft_scan.sh

* Update trivy_scan.sh

* Update upload_build_log.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Currency travis migration scanner (#2)

* Update upload_build_log.sh

* Update currency-build.yml

* Update currency-build.yml

* Create upload_source_code_scanner.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update trivy_code_scan.sh

* Update trivy_code_scan.sh

* Update currency-build.yml

* Update grype_code_scan.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update upload_source_code_scanner.sh

* Update currency-build.yml

* Update grype_code_scan.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update currency-build.yml

* Update upload_source_code_scanner.sh

* Update currency-build.yml

* Update and rename trivy_scan.sh to trivy_image_scan.sh

* Update currency-build.yml

* Update and rename upload_source_code_scanner.sh to upload_scanner_results.sh

* Update currency-build.yml

* Update currency-build.yml

* Update and rename grype_scan.sh to grype_image_scan.sh

* Update grype_image_scan.sh

* Update syft_scan.sh

* Rename syft_scan.sh to syft_image_scan.sh

* Update currency-build.yml

* Update currency-build.yml

* Update grype_image_scan.sh

* Update trivy_code_scan.sh

* Update grype_code_scan.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update trivy_image_scan.sh

* Update syft_image_scan.sh

* Update currency-build.yml

* Update pre_process.sh

* Update currency-build.yml

* Currency travis migration (ppc64le#3454)

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update read_buildinfo.sh

* Update currency-build.yml

* Update currency-build.yml

* Create upload_docker_image.sh

* Update currency-build.yml

* Update currency-build.yml

* Update read_buildinfo.sh

* Update build_docker.sh

* Update build_package.sh

* Update grype_code_scan.sh

* Update grype_scan.sh

* Update pre_process.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update upload_docker_image.sh

* Update currency-build.yml

* Update syft_scan.sh

* Update trivy_scan.sh

* Update upload_build_log.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Currency travis migration scanner (#2)

* Update upload_build_log.sh

* Update currency-build.yml

* Update currency-build.yml

* Create upload_source_code_scanner.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update trivy_code_scan.sh

* Update trivy_code_scan.sh

* Update currency-build.yml

* Update grype_code_scan.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update upload_source_code_scanner.sh

* Update currency-build.yml

* Update grype_code_scan.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update currency-build.yml

* Update upload_source_code_scanner.sh

* Update currency-build.yml

* Update and rename trivy_scan.sh to trivy_image_scan.sh

* Update currency-build.yml

* Update and rename upload_source_code_scanner.sh to upload_scanner_results.sh

* Update currency-build.yml

* Update currency-build.yml

* Update and rename grype_scan.sh to grype_image_scan.sh

* Update grype_image_scan.sh

* Update syft_scan.sh

* Rename syft_scan.sh to syft_image_scan.sh

* Update currency-build.yml

* Update currency-build.yml

* Update grype_image_scan.sh

* Update trivy_code_scan.sh

* Update grype_code_scan.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update trivy_image_scan.sh

* Update syft_image_scan.sh

* Update currency-build.yml

* Update pre_process.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update validate_builds_currency.py

* Update currency-build.yml

* Update currency-build.yml

* Update build_package.sh

* Update validate_builds_currency.py

* Update currency-build.yml

* Update currency-build.yml

* Update build_package.sh

* Update validate_builds_currency.py

* Update currency-build.yml

* Currency travis migration (ppc64le#3486)

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update read_buildinfo.sh

* Update currency-build.yml

* Update currency-build.yml

* Create upload_docker_image.sh

* Update currency-build.yml

* Update currency-build.yml

* Update read_buildinfo.sh

* Update build_docker.sh

* Update build_package.sh

* Update grype_code_scan.sh

* Update grype_scan.sh

* Update pre_process.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update upload_docker_image.sh

* Update currency-build.yml

* Update syft_scan.sh

* Update trivy_scan.sh

* Update upload_build_log.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update currency-build.yml

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Update pre_process.sh

* Currency travis migration scanner (#2)

* Update upload_build_log.sh

* Update currency-build.yml

* Update currency-build.yml

* Create upload_source_code_scanner.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update trivy_code_scan.sh

* Update trivy_code_scan.sh

* Update currency-build.yml

* Update grype_code_scan.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update upload_source_code_scanner.sh

* Update currency-build.yml

* Update grype_code_scan.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update currency-build.yml

* Update upload_source_code_scanner.sh

* Update currency-build.yml

* Update and rename trivy_scan.sh to trivy_image_scan.sh

* Update currency-build.yml

* Update and rename upload_source_code_scanner.sh to upload_scanner_results.sh

* Update currency-build.yml

* Update currency-build.yml

* Update and rename grype_scan.sh to grype_image_scan.sh

* Update grype_image_scan.sh

* Update syft_scan.sh

* Rename syft_scan.sh to syft_image_scan.sh

* Update currency-build.yml

* Update currency-build.yml

* Update grype_image_scan.sh

* Update trivy_code_scan.sh

* Update grype_code_scan.sh

* Update syft_code_scan.sh

* Update trivy_code_scan.sh

* Update trivy_image_scan.sh

* Update syft_image_scan.sh

* Update currency-build.yml

* Update pre_process.sh

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update validate_builds_currency.py

* Update currency-build.yml

* Update currency-build.yml

* Update build_package.sh

* Update validate_builds_currency.py

* Update currency-build.yml

* Update currency-build.yml

* Update build_package.sh

* Update validate_builds_currency.py

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update validate_builds_currency.py

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update validate_builds_currency.py

* Update currency-build.yml

* Update currency-build.yml

* Update currency-build.yml

* Update read_buildinfo.sh

* re-arranged the scripts in folders

* Update build_docker.sh

* Currency travis migration (ppc64le#3502)

* Update upload_scanner_results.sh

* Update upload_scanner_results.sh

* Update upload_scanner_results.sh

* Update upload_scanner_results.sh

* Update upload_scanner_results.sh

* Update upload_scanner_results.sh

* Update upload_scanner_results.sh

* Update upload_scanner_results.sh

* Rename upload_scanner_results.sh to upload_logs.sh

* Update upload_logs.sh

* Update currency-build.yml

* Delete script/upload-scripts/upload_build_log.sh

* Update currency-build.yml

* Update upload_docker_image.sh

* Update currency-build.yml

* Update upload_logs.sh

* Update upload_docker_image.sh

* Update upload_docker_image.sh

* Update upload_docker_image.sh

* Update upload_logs.sh

* Update upload_docker_image.sh

* Update currency-build.yml

* Rename upload_logs.sh to upload_file.sh

* Update currency-build.yml

---------

Co-authored-by: saiNammi <[email protected]>
  • Loading branch information
nikhil-kalbande and saiNammi authored Jan 2, 2024
1 parent a6c450f commit 246f845
Show file tree
Hide file tree
Showing 18 changed files with 830 additions and 18 deletions.
23 changes: 5 additions & 18 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,5 @@
---
arch: ppc64le
dist: focal
os: linux
language: shell

services:
- docker

before_install:
- docker pull registry.access.redhat.com/ubi8/ubi:8.7

script:
- sudo apt update -y && sudo apt-get install file -y
- pip3 install --upgrade requests
- pip3 install --upgrade docker
- python3 script/validate_builds.py $TRAVIS_PULL_REQUEST &
- SCRIPT_PID=$!;while ps -p $SCRIPT_PID > /dev/null;do echo "$SCRIPT_PID is running"; sleep 300; done; wait $SCRIPT_PID; my_pid_status=$?; travis_terminate $my_pid_status
import:
- source: travis-currency-ymls/pr-build.yml
if: type = pull_request
- source: travis-currency-ymls/currency-build.yml
if: type != pull_request
64 changes: 64 additions & 0 deletions script/build_docker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/bin/bash -e

version="$VERSION"
package_dirpath="$PKG_DIR_PATH"
config_file="build_info.json"
image_name=$IMAGE_NAME
build_docker=$BUILD_DOCKER

CUR_DIR=$(pwd)
cd $package_dirpath

# Using python script to find matched version string/key in build_info.json for version passed
match_version=$(python $CUR_DIR/script/match_version_buildinfo.py)

if [ $build_docker != false ];then
if [[ $(jq --arg ver "$match_version" '.[$ver]' $config_file) != null ]]; then
docker_builddir=$(jq -r --arg ver "$match_version" '.[$ver].dir' $config_file)
args=$(jq -r --arg ver "$match_version" '.[$ver].args' $config_file)
patches=$(jq -r --arg ver "$match_version" '.[$ver].patches' $config_file)
# By default send PACKAGE_VERSION argument.
build_args ="--build-arg PACKAGE_VERSION=$version"
if [ $args != null ]; then
for row in $(echo "$args" | jq -r 'to_entries[] | @base64'); do
key=$(echo "$row" | base64 -d | jq -r '.key')
value=$(echo "$row" | base64 -d | jq -r '.value')
build_args=$(echo $build_args --build-arg $key=$value )
done
fi
if [ $patches != null ]; then
for row in $(echo "$patches" | jq -r 'to_entries[] | @base64'); do
key=$(echo "$row" | base64 -d | jq -r '.key')
value=$(echo "$row" | base64 -d | jq -r '.value')
build_args=$(echo $build_args --build-arg $key=$value )
done
fi
if [[ $(jq --arg ver "$match_version" '.[$ver]' $config_file) != null ]] &&
[[ $(jq -r --arg ver "$match_version" '.[$ver].base_docker_image' $config_file) != null ]]; then
basename=$(jq -r --arg ver "$match_version" '.[$ver].base_docker_image' $config_file)
fi
cmd="$build_args -t $image_name $docker_builddir"
#final_upload_image_link=$(DOCKER_UPLOAD_LINK)/$image_name
docker_file_path="${package_dirpath}/Dockerfiles"
fi

cd Dockerfiles
echo "Deleting existing docker image"
docker rmi -f ${image_name}
#docker rmi -f ${basename}
echo "Building docker image"
echo "sudo docker build $build_args -t $image_name $docker_builddir"
echo "*************************************************************************************"
sudo docker build $build_args -t $image_name $docker_builddir
docker save -o "$HOME/build/$TRAVIS_REPO_SLUG/image.tar" $image_name
else
echo "Docker image is not supported"
fi

# # Publish code keeping commented for now
# if [ $? == 0 ]
# then
# sudo docker tag ${image_name} ${docker_upload_link}/${image_name}
# sudo docker login -u ${env.dockerHubUser} -p ${env.dockerHubPassword} ${docker_upload_link}
# sudo docker push ${docker_upload_link}/${image_name}
# fi
53 changes: 53 additions & 0 deletions script/build_package.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/bin/bash -e

sudo apt update -y && sudo apt-get install file -y
docker pull registry.access.redhat.com/ubi8/ubi:8.7
pip3 install --upgrade requests
pip3 install --upgrade docker

echo "Running build script execution in background for "$PKG_DIR_PATH$BUILD_SCRIPT" "$VERSION" "
echo "*************************************************************************************"

docker_image="registry.access.redhat.com/ubi8/ubi:8.7"
if [ "$NON_ROOT_BUILD" == "true" ];
then
echo "building docker image for non root user build"
docker build -t docker_non_root_image -f script/dockerfile_non_root .
docker_image="docker_non_root_image"
fi

python3 script/validate_builds_currency.py "$PKG_DIR_PATH$BUILD_SCRIPT" "$VERSION" "$docker_image" > build_log &

SCRIPT_PID=$!
while ps -p $SCRIPT_PID > /dev/null
do
echo "$SCRIPT_PID is running"
sleep 100
done
wait $SCRIPT_PID
my_pid_status=$?
build_size=$(stat -c %s build_log)

if [ $my_pid_status != 0 ];
then
echo "Script execution failed for "$PKG_DIR_PATH$BUILD_SCRIPT" "$VERSION" "
echo "*************************************************************************************"
if [ $build_size -lt 1800000 ];
then
cat build_log
else
tail -100 build_log
fi
exit 1
else
echo "Script execution completed successfully for "$PKG_DIR_PATH$BUILD_SCRIPT" "$VERSION" "
echo "*************************************************************************************"
if [ $build_size -lt 1800000 ];
then
cat build_log
else
tail -100 build_log
fi
fi
exit 0

6 changes: 6 additions & 0 deletions script/dockerfile_non_root
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
FROM registry.access.redhat.com/ubi8/ubi:8.7
RUN yum install -y sudo
RUN echo 'test_user ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers.d/test_user
RUN useradd -d /home/tester -m -s /bin/bash test_user
USER test_user
WORKDIR /home/tester
21 changes: 21 additions & 0 deletions script/match_version_buildinfo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import json
import os
import re
version = str(os.environ['VERSION'])
match_version=""
f = open('build_info.json')
data = json.load(f)
for key,value in data.items():
subKeys = [subKey.strip() for subKey in key.split(',')]
if version in subKeys:
match_version = key
break
else:
for subKey in subKeys:
regex_str = '^' + subKey.replace(".", "\\.").replace("*", ".*") + '$'
regex = re.compile(regex_str)
if regex.match(version):
match_version = key
if len(match_version) != 0:
break
print(match_version)
34 changes: 34 additions & 0 deletions script/pre_process.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#!/bin/bash -e
actual_package_name=$(awk -F'/' 'tolower($0) ~ /^# source repo.*github.com/{sub(/\.git/, "", $NF); print $NF}' $HOME/build/$TRAVIS_REPO_SLUG/$PKG_DIR_PATH$BUILD_SCRIPT)

cd $actual_package_name

IFS=',' read -ra langs <<< "$Languages"

for language in "${langs[@]}"; do
if [ "$language" == "python" ]; then
touch final-requirements
find ./ -type f -name '*requirements*.txt' -exec cat {} + >> final-requirements
mv final-requirements requirements.txt
elif [ "$language" == "javascript" ] || [ "$language" == "typescript" ]; then
nvm_path='/home/travis/.nvm/nvm.sh'
if [ -f "package-lock.json" ] || [ -f "yarn.lock" ]; then
sudo chown travis:travis -R .
echo '
if [ -f ${nvm_path} ]; then
source ${nvm_path}
else
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
source ~/.bashrc
nvm install 16
fi
npm install -g yarn
yarn import || true
' > generate.sh
chmod +x generate.sh
sudo ./generate.sh
sudo rm -rf node_modules/ package-lock.json
fi
fi
done

97 changes: 97 additions & 0 deletions script/read_buildinfo.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
#!/bin/bash -e
files_upload_link='https://163.69.91.4:8443/repository/currency-artifacts/docker-details/local/'
package_name=$(echo $PACKAGE_NAME | tr '[:upper:]' '[:lower:]')
image_name="icr.io/ppc64le-oss/$package_name-ppc64le:$VERSION"
url_prefix=${files_upload_link}${package_name}'-ppc64le/'${VERSION}
initial_char=${package_name:0:1}
package_dirpath="$initial_char/$package_name/"
buildinfo_path=$package_dirpath'build_info.json'
match_version=$VERSION


if [ -f $buildinfo_path ]; then
echo $package_dirpath 'exists'
else
package_dirpath="$initial_char/$PACKAGE_NAME"
echo "Correct package_dirpath is $package_dirpath"
fi

CUR_DIR=$(pwd)
cd $package_dirpath
echo "printing the list of contents"
pwd
ls -ltr
config_file='build_info.json'
if [ -f $config_file ]; then

jsonObj=$config_file
build_script=$(jq .build_script $jsonObj)

if $(jq 'has("use_non_root_user")' $jsonObj); then
nonRootBuild=$(jq .use_non_root_user $jsonObj)
fi

#default build_docker=true
build_docker=true
if $(jq 'has("docker_build")' $jsonObj); then
build_docker=$(jq .docker_build $jsonObj)
fi

#default validate_build_script=true
validate_build_script=true
if $(jq 'has("validate_build_script")' $jsonObj); then
validate_build_script=$(jq .validate_build_script $jsonObj)
fi
echo "Checking for string/pattern match for version in build_info.json"

if [[ $(jq --arg ver $VERSION '.[$ver]' $config_file) == null ]]; then
# Using python script to find matched version string/key in build_info.json for version passed
match_version=$(python $CUR_DIR/script/match_version_buildinfo.py)
echo "match_version = $match_version"

fi
#Getting specific build_script name for version
if [[ $(jq --arg ver "$match_version" '.[$ver]' $config_file) != null ]]; then
if [[ $(jq -r --arg ver "$match_version" '.[$ver].build_script' $config_file) != null ]]; then
build_script=$(jq -r --arg ver "$match_version" '.[$ver].build_script' $config_file)
fi

if [[ $(jq -r --arg ver "$match_version" '.[$ver].base_docker_image' $config_file) != null ]]; then
basename=$(jq -r --arg ver "$match_version" '.[$ver].base_docker_image' $config_file)
fi

if [[ $(jq -r --arg ver "$match_version" '.[$ver].base_docker_variant' $config_file) != null ]]; then
variant_str=$(jq -r --arg ver "$match_version" '.[$ver].base_docker_variant' $config_file)
case "$variant_str" in
"rhel")
variant=1
;;
"ubuntu")
variant=2
;;
"alpine")
variant=3
;;
*)
echo "No valid distro variant, picking default one"
variant=1
;;
esac
fi
fi
fi


echo "export VERSION=$VERSION" > $CUR_DIR/variable.sh
echo "export BUILD_SCRIPT=$build_script" >> $CUR_DIR/variable.sh
echo "export PKG_DIR_PATH=$package_dirpath" >> $CUR_DIR/variable.sh
echo "export IMAGE_NAME=$image_name" >> $CUR_DIR/variable.sh
#echo "export BUILD_DOCKER=$build_docker" >> $CUR_DIR/variable.sh
echo "export VALIDATE_BUILD_SCRIPT=$validate_build_script" >> $CUR_DIR/variable.sh
echo "export VARIANT=$variant" >> $CUR_DIR/variable.sh
echo "export BASENAME=$basename" >> $CUR_DIR/variable.sh
echo "export NON_ROOT_BUILD=$nonRootBuild" >> $CUR_DIR/variable.sh

chmod +x $CUR_DIR/variable.sh
cat $CUR_DIR/variable.sh
cd $CUR_DIR
15 changes: 15 additions & 0 deletions script/scanner-scripts/grype_code_scan.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash -e

validate_build_script=$VALIDATE_BUILD_SCRIPT
actual_package_name=$(awk -F'/' 'tolower($0) ~ /^# source repo.*github.com/{sub(/\.git/, "", $NF); print $NF}' $PKG_DIR_PATH$BUILD_SCRIPT)

cd package-cache

if [ $validate_build_script == true ];then
wget https://github.com/anchore/grype/releases/download/v0.67.0/grype_0.67.0_linux_ppc64le.tar.gz
tar -xf grype_0.67.0_linux_ppc64le.tar.gz
chmod +x grype
sudo mv grype /usr/bin
sudo grype -q -o cyclonedx-json dir:${actual_package_name} > grype_source_sbom_results.json
sudo grype -q -o json dir:${actual_package_name} > grype_source_vulnerabilities_results.json
fi
16 changes: 16 additions & 0 deletions script/scanner-scripts/grype_image_scan.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash -e

image_name=$IMAGE_NAME
build_docker=$BUILD_DOCKER

cd package-cache

if [ $build_docker == true ];then
wget https://github.com/anchore/grype/releases/download/v0.67.0/grype_0.67.0_linux_ppc64le.tar.gz
tar -xf grype_0.67.0_linux_ppc64le.tar.gz
chmod +x grype
sudo mv grype /usr/bin
sudo grype -q -s AllLayers -o cyclonedx-json ${image_name} > grype_image_sbom_results.json
sudo grype -q -s AllLayers -o json ${image_name} > grype_image_vulnerabilities_results.json
fi

15 changes: 15 additions & 0 deletions script/scanner-scripts/syft_code_scan.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash -e

validate_build_script=$VALIDATE_BUILD_SCRIPT
actual_package_name=$(awk -F'/' 'tolower($0) ~ /^# source repo.*github.com/{sub(/\.git/, "", $NF); print $NF}' $PKG_DIR_PATH$BUILD_SCRIPT)

cd package-cache

if [ $validate_build_script == true ];then
wget https://github.com/anchore/syft/releases/download/v0.90.0/syft_0.90.0_linux_ppc64le.tar.gz
tar -xf syft_0.90.0_linux_ppc64le.tar.gz
chmod +x syft
sudo mv syft /usr/bin
sudo syft -q -o cyclonedx-json dir:${actual_package_name} > syft_source_sbom_results.json
fi

16 changes: 16 additions & 0 deletions script/scanner-scripts/syft_image_scan.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash -e

image_name=$IMAGE_NAME
build_docker=$BUILD_DOCKER

cd package-cache


if [ $build_docker == true ];then
wget https://github.com/anchore/syft/releases/download/v0.90.0/syft_0.90.0_linux_ppc64le.tar.gz
tar -xf syft_0.90.0_linux_ppc64le.tar.gz
chmod +x syft
sudo mv syft /usr/bin
sudo syft -q -s AllLayers -o cyclonedx-json ${image_name} > syft_image_sbom_results.json
fi

16 changes: 16 additions & 0 deletions script/scanner-scripts/trivy_code_scan.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash -e

validate_build_script=$VALIDATE_BUILD_SCRIPT
actual_package_name=$(awk -F'/' 'tolower($0) ~ /^# source repo.*github.com/{sub(/\.git/, "", $NF); print $NF}' $PKG_DIR_PATH$BUILD_SCRIPT)

cd package-cache

if [ $validate_build_script == true ];then
wget https://github.com/aquasecurity/trivy/releases/download/v0.45.0/trivy_0.45.0_Linux-PPC64LE.tar.gz
tar -xf trivy_0.45.0_Linux-PPC64LE.tar.gz
chmod +x trivy
sudo mv trivy /usr/bin
sudo trivy -q fs --timeout 30m -f json ${actual_package_name} > trivy_source_vulnerabilities_results.json
sudo trivy -q fs --timeout 30m -f cyclonedx ${actual_package_name} > trivy_source_sbom_results.cyclonedx
fi

Loading

0 comments on commit 246f845

Please sign in to comment.