Script Update to include latest Exclude files for JDK versions #4897

KapilPowar · 2023-12-04T12:59:46Z

runtimes/backlog/issues/1241

Script Update to include latest exclude files from artifactory for JDK versions
CC: @llxia

KapilPowar · 2023-12-04T13:01:38Z

Here are the Grinder test runs
JCK_Sync #912 -- Latest exclude file available
JCK_Sync #909 -- Latest exclude file not available
JCK_Sync #913 -- No Exclude file present in Artifactory

buildenv/jenkins/jck_sync

jck/jck-semiauto-updater/jckupdater.sh

llxia · 2023-12-04T19:09:55Z

jck/jck-semiauto-updater/jckupdater.sh

+}
+
+getExcludeFiles(){
+	github_last_modified=$(curl -s -H "Authorization: token $GIT_TOKEN" "https://api.github.ibm.com/repos/runtimes/JCK$JCK_VERSION-unzipped/commits?path=excludes/jck$JCK_FOLDER_SUFFIX.jtx" | awk -F'"date":' '{if(NF>1){gsub(/[",]/,"",$2); print $2; exit}}')


Please do not hardcode a specific repo

It is not limited to one jck$JCK_FOLDER_SUFFIX.jtx. There could be other files.

In artifactory material we have only .jtx file in /exc . So we are comparing last modified of .jtx file from Git repo with artifactory material to decide if update is needed or not.

The script is also used by other vendors (e.g. Adoptium), not only ibm, so the link repo cannot be hardcoded.

Created a parameter GIT_EXCLUDE_API_URL to provide API url in JCK_Sync job.
JCK_Sync#928

jck/jck-semiauto-updater/jckupdater.sh

KapilPowar · 2023-12-11T11:43:51Z

JCK_Sync #924 -- Latest exclude file not available

JCK_Sync #921 -- Latest exclude file available

JCK_Sync #925 -- No Exclude file present in Artifactory

LongyuZhang

Also, when checking this pipeline, the result of 924 and 925 are red, but if the material is already up to date the job should be green.

buildenv/jenkins/jck_sync

jck/jck-semiauto-updater/jckupdater.sh

LongyuZhang · 2023-12-18T23:45:14Z

jck/jck-semiauto-updater/jckupdater.sh

+}
+
+getExcludeFiles(){
+	github_last_modified=$(curl -s -H "Authorization: token $GIT_TOKEN" "https://api.github.ibm.com/repos/runtimes/JCK$JCK_VERSION-unzipped/commits?path=excludes/jck$JCK_FOLDER_SUFFIX.jtx" | awk -F'"date":' '{if(NF>1){gsub(/[",]/,"",$2); print $2; exit}}')


The script is also used by other vendors (e.g. Adoptium), not only ibm, so the link repo cannot be hardcoded.

jck/jck-semiauto-updater/jckupdater.sh

KapilPowar · 2023-12-19T04:45:25Z

Also, when checking this pipeline, the result of 924 and 925 are red, but if the material is already up to date the job should be green.

In case of no new update is available, we are using Error condition to break out of flow and marking job as success. This is done as per @llxia suggestion.

llxia · 2023-12-29T16:35:36Z

jck/jck-semiauto-updater/jckupdater.sh

-		echo " JCK$JCK_VERSION $JCK_WITHOUT_BACKSLASH is latest and not in the repo $GIT_URL... Please proceed with download"
-		get_JAVA_SDK
+	else
+		if [ $result -ne 0 ]; then


In this PR, result and exclude_files are global variables and they are used in multiple places. And $result != 0 means we need to update the test material. Similarly, $exclude_files != 0 means we need to update the exclude files. I find it isn't very clear.

Please define boolean parameters isTestMaterialUpdateNeeded and isTestExcludeUpdateNeeded instead.

Using boolean parameters isTestMaterialUpdateNeeded and isTestExcludeUpdateNeeded.
isTestMaterialUpdateNeeded == 0 means we need to update the test material. Using 0 as true and 1 as false.

llxia · 2023-12-29T17:12:12Z

jck/jck-semiauto-updater/jckupdater.sh

 	fi
 }

 #install Java
-get_JAVA_SDK(){
+getJavaSDK() {


This is misleading. The function does get SDK or install java. It just tries to set JAVA_SDK_PATH=$WORKSPACE/../../../../jdkbinary/j2sdk-image if JAVA_HOME does not exist. And JAVA_HOME is a user-provided value.

aqa-tests/jck/jck-semiauto-updater/jckupdater.sh

Line 52 in a04569c

JAVA_HOME="$1"; shift;;

Please remove this function completely. Instead of

aqa-tests/jck/jck-semiauto-updater/jckupdater.sh

Line 52 in a04569c

JAVA_HOME="$1"; shift;;

, user can provide JAVA_SDK_PATH. If not, the default is $WORKSPACE/../../../../jdkbinary/j2sdk-image.
Also remove L282 to 287, change it to $JAVA_SDK_PATH/bin/java -jar $f -install shell_scripts -o $WORKSPACE/unpackjck

Changes done as per review. Removed get_JAVA_SDK().
$JAVA_SDK_PATH is the default unless user provides value.

jck/jck-semiauto-updater/jckupdater.sh

KapilPowar · 2024-01-08T11:36:13Z

JCK_Sync #939 -- Latest exclude file not available

JCK_Sync #937 -- Latest exclude file available

JCK_Sync #940 -- No Exclude file present in Artifactory

KapilPowar · 2024-01-16T05:40:15Z

Until further notice PR is moved to Draft.

Update Exclude files - Update script to include latest exclude files from artifactory for JDK versions Signed-off-by: Kapil Powar <[email protected]>

KapilPowar · 2024-01-23T09:02:28Z

JCK_Sync #1046

JCK_Sync #1047

KapilPowar · 2024-01-29T05:10:38Z

Moved to internal repo. Closing it.

KapilPowar changed the title ~~Update latest Exclude files for JDK versions~~ Script Update to include latest Exclude files for JDK versions Dec 4, 2023

KapilPowar force-pushed the ExcludeFiles branch from 67d2a5b to 5e143d4 Compare December 4, 2023 13:07

llxia requested changes Dec 4, 2023

View reviewed changes

llxia requested a review from LongyuZhang December 4, 2023 19:19

KapilPowar force-pushed the ExcludeFiles branch 4 times, most recently from 28065f6 to 5aabfa5 Compare December 11, 2023 11:06

KapilPowar requested a review from llxia December 11, 2023 11:46

LongyuZhang reviewed Dec 19, 2023

View reviewed changes

KapilPowar force-pushed the ExcludeFiles branch 2 times, most recently from 80b6a9a to a04569c Compare December 19, 2023 10:44

llxia requested changes Dec 29, 2023

View reviewed changes

KapilPowar force-pushed the ExcludeFiles branch from 538ebbe to ab27ca8 Compare January 8, 2024 11:36

KapilPowar requested a review from llxia January 8, 2024 11:37

KapilPowar force-pushed the ExcludeFiles branch from 6380429 to 9ef945d Compare January 16, 2024 05:38

KapilPowar marked this pull request as draft January 16, 2024 05:40

KapilPowar force-pushed the ExcludeFiles branch from e660083 to 8b930b0 Compare January 23, 2024 07:09

Update Exclude files

58bd207

Update Exclude files - Update script to include latest exclude files from artifactory for JDK versions Signed-off-by: Kapil Powar <[email protected]>

KapilPowar force-pushed the ExcludeFiles branch from 8b930b0 to 58bd207 Compare January 23, 2024 08:54

KapilPowar marked this pull request as ready for review January 23, 2024 09:02

KapilPowar closed this Jan 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Script Update to include latest Exclude files for JDK versions #4897

Script Update to include latest Exclude files for JDK versions #4897

KapilPowar commented Dec 4, 2023

KapilPowar commented Dec 4, 2023

llxia Dec 4, 2023

KapilPowar Dec 11, 2023

LongyuZhang Dec 18, 2023

KapilPowar Dec 19, 2023 •

edited by llxia

Loading

KapilPowar commented Dec 11, 2023 •

edited by llxia

Loading

LongyuZhang left a comment

LongyuZhang Dec 18, 2023

KapilPowar commented Dec 19, 2023

llxia Dec 29, 2023

KapilPowar Jan 8, 2024 •

edited

Loading

llxia Dec 29, 2023

KapilPowar Jan 8, 2024

KapilPowar commented Jan 8, 2024 •

edited by llxia

Loading

KapilPowar commented Jan 16, 2024

KapilPowar commented Jan 23, 2024 •

edited by smlambert

Loading

KapilPowar commented Jan 29, 2024

Script Update to include latest Exclude files for JDK versions #4897

Script Update to include latest Exclude files for JDK versions #4897

Conversation

KapilPowar commented Dec 4, 2023

KapilPowar commented Dec 4, 2023

llxia Dec 4, 2023

Choose a reason for hiding this comment

KapilPowar Dec 11, 2023

Choose a reason for hiding this comment

LongyuZhang Dec 18, 2023

Choose a reason for hiding this comment

KapilPowar Dec 19, 2023 • edited by llxia Loading

Choose a reason for hiding this comment

KapilPowar commented Dec 11, 2023 • edited by llxia Loading

LongyuZhang left a comment

Choose a reason for hiding this comment

LongyuZhang Dec 18, 2023

Choose a reason for hiding this comment

KapilPowar commented Dec 19, 2023

llxia Dec 29, 2023

Choose a reason for hiding this comment

KapilPowar Jan 8, 2024 • edited Loading

Choose a reason for hiding this comment

llxia Dec 29, 2023

Choose a reason for hiding this comment

KapilPowar Jan 8, 2024

Choose a reason for hiding this comment

KapilPowar commented Jan 8, 2024 • edited by llxia Loading

KapilPowar commented Jan 16, 2024

KapilPowar commented Jan 23, 2024 • edited by smlambert Loading

KapilPowar commented Jan 29, 2024

KapilPowar Dec 19, 2023 •

edited by llxia

Loading

KapilPowar commented Dec 11, 2023 •

edited by llxia

Loading

KapilPowar Jan 8, 2024 •

edited

Loading

KapilPowar commented Jan 8, 2024 •

edited by llxia

Loading

KapilPowar commented Jan 23, 2024 •

edited by smlambert

Loading