Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CryptoTests readlink: illegal option -- f #19164

Closed
pshipton opened this issue Mar 18, 2024 · 20 comments
Closed

CryptoTests readlink: illegal option -- f #19164

pshipton opened this issue Mar 18, 2024 · 20 comments

Comments

@pshipton
Copy link
Member

https://openj9-jenkins.osuosl.org/job/Test_openjdk11_j9_extended.functional_aarch64_mac_Nightly_testList_0/462 - mac11-aarch64-4
CryptoTests_0

21:44:11  Test results: passed: 32
21:44:11  Report written to /Users/jenkins/workspace/Test_openjdk11_j9_extended.functional_aarch64_mac_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest/test.1710553196/jdk/JTreport/html/report.html
21:44:11  Results written to /Users/jenkins/workspace/Test_openjdk11_j9_extended.functional_aarch64_mac_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest/test.1710553196/jdk/JTwork
21:44:11  ~/workspace/Test_openjdk11_j9_extended.functional_aarch64_mac_Nightly_testList_0/aqa-tests/TKG/output_17105515401191/CryptoTests_0
21:44:11  readlink: illegal option -- f
21:44:11  usage: readlink [-n] [file ...]
21:44:11  Missing tests.log!
21:44:11  -----------------------------------
21:44:11  CryptoTests_0_FAILED
@pshipton
Copy link
Member Author

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_extended.functional_aarch64_mac_Nightly_testList_0/464 - mac11-aarch64-2

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_extended.functional_s390x_linux_Nightly_testList_0/682 - ub20-390-2

23:05:51  Error:  cryptotest/CryptoTest.java
23:05:51  Test results: passed: 31; error: 1
23:05:51  Report written to /home/jenkins/workspace/Test_openjdk17_j9_extended.functional_s390x_linux_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest/test.1710557092/jdk/JTreport/html/report.html
23:05:51  Results written to /home/jenkins/workspace/Test_openjdk17_j9_extended.functional_s390x_linux_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest/test.1710557092/jdk/JTwork
23:05:51  Error: Some tests failed or other problems occurred.
23:05:51  ~/workspace/Test_openjdk17_j9_extended.functional_s390x_linux_Nightly_testList_0/aqa-tests/TKG/output_17105480336205/CryptoTests_0
23:05:51  renamed '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_s390x_linux_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest/test.1710557092.tar.gz' -> './test.1710557092.tar.gz'
23:05:51  -----------------------------------
23:05:51  CryptoTests_0_FAILED

@JasonFengJ9
Copy link
Member

JDK11 aarch64_mac

[2024-03-16T01:29:16.818Z] ~/workspace/Test_openjdk11_j9_dev.functional_aarch64_mac/aqa-tests/TKG/output_17105525213961/ssl-tests_0
[2024-03-16T01:29:16.818Z] readlink: illegal option -- f
[2024-03-16T01:29:16.818Z] usage: readlink [-n] [file ...]
[2024-03-16T01:29:16.818Z] Missing tests.log!
[2024-03-16T01:29:16.818Z] -----------------------------------
[2024-03-16T01:29:16.818Z] ssl-tests_0_FAILED

@pshipton
Copy link
Member Author

pshipton commented Mar 19, 2024

More failures today, excluding via adoptium/aqa-tests#5153

@llxia
Copy link
Contributor

llxia commented Mar 19, 2024

JasonFengJ9 pushed a commit to adoptium/aqa-tests that referenced this issue Mar 19, 2024
@pshipton
Copy link
Member Author

The OpenJ9 failures only started the night of March 18.

@pshipton
Copy link
Member Author

Shelley seems to have narrowed it down, adoptium/aqa-tests#5134 (comment)

sophiaxu0424 pushed a commit to sophiaxu0424/aqa-tests that referenced this issue Mar 19, 2024
@JasonFengJ9
Copy link
Member

JDK11 x86-64_mac

[2024-03-25T23:17:50.138Z] ~/workspace/Test_openjdk11_j9_extended.functional_x86-64_mac_testList_0/aqa-tests/TKG/output_17114078579034/CryptoTests_0
[2024-03-25T23:17:50.138Z] readlink: illegal option -- f
[2024-03-25T23:17:50.138Z] usage: readlink [-n] [file ...]
[2024-03-25T23:17:50.138Z] Missing tests.log!
[2024-03-25T23:17:50.138Z] -----------------------------------
[2024-03-25T23:17:50.138Z] CryptoTests_0_FAILED

@pshipton
Copy link
Member Author

pshipton commented Apr 1, 2024

@JasonFengJ9 as per adoptium/aqa-tests#5134 (comment) we excluded the test on amac, zlinux. Should we exclude it on xmac as well?

@JasonFengJ9
Copy link
Member

Should we exclude it on xmac as well?

The test in question passed in the latest JDK11 x86-64_mac weekly run, no need to exclude it for now.

@JasonFengJ9
Copy link
Member

JDK17 x86-64_mac(osxrt4)

[2024-04-02T01:53:09.453Z] readlink: illegal option -- f
[2024-04-02T01:53:09.453Z] usage: readlink [-n] [file ...]
[2024-04-02T01:53:09.453Z] Missing tests.log!
[2024-04-02T01:53:09.453Z] -----------------------------------
[2024-04-02T01:53:09.453Z] CryptoTests_0_FAILED

Should we exclude it on xmac as well?

The test in question passed in the latest JDK11 x86-64_mac weekly run, no need to exclude it for now.

Okay, it needs to be excluded as well, created

@judovana
Copy link

hi! I had removed the -f from default crypto test runner - rh-openjdk/CryptoTest@23496c5 . Maybe try to enable them again?

@llxia
Copy link
Contributor

llxia commented Apr 18, 2024

@judovana the test still failed.
https://openj9-jenkins.osuosl.org/job/Grinder/3481/console

00:02:07.973  readlink: illegal option -- f
00:02:07.973  usage: readlink [-n] [file ...]
00:02:10.499  treating jdk as: 11
00:02:10.499  Running with /Users/jenkins/workspace/Grinder/jdkbinary/j2sdk-image...
00:02:11.969  XML output  to /Users/jenkins/workspace/Grinder/aqa-tests/functional/security/Crypto/CryptoTest/test.1713451085/jdk/work
00:03:08.844  Passed: cryptotest/tests/AlgorithmParameterGeneratorTests.java
00:03:08.844  Passed: cryptotest/tests/AlgorithmParametersTests.java
00:03:08.844  Passed: cryptotest/tests/CertificateFactoryTests.java
00:03:08.844  Passed: cryptotest/tests/CertPathBuilderTests.java
00:03:08.844  Passed: cryptotest/tests/CertPathValidatorTests.java
00:03:09.857  Passed: cryptotest/tests/CertStoreTests.java
00:03:10.970  Passed: cryptotest/tests/CipherTests.java
00:03:10.970  Passed: cryptotest/tests/ConfigurationTests.java
00:03:12.032  Passed: cryptotest/tests/GssApiMechanismTests.java
00:03:13.193  Passed: cryptotest/tests/KeyAgreementTests.java
00:03:16.837  Passed: cryptotest/tests/KeyFactoryTests.java
00:03:17.994  Passed: cryptotest/tests/KeyGeneratorTests.java
00:03:19.164  Passed: cryptotest/tests/KeyInfoFactoryTests.java
00:03:20.484  Passed: cryptotest/tests/KeyManagerFactoryTests.java
00:03:24.190  Passed: cryptotest/tests/KeyPairGeneratorTests.java
00:03:25.200  Passed: cryptotest/tests/KeyStoreTests.java
00:03:26.253  Passed: cryptotest/tests/MacTests.java
00:03:26.253  Passed: cryptotest/tests/MessageDigestTests.java
00:03:27.466  Passed: cryptotest/tests/PolicyTests.java
00:03:28.455  Passed: cryptotest/tests/SaslClientFactoryTests.java
00:03:29.489  Passed: cryptotest/tests/SaslServerFactoryTests.java
00:03:29.489  Passed: cryptotest/tests/SecretKeyFactoryTests.java
00:03:30.601  Passed: cryptotest/tests/SecureRandomTests.java
00:03:57.452  Passed: cryptotest/tests/SignatureTests.java
00:03:58.596  Passed: cryptotest/tests/SSLContextTests.java
00:03:58.596  Passed: cryptotest/tests/TerminalFactoryTests.java
00:03:59.555  Passed: cryptotest/tests/TestProviders.java
00:04:01.672  Passed: cryptotest/tests/TestServices.java
00:04:01.672  Passed: cryptotest/tests/TransformServiceTests.java
00:04:01.672  Passed: cryptotest/tests/TrustManagerFactoryTests.java
00:04:02.834  Passed: cryptotest/tests/XMLSignatureFactoryTests.java
00:05:57.641  Passed: cryptotest/CryptoTest.java
00:05:57.641  Test results: passed: 32
00:05:57.641  Report written to /Users/jenkins/workspace/Grinder/aqa-tests/functional/security/Crypto/CryptoTest/test.1713451085/jdk/report/html/report.html
00:05:57.641  Results written to /Users/jenkins/workspace/Grinder/aqa-tests/functional/security/Crypto/CryptoTest/test.1713451085/jdk/work
00:05:57.641  ~/workspace/Grinder/aqa-tests/TKG/output_17134510838640/CryptoTests_0
00:05:57.641  Missing tests.log!
00:05:57.641  -----------------------------------
00:05:57.641  CryptoTests_0_FAILED
00:05:57.641  -----------------------------------

@judovana
Copy link

judovana commented Apr 18, 2024

wou. The test.log is what is visible on that snippet.

 $FOLDER | tee test.${TIME}/tests.log || r=$?

There is that

if [ ! `readlink $READLINK_F ${SCRIPT_DIR}` == `pwd`  ] ;

which must return wrongly on two already canonicalised dirs. So the following mv ${SCRIPT_DIR}/test.${TIME} . was in missed branch.

I'm wondering what the zos may be doing so differently. The pssible fix for this would be to move that Missing tests.log! efore the move happenes. But it is intentionally after to verify the move. So If I do that, the junit results would be missing.

Do you think you can run it on that machine without aqavit with bash -x (or in adjusted aqavit) or give me access to that machine?

llxia added a commit to llxia/aqa-tests that referenced this issue Apr 18, 2024
- add CryptoTests_jtreg back (based on the original change https://github.com/adoptium/aqa-tests/pull/5134/files)
- CryptoTests_jtreg directly triggers the tests via jtreg and it runs against all vendors except redhat
- CryptoTests only runs for vendor=redhat
- re-enable CryptoTests_jtreg on aarch64_mac|x86-64_mac|s390x_linux (related to run.sh issue. See CryptoTests readlink: illegal option -- f eclipse-openj9/openj9#19164)
- add retry for git clone to mitigate network issue
- add check CryptoTest repo sha (sha will be captured in TAP file)

Signed-off-by: Lan Xia <[email protected]>
@llxia
Copy link
Contributor

llxia commented Apr 18, 2024

CryptoTests_jtreg is created to trigger the crypto tests via jtreg directly. I re-enabled crypto tests on aarch64_mac|x86-64_mac|s390x_linux via CryptoTests_jtreg. CryptoTests that uses run.sh will be limited to vendor=redhat only. See adoptium/aqa-tests#5240.

@llxia
Copy link
Contributor

llxia commented Apr 18, 2024

set KEEP_REPORTDIR = true for extended.functional: ibmruntimes/ci-jenkins-pipelines#220

@pshipton
Copy link
Member Author

This happened last night
https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/655

21:52:51  TESTING:
21:52:51  ~/workspace/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest ~/workspace/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/aqa-tests/TKG/output_17134033039768/CryptoTests_0
21:52:51  Non cygwin system!
21:52:51  treating jdk as: 8
21:52:51  Running with /home/jenkins/workspace/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/jdkbinary/j2sdk-image...
21:52:52  XML output  to /home/jenkins/workspace/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest/test.1713405171/jdk/JTwork
21:52:52  Error: Unexpected exception occurred! java.lang.NullPointerException
21:52:52  java.lang.NullPointerException
21:52:52  	at com.sun.javatest.regtest.config.TestProperties$Cache.getEntryInternal(TestProperties.java:508)
21:52:52  	at com.sun.javatest.regtest.config.TestProperties$Cache.getEntryInternal(TestProperties.java:508)
21:52:52  	at com.sun.javatest.regtest.config.TestProperties$Cache.getEntry(TestProperties.java:500)
21:52:52  	at com.sun.javatest.regtest.config.TestProperties.getEntry(TestProperties.java:168)
21:52:52  	at com.sun.javatest.regtest.config.TestProperties.getTestNGRoot(TestProperties.java:123)
21:52:52  	at com.sun.javatest.regtest.config.RegressionTestFinder.scanFile(RegressionTestFinder.java:142)
21:52:52  	at com.sun.javatest.finder.TagTestFinder.scan(TagTestFinder.java:114)
21:52:52  	at com.sun.javatest.TestFinder.read(TestFinder.java:432)
21:52:52  	at com.sun.javatest.TRT_TreeNode.processFile(TRT_TreeNode.java:1255)
21:52:52  	at com.sun.javatest.TRT_TreeNode.scanIfNeeded(TRT_TreeNode.java:805)
21:52:52  	at com.sun.javatest.TRT_TreeNode.getChildCount(TRT_TreeNode.java:291)
21:52:52  	at com.sun.javatest.TRT_Iterator$PseudoFrame.nextIndex(TRT_Iterator.java:998)
21:52:52  	at com.sun.javatest.TRT_Iterator.findNext(TRT_Iterator.java:727)
21:52:52  	at com.sun.javatest.TRT_Iterator.init(TRT_Iterator.java:697)
21:52:52  	at com.sun.javatest.TRT_Iterator.<init>(TRT_Iterator.java:133)
21:52:52  	at com.sun.javatest.TestResultTable.getIterator(TestResultTable.java:320)
21:52:52  	at com.sun.javatest.TestResultTable.getIterator(TestResultTable.java:1049)
21:52:52  	at com.sun.javatest.Harness.createTreeIterator(Harness.java:874)
21:52:52  	at com.sun.javatest.Harness.runTests(Harness.java:753)

smlambert pushed a commit to adoptium/aqa-tests that referenced this issue Apr 19, 2024
- add CryptoTests_jtreg back (based on the original change https://github.com/adoptium/aqa-tests/pull/5134/files)
- CryptoTests_jtreg directly triggers the tests via jtreg and it runs against all vendors except redhat
- CryptoTests only runs for vendor=redhat
- re-enable CryptoTests_jtreg on aarch64_mac|x86-64_mac|s390x_linux (related to run.sh issue. See CryptoTests readlink: illegal option -- f eclipse-openj9/openj9#19164)
- add retry for git clone to mitigate network issue
- add check CryptoTest repo sha (sha will be captured in TAP file)

Signed-off-by: Lan Xia <[email protected]>
@judovana
Copy link

This happened last night https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/655

21:52:51  TESTING:
21:52:51  ~/workspace/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest ~/workspace/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/aqa-tests/TKG/output_17134033039768/CryptoTests_0
21:52:51  Non cygwin system!
21:52:51  treating jdk as: 8
21:52:51  Running with /home/jenkins/workspace/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/jdkbinary/j2sdk-image...
21:52:52  XML output  to /home/jenkins/workspace/Test_openjdk8_j9_extended.functional_aarch64_linux_Nightly_testList_0/aqa-tests/functional/security/Crypto/CryptoTest/test.1713405171/jdk/JTwork
21:52:52  Error: Unexpected exception occurred! java.lang.NullPointerException
21:52:52  java.lang.NullPointerException
21:52:52  	at com.sun.javatest.regtest.config.TestProperties$Cache.getEntryInternal(TestProperties.java:508)
21:52:52  	at com.sun.javatest.regtest.config.TestProperties$Cache.getEntryInternal(TestProperties.java:508)
21:52:52  	at com.sun.javatest.regtest.config.TestProperties$Cache.getEntry(TestProperties.java:500)
21:52:52  	at com.sun.javatest.regtest.config.TestProperties.getEntry(TestProperties.java:168)
21:52:52  	at com.sun.javatest.regtest.config.TestProperties.getTestNGRoot(TestProperties.java:123)
21:52:52  	at com.sun.javatest.regtest.config.RegressionTestFinder.scanFile(RegressionTestFinder.java:142)
21:52:52  	at com.sun.javatest.finder.TagTestFinder.scan(TagTestFinder.java:114)
21:52:52  	at com.sun.javatest.TestFinder.read(TestFinder.java:432)
21:52:52  	at com.sun.javatest.TRT_TreeNode.processFile(TRT_TreeNode.java:1255)
21:52:52  	at com.sun.javatest.TRT_TreeNode.scanIfNeeded(TRT_TreeNode.java:805)
21:52:52  	at com.sun.javatest.TRT_TreeNode.getChildCount(TRT_TreeNode.java:291)
21:52:52  	at com.sun.javatest.TRT_Iterator$PseudoFrame.nextIndex(TRT_Iterator.java:998)
21:52:52  	at com.sun.javatest.TRT_Iterator.findNext(TRT_Iterator.java:727)
21:52:52  	at com.sun.javatest.TRT_Iterator.init(TRT_Iterator.java:697)
21:52:52  	at com.sun.javatest.TRT_Iterator.<init>(TRT_Iterator.java:133)
21:52:52  	at com.sun.javatest.TestResultTable.getIterator(TestResultTable.java:320)
21:52:52  	at com.sun.javatest.TestResultTable.getIterator(TestResultTable.java:1049)
21:52:52  	at com.sun.javatest.Harness.createTreeIterator(Harness.java:874)
21:52:52  	at com.sun.javatest.Harness.runTests(Harness.java:753)

This happens when the jtreg.run runs from different location then expected or if the target directory with TEST.ROOT isnot around. The only shared chaneg which would affect both CryptoTests and CryptoTests_jtreg seesm to be https://github.com/adoptium/aqa-tests/pull/5240/files#diff-beab30a233df4a27d6260ab0d667970c197c7827c6869ca52cebaa20ff2070ecR45

@judovana
Copy link

judovana commented Apr 19, 2024

CryptoTests_jtreg is created to trigger the crypto tests via jtreg directly. I re-enabled crypto tests on aarch64_mac|x86-64_mac|s390x_linux via CryptoTests_jtreg. CryptoTests that uses run.sh will be limited to vendor=redhat only. See adoptium/aqa-tests#5240.

By doing this, you moreover killed any possible development (for this case) of the wrapper. Yes, you can always fork. But it is always better to tune up original project then create fork. In addition I had committed to work on fixing it for you

Note that I'm running it on aarch64_mac,x86-64_mac and s390x_linux and it is passing. So it is yours machines which probably needs a bit of love.

@llxia
Copy link
Contributor

llxia commented Apr 30, 2024

Just for the record, we switched to using CryptoTests_jtreg . It directly triggers crypto tests in https://github.com/rh-openjdk/CryptoTest/ repo via jtreg. We do not use https://github.com/rh-openjdk/CryptoTest/blob/master/run.sh, so we should not have the above issue. The above-excluded platforms are re-enabled in CryptoTests_jtreg. For details, please see adoptium/aqa-tests#5240

With jtreg, JUnit output can be found at the job Test Result. (i.e., https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk17_j9_extended.functional_x86-64_mac_testList_0/640/testReport/)

I will close this issue.

@llxia llxia closed this as completed Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants