jdk 17 jdk_security_infra_1 tests fail on xlinux

[Haroon-Khel]

Execution failed: main' threw exception: java.lang.RuntimeException: TEST FAILED: couldn't determine EE certificate status

https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_x86-64_linux_testList_0/15/testReport/

security/infra/java/security/cert/CertPathValidator/certification/DTrustCA.java.DTrustCA
security/infra/java/security/cert/CertPathValidator/certification/HaricaCA.java.HaricaCA
security/infra/java/security/cert/CertPathValidator/certification/SSLCA.java.SSLCA

https://bugs.openjdk.java.net/browse/JDK-8194170 suggests it is a machine issue

rhaladeRajan Halade added a comment - 2017-12-28 19:29
These test failures are result of incorrect proxy setup. Please make sure proxy on test machines is set to be able to reach external servers.

Related issue adoptium/aqa-tests#2074

[Haroon-Khel]

It has been found that these tests fail only on our docker nodes, such as test-docker-ubuntu2010-x64-3. It passes on test-packet-ubuntu1604-x64-1.

I have spun up a docker container on test-packet-ubuntu1604-x64-1, and on that it passes. So I think the error lies within the network config of the host machines of the docker containers on which the tests fail.

Those machines are dockerhost-equinix-ubuntu2004-x64-1 and 2

[sophia-guo]

security/infra/java/security/cert/CertPathValidator/certification/EntrustCA.java failed and passed on same docker nodes test-marist-ubuntu1804-s390x-3.

[sxa]

Needs retesting on a variety of current nodes.

[Haroon-Khel]

security/infra/java/security/cert/CertPathValidator/certification/SSLCA.java is no longer in JDK17u https://github.com/adoptium/jdk17u/tree/master/test/jdk/security/infra/java/security/cert/CertPathValidator/certification

While rerunning the other 2 tests, I get a strange failure
https://ci.adoptium.net/job/Grinder/11633/console

17:36:10  Test results: no tests selected
17:36:19  Report written to /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17321241637870/jdk_custom_0/report/html/report.html
17:36:19  Results written to /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_17321241637870/jdk_custom_0/work
17:36:19  -----------------------------------
17:36:19  jdk_custom_0_FAILED
17:36:19  -----------------------------------

No tests selected is a strange error to have

[sxa]

Noting that jdk_security_infra_1 is showing as DISABLED if you try to run it. Ref: https://ci.adoptium.net/job/Grinder/11667/console

[smlambert]

And if you look in the output of the Grinder, we also list the issue that it is excluded under
adoptium/aqa-tests#4808

[sxa]

👍 Not seen that before. Useful! I hadn't looked up from the summary.

jdk_security_infra_1 Start Time: Thu Nov 21 08:09:54 2024 Epoch Time (ms): 1732198194756
14:09:55  jdk_security_infra_1_DISABLED
14:09:55  Disabled Reason:
14:09:55  https://github.com/adoptium/aqa-tests/issues/4808
14:09:55  jdk_security_infra_1 Finish Time: Thu Nov 21 08:09:54 

[sxa]

I'm going to try and summarise what I believe is the current situation. See also slack thread:

• The tests were excluded in AQA as part of https://github.com/adoptium/aqa-tests/pull/5014/files at the start of 2024 (Issue )
• This was likely a problem when https://bugs.openjdk.org/browse/JDK-8308592 was introduced. The failures seem to be due to Temurin's use of a custom (Mozilla) cacerts bundle.
• Shortly afterwards, a proposal to mark them as "manual" was made was made, and this was merged into the jdk24 codebase in June 2024, and backported to jdk17u in 8334441: Mark tests in jdk_security_infra group as manual openjdk/jdk17u#393 - this should remove it from the automation.

Running jdk_security_infra_1 with a revert of a45c03cd and 557cd90 via Grinder#11689 hit a FAILED state with the message No Tests Selected, however it may be possible to partially revert those given the upstream changes sine the exclusions were put into AQA.

Potentially also related: adoptium/aqa-tests#2123

[sxa]

The comment at #2363 (comment) intrigues me though. Given the nature of the problem as described in the issues in the previous comment, it seems unlikely that this would be machine specific, so it may be worth running it (manually) on a selection of machines to see if we have an underlying issue here, but given the exclusions I would now consider this low priority for this iteration :-)

Sample reproducer: make _jdk_custom JDK_CUSTOM_TARGET=security/infra/java/security/cert/CertPathValidator/certification/HaricaCA.java (After following setup instructions from https://github.com/gdams/openjdk-infrastructure/blob/win/FAQ.md#how-do-i-replicate-a-test-failure)