CA interoperability tests fail

[sophia-guo]

CA interoperability tests fail on all platforms with error message

**Execution failed: `main' threw exception: java.lang.NullPointerException: Cannot invoke "java.security.cert.X509Certificate.getSubjectX500Principal()" because "this.rootCertificate" is null

https://ci.adoptium.net/job/Test_openjdk21_hs_extended.openjdk_x86-64_linux_testList_1/24/testReport/junit/security_infra_java_security_cert_CertPathValidator_certification_CAInterop/java_amazonrootca1/CAInterop_amazonrootca1/

The new framework for interoperability testing https://bugs.openjdk.org/browse/JDK-8308592.

Similar https://bugs.openjdk.org/browse/JDK-8316381

[smlambert]

These tests should get excluded if the new framework has not been delivered to our upstream mirrors. We should not be going into another release period with all of these tests failing as it makes it very difficult to see other unknown, misunderstood failures.

[sophia-guo]

Tests failed as the new framework merged. May need to open the issue in JBS. Will exclude in this release.

[sophia-guo]

Those 44 tests are in one file. It is currently not possible to exclude all the tests in a file with a single entry. See CODETOOLS-7902265. Have to exclude one by one

[sophia-guo]

same for 11, 17

[smlambert]

There are 45 testcases in the jdk_security_infra target, 44 are failing. We can exclude the entire jdk_security_infra target in the playlist and should link to an upstream bug that gets raised to report the issue (assuming it is an upstream issue and not our mirror failing to pick up the new framework).

From recent TAP file https://ci.adoptium.net/job/Test_openjdk21_hs_extended.openjdk_x86-64_linux_testList_1/33/tapResults/:

Failed test cases: 
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#amazonrootca1
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#emsignrootcag1
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#certignarootca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#amazonrootca3
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#affirmtrustpremiumeccca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#globalsigneccrootcar4
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#affirmtrustpremiumca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#comodorsaca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#globalsignrootcar6
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#actalisauthenticationrootca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#buypassclass2ca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#comodoeccca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#affirmtrustcommercialca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#affirmtrustnetworkingca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#digicerttlsrsarootg5
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#amazonrootca4
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#gtsrootcar2
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#godaddyrootg2ca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#buypassclass3ca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#entrustrootcaec1
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#entrustrootcag4
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#digicerttlseccrootg5
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#teliarootcav2
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#quovadisrootca3g3
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#starfieldrootg2ca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#teliasonerarootcav1
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#emsigneccrootcag3
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#usertrusteccca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#microsoftrsa2017
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#quovadisrootca2g3
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#gtsrootecccar4
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#letsencryptisrgx1
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#microsoftecc2017
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#twcaglobalrootca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#usertrustrsaca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#sslrootrsaca
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#letsencryptisrgx2
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#gtsrootecccar3
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#quovadisrootca1g3
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#sslrooteccca
TEST: security/infra/java/security/cert/CertPathValidator/certification/DigicertCSRootG5.java
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#sslrootevrsaca
TEST: security/infra/java/security/cert/CertPathValidator/certification/EmSignRootG2CA.java
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#gtsrootcar1
TEST: security/infra/java/security/cert/CertPathValidator/certification/CAInterop.java#amazonrootca2
Test results: passed: 1; failed: 45

[smlambert]

Ok, I see you went the ProblemList route, now I see your PR.

[macarte]

We just tested 21 as we also had this issue.

I don't know the background on why we've been generating the cacerts file and replacing the one thats built by default, however if you run the tests with the default cacert then the tests pass (we confirmed that the alias being searched for is in the default cacert file and not the one generated from the mozilla source

I don't know (again because of the background) which cacert file (if any) would be needed for TCK

[smlambert]

We will see what happens in our dry run tomorrow. These jtreg tests will be excluded and investigated (sounds like an area of improvement upstream to be able to support or skip in the event of other cacerts in use).

[sophia-guo]

Note for now we might just disable those tests for Temurin as https://github.com/adoptium/temurin-build/tree/master/security

[smlambert]

Given these test cases are not valid for any vendor (including Adoptium) who has their own CA certs, this may become a permanent exclude (where we track it against a closed issue with no intention to re-enable).