Error page tests

[janbartel]

The following tests fail on jetty-11:

com/sun/ts/tests/servlet/spec/errorpage/URLClient.java.heirarchyErrorMatchTest
com/sun/ts/tests/servlet/spec/errorpage/URLClient.java.servletToDifferentErrorPagesTest
com/sun/ts/tests/servlet/spec/errorpage/URLClient.java.wrappedExceptionTest

This is because the tests throw a ServletException wrapping an IllegalThreadStateException and IllegalStateException respectively. Jetty sets the request attributes jakarta.servlet.error.exception_type and jakarta.servlet.error.exception to the ServletException, whereas the tck is testing that they are set to the unwrapped Exception. The servlet spec does not say what they should be set to. I have raised an issue on the servlet-api to clarify what they should be: jakartaee/servlet#388.

Note that these tests do not fail on jetty-10, because the earlier 4.0.0 version of the tck had a typo in javax.servlet.error.exception_type (was mispelt as javax.serlvet.error.exception_type) producing a value of null that the test was then incorrectly testing for, instead of the actual Exception. That was fixed for tck 5.0.0 by jakartaee/platform-tck#254.

[olamy]

we unwrapped the exception to find the error redirect location so would be logic to use the same unwrapped exception as the attribute.

[joakime]

Still test failures on these, latest build completed 22 minutes ago.

com/sun/ts/tests/servlet/spec/errorpage/URLClient.java.heirarchyErrorMatchTest

[javatest.batch] 02-10-2021 13:23:26:  [HttpRequest] Dispatching request: 'GET /servlet_spec_errorpage_web/TestServlet?testname=heirarchyErrorMatchTest HTTP/1.1' to target server at 'localhost:8080'
[javatest.batch] 02-10-2021 13:23:26:  TRACE: ########## The real value set: false
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - >> "GET /servlet_spec_errorpage_web/TestServlet?testname=heirarchyErrorMatchTest HTTP/1.1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - >> "Host: localhost:8080[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - >> "[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - << "HTTP/1.1 404 Not Found[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - << "HTTP/1.1 404 Not Found[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - << "Content-Type: text/html;charset=iso-8859-1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - << "Content-Length: 475[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - << "Server: Jetty(11.0.1-SNAPSHOT)[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  TRACE: [WIRE] - << "[\r][\n]"
[javatest.batch] 02-10-2021 13:23:26:  ERROR: [WebValidatorBase] Unexpected Status Code recieved from server.  Expected '500' received '404'
[javatest.batch] 02-10-2021 13:23:26:  ERROR: Test case throws exception: [BaseUrlClient] heirarchyErrorMatchTest failed! Check output for cause of failure.
[javatest.batch] 02-10-2021 13:23:26:  ERROR: Exception at:  
[javatest.batch] 02-10-2021 13:23:26:  ERROR: com.sun.ts.tests.common.webclient.TestFailureException: Test FAILED!
[javatest.batch] 	at com.sun.ts.tests.common.webclient.WebTestCase.execute(WebTestCase.java:180)
[javatest.batch] 	at com.sun.ts.tests.common.webclient.BaseUrlClient.invoke(BaseUrlClient.java:455)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.heirarchyErrorMatchTest(URLClient.java:125)
[javatest.batch] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[javatest.batch] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[javatest.batch] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[javatest.batch] 	at java.lang.reflect.Method.invoke(Method.java:498)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.run(EETest.java:596)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.run(EETest.java:337)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.run(URLClient.java:51)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.main(URLClient.java:37)
[javatest.batch] 

com/sun/ts/tests/servlet/spec/errorpage/URLClient.java.servletToDifferentErrorPagesTest

[javatest.batch] 02-10-2021 13:23:27:  [HttpRequest] Dispatching request: 'GET /servlet_spec_errorpage_web/TestServlet?testname=servletErrorPageTest HTTP/1.1' to target server at 'localhost:8080'
[javatest.batch] 02-10-2021 13:23:27:  TRACE: ########## The real value set: false
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - >> "GET /servlet_spec_errorpage_web/TestServlet?testname=servletErrorPageTest HTTP/1.1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - >> "Host: localhost:8080[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - >> "[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - << "HTTP/1.1 404 Not Found[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - << "HTTP/1.1 404 Not Found[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - << "Content-Type: text/html;charset=iso-8859-1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - << "Content-Length: 475[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - << "Server: Jetty(11.0.1-SNAPSHOT)[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  TRACE: [WIRE] - << "[\r][\n]"
[javatest.batch] 02-10-2021 13:23:27:  ERROR: [WebValidatorBase] Unexpected Status Code recieved from server.  Expected '500' received '404'
[javatest.batch] 02-10-2021 13:23:27:  ERROR: Test case throws exception: [BaseUrlClient] servletErrorPageTest failed! Check output for cause of failure.
[javatest.batch] 02-10-2021 13:23:27:  ERROR: Exception at:  
[javatest.batch] 02-10-2021 13:23:27:  ERROR: com.sun.ts.tests.common.webclient.TestFailureException: Test FAILED!
[javatest.batch] 	at com.sun.ts.tests.common.webclient.WebTestCase.execute(WebTestCase.java:180)
[javatest.batch] 	at com.sun.ts.tests.common.webclient.BaseUrlClient.invoke(BaseUrlClient.java:455)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.servletToDifferentErrorPagesTest(URLClient.java:77)
[javatest.batch] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[javatest.batch] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[javatest.batch] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[javatest.batch] 	at java.lang.reflect.Method.invoke(Method.java:498)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.run(EETest.java:596)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.run(EETest.java:337)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.run(URLClient.java:51)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.main(URLClient.java:37)
[javatest.batch] 

com/sun/ts/tests/servlet/spec/errorpage/URLClient.java.wrappedExceptionTest

[javatest.batch] 02-10-2021 13:23:30:  [HttpRequest] Dispatching request: 'GET /servlet_spec_errorpage_web/WrappedException HTTP/1.1' to target server at 'localhost:8080'
[javatest.batch] 02-10-2021 13:23:30:  TRACE: ########## The real value set: false
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - >> "GET /servlet_spec_errorpage_web/WrappedException HTTP/1.1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - >> "Host: localhost:8080[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - >> "[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - << "HTTP/1.1 404 Not Found[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - << "HTTP/1.1 404 Not Found[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - << "Content-Type: text/html;charset=iso-8859-1[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - << "Content-Length: 480[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - << "Server: Jetty(11.0.1-SNAPSHOT)[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  TRACE: [WIRE] - << "[\r][\n]"
[javatest.batch] 02-10-2021 13:23:30:  ERROR: [WebValidatorBase] Unexpected Status Code recieved from server.  Expected '500' received '404'
[javatest.batch] 02-10-2021 13:23:30:  ERROR: Test case throws exception: [BaseUrlClient] null failed! Check output for cause of failure.
[javatest.batch] 02-10-2021 13:23:30:  ERROR: Exception at:  
[javatest.batch] 02-10-2021 13:23:30:  ERROR: com.sun.ts.tests.common.webclient.TestFailureException: Test FAILED!
[javatest.batch] 	at com.sun.ts.tests.common.webclient.WebTestCase.execute(WebTestCase.java:180)
[javatest.batch] 	at com.sun.ts.tests.common.webclient.BaseUrlClient.invoke(BaseUrlClient.java:455)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.wrappedExceptionTest(URLClient.java:154)
[javatest.batch] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[javatest.batch] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[javatest.batch] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[javatest.batch] 	at java.lang.reflect.Method.invoke(Method.java:498)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.run(EETest.java:596)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
[javatest.batch] 	at com.sun.ts.lib.harness.EETest.run(EETest.java:337)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.run(URLClient.java:51)
[javatest.batch] 	at com.sun.ts.tests.servlet.spec.errorpage.URLClient.main(URLClient.java:37)
[javatest.batch] 

[joakime]

Tossing this in here for reference, this is the code tree for the Jetty 11 (TCK 9.0.0 / Servlet 5.0.1) for these specific tests.

https://github.com/eclipse-ee4j/jakartaee-tck/tree/9.0.0/src/com/sun/ts/tests/servlet/spec/errorpage

[joakime]

Noticed this in the jetty logs.

2021-02-10 12:31:25,730 [main] WARN  org.eclipse.jetty.webapp.WebAppContext [] - Failed startup of context o.e.j.w.WebAppContext@24534cb0{/servlet_spec_errorpage_web,null,STOPPED}{/home/jenkins/agent/workspace/tck/servlettck-run/jetty-home/target/jetty-base/webappsservlet_spec_errorpage_web.war}
java.io.FileNotFoundException: /home/jenkins/agent/workspace/tck/servlettck-run/jetty-home/target/jetty-base/webappsservlet_spec_errorpage_web.war
	at org.eclipse.jetty.webapp.WebInfConfiguration.unpack(WebInfConfiguration.java:399) ~[jetty-webapp-11.0.1-SNAPSHOT.jar:11.0.1-SNAPSHOT]
	at org.eclipse.jetty.webapp.WebInfConfiguration.preConfigure(WebInfConfiguration.java:56) ~[jetty-webapp-11.0.1-SNAPSHOT.jar:11.0.1-SNAPSHOT]

Pushed a fix for the new XML deployable on commit 81b159b