Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a lot more versions/variants (more Java versions, more base images, more OpenJDK implementations) #299

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

tianon
Copy link
Member

@tianon tianon commented May 8, 2023

This implements a large number of new variants based on OpenJDK alternatives listed in https://hub.docker.com/_/openjdk (more Corretto, Temurin, Semeru, SAP, etc).

This is not intended as a target for merging, but rather as a placeholder for discussion (to make the scope of the problem more obvious than it has previously been).

See especially #248 (comment), which summarizes this well:

Sorry, our current "support matrix" is already out of control, so we're still trying to decide the best path forward. 🙇

To give you an idea the scale I'm talking about, we currently support four concrete versions of Tomcat across ~3 versions of Java (both JDK and JRE) on up to three different implementation+distro combinations. That's ~49 different builds of Tomcat, which is then multiplied by the list of supported architectures for each image we're FROM. 😬

If we take this PR to the extended conclusion, that balloons to ~6 versions of Java (again, ~two builds each, JDK and JRE), from ~5 vendors across even more variants -- I have a test branch that includes those full changes, and it takes the matrix of supported versions from 49 all the way up to 228 (which still doesn't factor in the way the matrix expands along the supported architectures axis -- that's just explicit tag combinations that all support at least amd64). Limiting that to just the three LTS versions of Java takes us down to 147, but that's still really unmanageable (even the 49 we're at now is pretty high up there). 😮‍💨

One thing we've considered is updating the templating to generate all 228 variants, but only officially build/publish one or two "tracks" (perhaps just Temurin, for example). That would take our actual supported list way down, but I wouldn't want to publish that many Dockerfiles unless our automated builds are also testing them, and at 228 we'd be pushing the limits of what GitHub Actions is willing to accept. 😱

So, as you can see, this isn't as simple as just merging this PR. 😞

Example docker build command to build a version/variant from this PR (adjust as needed for other combinations):

docker build --pull 'https://github.com/docker-library/tomcat.git#refs/pull/299/head:10.1/jdk17/semeru-jammy'

@veselov
Copy link
Contributor

veselov commented May 9, 2023

To make sure I get this right - the only problem/bottleneck is the GitHub actions limit? Have you tried reaching out to GitHub and request more capacity considering the impact of this project? Or to the ASF to pick up some of this load? If they won't bother, it should be possible to find a commercial sponsor to contribute the build systems for something like this.

@tianon
Copy link
Member Author

tianon commented May 9, 2023

No, the GitHub Actions limits are only an example to help illustrate how large this matrix becomes -- the bigger issue is the load in both maintenance and downstream in the official images program.

@tianon
Copy link
Member Author

tianon commented May 9, 2023

For a user-facing impact, see the tag listing in https://github.com/docker-library/docs/blob/eb34efeaf92428afecd5375459c3fe3e657bb9f2/tomcat/README.md#supported-tags-and-respective-dockerfile-links which is already really unwieldy. 😞

Here's what it would look like with this PR:

very long tag listing example

Supported tags and respective Dockerfile links

@mstoodle
Copy link

mstoodle commented May 9, 2023

I understand the challenge with many different variations to support. One thing I would like to point out is that all the variations mentioned but one are variations of OpenJDK using the Hotspot JVM. From that perspective, they are quite similar (even intentionally, perhaps). But IBM Semeru Runtimes Open Edition is the only JDK variant that uses OpenJDK class libraries with the Eclipse OpenJ9 JVM. As such, it's a valuable variant simply for the significant difference it represents. OpenJ9 can provide different performance characteristics (faster startup and lower memory consumption) that some users might find attractive[1]. While it might not be practical to include all the different variations, adding an IBM Semeru Runtime Open Edition variant might be worth it.

I'm the chief architect for IBM Semeru Runtimes and a project lead at Eclipse OpenJ9 (and OMR) so if you have any questions, I would be happy to help.

[1] https://blog.openj9.org/2019/11/20/tomcat-tradelite7-performance-of-openj9-with-default-shared-cache/

@offerbaruch
Copy link

To continue on the last comment, OpenJ9 has important performance features for s390x and we would really appreciate a docker container with Semeru for 390x which is currently not available.

@veselov
Copy link
Contributor

veselov commented May 9, 2023

Honestly, the number of tags showing on the Dockerhub doesn't scare me, nor do I think it's getting out of control. There is a significant value in all the variants. Just as we are picky and only want Corretto on AL, there are specific reasons why others want other Xs on Ys.

If the tag list is to be trimmed, I'd argue it can be trimmed horizontally. I.e., :

  • 10.1.8-jdk20-temurin-jammy,
  • 10.1-jdk20-temurin-jammy,
  • 10-jdk20-temurin-jammy
  • jdk20-temurin-jammy
  • 10.1.8-jdk20-temurin,
  • 10.1-jdk20-temurin,
  • 10-jdk20-temurin,
  • jdk20-temurin,
  • 10.1.8-jdk20,
  • 10.1-jdk20,
  • 10-jdk20,
  • jdk20

I see value in having 10.1.8, 10.1 and 10 variants.
The rest seem to be excessive. I.e., it assumes that:

  • Tomcat 10 is the default, if not specified
  • Temurin is the default JVM, if not specified
  • Jammy is the default distro, if not specified

I think that folks that use those would need to say, at least, which major version of Tomcat they want, which VM and distro they want. Things are just too different between the variations here, and one truly doesn't care they might as well pick randomly. So this list can be shortened to:

  • 10.1.8-jdk20-temurin-jammy,
  • 10.1-jdk20-temurin-jammy,
  • 10-jdk20-temurin-jammy

And the last one is questionable as well, as there is enough incompatibilities between minor versions of Tomcat, it should be expected that it's specified.

Of course, reducing this list now would throw a wrench to folks who're already using the shortcuts, as they would stop pulling the "latest" things for them.

@veselov
Copy link
Contributor

veselov commented May 9, 2023

Regarding the maintenance - I agree that this is a problem, considering just the changes that were needed for yum->dnf support. I think that the current framework is not too maintenance-friendly. I think that if the script was broken up into individual script files for each distro, things would be easier. If a distro stops working, the builds are not published (for that distro) until somebody goes and fixes it. Yes, there will be more duplication in the script code, but it will be offset by not having the as much work in the JQ files, and reduced regressions.

@tianon tianon force-pushed the tomorecat branch 2 times, most recently from 76b2c48 to 7190172 Compare May 15, 2023 22:12
@tianon tianon force-pushed the tomorecat branch 3 times, most recently from 3ee301f to ec42dae Compare June 13, 2023 18:53
@aavileli

This comment was marked as off-topic.

@cjsr
Copy link

cjsr commented Aug 7, 2023

We are looking to migrate our applications to Spring 6, servlet spec 6.0 and thus Tomcat 10.1.x, however since we are on AWS we would be looking to use Corretto jdk 17 and Amazon Linux.
Might there be an update as to where the support matrix is headed @tianon?

@tianon tianon force-pushed the tomorecat branch 2 times, most recently from a3bdc70 to f0a883c Compare December 7, 2023 19:50
@tianon tianon force-pushed the tomorecat branch 2 times, most recently from f5f54dd to 7163a4f Compare December 18, 2023 23:00
@yosifkit yosifkit mentioned this pull request Mar 4, 2024
@tianon tianon force-pushed the tomorecat branch 3 times, most recently from 7781cb0 to 43cda0f Compare March 18, 2024 22:52
@CL-Jeremy
Copy link

Thanks a ton for your work (in progress)! I saw the addition of AL2023 that looks very promising on the RedHat/RPM side. I'd just like to also have Temurin UBI9 (RedHat Universal Base Image 9, a freely distributable base image officially supported by Adoptium) which is available at https://github.com/adoptium/containers for JDK11/17/21. Hopefully that doesn't involve too much work? Thanks in advance.

@tianon
Copy link
Member Author

tianon commented Mar 20, 2024

I'm not really at a place to do that validation work right at the moment, but if someone else could take one of the Dockerfiles in this PR and validate that the changes necessary to support it are minimal and work (ideally just changing FROM 🤞), I'd be willing to consider adding it to my matrix here.

@CL-Jeremy
Copy link

Regarding my last request, in case you missed it: infosiftr#2

Co-Authored-By: Mike L (Add temurin-ubi9-minimal) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants