Migrate to jdk.javadoc.doclet API for Java 9 and later

[msgilligan]

When running Asciidoclet under Java 9 I get the following message

javadoc: warning - The old Doclet and Taglet APIs in the packages
com.sun.javadoc, com.sun.tools.doclets and their implementations
are planned to be removed in a future JDK release. These
components have been superseded by the new APIs in jdk.javadoc.doclet.
Users are strongly recommended to migrate to the new APIs.

At some point Asciidoclet should be ported to the new API.

[msgilligan]

Trying to build Asciidoclet under JDK 9 (or 10), results in the following error message:

[ERROR] Failed to execute goal on project asciidoclet: Could not resolve dependencies for project org.asciidoctor:asciidoclet:jar:1.5.5-SNAPSHOT: Could not find artifact com.sun.tools:tools:jar:9 at specified path /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/../lib/tools.jar -> [Help 1]

[johncarl81]

Yeah, from what I've seen it's a non-trivial migration. I hope it would be backwards compatible with < Java 9. I would love to see the dependency on tools.jar to go away.

[msgilligan]

I don't think it is backwards compatible in terms of compiling or running on pre-Java 9. In other words the api is @since 9. So implementing this API will create a new major version of Asciidoclet. However I believe it should be possible to generate Java 7/8 javadoc output with JDK 9+ as long as the source and target versions are set correctly.

The old (current) Asciidoclet should continue to run on newer JDKs until com.sun.javadoc is removed "in a future JDK release".

(This is why I propose two "phases" to Java 9 support in Issue #76)

[johncarl81]

I've been tinkering around with the new Javadoclet api. Bad news is that we won't have the (hack) ability to override the default comment text. But, this might mean we should take a more direct approach... maybe proxying the elements api everywhere a javadoc is accessed via the HtmlDoclet?

[msgilligan]

we won't have the (hack) ability to override the default comment text

What do you mean by that?

[johncarl81]

I mean, I don't know if the original intention of a javadoc doclet was to give the ability to swap in a different rendering engine for javadocs as we have. The API feels like it wasn't designed for this, as we replace the raw javadoc text with our rendered text and then pass it on to the default javadoc renderer. We would have to find a similar technique in the new javadoc API and the examples I've seen don't cover this sort of functionality.

[msgilligan]

Well worst case we could take the code for the Standard Doclet and modify it, right?

[johncarl81]

Yeah, I think you're right. Plus we can use the standard doclet (HtmlDoclet) as guidance to how to write in the new API.

For an implementation, my initial thought is to add a proxy layer into the elements api to process the javadocs from asciidoc to html. We could then hand this elements proxy into the HtmlDoclet for final rendering. This would benefit from leveraging the features of the HtmlDoclet while not inheriting the maintanence. Since one goal of the new doclet is to avoid a static API, on principal it should be possible.
WDYT @msgilligan?

[msgilligan]

You're talking about the core of the implementation here -- some fairly tricky code. I haven't looked at it much -- so my opinion isn't that valuable (at least yet)

I'm actually thinking we might want to create a little design document and talk about goals and architecture a little before getting started.

One goal I'd like to see is code that could be also be used for a new GroovyDoc that supports Asciidoclet. In fact, we might even be able to propose to the Groovy team to make Asciidoclet the standard format for Groovy. (Especially now that they have Kotlin-envy and Kotlin has a built-in markdown syntax for KDoc) This might be a little too ambitious for our first cut, but I'm wondering what (if anything) could be reusable for GroovyDoc.

Back to the question of proxy layer -- I think it makes sense and it might also be reusable for Groovy which would be cool.

[johncarl81]

I agree 100% @msgilligan, this deserves some planning w/ a design doc, and the Groovy possibility sounds fantastic.

[johncarl81]

If you're interested @msgilligan, I was able to convert one of my other doclets over to Java 9 here. The meat of it here is getting the Javadoc comment via the Elements utility and processing it. In export doclet we just drop it to a file, but this could be the proxy location where we pass it through Asciidoctor to create the output html.