Support markdown syntax.

[jordwalke]

Many in the web community use markdown syntax for their source documentation.

I know there is not a clean one to one mapping between markdown and ocamldoc syntax, but it would be nice to support even just the most basic markdown syntax.

A good way to tell if markdown syntax should be used is if the comments are of the form:

(**
 * This is a markdown comment. Typically people will always have leading
 * stars to the left, aligned in a perfect vertical line. That is probably pretty
 * well distinguished from a list of ocamldoc items.
 *)

I've noticed that most ocamldoc comments are of the form:

(** This is an ocamldoc comment. aslkdjf laskdjf lkasj dflkasj dflkasj df
    alsdkfj alskdjf lkasdj flkasj dflkaj sdflkj asdlkfj asdlfkj asd *)

Which allows you to pretty easily distinguish a list in ocamldoc format:

(** This is an ocamldoc comment
    * This is the first item in a list
    * Second item in a list *)

Is the markdown detection I suggested in the first example fairly robust?

[lpw25]

The systems I know of do not use markdown, but markdown plus some extra stuff because markdown doesn't let you express everything you need for API docs, so we'd need to give that some thought. There is also the question of which flavour markdown to use, and probably some debate to be had about markdown vs some of the other popular alternatives (e.g. rst, asciidoc, org mode).

Is the markdown detection I suggested in the first example fairly robust?

ocamldoc always supported a mode where comments had this other format, so we'd need to check to see whether much existing code is using it.

A simpler alternative to this might be to just add some of the classic markdown syntax to odoc's syntax. I think that only the reference syntax directly conflicts with existing stuff.

[avsm]

On the topic of Markdown parsing, the omd parser has recently had a rewrite and is CommonMark compliant. However, it still has a few missing features before trunk can be released, so those interested in Markdown in odoc are encouraged to contribute fixes over there so that we have a robust implementation in OCaml first.

[dbuenzli]

I don't think it's a good idea to multiply syntaxes. Use of the ocamldoc language is something that exists consistently in the community, adding new syntaxes would only add more entropy and inconsistencies across code bases. I think if you would like to change this you should pick a new one, convince the community it's a good one and provide a source level upgrade compatibility story.

But then I'm not sure markdown is such a good idea. For example inline code block are not well adapted to OCaml, I can never remember how to write a polymorphic variant with it. And as @lpw25 mentioned it misses important features that IIRC even CommonMark failed to add (e.g. section ids and references) so you'll need to invent something anyways at which point you can ask whether simply using ocamldoc isn't fine after all.

I'm surprised by @lpw25's comment regarding that ocamldoc "mode", I thought that was never supported see #267 which asks for it (but I think should not be done).

[jonludlam]

Along similar lines, I would definitely like to see odoc able to support a 'url resolution' mode, where you could use it as a preprocessor to resolve odoc references (written as a url). This would allow support documentation to link to types/modules/etc whilst being written in markdown/rst/whatever.

Even better, eventually we could run it in a mode where the links would be resolved to a canonical source - docs.ocaml.org for example - and then you could use them in blog posts and the like.

[dbuenzli]

@jonludlam something similar is requested here.

[jonludlam]

Ah yes, I'd missed that somehow. I'm thinking of some sort of uri schema: odoc:module-M.module-type-S.type-t or similar that could be replaced in one pass through a markdown/rst doc though.

[dbuenzli]

Ah yes, I'd missed that somehow. I'm thinking of some sort of uri schema: odoc:module-M.module-type-S.type-t or similar that could be replaced in one pass through a markdown/rst doc though.

Why not; the argument on the cli of the request I mentioned could actually have that syntax.

Note however that mld files are hard to beat at the documentation game since they allow checked xrefs to be performed between your API docs and your manual. They are also developer friendly since all it takes for them to be made available in rendered form to the end users is to install their sources along side the .cmti files.

I think we should rather put more effort in fleshing out the .mld support which is currently really rough on the edge than trying to support every other documentation language out there in way that is bound to be inferior from a documentation maintenance and distribution perspective.

[Khady]

A good way to tell if markdown syntax should be used is if the comments are of the form: `(** * This is a markdown comment. Typically people will always have leading * stars to the left, aligned in a perfect vertical line. That is probably pretty * well distinguished from a list of ocamldoc items. *)`

Recent versions of tuareg are using this style of comment by default. So I don't think that you could use this style as a pattern to detect what you want. https://github.com/ocaml/tuareg/blob/master/tuareg.el#L2453 In addition, it seems that using this style of comment with leading characters is bad for accessibility reasons. At least for users relying on a text-to-speach program.

[jordwalke]

@Khady

In addition, it seems that using this style of comment with leading characters is bad for accessibility reasons. At least for users relying on a text-to-speach program.

Why does it seem to be good enough for all the other languages' comments? (They also have // line comments).

@Khady: For taureg mode, shouldn't it only insert the * if the comment is not a special comment (** ? (That was my impression after reading through the ocamldoc documentation).

[jordwalke]

@lpw25 I definitely agree markdown alone won't be sufficient. Maybe we could come up with a list of features that aren't directly available in markdown, but is available in ocamldoc, and then if we can come up with at least one way to create those features as special cases of markdown (not even implementing them, just showing it's possible), it would be enough to at least get the ball rolling on the most basic features, then encourage people to help fill out the longer tail of features.

Merely adding the features to ocamldoc sounds appealing but I realized that one of the most important/common markdown constructs directly clashes with ocamldoc - the URL: [text](http://www.google.com). (Also polymorphic variants collide with markdowns inline code).
Linked content can also contain "code style" backticks too:

[`codeSyntax`](http://www.google.com)

[jordwalke]

Another solution is to just create something that is "markdown-like". It wouldn't be a superset of ocamldoc (because of the conflicts I mentioned). But this other mode would let 90% of the things people try to do, just work on the first try. The primary conflicts are links, and inline code examples. Most of ocamldoc's { } style rules could continue to be interpreted as ocamldoc styles.

This purpose of this feature request is to help newcomers by making their workflows more familiar.

I think if odoc had a mode that supported the following, in addition to most of the features of ocamldoc (where it doesn't conflict) most newcomers would be happy:

`code`

[link](url)

*italic*

**bold**

```bash
CODE_FENCES="${VAR}"
```

1. Numbered
2. List

[jordwalke]

I've created a separate task for the Reason repo to explore performing this parsing transformation in the Reason parser itself. The upside would be that all the tools like odoc and merlin will just work. The downside is that it creates different conventions for Reason users than existing OCaml users. However that tends to match the interests/demands of OCaml and Reason users respectively, and the implementation would perform an automated translation at least in one direction. Does this sound like a better approach?

[dbuenzli]

@jordwalke to be honest I'm not sure you are making your users a service.

The familiarity you are seeking for will quickly vanish as soon as those will have to write a section heading or references. They will also likely read code elsewhere and will meet an unfamiliar syntax so they will have to learn it anyways. But meanwhile you diluted the semantics of the language by providing more than one way of performing the same thing which is a cognitive and usability burden for everyone.

That said from the above list you gave it would be interesting to keep the code syntax proposal, if it can fit in the current ocamldoc syntax. This would allow to distinguish from [] in the same way we disinguish between {[ and {v (and thus allow for example highlighting and/or identifier linking in [] but no in code).

[jordwalke]

to be honest I'm not sure you are making your users a service.

Some more context: This is a highly requested feature for new Reason syntax users. (Reason is a gateway for the web community). People writing Reason tend to write doc comments in markdown even if they are aware there is some other syntax (ocamldoc) for doing so. It's natural to them and they want their code to be more approachable to their non-Reason, non-OCaml coworkers who will not take the time to look up conventions of ocamldoc. (Rust also uses markdown as the convention). Markdown doc comments remove one more point of friction for newcomers.

The familiarity you are seeking for will quickly vanish as soon as those will have to write a section heading or references.

The goal would be to increase support for mapping over markdown into ocamldoc including heading, and section heading references, and where there is no markdown equivalent, use ocamldoc syntax (unless someone has better ideas). With that solution, the user's familiarity shouldn't vanish because most of the things they can do in markdown would still work. They just would need to learn new things to do things markdown can't do. This is how various markdown blogging engines work to add extended features (Docsify, Docusaurus) so that concept shouldn't be totally foreign. We can't remove all the unfamiliarity, but why not remove as much as possible, especially when it is relatively easy to do so?

[jordwalke]

Btw, I agree about downsides of confusion for there being two ways to do things, but the appeal of doing it at the Reason syntax toolchain is that you already have different syntax convention for .re files so at least there is a clear context/boundary. If it was done in odoc itself I could imagine it being more confusing when some .ml files have ocamldoc, and some have markdown. Today Reason users already use markdown - so having Reason parser convert markdown (as much as possible) to ocamldoc doesn't really change anything, it just helps odoc get wider usage.
Edit: I don't love the fact that it would require more processing time during parsing, whereas doing it in odoc wouldn't be in the critical build path, but I think it could be worth it.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[dbuenzli]

I remain doubtful about the idea but people interested in the idea can give it a try with ocaml-doc/odoc-parser#15