Warnings to unresolved references make odoc unusable

[dbuenzli]

I was pretty sure there was an issue about this. But I can't find one. So there it is.

I think this should really be acted upon and prioritized as odoc becomes quickly unusable when working on partial docsets¹ and this problem has been here for ages.

Let's take a simple example:

> cat m.mli
(** My doc.

    The max [int] value is {!Sys.max_int}. *)

This will typically report this warning:

File "lib/m.mli", line 3, characters 27-41:
Warning: Failed to resolve reference unresolvedroot(Sys).max_int Couldn't find "Sys"

I really don't have the compilation pipeline of odoc in my head right now but I suspect there is a point where we can record this symbol exists despite the fact that no documentation is going to be generated for it by simply providing an appropriate include into the stdlib's library directory or the path to the Sys.cm[i,ti] file.

I'm pretty sure people working on odoc have a better idea than I have of when/where in the pipeline this should be done (I vaguely remember a discussion with @trefis a few years ago where he indicated this would be possible and not too hard).

(This will not work as is stands for external references into .mld but in my case it would, most of the time remove 100% of these warnings).

Footnotes

1. Case in point I have a medium scale project that generates 122 warnings from links into other libraries and the Stdlib and it is of course no longer possible to sift through that to find out if a couple of doc strings I just wrote made wrong or ambiguous links. ↩

[jonludlam]

Thanks for bringing this up - I totally agree that it's really important to focus on issues like this, otherwise we're putting people off making good docs.

I'm very skeptical of trying to solve the issue in the way you're suggesting though - essentially I don't believe we can do anything less than generating the correct odoc files for your dependencies. I think the only reliable way to distinguish between references that will and won't resolve is to do a complete docs build for all your dependencies, and then try to resolve the references!

Now, a second issue is that of partial docsets. I'm going to assume that the main reason for having partial docsets is that someone would like to publish the docs for their libraries without having to additionally publish the docs for all of their dependencies. For this, if we start from the premise that we've built docs for all our deps locally, we've got a couple of options:

1. We could continue to do what many people do today: publish docs where all links that would go to other packages are rendered as unresolved links. This would be pretty straightforward.
2. We could assume that links to other packages could be rewritten as links to ocaml.org/p/.... The downside to this is that there's no guarantee the links will be exactly correct.

I'm leaning in the second direction, but it seems likely that the path to get there would mean that implementing '1' would be trivial.

Building a complete doc set is obviously not easy, but there are already several options available. Obviously there's your own odig, which is really nicely suited to producing docs for packages once they've been installed, but not quite so useful when you're still developing the docs though. There is the dune prototype -- dune build @doc-new -- which isn't quite so battle tested, and is missing some features, but very useful (when it works) for developing your docs. Then we're also developing a driver within the odoc repo itself, although this is more intended for developing features in odoc than as a dev tool for documentation authors -- though this is likely to be the most 'feature complete' driver at any moment, and is likely a good candidate for building docs for publishing. It could even end up replacing chunks of voodoo which is what we use to build the docs for ocaml.org.

I'm curious what you're using to build and test your docs?

Anyway, clearly, fixing this isn't the only thing we need to do to make the experience of writing docs better - the duplicates @dlesbre mentioned in #1127 need a separate fix - but I'm very keen to work on these sorts of issues with high priority, as you suggest.

[dbuenzli]

essentially I don't believe we can do anything less than generating the correct odoc files for your dependencies.

If your positive that's the way, then so be it (though I fear a bit that the time to generate simple docs can explode a bit if you depend on large stuff like core). However it would perhaps be nice to try to characterize a bit more what counts as a dependency and how to get them:

1. Regarding the dependencies to render module signatures, I suspect it corresponds to the dependencies for the compilation phase we get via the new -H flag, see here.
2. Then you have the links to the .mld pages of other packages and possible references to modules that are not part of code dependencies of the modules. We need odoc to be able to spit them out in some way for the driver to resolve them.

Regarding partial docsets I think we should really have them, I'm fine with 1. as far as generation is concerned rather than trying to be smart but failing too often.

I'm curious what you're using to build and test your docs?

At the moment I'm using the brzo tool. It was not made for that but it works quite well in practice since my packages are rather simple. The brzo driver, which shares some of the code with odig via the b0 project is here (high-level description). I guess I could add the dependencies that brzo found to get rid of the warnings but then I'd really like to be able to generate partial docsets.

[NicNomadic]

1. We could continue to do what many people do today: publish docs where all links that would go to other packages are rendered as unresolved links. This would be pretty straightforward.

[I react here as the solution to this issue could also solve #1144]

In think this simple solution would be acceptable (and even good!) if we can define somewhere (options, config file, etc) a list of external libraries. Otherwise, how can odoc distinguish between an external lib and a misspelled lib among the ones we are just submitting?