Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow doctest to refer to external crates (in context of a Cargo project) #19

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

bobhy
Copy link

@bobhy bobhy commented Dec 8, 2024

Addresses issue #9 (maybe?)
Certainly allows me to use external crates naturally in my book samples, when embedded in a Cargo project.

The symptom I had was compiler "use of undefined external crate", even though the crate was correctly configured as a dependency in my Cargo.toml.

With this fix:

  1. explicitly configure manifest_dir in your book.toml (this tells mdbook-keeper to read the Cargo.toml, which it does not do by default).
  2. optionally, also configure target_dir. It now defaults to target/debug (but only if manifest_dir was configured above).
  3. Develop your doctests under mdbook serve as usual. Refrain from running mdbook test! This runs mdbook-keeper' (which works) but then a second test pass via rustdocwithinmdbook, which will generate all the old external crate` errors, making you think the plug in is not working (I was doing this for a while and was horribly confused).

Summary of changes

The big issue is that mdbook-keeper wasn't invoking rustc with --externs, because it wasn't finding the extern dependencies that were declared in Cargo.toml. The format of the package ID in the Fingerprint seems to have changed since rust-skeptic analyzed it: it no longer uses whitespace delimiters, rather it seems to be using <registryOrFilePath>#<libname>@<version> (or <registryOrFilePath>#<version> when package inherits version from the Cargo workspace definition, see comments in LockedDep iterator). This is all reverse engineering, I make no promises I got it right, or that it will stay right.

Future directions

I'd like to move this logic into mdbook itself and avoid the need for mdbook-keeper when all you want is external crates in book code samples (and who doesn't?) As the redundant mdbook test behavior shows, the preprocessor plugin interface is not the best place for the doctest logic.
But the current dependency parsing logic depends on unstable internal interfaces from Cargo, I can well understand mdbook team will be reluctant, we need a stable interface from Cargo.
I'm currently looking at the 'Compiler' interface in the Cargo api to see if it can be used to invoke mdBook; that would get Cargo to provide the list of -L and --externargs that we need. But the moment, it looks like it's forrustdocandrustc` only. Any advice would be appreciated!

bobhy added 3 commits December 7, 2024 23:11
…t` from running rudundantly and producing bogus errors.
1. LockedDep iterator: track change in metadata for dependency ID.
it's no longer whitespace based.
2. (same): also handle package in cargo workspace inheriting version
from workspace (ID doesn't have `@`)
3. KeeperConfigParser: change default for target_dir to `target/debug`
if manifest_dir is specified, leave it as `target` if not. (arguable)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant