quarto run fails because of --cached-only and dependency version unfound

[cderv]

@cscheid This happens only with our bundled version where we activate --cached-only. This is why our test is passing, especially

quarto-cli/tests/smoke/run/stdlib-run-version.test.ts

Lines 1 to 24 in 464c18c

	/*
	* stdlib-run-version.test.ts
	*
	* Copyright (C) 2022 Posit Software, PBC
	*
	*/
	import { execProcess } from "../../../src/core/process.ts";
	import { assert } from "testing/asserts";
	import { unitTest } from "../../test.ts";
	unitTest("stdlib-run-version", async () => {
	const result = await execProcess({
	cmd: [
	"quarto",
	"run",
	"docs/run/test-stdlib.ts",
	],
	});
	console.log({result})
	assert(result.success);
	}, {
	ignore: Deno.build.os == "windows",
	});

Trying to run the file using last prerelease I get

❯ C:\Users\chris\AppData\Local\qvm\versions\v1.6.30\bin\quarto run .\tests\docs\run\test-stdlib.ts
error: JSR package version manifest for '@std/[email protected]' failed to load: Specifier not found in cache: "https://jsr.io/@std/io/0.224.9_meta.json", --cached-only is specified.
    at file:///C:/Users/chris/Documents/DEV_R/quarto-cli/tests/docs/run/test-stdlib.ts:1:27

My understanding is the following

• We are using bundling versions and it seems we do bundle 0.224.8
  

  quarto-cli/src/dev_import_map.json

  Line 19 in 464c18c

  "io/": "./vendor/jsr.io/@std/io/0.224.8/mod.ts/",

• However the spec for stdlib/io used for loading the dep in a quarto run script is
  

  quarto-cli/src/resources/deno_std/run_import_map.json

  Line 30 in 464c18c

  "stdlib/io": "jsr:/@std/io@^0.224.0",

  
  Using the ^ means that 0.224.9 is valid and it seems Deno wants do use it, but can't find it in the cache as expected.

Maybe I don't get it right, but it seems that Deno may resolve the specifier, unrelated to the cache, instead of looking for a version in cache that respect ^0.224.0 🤔

I did the test with deno run, using deno binary bundled in quarto and using this file

import { format } from "jsr:@std/[email protected]"

and in a clean cache (I deleted the all Deno cache directory)

• First run using cached only fails

  ❯ C:\Users\chris\scoop\apps\quarto-prerelease\current\bin\tools\x86_64\deno.exe run --cached-only test.ts
  error: JSR package version manifest for '@std/[email protected]' failed to load: Specifier not found in cache: "https://jsr.io/@std/io/0.224.8_meta.json", --cached-only is specified.
      at file:///C:/Users/chris/Documents/DEV_OTHER/00-TESTS/test-quarto/test.ts:1:24
  

• Second run without --cached-only download the dependency
• For third run I changed to jsr:@std/io@^0.224.0 in the file

  import { format } from "jsr:@std/io@^0.224.0"

  and it fails the same as with Quarto

  ❯ C:\Users\chris\scoop\apps\quarto-prerelease\current\bin\tools\x86_64\deno.exe run --cached-only test.ts
  error: Specifier not found in cache: "https://jsr.io/@std/io/0.224.9/mod.ts", --cached-only is specified.
      at file:///C:/Users/chris/Documents/DEV_OTHER/00-TESTS/test-quarto/test.ts:1:24
  

This is directly impacting tools like shinylive which needs to be updated anyway to be working with Quarto 1.6
This is blocking quarto-ext/shinylive#65

[cscheid]

Confirmed fixed by @cderv in the latest 1.6 prereleases.

[cderv]

@cscheid I am reopening because I found this is not totally fixed while I investigated shinylive change to make for new Deno run deps (working on posit-dev/shinylive#185 with @georgestagg).

Sorry to have missed that a few weeks ago.. I am not sure what change - I was pretty sure to have tested the initial fix !
But it seems running with an installed quarto the file from within quarto-cli project directly is not throwing the issue (while from anywhere it is)

So take the same test.ts file content than tests\docs\run\test-stdlib.ts

import { readLines } from "stdlib/io";

So copy it in a clean directory (I mean outside of quarto-cli project) and using a bundled version (not dev version) it still errors

❯ quarto run test.ts
error: JSR package version manifest for '@std/[email protected]' failed to load: Specifier not found in cache: "https://jsr.io/@std/bytes/1.0.4_meta.json", --cached-only is specified.
    at https://jsr.io/@std/io/0.224.8/read_range.ts:3:35

This is because of the dependency tree.

• std/[email protected] depends on std/bytes@^1.0.2 : https://jsr.io/@std/[email protected]/dependencies
• But the PR use strict version specifiers in import maps #11222 only fixes the specifier part which is now for stdlib/io
  

  quarto-cli/src/resources/deno_std/run_import_map.json

  Line 30 in bf93178

  "stdlib/io": "jsr:/@std/[email protected]",

  
  which was updated in the PR use strict version specifiers in import maps #11222 in addition of removing specifier from std/[email protected] to std/[email protected]
• Though dependency was not updated as we still vendor in cache std/[email protected] (which was used by previous std/[email protected])
  

  quarto-cli/src/resources/deno_std/run_import_map.json

  Line 5 in bf93178

  "stdlib/bytes": "jsr:/@std/[email protected]",

So we probably need to check the dependency tree of what we do import for cached only to run.

[cderv]

In fact we do have two versions of bytes in https://github.com/quarto-dev/quarto-cli/tree/main/src/vendor/jsr.io/%40std/bytes
(is that expected ? 🤔 )

And, std/io requires concat and copy and they are in https://github.com/quarto-dev/quarto-cli/tree/main/src/vendor/jsr.io/%40std/bytes/1.0.2 so they are correctly vendored (I think)

Though the run_import_map.json only resolve to the 0.224.0 one. I guess I don't know yet enough to correctly fix this myself 😅

[cscheid]

I just tested the 1.6.37 release, and this works there. Unfortunately, the first time you run a project script that imports (some) files from the standard library you might see output like this:

$ quarto run test.ts
Download https://jsr.io/@std/bytes/1.0.4_meta.json
Download https://jsr.io/@std/bytes/1.0.4/copy.ts
Download https://jsr.io/@std/bytes/1.0.4/concat.ts
$ quarto run test.ts
$

Further runs, however, will use the recently-downloaded files from the cache.