Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unlocked deps #393

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions design/mvp/WIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -661,6 +661,80 @@ world w2 {
> configure that a `use`'d interface is a particular import or a particular
> export.

## Unlocked Dependency Imports

When working with a registry, the keyword `unlocked-dep` is available to specify dependencies with package name and version requirements. For example:

```wit
world w {
unlocked-dep foo:bar@{>=x.x.x <y.y.y};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intended to support foo:bar here exactly? Or is a /interface required as well?

Copy link
Member

@lukewagner lukewagner Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that, when /interface is absent, the expectation could be that a registry is used to resolve ns:package to a component and the interface is the exports of that component (which have no name).

More hypothetically, if we get unnamed interfaces/worlds in WIT, /interface could also be absent without relying on a registry by using a nested package of the form:

package foo:bar {
  interface { ... }
}
world w {
  unlocked-dep foo:bar@{...}
}

WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But perhaps this is a good opportunity to make clear that the optional /interface is possible as well, and maybe talk a bit about in which cases toolchains would create an unlocked-dep with and without an interface projection? If we wanted to scope it in, folks could specify an interface when they add a dep and tools could only grab interfaces they need from the registry. And also the lock or bundle command could shake out unused interfaces in cases as well. May be more than we want for the initial discussion, but I was thinking that folks may find it confusing to only see a component foo:bar when they're accustomed to depending on interfaces at the moment.

Copy link
Member

@lukewagner lukewagner Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamentally, an unlocked-dep (in the component importname) names a component via package name and so, for a simple illustrative example like this, I think makes sense to similarly start with unlocked-dep naming a package. (Yes, it's different that regular interface imports, but being different is the point.) The /interface only shows up after inlining registry contents and is only necessary due to current expressive limitations of WIT; ideally it wouldn't ever be necessary.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this spec out the interface { ... } change as well in that case? That's not currently implemented or sketched out here, so I think that should be included too if that's the intention. (also could this update the ebnf for unlocked-dep in worlds too?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be up for that; it would simplify things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great I should be able to add some of these updates soon.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an alternative to adding unnamed interfaces/worlds (which open up another set of design questions), we could instead add a bit of also-useful syntax that allows the WIT to capture precisely how the range query was resolved:

package gh:sqlite@1.1.1 {
  interface exports { ... }
}
world w {
  import dependency gh:sqlite@{>=1.0.0} = gh:sqlite/[email protected];
}

This = <pkgname> right-hand-side of the import dependency is necessary for advanced cases where the WIT needs to nest multiple versions of the same package such that the <pkgnamequery> alone would be ambiguous. Importantly for our purposes here, though, none of the name/version info in the = <pkgname> shows up in the import so our world w above just contains (import "unlocked-dep=<gh:sqlite@{>=1.0.0}>" (instance ...)) (no /exports, no 1.1.1), and we didn't need anonymous interfaces to achieve that.

}
```

The binary format has a corresponding [import definition](Explainer.md#import-and-export-definitions) and this WIT syntax informs
bindgen tooling that it should be used.

The point of `unlocked-dep` is to specify a dependency on a _component implementation_ (or a semver *range* of implementations), rather than on an abstract WIT interface (with an unspecified implementation).

### Example Unlocked Dependency Workflow
Let's say someone authoring a Rust component `my:component` targeting the `wasi:http/proxy` world adds a dependency on another component `foo:bar` by adding the following (hypothetical) lines to their `Cargo.toml`:

```
[package.metadata.component.target]
target = "wasi:http/[email protected]"

[package.metadata.component.dependencies]
"foo:bar" = "1.2"
The language toolchain would first generate a new world that augments the target world with an additional unlocked dependency:
```wit
package my:component;
world generated-world {
include wasi:http/[email protected];
unlocked-dep foo:bar@{>=1.2.0};
}
```
macovedj marked this conversation as resolved.
Show resolved Hide resolved

Now say that `foo:[email protected]` implements the following `exports` interface:
```wit
package foo:[email protected];
interface exports {
calc: func(x: u64) -> u64;
}
```

The next step is to expand `generated-world` with nested packages fetched from all the relevant registries, producing an all-in-one WIT file with no external references:
```wit
package my:component;

package wasi:[email protected] {
...
world proxy { ... }
}

package foo:[email protected] {
interface exports {
calc: func(x: u64) -> u64;
}
}

world generated-world {
include wasi:http/[email protected];
unlocked-dep foo:bar/exports@{>=1.2.0};
}
```
Note that `exports` is an arbitrary name and can be anything because the WIT bindings generation will always strip off the final interface name, leaving only the package name. In particular, the Component Model type for this world is:
```wat
(component
(import "wasi:http/[email protected]" (instance ...))
(import "wasi:http/[email protected]" (instance ...))
(import "unlocked-dep=<foo:bar@{>=1.2.0}>" (instance
(export "calc" (func (param "x" u64) (result u64)))
))
(export "wasi:http/[email protected]" (instance ...))
)
```

A wasm component that contains `unlocked-dep` imports is referred to as an "unlocked component". Unlocked components are what you normally would want to publish to a registry, since it allows users of the unlocked component to perform the final dependency solving across a DAG of components.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to my comment on this PR, but an "unlocked component" really doesn't seem self-explanatory to me. Language like, "coupled component" or "tightly coupled component" seem to more readily express the fact that the component depends on implementations rather than interfaces.

## WIT Functions
[functions]: #wit-functions

Expand Down
Loading