Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new: Use temporary, backported trivial caching implemention in extism #307

Closed
wants to merge 1 commit into from

Conversation

johnpyp
Copy link

@johnpyp johnpyp commented Nov 29, 2023

This pr resolves extism from a temporarily-created branch backport-cache off of v0.5.x. More holistic caching improvements are already merged on extism main, which is for the upcoming 1.0 release.

The 1-line change to prompt wasmtime to use caching by default results in significant tool resolution performance gains.

In my testing, trivial calling of node goes from ~100ms to ~18.5ms.

No config change is needed, wasmtime will do the caching transparently in the background. The behavior can also be optionally disabled by editing the user-global wasmtime config, if issues arise.

Tested on linux, not sure how or if it'll work on Windows.


hyperfine tests:

# Latest proto
hyperfine --warmup 10 -- 'proto run node -- --version'
Benchmark 1: proto run node -- --version
  Time (mean ± σ):      99.2 ms ±   6.3 ms    [User: 662.5 ms, System: 79.4 ms]
  Range (min … max):    91.3 ms … 119.7 ms    29 runs


# With the change on main
hyperfine --warmup 10 -- './target/release/proto run node -- --version'
Benchmark 1: ./target/release/proto run node -- --version
  Time (mean ± σ):      18.5 ms ±   1.0 ms    [User: 7.9 ms, System: 12.7 ms]
  Range (min … max):    17.1 ms …  23.8 ms    146 runs

Results in significant tool resolution performance gains.

In my testing, trivial calling of `node` goes from ~100ms to ~18.5ms.

No config change is needed, wasmtime will do the caching transparently in the background.

Tested on linux, not sure how or if it'll work on Windows.
@@ -8,7 +8,7 @@ cached = "0.46.1"
clap = "4.4.8"
clap_complete = "4.4.4"
convert_case = "0.6.0"
extism = { version = "0.5.4" }
extism = { git = "https://github.com/extism/extism.git", branch = "backport-cache"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't publish to crates.io when using the git pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
extism = { git = "https://github.com/extism/extism.git", branch = "backport-cache"}
extism = "0.5.5"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We released a new stable you can try

@milesj
Copy link
Contributor

milesj commented Nov 29, 2023

Don't we need to set the extism cache env var for this to work?

@johnpyp
Copy link
Author

johnpyp commented Nov 29, 2023

Don't we need to set the extism cache env var for this to work?

The implementation extism is using now is using a native wasmtime "caching" function rather than manually doing so like with the previous implementation that had the cache env var. Docs: https://docs.wasmtime.dev/api/wasmtime/struct.Config.html#method.cache_config_load_default

What actually happens in this case is that if there's a ~/.config/wasmtime/config.toml specifying cache behavior, it'll use that. If there's no config.toml file, the cache is considered enabled by default and the rest of the values specified here are treated as defaults too. It uses a default cache directory (on my linux machine, ~/.cache/wasmtime) for the AOT-compiled artifacts. Technically, the AOT behavior is now an implementation detail of wasmtime using the cache api rather than AOT directly, but it's effectively the same.

Previously, wasmtime would never care about the config.toml file and caching would just be disabled by default.

@milesj
Copy link
Contributor

milesj commented Nov 30, 2023

Closing in favor of #314

@milesj milesj closed this Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants