podman machine for macOS and Windows with wasm support

[benoitf]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

I would like to use wasm when working with Podman engine.
But when using the default podman machine it's failing as there is no wasm runtime in the podman machine

Steps to reproduce the issue:

1. use like '--runtime=io.containerd.wasmedge.v1' or --runtime /usr/bin/crun-wasmflag and --platform=wasi/wasm32

Describe the results you received:
I received either crun: could not load libwasmedge.so.0: libwasmedge.so.0: cannot open shared object file
or Error: unknown flag: --runtime

Describe the results you expected:

Work out of the box using Podman machine on macOS and Windows

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

(paste your output here)

Output of podman info:

(paste your output here)

Package info (e.g. output of rpm -q podman or apt list podman or brew info podman):

(paste your output here)

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes/No

Additional environment details (AWS, VirtualBox, physical, etc.):

[gbraad]

For Fedora and Fedora CoreOS

/cc: @n1hility

[rhatdan]

@dustymabe WDYT?
@giuseppe @flouthoc What package should we add?

[rhatdan]

@dustymabe where is the site to request additional packages in coreos?

[rhatdan]

coreos/fedora-coreos-tracker#1375

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@giuseppe @flouthoc Could you see if we can get rid of the python requirement in crun-wasm.

[giuseppe]

@hydai would it be possible for wasmedge to drop the requirement for python?

[hydai]

Hi @giuseppe

1. WasmEdge doesn't contain or rely on python in our core source code. Only a standalone installer script for users who want to install wasmedge.
2. We are also not asking for python requirements in the spec file: https://src.fedoraproject.org/rpms/wasmedge/blob/rawhide/f/wasmedge.spec#_14

Could you identify which stages wasmedge ask for this?

[giuseppe]

it seems to drag python3 as an indirect dependency:

$ podman run --rm registry.fedoraproject.org/fedora-minimal:37 microdnf install wasmedge
Downloading metadata...
Downloading metadata...
Downloading metadata...
Package                                          Repository     Size
Installing:                                                         
 expat-2.5.0-1.fc37.x86_64                       updates    113.0 kB
 fmt-9.1.0-1.fc37.x86_64                         fedora     121.0 kB
 libb2-0.98.1-7.fc37.x86_64                      fedora      25.5 kB
 libedit-3.1-43.20221009cvs.fc37.x86_64          updates    108.2 kB
 libgomp-12.2.1-4.fc37.x86_64                    updates    302.5 kB
 libnsl2-2.0.0-4.fc37.x86_64                     fedora      30.9 kB
 libtirpc-1.3.3-0.fc37.x86_64                    fedora      95.5 kB
 libxcrypt-compat-4.4.33-4.fc37.x86_64           updates     92.9 kB
 lld-15.0.7-1.fc37.x86_64                        updates     27.6 kB
 lld-libs-15.0.7-1.fc37.x86_64                   updates      1.4 MB
 llvm-15.0.7-1.fc37.x86_64                       updates     14.5 MB
 llvm-libs-15.0.7-1.fc37.x86_64                  updates     26.5 MB
 mpdecimal-2.5.1-4.fc37.x86_64                   fedora     104.1 kB
 python-pip-wheel-22.2.2-3.fc37.noarch           updates      1.5 MB
 python-setuptools-wheel-62.6.0-2.fc37.noarch    fedora     729.0 kB
 python-unversioned-command-3.11.1-3.fc37.noarch updates     10.7 kB
 python3-3.11.1-3.fc37.x86_64                    updates     28.0 kB
 python3-libs-3.11.1-3.fc37.x86_64               updates     10.1 MB
 spdlog-1.10.0-3.fc37.x86_64                     fedora     150.4 kB
 wasmedge-0.11.2-1.fc37.x86_64                   updates    537.4 kB
Transaction Summary:
 Installing:       20 packages
 Reinstalling:      0 packages
 Upgrading:         0 packages
 Obsoleting:        0 packages
 Removing:          0 packages
 Downgrading:       0 packages
Is this ok [y/N]: 

it seems to be a dependency of llvm

[hydai]

Okay, seems like llvm has python deps.
We can provide a minimal version of wasmedge which remove the llvm and lld dependencies. Without this, wasmedge will not leverage the maximum performance by generating some native codes. However, it can still run wasm applications.

Does this work for you? If so, we can have it in the next release.

[giuseppe]

yes, that sounds like a decent compromise to start playing with it on CoreOS as well. Thanks for having a look

[rhatdan]

Just change the spec file to from Requires llvm to Recommends llvm. CoreOS Already blocks Recommends. Then most people who install wasmedge get what they want, good performance, and CoreOS gets to avoid installing Python.

[dm4]

Because the current wasmedgec requires LLVM, we are planning to add a new runtime-only package for WasmEdge, possibly called wasmedge-rt. Since both wasmedge and wasmedge-rt will include a binary file with the same name, libwasmedge.so.0.0.1, they will conflict with each other.

However, to remove the LLVM dependency, the CMake compile options for wasmedge and wasmedge-rt will differ. Specifically, WASMEDGE_BUILD_AOT_RUNTIME=OFF needs to be added for wasmedge-rt. Is there a way to execute different build commands for various packages in the same %build section of the RPM spec file?

I appreciate your help!

[giuseppe]

could the LLVM dependency be loaded at runtime similarly to what crun does with wasmedge? If that is possible, it would simplify packaging wasmedge and llvm support could be added later.

Otherwise, not sure how to achieve that just from the spec file. @lsm5 do you have any suggestions?

[dm4]

I just tried to add the wasmedge-rt package to our spec file and not sure if I did it right.
This is my PR: https://src.fedoraproject.org/rpms/wasmedge/pull-request/2
@hydai @lsm5 Do you have any suggestions about this PR? Thanks!

[benoitf]

hello, the PR from the previous comment has been merged
does it mean support is now available ? or does it require something more ?

[vrothberg]

@lsm5, do you know the latest state on this issue?

[lsm5]

I'm probably more of a middleman here. The wasmedge-rt subpackage does exist https://src.fedoraproject.org/rpms/wasmedge/blob/rawhide/f/wasmedge.spec#_36 . If I'm reading this right, podman-desktop would need to use wasmedge-rt package instead of wasmedge.

@benoitf please let me know if there's anything else you're looking for in this case.

[benoitf]

issue was about that wasmedge (or wasmedge-rt) would be installed by default on each macOS podman machine (so no extra setup to perform)

[lsm5]

I'm probably more of a middleman here. The wasmedge-rt subpackage does exist https://src.fedoraproject.org/rpms/wasmedge/blob/rawhide/f/wasmedge.spec#_36 . If I'm reading this right, podman-desktop would need to use wasmedge-rt package instead of wasmedge.

@benoitf please let me know if there's anything else you're looking for in this case.

One potential issue could be the handling of wasmedge dependency in crun. Please let me know if you have trouble installing wasmedge-rt.

[lsm5]

issue was about that wasmedge (or wasmedge-rt) would be installed by default on each macOS podman machine (so no extra setup to perform)

@benoitf ack, so are we good to close this? wasmedge-rt installation on podman machine by default would probably be an FCOS issue, so we'll need to take it there if need be.

[lsm5]

@benoitf ack, so are we good to close this? wasmedge-rt installation on podman machine by default would probably be an FCOS issue, so we'll need to take it there if need be.

Guess wasmedge-rt might also need to be handled in #19477

[lsm5]

Guess wasmedge-rt might also need to be handled in #19477

Looking at the Containerfile included in #19477, it only has podman-next specific stuff, and quay.io/fedora/fedora-coreos:stable as the base image. I'm not sure how things go, but I suspect wasmedge-rt will be handled in the coreos base image itself, so nothing relevant to #19477 .

[benoitf]

I thought I had commented that we should keep this issue as we need wasm both in fedora (for Windows VM) and inside FCOS (for macOS VM)

[benoitf]

and yes it's not related to #19477

Here is the expectation: podman machine init , podman machine start and then you can run commands with wasm with podman run

[lsm5]

I thought I had commented that we should keep this issue as we need wasm both in fedora (for Windows VM) and inside FCOS (for macOS VM)

ack. The fedora (for Windows VM) part is done as wasmedge-rt is available on Fedora already. The FCOS part needs to be handled on the coreos side (or whoever maintains fcos images), it can't be handled in podman, someone please correct me if I'm wrong. AFAICT, we don't maintain any coreos images. So, as I see it, there's nothing else left to do on the podman side. Please let me know if anything else is still unresolved.

[benoitf]

but this issue is like the 'track' of external issues.

We can't close it until podman machine on both Windows and macOS provides wasm engine out of the box.

if there are dependencies, we can list them here.
like 'need to update https://github.com/containers/podman-wsl-fedora' and FCOS image listed in https://builds.coreos.fedoraproject.org/streams/testing.json like https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/38.20230819.2.0/aarch64/fedora-coreos-38.20230819.2.0-qemu.aarch64.qcow2.xz including wasm runtime

[lsm5]

but this issue is like the 'track' or external issues.

We can't close it until podman machine on both Windows and macOS provides wasm engine out of the box.

if there are dependencies, we can list them here. like 'need to update https://github.com/containers/podman-wsl-fedora' and FCOS image listed in https://builds.coreos.fedoraproject.org/streams/testing.json like https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/38.20230819.2.0/aarch64/fedora-coreos-38.20230819.2.0-qemu.aarch64.qcow2.xz including wasm runtime

@benoitf sgtm, thanks

[rhatdan]

@dustymabe FYI

[dustymabe]

can someone summarize and post an update to coreos/fedora-coreos-tracker#1375 ?

[benoitf]

available since podman machine v4.8