Add compilation to NodeJS module #111

smiasojed · 2024-11-06T14:06:18Z

Cross-compile the resolc binary to WebAssembly (WASM) using Emscripten.

athei · 2024-11-08T15:07:01Z

I guess it was an accident that you checked in the build outputs?

athei

I think after this initial PR you should look into cross language LTO to bring the code size down (for web). As of right now the release profile isn't even using LTO for the rust code. Adding wasm-opt to the pipeline will also help a lot.

js/resolc.wasm

.github/workflows/build-revive-wasm.yml

smiasojed · 2024-11-08T15:24:28Z

I guess it was an accident that you checked in the build outputs?

I did it to share the output with @pgherveou

.github/workflows/build-revive-wasm.yml

athei · 2024-11-18T11:39:10Z

.github/workflows/build-revive-wasm.yml

+          # Install Emscripten
+          git clone https://github.com/emscripten-core/emsdk.git
+          cd emsdk
+          ./emsdk install ${{ env.EMSCRIPTEN_VERSION }}
+          ./emsdk activate ${{ env.EMSCRIPTEN_VERSION }}


We should check out a specific tag here. Otherwise we build against a different version every time.

we are using the version defined in EMSCRIPTEN_VERSION env which is set now to the rev fd61bacaf40131f74987e649a135f1dd559aff60

athei · 2024-11-18T11:57:18Z

.github/workflows/build-revive-wasm.yml

+          path: |
+            ${{ env.REVIVE_WASM_INSTALL_DIR }}/resolc.js
+            ${{ env.REVIVE_WASM_INSTALL_DIR }}/resolc.wasm


Are those all the files that are needed? What about the hand written javascript files in /js? Or are you planning to do the full npm package release work flow as follow up?

I will add npm package with its workflow in next PR. Now we need these 2 files only.

crates/solidity/Cargo.toml

xermicus

Thanks for pulling this over the line! Here's a quick first review and some nitpicks

xermicus · 2024-11-19T12:57:19Z

README.md

+
+### Cross-compilation to WASM
+
+Cross-compiles the Revive recompiler to WASM to enable it to run in a Node.js or browser environment.


Suggested change

Cross-compiles the Revive recompiler to WASM to enable it to run in a Node.js or browser environment.

Cross-compiles the revive compiler to Wasm for running it in a Node.js or browser environment.

xermicus · 2024-11-19T13:00:48Z

build-llvm.sh


 # Build LLVM, clang
 LLVM_SRC_PREFIX=${PWD}/llvm-project
 LLVM_SRC_DIR=${LLVM_SRC_PREFIX}/llvm
-LLVM_BUILD_DIR=${PWD}/build/llvm


Why is this changed? Before the target build directory was outside the llvm-project source directory

I have aligned it to this what i have in emscripten build:
LLVM_NATIVE=$LLVM_SRC/build-native
LLVM_WASM=$LLVM_SRC/build-wasm
We can reorganise it if you want

Yes please I think it is a standard practice to keep the build outside the source, i.e. leave it as it is now.

xermicus · 2024-11-19T13:04:01Z

build-llvm.sh

@@ -5,17 +5,12 @@ set -euo pipefail
 INSTALL_DIR="${PWD}/llvm18.0"
 mkdir -p ${INSTALL_DIR}

-
-# Clone LLVM 18 (any revision after commit bd32aaa is supposed to work)
-if [ ! -d "llvm-project" ]; then


Should clone into $LLVM_SRC_PREFIX instead of a hardcoded path (if it goes into a dedicated script it should be an argument instead)

crates/lld-sys/build.rs

xermicus · 2024-11-19T13:12:19Z

crates/solidity/src/process/mod.rs

-            )
-        })?;
-    Ok(output)
+pub trait Process {


Trying to understand why is this a trait? Because everywhere else it seems we decide on the configured target. But the trait makes it theoretically possible to use the worker process in native and vice-versa?

It doesn't need to be a trait now. I was considering introducing a Config trait with associated types for the compiler and process, but I haven't decided to add it yet. it should reduce the number of #cfgs.

Ah okay, I think both would work but I think the set of targets we want to support is finite so the feature gates should do fine. No hard opinion on which solution, but can you please make it consistent everywhere?

I don't see any issues with having a trait that provides a common interface for both native and worker processes. Would you like me to remove it? Similarly both SolcCompiler and SoljsonCompiler implement the Compiler trait, but in this case it is already used.

I'm just trying to understand the use case for the trait? I.e. it seems like we can just have the Wasm version gated behind a feature.

This makes it consistent with the rest of the codebase, where we gate the emscripten specific code behind features.

I guess the trait would be useful if we need to add many more but I can't think of anything else?

If it works and you want to leave it, fine for me. I just noticed this is a bit inconsistent.

emscripten-build-llvm.sh

Co-authored-by: Cyrill Leutwiler <[email protected]>

xermicus

LGTM

clone-llvm.sh

Co-authored-by: Cyrill Leutwiler <[email protected]>

smiasojed added 20 commits August 29, 2024 17:28

Initial wasm support

5ac67bd

Merge remote-tracking branch 'origin/main' into resolc.js

88a888d

Fix Cargo.toml

9b23e19

Make native version to compaile again

4f6debc

Fmt

d260472

Update solc compiler version check

b7b28ef

Make wasm version to compile

b6baf6c

Add temoprary wasm compilation output

a934ec2

Fix compilation for wasm target

94445ba

Add GHA for wasm target

c51d50b

Update wasm GHA

c0a82ce

Update GHA for wasm build

8c7d18a

Use bash in GHA

007b79e

Fix GHA emsdk path

1837643

Update cmake versions in GHA

7f3d0ce

Install llvm in GHA

9a8003a

Remove old llvm-15 from GHA

2551769

Remove llvm-15 env from GHA

90423ff

Switch GHA to ubuntu

677aedc

Fix deps in GHA

8a22587

athei reviewed Nov 8, 2024

View reviewed changes

js/resolc.wasm Outdated Show resolved Hide resolved

.github/workflows/build-revive-wasm.yml Outdated Show resolved Hide resolved

.github/workflows/build-revive-wasm.yml Outdated Show resolved Hide resolved

smiasojed added 5 commits November 12, 2024 09:47

Log LLVM version in GHA

93d2f3b

Add Missing dep to GHA

6e6fe20

Add ltinfo dep to GHA

f59b47d

Removed not needed libs from linking process

64fefe7

Update GHA

6d16790

athei reviewed Nov 12, 2024

View reviewed changes

.github/workflows/build-revive-wasm.yml Show resolved Hide resolved

Fix CI for wasm path

14991f4

smiasojed added 2 commits November 18, 2024 11:07

Remove libsolc crate

87dd77b

Refactor soljson compiler

d88ddb2

smiasojed marked this pull request as ready for review November 18, 2024 11:20

smiasojed requested review from xermicus and athei November 18, 2024 11:21

athei reviewed Nov 18, 2024

View reviewed changes

smiasojed added 3 commits November 18, 2024 13:59

Fix parallel feature

3072c03

Add worker code to resolc.js

63da721

Add test with standard json args

39e5047

xermicus reviewed Nov 19, 2024

View reviewed changes

smiasojed and others added 8 commits November 20, 2024 10:27

Add stdin support

f36d62c

Fix stdin support

cdebf69

Improve description

7740626

Merge branch 'main' into resolc.js

fcbe00f

Update readme file

4918507

Add directory arg support to clone-llvm

0a82add

Fix build path

a772ff1

Fix build script

32d4b23

xermicus reviewed Nov 21, 2024

View reviewed changes

emscripten-build-llvm.sh Outdated Show resolved Hide resolved

smiasojed and others added 2 commits November 21, 2024 12:49

Update emscripten-build-llvm.sh

8990b2a

Co-authored-by: Cyrill Leutwiler <[email protected]>

Update comment

05925f2

xermicus approved these changes Nov 22, 2024

View reviewed changes

clone-llvm.sh Show resolved Hide resolved

smiasojed and others added 4 commits November 22, 2024 19:52

Update clone-llvm.sh

892c9b5

Co-authored-by: Cyrill Leutwiler <[email protected]>

Add support for esm and cjs modules

4e02465

Merge remote-tracking branch 'origin/main' into resolc.js

229c0d4

Fix npm build

4c0c74f

smiasojed requested a review from athei November 25, 2024 18:37

athei approved these changes Nov 26, 2024

View reviewed changes

xermicus approved these changes Nov 26, 2024

View reviewed changes

smiasojed merged commit 1eb1083 into paritytech:main Nov 26, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add compilation to NodeJS module #111

Add compilation to NodeJS module #111

smiasojed commented Nov 6, 2024 •

edited

Loading

athei commented Nov 8, 2024

athei left a comment •

edited

Loading

smiasojed commented Nov 8, 2024

athei Nov 18, 2024

smiasojed Nov 18, 2024

athei Nov 18, 2024

smiasojed Nov 18, 2024 •

edited

Loading

xermicus left a comment

xermicus Nov 19, 2024

smiasojed Nov 21, 2024

xermicus Nov 19, 2024

smiasojed Nov 19, 2024

xermicus Nov 21, 2024

smiasojed Nov 21, 2024

xermicus Nov 19, 2024

smiasojed Nov 21, 2024

xermicus Nov 19, 2024 •

edited

Loading

smiasojed Nov 19, 2024

xermicus Nov 21, 2024 •

edited

Loading

smiasojed Nov 21, 2024

xermicus Nov 21, 2024

xermicus left a comment


		### Cross-compilation to WASM

		Cross-compiles the Revive recompiler to WASM to enable it to run in a Node.js or browser environment.

	Cross-compiles the Revive recompiler to WASM to enable it to run in a Node.js or browser environment.
	Cross-compiles the revive compiler to Wasm for running it in a Node.js or browser environment.

Add compilation to NodeJS module #111

Add compilation to NodeJS module #111

Conversation

smiasojed commented Nov 6, 2024 • edited Loading

athei commented Nov 8, 2024

athei left a comment • edited Loading

Choose a reason for hiding this comment

smiasojed commented Nov 8, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smiasojed Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

xermicus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xermicus Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xermicus Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xermicus left a comment

Choose a reason for hiding this comment

smiasojed commented Nov 6, 2024 •

edited

Loading

athei left a comment •

edited

Loading

smiasojed Nov 18, 2024 •

edited

Loading

xermicus Nov 19, 2024 •

edited

Loading

xermicus Nov 21, 2024 •

edited

Loading