wip, adding rust

[ZuseZ4]

No description provided.

[ZuseZ4]

For the Rust side we have changes to rustc itself, so there is no way to just link Enzyme in.
Shall I thus just change
setProperty "compiler.clang$version-enzyme-$branch.exe" "/opt/compiler-explorer/$compiler/bin/clang++" "/tmp/ce/c++.enzyme.properties" to link to the cargo (rustc?) binary in the /app folder?

Also, could you make me a contributor please, so I can create PRs against this repo directly and see what CI says?

@tgymnich
cc @wsmoses

[jedbrown]

I'd like to get this going and would be able to host the builder for new rust-enzyme images. Are there docs somewhere for what is needed, or someone who can advise assuming I can build a Rust toolchain in whatever environment you like?

[wsmoses]

@jedbrown @ZuseZ4 you both now have write permissions to the repo.

Not sure what else is required, would probably be a question for @tgymnich

[tgymnich]

For deployment somebody just needs to run update.sh on the enzyme-ce machine (just ping me). Also feel free to reach out if I can help in any way.

[wsmoses]

If possible, I'd like to reduce the frequency of the rust rebuilds to nightly (rather than the Enzyme rebuild which does much more frequently), to ensure latency of EnzymeCore updates is reasonable.

[ZuseZ4]

I don't think it can be reasonably built from scratch on a 4c gh runner, so I think for now it's better if Jed hosts it completely and we just download artifacts here. I feel like once Enzyme can be build against the pre-build LLVM from Rust's official download-CI it would make sense to move it over.

Also, Enzyme is just a .so here, so I assume it should later be possible to silently delete it and replace it by a newer LLVMEnzyme.so, without rebuilding Rust (assuming no breaking Enzym change).

[wsmoses]

Is there anything preventing that right now? Enzyme works on llvm main

[ZuseZ4]

Yes, it's the discussion we had here: Nvm,we had it in an Enzyme weekly meeting.
Let me create a EnzymeAD/rust issue.

[wsmoses]

I think you forgot to add a link since I don't see anything after the colon. In any case, if you're referring to the MSVC issue, that only makes it difficulty to build with MSVC in windows. Since the this is built with clang/gcc (and on Linux), it should be fine.

[ZuseZ4]

created Rust issue (though it could live in Enzyme core too)
EnzymeAD/rust#79

[ZuseZ4]

Given that there is no progress on building Enzyme on Rust's LLVM, lets keep the Rust version fixed.
@jedbrown As you just mentioned in the chat, can you precompile Rust/LLVM/Enzyme, s.t. we can add the artifacts here? On every Enzyme update I would then just delete and replace the old LLVMEnzyme-.so
cc @tgymnich, since I don't know what's the best way to add the precompiled Rust to the image.

[tgymnich]

@ZuseZ4 Just download it inside https://github.com/EnzymeAD/enzyme-explorer/blob/main/builder/install-compilers.sh. The binary should run on ubuntu 22.
Just put somewhere in /opt/compiler-explorer/ and add a version number. This is how the dir currently looks like.

root@1124d56899b4:/app# cd /opt/compiler-explorer/
clang-10.0.1/                    clang-15.0.0/                    clang-9.0.1/                     clang-assertions-trunk-20240130/ gcc-10.2.0/                      main/
clang-11.0.1/                    clang-16.0.0/                    clang-assertions-trunk/          clang-assertions-trunk-20240206/ julia-1.8.5/                     mlir-16.0.0/
clang-12.0.1/                    clang-17.0.1/                    clang-assertions-trunk-20230925/ clang-assertions-trunk-20240218/ julia-1.9.3/                     staging/
clang-13.0.1/                    clang-7.1.0/                     clang-assertions-trunk-20231007/ clang-assertions-trunk-20240225/ juliapackages/
clang-14.0.0/                    clang-8.0.1/                     clang-assertions-trunk-20231218/ cuda/                            libs/

[ZuseZ4]

@jedbrown Shall we give this another try, now that we have a working Rust CI? I assume it should be possible to just pull in the latest artifact from /rust?

[jedbrown]

Thanks. It's probably possible to tar up the whole build directory, but I wanted to get x dist working because that's clearly the right way to distribute, and necessary for upstreaming.

[ZuseZ4]

@tgymnich @wsmoses A rustc dev fixed the dist build for us, so now we have a rust.tar.gz file with 320MB which includes all that we need. We'd just need to download and unpack it. Do you have a good place to where we could upload the tar, since we currently still generate those by hand?

[tgymnich]

@ZuseZ4 you could add the rustc tar as a binary artefact to a release.

[tgymnich]

closed-ish by 6f73fd0

[wsmoses]

hm something is still awry:

error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0463`.
Compiler returned: 1

[ZuseZ4]

Yep, but we're aware of it and looking into fixes.