Panic at program execution end with tfhe::generate_keys before rust 1.83
#1687
Labels
Comments
|
That is very very weird to say the least. do you have any non standard configuration ? which hardware/OS (model and version) are you using ? could you try updating your rust version with rustup update ? also cargo clean and cargo update if you have the chance and try again |
|
Also if you have a full backtrace, it’s not clear what code is triggering the panic |
|
I think there may have been a faulty linker packaged with xcode tools on Apple mac at some point, could be worth to check if there is a way to update the dev tools coming from Apple |
|
No non-standard hardware or config. Running macOS 14.1.2 on Apple M1. Here's a repro: https://github.com/amaury1093/tfhe-rs-panic. Backtrace (tried after cargo clean and cargo update): Also happens after: I'm updating Apple dev tools to see. I'll also ask other macOS users around me to try, and report back. |
|
Looks like the crash is during the server key drop that’s stored in an Arc, or some of our thread local « engines » |
|
We’ll try to repro |
|
Running in release works fine I assume ? |
Correct. I also updated to macOS to 15.0.1 along with the Apple command line tools. Panic is still here. |
|
someone tried to run this on an M1 mac in debug and could not repro the crash :/ how did you install rustup ? |
It was a long time ago, but I'm sure it was following https://www.rust-lang.org/tools/install. @IceTDrinker I know it can be hard to debug without repro. Let me know what info I can give you. However, it seems like it's only reproducible on my M1, so I'm happy to leave this issue in the background or even close it for now, until someone else confirms too. For now I'll simply run in release mode. |
|
from where I'm standing it looks like a potential issue with your machine only, so maybe some peculiar setup having an issue, I would maybe try resintalling rust if for whatever reason a binary for the std is corrupted |
|
hey @amaury1093 unless you manage to have a repro on another machine I think we'll close that issue as it seems related to your particular setup (maybe some corrupted binary somewhere, really can't say) |
|
Sounds good, go ahead and close it. |
|
Running into the same issue with MacOs 13.5.1 on the latest rustc 1.82.0. The quickstart example fails with the same error messages as @amaury1093's repo. Likewise, running |
|
hello @peitalin thanks for sharing, could you both share the version of the linker you are using ? |
|
Sure:
|
|
could you try running the minimal repro with RUST_BACKTRACE=full ? |
|
Running @amaury1093's repo: |
|
ok looks like something goes wrong when the engine gets populated 🧐 |
|
@amaury1093 could you run the same please with the full backtrace ? |
|
@peitalin do you have the Done. print in your terminal ? |
|
|
interesting you both have issues with thread locals, but not at the same point in the code |
IceTDrinker
added
bug
|
for now triaging as a bug in a dep/build tool, asked on Zulip to see if some rust experts may know what's going on here |
|
Can anyone who is able to provoke a crash run the program in gdb and show the backtrace in lldb? I think that will print the arguments to Something like |
|
Hello, I've run Then running |
|
Looks like the check expects a 16 bytes alignement but gets a pointer with insufficient alignment with only an alignment of 8 bytes |
|
Can someone who is running into the crash try running this program? #[repr(align(16))]
struct VeryAligned {
_data: [u8; 16],
}
thread_local! {
static LOCAL: VeryAligned = VeryAligned { _data: [0u8; 16] };
}
fn main() {
LOCAL.with(|local| println!("{:?}", local as *const VeryAligned));
}If the problem is just insufficiently-aligned thread-locals, this should make it clear that's what is happening. In addition, can someone who is seeing the crash try to reproduce the crash on beta (1.83.0) and nightly? |
|
Running the above program @saethlin posted on Running the program on |
|
Alright, then I guess it’s a matter of the patch hitting stable which should be in 1.83 at the latest, will keep this open until it’s released and update the title of the issue |
IceTDrinker
changed the title
tfhe::generate_keystfhe::generate_keys before rust 1.83
|
@amaury1093 and @peitalin could you try to repro with rust 1.83 ? Normally this has been fixed on the rustc side |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
Followed docs. When calling
tfhe::generate_keys, I get a panic at the end of the program execution (not during the function call). The panic is:To Reproduce
Add the correct tfhe-rs version:
Then in
src/main.rs:Expected behaviour
No error.
Actual behavior
The program ends, the keys are generated, but there's a panic at the end, after the call to generate_keys. See that the "Done." line is correctly printed.
Configuration(please complete the following information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: