Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failures on the (big-endian) s390x architecture #23

Closed
musicinmybrain opened this issue May 7, 2024 · 16 comments · Fixed by Blosc/c-blosc2#606 or #24
Closed

Test failures on the (big-endian) s390x architecture #23

musicinmybrain opened this issue May 7, 2024 · 16 comments · Fixed by Blosc/c-blosc2#606 or #24

Comments

@musicinmybrain
Copy link
Contributor

While testing blosc2-rs in COPR for Fedora, with the src/lib.rs changes from #22, I found that there are several test failures on the s390x architecture. Since this is the only big-endian primary architecture in Fedora, these are likely due to implicit assumptions about host endianness.

I don’t have easy interactive access to real s390x hardware, but I can run any experiments I need to in the form of package builds. Also,

sudo dnf install qemu-user-static
mock -r fedora-rawhide-s390x --dnf -i package1 package2 package3
mock -r fedora-rawhide-s390x --shell --enable-network

is a convenient approach to debugging interactively in an emulated environment on a Fedora workstation. (If you haven’t used mock before, you’ll need to install it, run sudo useradd -G mock -a $(whoami), and then log out and back in to pick up the new supplementary group.)

I will paste the full test output into a comment.

@musicinmybrain
Copy link
Contributor Author

running 10 tests
test tests::test_basic_roundtrip ... ok
test tests::test_basic_roundtrip_into ... ok
test tests::test_compress_ctx ... ok
test tests::test_compress_into_ctx ... ok
test tests::test_decompress_ctx ... ok
test tests::test_decompress_into_ctx ... ok
test tests::test_get_complib_version_string ... ok
test tests::test_get_version_string ... ok
test tests::test_schunk_basic ... FAILED
test tests::test_schunk_write ... ok

failures:

---- tests::test_schunk_basic stdout ----
Error: Other("Failed to get schunk from buffer; might not be valid buffer for schunk")


failures:
    tests::test_schunk_basic

test result: FAILED. 9 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

error: test failed, to rerun pass `--lib`
   Doc-tests blosc2
     Running `/usr/bin/rustdoc --edition=2021 --crate-type lib --crate-name blosc2 --test src/lib.rs --test-run-directory /builddir/build/BUILD/blosc2-rs-0.2.6+2.14.3 -L dependency=/builddir/build/BUILD/blosc2-rs-0.2.6+2.14.3/target/rpm/deps -L dependency=/builddir/build/BUILD/blosc2-rs-0.2.6+2.14.3/target/rpm/deps -L /usr/lib64 -L native=/usr/lib64 --extern blosc2=/builddir/build/BUILD/blosc2-rs-0.2.6+2.14.3/target/rpm/deps/libblosc2-5d9a57e7b057098b.rlib --extern blosc2_sys=/builddir/build/BUILD/blosc2-rs-0.2.6+2.14.3/target/rpm/deps/libblosc2_sys-e59d33f9d181775d.rlib --extern ctor=/builddir/build/BUILD/blosc2-rs-0.2.6+2.14.3/target/rpm/deps/libctor-cd001ce23d059acc.so --extern parking_lot=/builddir/build/BUILD/blosc2-rs-0.2.6+2.14.3/target/rpm/deps/libparking_lot-37cfcfee224bec74.rlib --extern rand=/builddir/build/BUILD/blosc2-rs-0.2.6+2.14.3/target/rpm/deps/librand-862b4067dc29d4ca.rlib -C embed-bitcode=no --cfg 'feature="default"' --cfg 'feature="shared"' --error-format human`

running 16 tests
test src/lib.rs - Context::get_cparams (line 1298) ... ok
test src/lib.rs - CParams (line 1164) ... ok
test src/lib.rs - DParams (line 1257) ... ok
test src/lib.rs - Context::get_dparams (line 1319) ... ok
test src/lib.rs - getitems (line 1445) ... ok
test src/lib.rs - schunk::Chunk (line 435) ... ok
test src/lib.rs - schunk::Chunk::compress (line 678) ... ok
test src/lib.rs - schunk::Chunk::compression_ratio (line 661) ... ok
test src/lib.rs - schunk::Chunk::from_vec (line 486) ... ok
test src/lib.rs - schunk::Chunk::decompress (line 700) ... FAILED
test src/lib.rs - schunk::Chunk::len (line 527) ... ok
test src/lib.rs - schunk::Chunk::repeatval (line 591) ... FAILED
test src/lib.rs - schunk::Chunk::uninit (line 558) ... FAILED
test src/lib.rs - schunk::Chunk::zeros (line 626) ... FAILED
test src/lib.rs - schunk::Storage (line 354) ... ok
test src/lib.rs - schunk::Storage::get_urlpath (line 389) ... ok

failures:

---- src/lib.rs - schunk::Chunk::decompress (line 700) stdout ----
Test executable failed (exit status: 101).

stderr:
thread 'main' panicked at src/lib.rs:9:49:
called `Result::unwrap()` on an `Err` value: Other("Appears this buffer is not a valid blosc2 chunk")
stack backtrace:
   0:      0x2aa027b7584 - std::backtrace_rs::backtrace::libunwind::trace::hb5658511ef1cc65f
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/../../backtrace/src/backtrace/libunwind.rs:105:5
   1:      0x2aa027b7584 - std::backtrace_rs::backtrace::trace_unsynchronized::h4eb31789c87f8ba2
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:      0x2aa027b7584 - std::sys_common::backtrace::_print_fmt::hfb5971ceebe2fba6
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:68:5
   3:      0x2aa027b7584 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h8f7d128a464dda6b
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:44:22
   4:      0x2aa027e37e8 - core::fmt::rt::Argument::fmt::h09008b0c06b729ef
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/fmt/rt.rs:142:9
   5:      0x2aa027e37e8 - core::fmt::write::h1bb9cd62fc10df78
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/fmt/mod.rs:1153:17
   6:      0x2aa027b52da - std::io::Write::write_fmt::h8d0716a1b140f626
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/io/mod.rs:1843:15
   7:      0x2aa027b9294 - std::sys_common::backtrace::_print::h04da0dd92e62fd13
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:47:5
   8:      0x2aa027b9294 - std::sys_common::backtrace::print::h7e718644247fb4ad
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:34:9
   9:      0x2aa027b9294 - std::panicking::default_hook::{{closure}}::h5b26cd3df3eac6e7
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:272:22
  10:      0x2aa027b8ca0 - std::panicking::default_hook::h5264eef6eea67294
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:292:9
  11:      0x2aa027ba698 - std::panicking::rust_panic_with_hook::h127de1ecb920b53d
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:779:13
  12:      0x2aa027ba518 - std::panicking::begin_panic_handler::{{closure}}::h8cfaf16f5b909a93
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:657:13
  13:      0x2aa027b7c60 - std::sys_common::backtrace::__rust_end_short_backtrace::h90047c226154dead
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:171:18
  14:      0x2aa027ba232 - rust_begin_unwind
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:645:5
  15:      0x2aa0278affe - core::panicking::panic_fmt::h2785b558eed7326d
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/panicking.rs:72:14
  16:      0x2aa0278b444 - core::result::unwrap_failed::hedeabac2d80b7732
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/result.rs:1654:5
  17:      0x2aa02790e14 - rust_out::main::_doctest_main_src_lib_rs_700_0::h0fbc1e903d2b79f3
  18:      0x2aa02790d80 - rust_out::main::h3215e911437994eb
  19:      0x2aa0278c09c - core::ops::function::FnOnce::call_once::h150faa365c92f171
  20:      0x2aa0278b710 - std::sys_common::backtrace::__rust_begin_short_backtrace::h9f29760b9548c60d
  21:      0x2aa0278b776 - std::rt::lang_start::{{closure}}::hbc040bc412671f75
  22:      0x2aa027b96d2 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h947bba242db2ccac
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/ops/function.rs:284:13
  23:      0x2aa027b96d2 - std::panicking::try::do_call::h147138c07f28c4ea
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:552:40
  24:      0x2aa027cc4c0 - __rust_try
  25:      0x2aa027b962c - std::panicking::try::h740fd007af8c33c5
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:516:19
  26:      0x2aa027b962c - std::panic::catch_unwind::he8bb01c81fc1bf81
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panic.rs:146:14
  27:      0x2aa027b962c - std::rt::lang_start_internal::{{closure}}::h91500b561960fb55
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/rt.rs:148:48
  28:      0x2aa027b962c - std::panicking::try::do_call::h075f68ab3a823114
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:552:40
  29:      0x2aa027cc4c0 - __rust_try
  30:      0x2aa027b25d0 - std::panicking::try::h71d5d2cfba89358e
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:516:19
  31:      0x2aa027b25d0 - std::panic::catch_unwind::h09ffabd68cb06711
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panic.rs:146:14
  32:      0x2aa027b25d0 - std::rt::lang_start_internal::h0efd3dfcf768b280
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/rt.rs:148:20
  33:      0x2aa0278b74a - std::rt::lang_start::h125d75d0038ce58c
  34:      0x2aa027912d2 - main
  35:      0x3ffa6db4b8c - __libc_start_call_main
  36:      0x3ffa6db4c8e - __libc_start_main_impl
  37:      0x2aa0278b5f0 - <unknown>
  38:                0x0 - <unknown>


---- src/lib.rs - schunk::Chunk::repeatval (line 591) stdout ----
Test executable failed (exit status: 101).

stderr:
thread 'main' panicked at src/lib.rs:9:1:
assertion `left == right` failed
  left: 268435456
 right: 16
stack backtrace:
   0:      0x2aa309358c4 - std::backtrace_rs::backtrace::libunwind::trace::hb5658511ef1cc65f
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/../../backtrace/src/backtrace/libunwind.rs:105:5
   1:      0x2aa309358c4 - std::backtrace_rs::backtrace::trace_unsynchronized::h4eb31789c87f8ba2
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:      0x2aa309358c4 - std::sys_common::backtrace::_print_fmt::hfb5971ceebe2fba6
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:68:5
   3:      0x2aa309358c4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h8f7d128a464dda6b
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:44:22
   4:      0x2aa3096abf8 - core::fmt::rt::Argument::fmt::h09008b0c06b729ef
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/fmt/rt.rs:142:9
   5:      0x2aa3096abf8 - core::fmt::write::h1bb9cd62fc10df78
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/fmt/mod.rs:1153:17
   6:      0x2aa3093381a - std::io::Write::write_fmt::h8d0716a1b140f626
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/io/mod.rs:1843:15
   7:      0x2aa309375d4 - std::sys_common::backtrace::_print::h04da0dd92e62fd13
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:47:5
   8:      0x2aa309375d4 - std::sys_common::backtrace::print::h7e718644247fb4ad
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:34:9
   9:      0x2aa309375d4 - std::panicking::default_hook::{{closure}}::h5b26cd3df3eac6e7
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:272:22
  10:      0x2aa30936fe0 - std::panicking::default_hook::h5264eef6eea67294
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:292:9
  11:      0x2aa309389d8 - std::panicking::rust_panic_with_hook::h127de1ecb920b53d
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:779:13
  12:      0x2aa30938858 - std::panicking::begin_panic_handler::{{closure}}::h8cfaf16f5b909a93
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:657:13
  13:      0x2aa30935fa0 - std::sys_common::backtrace::__rust_end_short_backtrace::h90047c226154dead
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:171:18
  14:      0x2aa30938572 - rust_begin_unwind
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:645:5
  15:      0x2aa3090affe - core::panicking::panic_fmt::h2785b558eed7326d
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/panicking.rs:72:14
  16:      0x2aa3090b418 - core::panicking::assert_failed_inner::ha24537816496d5be
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/panicking.rs:343:17
  17:      0x2aa3090ccbe - core::panicking::assert_failed::hd17b97a569442ff5
  18:      0x2aa3090ff50 - rust_out::main::_doctest_main_src_lib_rs_591_0::h52bfbbfdace4813b
  19:      0x2aa3090fcf0 - rust_out::main::h3215e911437994eb
  20:      0x2aa3090bf2c - core::ops::function::FnOnce::call_once::h150faa365c92f171
  21:      0x2aa3090b810 - std::sys_common::backtrace::__rust_begin_short_backtrace::h9f29760b9548c60d
  22:      0x2aa3090b876 - std::rt::lang_start::{{closure}}::hbc040bc412671f75
  23:      0x2aa30937a12 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h947bba242db2ccac
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/ops/function.rs:284:13
  24:      0x2aa30937a12 - std::panicking::try::do_call::h147138c07f28c4ea
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:552:40
  25:      0x2aa3094a800 - __rust_try
  26:      0x2aa3093796c - std::panicking::try::h740fd007af8c33c5
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:516:19
  27:      0x2aa3093796c - std::panic::catch_unwind::he8bb01c81fc1bf81
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panic.rs:146:14
  28:      0x2aa3093796c - std::rt::lang_start_internal::{{closure}}::h91500b561960fb55
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/rt.rs:148:48
  29:      0x2aa3093796c - std::panicking::try::do_call::h075f68ab3a823114
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:552:40
  30:      0x2aa3094a800 - __rust_try
  31:      0x2aa309311d0 - std::panicking::try::h71d5d2cfba89358e
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:516:19
  32:      0x2aa309311d0 - std::panic::catch_unwind::h09ffabd68cb06711
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panic.rs:146:14
  33:      0x2aa309311d0 - std::rt::lang_start_internal::h0efd3dfcf768b280
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/rt.rs:148:20
  34:      0x2aa3090b84a - std::rt::lang_start::h125d75d0038ce58c
  35:      0x2aa30910192 - main
  36:      0x3ff94bb4b8c - __libc_start_call_main
  37:      0x3ff94bb4c8e - __libc_start_main_impl
  38:      0x2aa3090b700 - <unknown>
  39:                0x0 - <unknown>


---- src/lib.rs - schunk::Chunk::uninit (line 558) stdout ----
Test executable failed (exit status: 101).

stderr:
thread 'main' panicked at src/lib.rs:9:1:
assertion `left == right` failed
  left: 167772160
 right: 10
stack backtrace:
   0:      0x2aa3b3b4264 - std::backtrace_rs::backtrace::libunwind::trace::hb5658511ef1cc65f
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/../../backtrace/src/backtrace/libunwind.rs:105:5
   1:      0x2aa3b3b4264 - std::backtrace_rs::backtrace::trace_unsynchronized::h4eb31789c87f8ba2
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:      0x2aa3b3b4264 - std::sys_common::backtrace::_print_fmt::hfb5971ceebe2fba6
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:68:5
   3:      0x2aa3b3b4264 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h8f7d128a464dda6b
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:44:22
   4:      0x2aa3b3e04c8 - core::fmt::rt::Argument::fmt::h09008b0c06b729ef
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/fmt/rt.rs:142:9
   5:      0x2aa3b3e04c8 - core::fmt::write::h1bb9cd62fc10df78
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/fmt/mod.rs:1153:17
   6:      0x2aa3b3b21ba - std::io::Write::write_fmt::h8d0716a1b140f626
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/io/mod.rs:1843:15
   7:      0x2aa3b3b5f74 - std::sys_common::backtrace::_print::h04da0dd92e62fd13
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:47:5
   8:      0x2aa3b3b5f74 - std::sys_common::backtrace::print::h7e718644247fb4ad
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:34:9
   9:      0x2aa3b3b5f74 - std::panicking::default_hook::{{closure}}::h5b26cd3df3eac6e7
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:272:22
  10:      0x2aa3b3b5980 - std::panicking::default_hook::h5264eef6eea67294
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:292:9
  11:      0x2aa3b3b7378 - std::panicking::rust_panic_with_hook::h127de1ecb920b53d
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:779:13
  12:      0x2aa3b3b71f8 - std::panicking::begin_panic_handler::{{closure}}::h8cfaf16f5b909a93
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:657:13
  13:      0x2aa3b3b4940 - std::sys_common::backtrace::__rust_end_short_backtrace::h90047c226154dead
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:171:18
  14:      0x2aa3b3b6f12 - rust_begin_unwind
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:645:5
  15:      0x2aa3b38afde - core::panicking::panic_fmt::h2785b558eed7326d
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/panicking.rs:72:14
  16:      0x2aa3b38b2e8 - core::panicking::assert_failed_inner::ha24537816496d5be
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/panicking.rs:343:17
  17:      0x2aa3b38c88e - core::panicking::assert_failed::hd17b97a569442ff5
  18:      0x2aa3b38eac8 - rust_out::main::_doctest_main_src_lib_rs_558_0::h9ab4c983158d31d4
  19:      0x2aa3b38e8a0 - rust_out::main::h3215e911437994eb
  20:      0x2aa3b38bbdc - core::ops::function::FnOnce::call_once::h150faa365c92f171
  21:      0x2aa3b38b6e0 - std::sys_common::backtrace::__rust_begin_short_backtrace::h9f29760b9548c60d
  22:      0x2aa3b38b746 - std::rt::lang_start::{{closure}}::hbc040bc412671f75
  23:      0x2aa3b3b63b2 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h947bba242db2ccac
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/ops/function.rs:284:13
  24:      0x2aa3b3b63b2 - std::panicking::try::do_call::h147138c07f28c4ea
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:552:40
  25:      0x2aa3b3c91a0 - __rust_try
  26:      0x2aa3b3b630c - std::panicking::try::h740fd007af8c33c5
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:516:19
  27:      0x2aa3b3b630c - std::panic::catch_unwind::he8bb01c81fc1bf81
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panic.rs:146:14
  28:      0x2aa3b3b630c - std::rt::lang_start_internal::{{closure}}::h91500b561960fb55
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/rt.rs:148:48
  29:      0x2aa3b3b630c - std::panicking::try::do_call::h075f68ab3a823114
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:552:40
  30:      0x2aa3b3c91a0 - __rust_try
  31:      0x2aa3b3afb70 - std::panicking::try::h71d5d2cfba89358e
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:516:19
  32:      0x2aa3b3afb70 - std::panic::catch_unwind::h09ffabd68cb06711
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panic.rs:146:14
  33:      0x2aa3b3afb70 - std::rt::lang_start_internal::h0efd3dfcf768b280
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/rt.rs:148:20
  34:      0x2aa3b38b71a - std::rt::lang_start::h125d75d0038ce58c
  35:      0x2aa3b38eb32 - main
  36:      0x3ffa14b4b8c - __libc_start_call_main
  37:      0x3ffa14b4c8e - __libc_start_main_impl
  38:      0x2aa3b38b5d0 - <unknown>
  39:                0x0 - <unknown>


---- src/lib.rs - schunk::Chunk::zeros (line 626) stdout ----
Test executable failed (exit status: 101).

stderr:
thread 'main' panicked at src/lib.rs:9:46:
called `Result::unwrap()` on an `Err` value: Other("Appears this buffer is not a valid blosc2 chunk")
stack backtrace:
   0:      0x2aa03e34294 - std::backtrace_rs::backtrace::libunwind::trace::hb5658511ef1cc65f
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/../../backtrace/src/backtrace/libunwind.rs:105:5
   1:      0x2aa03e34294 - std::backtrace_rs::backtrace::trace_unsynchronized::h4eb31789c87f8ba2
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:      0x2aa03e34294 - std::sys_common::backtrace::_print_fmt::hfb5971ceebe2fba6
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:68:5
   3:      0x2aa03e34294 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h8f7d128a464dda6b
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:44:22
   4:      0x2aa03e604f8 - core::fmt::rt::Argument::fmt::h09008b0c06b729ef
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/fmt/rt.rs:142:9
   5:      0x2aa03e604f8 - core::fmt::write::h1bb9cd62fc10df78
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/fmt/mod.rs:1153:17
   6:      0x2aa03e321ea - std::io::Write::write_fmt::h8d0716a1b140f626
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/io/mod.rs:1843:15
   7:      0x2aa03e35fa4 - std::sys_common::backtrace::_print::h04da0dd92e62fd13
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:47:5
   8:      0x2aa03e35fa4 - std::sys_common::backtrace::print::h7e718644247fb4ad
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:34:9
   9:      0x2aa03e35fa4 - std::panicking::default_hook::{{closure}}::h5b26cd3df3eac6e7
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:272:22
  10:      0x2aa03e359b0 - std::panicking::default_hook::h5264eef6eea67294
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:292:9
  11:      0x2aa03e373a8 - std::panicking::rust_panic_with_hook::h127de1ecb920b53d
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:779:13
  12:      0x2aa03e37228 - std::panicking::begin_panic_handler::{{closure}}::h8cfaf16f5b909a93
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:657:13
  13:      0x2aa03e34970 - std::sys_common::backtrace::__rust_end_short_backtrace::h90047c226154dead
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/sys_common/backtrace.rs:171:18
  14:      0x2aa03e36f42 - rust_begin_unwind
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:645:5
  15:      0x2aa03e0afde - core::panicking::panic_fmt::h2785b558eed7326d
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/panicking.rs:72:14
  16:      0x2aa03e0b424 - core::result::unwrap_failed::hedeabac2d80b7732
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/result.rs:1654:5
  17:      0x2aa03e0e954 - rust_out::main::_doctest_main_src_lib_rs_626_0::h8513b6ea61f8b2dd
  18:      0x2aa03e0e8c0 - rust_out::main::h3215e911437994eb
  19:      0x2aa03e0bbdc - core::ops::function::FnOnce::call_once::h150faa365c92f171
  20:      0x2aa03e0b6e0 - std::sys_common::backtrace::__rust_begin_short_backtrace::h9f29760b9548c60d
  21:      0x2aa03e0b746 - std::rt::lang_start::{{closure}}::hbc040bc412671f75
  22:      0x2aa03e363e2 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h947bba242db2ccac
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/core/src/ops/function.rs:284:13
  23:      0x2aa03e363e2 - std::panicking::try::do_call::h147138c07f28c4ea
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:552:40
  24:      0x2aa03e491d0 - __rust_try
  25:      0x2aa03e3633c - std::panicking::try::h740fd007af8c33c5
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:516:19
  26:      0x2aa03e3633c - std::panic::catch_unwind::he8bb01c81fc1bf81
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panic.rs:146:14
  27:      0x2aa03e3633c - std::rt::lang_start_internal::{{closure}}::h91500b561960fb55
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/rt.rs:148:48
  28:      0x2aa03e3633c - std::panicking::try::do_call::h075f68ab3a823114
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:552:40
  29:      0x2aa03e491d0 - __rust_try
  30:      0x2aa03e2fba0 - std::panicking::try::h71d5d2cfba89358e
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panicking.rs:516:19
  31:      0x2aa03e2fba0 - std::panic::catch_unwind::h09ffabd68cb06711
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/panic.rs:146:14
  32:      0x2aa03e2fba0 - std::rt::lang_start_internal::h0efd3dfcf768b280
                               at /builddir/build/BUILD/rustc-1.78.0-src/library/std/src/rt.rs:148:20
  33:      0x2aa03e0b71a - std::rt::lang_start::h125d75d0038ce58c
  34:      0x2aa03e0eb62 - main
  35:      0x3ff93bb4b8c - __libc_start_call_main
  36:      0x3ff93bb4c8e - __libc_start_main_impl
  37:      0x2aa03e0b5d0 - <unknown>
  38:                0x0 - <unknown>



failures:
    src/lib.rs - schunk::Chunk::decompress (line 700)
    src/lib.rs - schunk::Chunk::repeatval (line 591)
    src/lib.rs - schunk::Chunk::uninit (line 558)
    src/lib.rs - schunk::Chunk::zeros (line 626)

test result: FAILED. 12 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.00s

error: doctest failed, to rerun pass `--doc`
error: 2 targets failed:
    `--lib`
    `--doc`

@milesgranger
Copy link
Owner

Maybe/probably related Blosc/c-blosc2#467, maybe I'll take a stab at it.

@musicinmybrain
Copy link
Contributor Author

Maybe/probably related Blosc/c-blosc2#467, maybe I'll take a stab at it.

Hmm, I think you’re right. Test failures are ignored on s390x in Fedora’s blosc2 package because of this issue.

It looks like there’s no reason to believe there is anything wrong with your bindings, and no way to tell one way or another until the issues in c-blosc2 are fixed.

I don’t think I have time to fix c-blosc2 right now. I’m not thrilled with the idea of shipping a known broken package on s390x, but I suppose that is already the status quo for everything that links blosc2 on that architecture, so I suppose I can live with skipping the tests on s390x and moving on.

@musicinmybrain
Copy link
Contributor Author

(On the other hand, if Blosc/c-blosc2@8b5ea6f is really the only part that is broken, maybe I can pull together a PR without spending too much time on it.)

@milesgranger
Copy link
Owner

milesgranger commented May 8, 2024

I'm not convinced it's that bit of code anymore, I've narrowed it down to something funky going on here:
https://github.com/Blosc/c-blosc2/blob/df889b5dbe0f8fd651e9b5f41edf5dd6d792d147/blosc/frame.c#L895

If one changes trailer_len = 35 right after this call to from_big, then test_schunk_basic passes, which is the same value for trailer_len that testing on x86_64 will return. However, on s390x, it'll have trailer_len = 0 which bungs up everything else.

I'm testing these changes w/ cross test --target s390x-unknown-linux-gnu --release schunk_basic -- --nocapture (cross) So far everything else preceding this call to from_big appears the same, definitely the values being provided at least, so would have expected the same output at this point. I'm reaching the limits of my current C knowledge though.


Disregard, I see now that after into_vec, on little endian there is the trailer len stored at index 194, whereas on big endian that index value is 0 (in fact, the expected value 35 isn't anywhere to be found) and thus where the function is also returning 0.

@milesgranger
Copy link
Owner

@musicinmybrain if you can give main a go when you have time, I think/hope we're good here now. 🙏

@musicinmybrain
Copy link
Contributor Author

I will look into it ASAP! It’s a little tricker for me since I am using an unbundled blosc2 C library, so I need to try building a patched copy of that first, but I will certainly give it a try when I have a little time.

@milesgranger
Copy link
Owner

No rush at all, I'm not pressured to release anything. 👍

@musicinmybrain
Copy link
Contributor Author

I just got around to testing this. I applied https://github.com/Blosc/c-blosc2/pull/606.patch to the 2.14.4 release of blosc2, rebuilt it, and test-built rust-blosc2-sys and rust-blosc2-rs 0.2.6 packages with the patched system blosc2: https://copr.fedorainfracloud.org/coprs/music/libcramjam/packages/

I am still seeing a bunch of failures on s390x, and I’m now seeing a segfault on the other architectures.

I’d like to provide more helpful feedback, but I’m not sure where to start…

I was curious what would happen if I used the bundled c-blosc2 (not that I want to package this that way), but I couldn’t get the blosc2-sys crate to build that way either as an RPM or in a plain git checkout:

CMake Error: INSTALL(EXPORT) given unknown export "Blosc2Targets"

@milesgranger
Copy link
Owner

That kinda looks like neither shared or static feature was set. I've only encountered that when neither feature was set. Outside of that, I'm not sure.

@musicinmybrain
Copy link
Contributor Author

I just got around to testing this. I applied https://github.com/Blosc/c-blosc2/pull/606.patch to the 2.14.4 release of blosc2, rebuilt it, and test-built rust-blosc2-sys and rust-blosc2-rs 0.2.6 packages with the patched system blosc2: https://copr.fedorainfracloud.org/coprs/music/libcramjam/packages/

I am still seeing a bunch of failures on s390x, and I’m now seeing a segfault on the other architectures.

I finally got back to this with fresh eyes. The segfaults, it turns out, were my fault. I was patching build.rs in the blosc2-sys to re-generate the bindings with bindgen unconditionally, regardless of whether or not the regenerate-bindings feature was enabled. That was a reasonable thing to do for distribution packaging, because I’m trying to always use the system blosc2 library, regardless of whether that is exactly the same as the one you bundled in the crate, and so the bindings should always be regenerated to be sure they match. The problem was:

  1. I failed to rm src/bindings.rs, which would have proved the distributed bindings.rs was unused. Instead, the build could succeed even if it was accidentally not using regenerated bindings.
  2. I missed patching src/lib.rs to also ignore regenerate-bindings and unconditionally use the bindings.rs generated in build.rs.

The excellent news is that once I fixed this, the segfaults went away and the rust-blosc2-rs tests started passing on all Fedora releases for all but s390x

Now I need to see if I can get s390x tests passing. I tried it with a blosc2 that was patched with Blosc/c-blosc2#606, but I think I need the changes to blosc2-rs’s lib.rs from #24, too. (I see that your fix was released in blosc2 2.15, so updating the Fedora package may make more sense than backporting the patch, once I can prove it works.)

@milesgranger
Copy link
Owner

That's great!
I also have a few backlog items to address before doing another release so I'd say we're in no rush at the moment unless people are pressing for Python3.13 support. But I'm a bit stacked with chores/life presently, maybe in a month or two I'll be ready. :)

@musicinmybrain
Copy link
Contributor Author

musicinmybrain commented Aug 30, 2024

Ok, so if I [backport both blosc2 patches] to blosc2 2.14.2, plus the lib.rs changes from #24 in this package, then I still get rust-blosc2-rs test failures on s390x package:

failures:

---- src/lib.rs - schunk::Chunk::decompress (line 700) stdout ----
Test executable failed (exit status: 101).

stderr:
thread 'main' panicked at src/lib.rs:9:49:
called `Result::unwrap()` on an `Err` value: Other("Appears this buffer is not a valid blosc2 chunk")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


---- src/lib.rs - schunk::Chunk::repeatval (line 591) stdout ----
Test executable failed (exit status: 101).

stderr:
thread 'main' panicked at src/lib.rs:9:1:
assertion `left == right` failed
  left: 268435456
 right: 16
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


---- src/lib.rs - schunk::Chunk::uninit (line 558) stdout ----
Test executable failed (exit status: 101).

stderr:
thread 'main' panicked at src/lib.rs:9:1:
assertion `left == right` failed
  left: 167772160
 right: 10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


---- src/lib.rs - schunk::Chunk::zeros (line 626) stdout ----
Test executable failed (exit status: 101).

stderr:
thread 'main' panicked at src/lib.rs:9:46:
called `Result::unwrap()` on an `Err` value: Other("Appears this buffer is not a valid blosc2 chunk")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace



failures:
    src/lib.rs - schunk::Chunk::decompress (line 700)
    src/lib.rs - schunk::Chunk::repeatval (line 591)
    src/lib.rs - schunk::Chunk::uninit (line 558)
    src/lib.rs - schunk::Chunk::zeros (line 626)

I don’t think I missed anything – not sure what’s happening here.

I think I’ll work on getting at least Rawhide updated with the latest release 2.15.1 of blosc2, which should contain all of your patches, and see if that works any better than the backport.

@musicinmybrain
Copy link
Contributor Author

I see similar results with blosc2 2.15.1: https://copr.fedorainfracloud.org/coprs/music/libcramjam/build/7955601/

Not sure what to try next. Having a rust-blosc2-rs that’s ExcludeArch: s390x is possible, but this would then make rust-libcramjam and python-cramjam ExcludeArch too, and that would impact a tree of dependent Python packages that currently work on s390x.

Skipping the tests and shipping a rust-blosc2-rs that’s known-broken on s390x is possible too, I guess. I don’t like it, but it’s what the blosc2 package already does, and it would preserve the ability of other packages to depend on python-cramjam for compression formats that do work on s390x.

I’m curious, though, why this seems to be fixed for you but not for me.

@milesgranger
Copy link
Owner

I’m curious, though, why this seems to be fixed for you but not for me.

Ya, that's a mystery, but it's this workflow that works after doing those upstream changes. Although I think what blosc-2 package tests on that system is not what I was fixing, but was similar: Blosc/c-blosc2#467, which is a file-backed schunk; maybe that's what's continuing to fail? I'm not claiming all blosc2 tests pass for me, just the blosc2-rs tests.

Which there, they pass on s390x as shown in that workflow.

@milesgranger
Copy link
Owner

With that said though, I won't be too broken up about it to rip out blosc2 from cramjam (or just ship it default without); until someone really presses for it. People should probably use the official Python-Blosc2 package anyway maybe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants