macos arm64 compatibility

[watsy0007]

hi, i try run it on my macos immediately when i saw #37 merged

i get the same error

this is my mix.exs deps

{:lz4, github: "rabbitmq/lz4-erlang", ref: "01765f8"}

compile log

/Users/watsy0007/code/chainindex/porter_oss/deps/lz4
compiling static library
clang: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
compiling dynamic library 1.9.2
creating versioned links
cc -O3   -shared -fPIC -I../lib -DXXH_NAMESPACE=LZ4_  -c -o bench.o bench.c
clang: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
cc -O3   -shared -fPIC -I../lib -DXXH_NAMESPACE=LZ4_  -c -o datagen.o datagen.c
clang: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
cc -O3   -shared -fPIC -I../lib -DXXH_NAMESPACE=LZ4_  -c -o lz4cli.o lz4cli.c
clang: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
cc -O3   -shared -fPIC -I../lib -DXXH_NAMESPACE=LZ4_  -c -o lz4io.o lz4io.c
clang: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
cc -O3   -shared -fPIC -I../lib -DXXH_NAMESPACE=LZ4_  ../lib/lz4.o ../lib/lz4frame.o ../lib/lz4hc.o ../lib/xxhash.o bench.o datagen.o lz4cli.o lz4io.o -o lz4
make[1]: Nothing to be done for `all'.
 DEPEND host_triple.d
 ERLC   host_triple.erl
 APP    host_triple.app.src
 DEPEND lz4.d
 C      lz4.c
 C      lz4f.c
 C      nif_helpers.c
 LD     lz4_nif.so
ld: warning: ignoring file /Users/watsy0007/code/chainindex/porter_oss/deps/lz4/deps/lz4_src/lib/liblz4.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/watsy0007/.asdf/installs/erlang/25.2/lib/erl_interface-5.3/lib/libei.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64

error log

17:55:44.273 [debug] lz4: Trying to load "lz4_nif" from "/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/aarch64-apple-darwin22.2.0"
17:55:44.277 [debug] lz4: Couldn't load "lz4_nif" from "/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/aarch64-apple-darwin22.2.0":
{:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/aarch64-apple-darwin22.2.0/lz4_nif.so, 0x0002): tried: \'/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/a
arch64-apple-darwin22.2.0/lz4_nif.so\' (no such file), \'/System/Volumes/Preboot/Cryptexes/OS/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/aarch64-apple-darwin22.2.0/lz4_nif.so\' (no such file), \'/Users/watsy000
7/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/aarch64-apple-darwin22.2.0/lz4_nif.so\' (no such file)\''}}
17:55:44.278 [debug] lz4: Trying to load "lz4_nif" from "/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv"
17:55:44.278 [debug] lz4: Couldn't load "lz4_nif" from "/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv":
{:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/lz4_nif.so, 0x0002): tried: \'/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/lz4_nif.so\' (mach-o file, b
ut is an incompatible architecture (have \'x86_64\', need \'arm64\')), \'/System/Volumes/Preboot/Cryptexes/OS/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/lz4_nif.so\' (no such file), \'/Users/watsy0007/code/chai
nindex/porter_oss/_build/dev/lib/lz4/priv/lz4_nif.so\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64\')), \'/Users/watsy0007/code/chainindex/porter_oss/deps/lz4/priv/lz4_nif.so\' (mach-o file, but is
 an incompatible architecture (have \'x86_64\', need \'arm64\')), \'/System/Volumes/Preboot/Cryptexes/OS/Users/watsy0007/code/chainindex/porter_oss/deps/lz4/priv/lz4_nif.so\' (no such file), \'/Users/watsy0007/code/chainindex/porter_o
ss/deps/lz4/priv/lz4_nif.so\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64\'))\''}}
17:55:44.279 [warning] The on_load function for module lz4_nif returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'dlopen(/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/lz4_nif.so, 0x0002): tried: \'/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/lz4_nif.so\' (mach-o file, b
ut is an incompatible architecture (have \'x86_64\', need \'arm64\')), \'/System/Volumes/Preboot/Cryptexes/OS/Users/watsy0007/code/chainindex/porter_oss/_build/dev/lib/lz4/priv/lz4_nif.so\' (no such file), \'/Users/watsy0007/code/chai
nindex/porter_oss/_build/dev/lib/lz4/priv/lz4_nif.so\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64\')), \'/Users/watsy0007/code/chainindex/porter_oss/deps/lz4/priv/lz4_nif.so\' (mach-o file, but is
 an incompatible architecture (have \'x86_64\', need \'arm64\')), \'/System/Volumes/Preboot/Cryptexes/OS/Users/watsy0007/code/chainindex/porter_oss/deps/lz4/priv/lz4_nif.so\' (no such file), \'/Users/watsy0007/code/chainindex/porter_o
ss/deps/lz4/priv/lz4_nif.so\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64\'))\''}}

sorry for my lack of c++ knowledge, Are there any steps I'm doing wrong?

[michaelklishin]

This NIF currently tries to load lz4_nif.so which is a Linux-specific binary file.

[watsy0007]

thanks for reply, are there any plan to support more architectures?

[HoloRin]

#37 was a bazel-only change, so would not apply when this library is included via mix in that way

[pgorczak]

The reason for lz4_nif.so having the wrong architecture is because the currently included version of erlang.mk always sets -arch x86_64 when it detects a Darwin/MacOS platform.

This has been fixed upstream in ninenines/erlang.mk@94718f7

After removing the hardcoded flags, I could build and use the package on an M1 machine.

[michaelklishin]

@pgorczak can you give #84 a try?

[pgorczak]

The upgrade broke something else for me. Erlang.mk now fails at host_triple with

...
 DEP    host_triple (0.1.0)
Evaluating config script ".../lz4/.erlang.mk/rebar3/_build/default/lib/rebar/src/rebar.app.src.script"
===> Verifying dependencies...
===> Failed to update package host_triple from repo hexpm
escript: exception error: no match of right hand side value
                 {error,
                     {rebar_app_utils,
                         {missing_package,<<"host_triple">>,<<"0.1.0">>}}}
  in function  erl_eval:expr/6 (erl_eval.erl, line 498)
  in call from escript:eval_exprs/5 (escript.erl, line 869)
  in call from erl_eval:local_func/8 (erl_eval.erl, line 646)
  in call from escript:interpret/4 (escript.erl, line 780)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1
  in call from init:do_boot/3
...

Followed by

Error: No Makefile to build dependency .../lz4/deps/host_triple.

[pgorczak]

The package lz4/deps/host_triple did get downloaded though

[pgorczak]

I think the error happens during autopatch (see 7.10). I can reproduce it when just running make fetch-deps after removing the deps directory.

[michaelklishin]

Yup, host_triple can be "new" (newer than the last update of this package) and may need updating to work as a dependency in an erlang.mk-managed codebase.

[pgorczak]

Finally got around to digging into the remaining issue. It's because I'm using the library through mix and mix was trying to compile it with rebar (probably because it detected the rebar config file). I needed to specify the dependency as

{:lz4, git: "https://github.com/rabbitmq/lz4-erlang.git", tag: "v1.9.4.1", manager: :make}

I can confirm the project compiles (with some warnings) on Erlang/OTP 26 on Apple Silicon (macOS 14.2).