Failure to compile statically against binutils >= 2.40 (libbfd)

[qmonnet]

Someone reported an issue with binutils 2.40 (released mid-January 2023).

Reference: https://build.opensuse.org/package/live_build_log/openSUSE:Factory:Staging:B/bpftool/standard/x86_64

[   77s] gcc -O2 -W -Wall -Wextra -Wno-unused-parameter
    -Wno-missing-field-initializers -Wbad-function-cast
    -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self
    -Wmissing-declarations -Wmissing-prototypes -Wno-system-headers
    -Wold-style-definition -Wpacked -Wredundant-decls -Wstrict-prototypes
    -Wswitch-default -Wundef -Wwrite-strings -Wformat -Wno-type-limits
    -Wstrict-aliasing=3 -Wshadow -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ -I.
    -I/home/abuild/rpmbuild/BUILD/tools/bpf/bpftool/libbpf/include
    -I/home/abuild/rpmbuild/BUILD/kernel/bpf/
    -I/home/abuild/rpmbuild/BUILD/tools/include
    -I/home/abuild/rpmbuild/BUILD/tools/include/uapi -DDISASM_FOUR_ARGS_SIGNATURE
    -DDISASM_INIT_STYLED -DHAVE_LIBBFD_SUPPORT -DBPFTOOL_WITHOUT_SKELETONS  btf.o
    btf_dumper.o cfg.o cgroup.o common.o feature.o gen.o iter.o json_writer.o
    link.o main.o map.o map_perf_ring.o net.o netlink_dumper.o perf.o pids.o prog.o
    struct_ops.o tracelog.o xlated_dumper.o jit_disasm.o disasm.o
    /home/abuild/rpmbuild/BUILD/tools/bpf/bpftool/libbpf/libbpf.a -lelf -lz -lbfd
    -ldl -lopcodes -liberty -o bpftool
[   77s] /usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: /usr/lib64/gcc/x86_64-suse-linux/12/../../../../lib64/libbfd.a(elfxx-x86.o): in function `_bfd_x86_elf_write_sframe_plt':
[   77s] /home/abuild/rpmbuild/BUILD/binutils-2.40/build-dir/bfd/../../bfd/elfxx-x86.c:1977: undefined reference to `sframe_encoder_write'
[   77s] /usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: /home/abuild/rpmbuild/BUILD/binutils-2.40/build-dir/bfd/../../bfd/elfxx-x86.c:1983: undefined reference to `sframe_encoder_free'
[   77s] /usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: /usr/lib64/gcc/x86_64-suse-linux/12/../../../../lib64/libbfd.a(elfxx-x86.o): in function `_bfd_x86_elf_create_sframe_plt':
[   77s] /home/abuild/rpmbuild/BUILD/binutils-2.40/build-dir/bfd/../../bfd/elfxx-x86.c:1874: undefined reference to `sframe_encode'

as seen, new binutils comes with a new library called sframe and the
aforementioned issue can be fixed by adding '-lsframe' to the command line.

Sounds like we'll need to add new probes to https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/build/feature/Makefile#n246, and update bpftool's Makefile accordingly.

I don't know if this library comes with binutils-dev or if it needs to be installed separately. I haven't reproduced yet.

[marxin]

I don't know if this library comes with binutils-dev or if it needs to be installed separately. I haven't reproduced yet.

Hey. The library is provided by the very same package as libbfd.a (it's binutils package on openSUSE). And as seen (built from source), the libsframe.so.* is dependency of libbfd.so.* right now:

$ binutils/objdir/bfd> ldd ./.libs/libbfd.so | grep sframe
	libsframe.so.0 => /home/marxin/Programming/binutils/objdir/libsframe/.libs/libsframe.so.0 (0x00007ffff7d86000)

So yes, the Makefile needs to be tweaked a bit more ;)

[qmonnet]

For anyone else wondering what this sframe is about:

• https://www.phoronix.com/news/GNU-Binutils-SFrame
• https://lore.kernel.org/bpf/[email protected]/T/#u

[marxin]

One more note about what happens with -lbfd. In the case of openSUSE, we intentionally don't ship shared library libbfd.so, but only a static version that we link against. Unfortunately, unlike dynamic libraries, transitive dependencies are not added automatically for static libs (libsframe) and thus we end up with unresolved symbols from the library.

[qmonnet]

Are you saying that the issue only occurs with shared libbfd? (Sorry, I still haven't found the time to reproduce.)

[marxin]

I'm saying it happens with a static libbfd library (that's what we provide in openSUSE).

[qmonnet]

(That's what I meant 🤦.)
OK, thank you

[marxin]

And the situation gets even more complicated when it comes to the latest release. libbfd newly supports zstd compression and thus one ends with:

[   71s] /usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: /usr/lib64/gcc/x86_64-suse-linux/12/../../../../lib64/libbfd.a(compress.o): in function `decompress_contents':
[   71s] /home/abuild/rpmbuild/BUILD/binutils-2.40/build-dir/bfd/../../bfd/compress.c:517: undefined reference to `ZSTD_decompress'
[   71s] /usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: /home/abuild/rpmbuild/BUILD/binutils-2.40/build-dir/bfd/../../bfd/compress.c:519: undefined reference to `ZSTD_isError'
[   71s] /usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: /usr/lib64/gcc/x86_64-suse-linux/12/../../../../lib64/libbfd.a(compress.o): in function `bfd_compress_section_contents':
[   71s] /home/abuild/rpmbuild/BUILD/binutils-2.40/build-dir/bfd/../../bfd/compress.c:656: undefined reference to `ZSTD_compress'
[   71s] /usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: /home/abuild/rpmbuild/BUILD/binutils-2.40/build-dir/bfd/../../bfd/compress.c:661: undefined reference to `ZSTD_isError'

Note the binutils folks are discussion a proper static libs dependencies:
https://sourceware.org/pipermail/binutils/2023-February/126155.html

[qmonnet]

I still need to get back to this eventually. We have another issue dedicated to zstd now, #152