libfaustwithllvm.a has bogus symbols on arm64

[turian]

arm64 is not supported yet: #951 for shipping aarm64 libfaustwithllvm.a

And the wiki documentation gives no pointers how this should be specifically built.

I am tearing my hair out trying to build a Dockerfile to build everything under arm64 Ubuntu.

Here is what I currently have:

# Using Ubuntu 22.04 as the base image
FROM ubuntu:22.04

LABEL maintainer="[email protected]" \
      version="0.1" \
      description=""

# Set the working directory
WORKDIR /root/

# Set the environment variables
ENV LANG=C.UTF-8 \
    TZ=Etc/UTC \
    DEBIAN_FRONTEND=noninteractive \
    PYTHONLIBPATH=/usr/lib/python3.10 \
    PYTHONINCLUDEPATH=/usr/include/python3.10

# Setting the timezone and installing essential packages
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone \
    && apt-get update \
    && apt-get install -y lsb-release software-properties-common wget python3-pip python3-dev git build-essential cmake g++ make nasm curl unzip libgl1-mesa-dev \
    && bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" \
    && apt-get install -y lv2-c++-tools libgtkmm-2.4-1v5 pkg-config lv2-dev libgtkmm-2.4-dev \
       libsndfile1 libx11-dev libxrandr-dev libxinerama-dev libxrender-dev libxcomposite-dev libxcursor-dev libfreetype6-dev libsndfile1-dev \
       libvorbis-dev libopus-dev libflac-dev libasound2-dev alsa-utils

# Download and extract Faust
RUN wget https://github.com/grame-cncm/faust/releases/download/2.69.3/faust-2.69.3.tar.gz \
    && tar zxvf faust-2.69.3.tar.gz

# Install additional dependencies
RUN apt-get install -y libpolly-17-dev

# Set up symlink for llvm-config if necessary
RUN ln -s /usr/bin/llvm-config-17 /usr/bin/llvm-config


# Log LLVM version and installations
RUN echo "LLVM Versions Installed:" > /root/build_logs.txt \
    && apt list --installed | grep llvm >> /root/build_logs.txt \
    && echo "\nllvm-config version:" >> /root/build_logs.txt \
    && llvm-config --version >> /root/build_logs.txt

# Build Faust and log the process
RUN echo "\nBuilding Faust..." >> /root/build_logs.txt \
    && mkdir faust-2.69.3/build/lib \
    && cd faust-2.69.3/build/ \
    && { cmake . -DINCLUDE_LLVM=ON -DINCLUDE_STATIC=ON && make && make -f Make.llvm.static && make install; } >> /root/build_logs.txt 2>&1

# Clone repositories and install Python packages
RUN echo "\nCloning repositories and installing Python packages..." >> /root/build_logs.txt \
    && git clone https://github.com/zynthian/moog.git \
    && pip3 install pedalboard \
    && git clone https://github.com/DBraun/DawDreamer.git \
    && cd DawDreamer \
    && git submodule init \
    && git submodule update

# Ensure the DawDreamer/Builds/LinuxMakefile directory exists and list its contents for verification
RUN echo "\nChecking DawDreamer/Builds/LinuxMakefile directory..." >> /root/build_logs.txt \
    && ls -la DawDreamer/Builds/LinuxMakefile >> /root/build_logs.txt 2>&1

## Copy the build logs to the output directory
RUN cp /root/build_logs.txt /output/

## Run make with VERBOSE and CONFIG in the DawDreamer build directory and log the output
RUN echo "\nBuilding DawDreamer..." >> /root/build_logs.txt \
    && cd DawDreamer/Builds/LinuxMakefile \
    && make VERBOSE=1 CONFIG=Release LIBS="-lstdc++fs" CXXFLAGS="-I../../alsa-lib/include -I/usr/include/python3.10 -I$PYTHONINCLUDEPATH"  LDFLAGS="-L/__w/DawDreamer/DawDreamer/alsa-lib/src -L$PYTHONLIBPATH -L/root/faust-2.69.3/lib -L/root/faust-2.69.3/build/lib/" >> /root/build_logs.txt 2>&1

# Move the built library
RUN cd DawDreamer/Builds/LinuxMakefile && mv build/libdawdreamer.so ../../dawdreamer/dawdreamer.so

## Build and install the Python package and log the output
RUN cd DawDreamer && python3 setup.py build >> /root/build_logs.txt 2>&1
RUN cd DawDreamer && python3 setup.py install >> /root/build_logs.txt 2>&1

## Test import of DawDreamer and log the output
RUN { python3 -c "import dawdreamer"; } >> /root/build_logs.txt 2>&1 || echo "Import failed" >> /root/build_logs.txt

## Final command
#CMD ["cat", "/root/build_logs.txt"]

It ultimately fails with

 > [14/14] RUN python3 -c "import dawdreamer; print('Successfully imported dawdreamer')":
0.264 Traceback (most recent call last):
0.264   File "<string>", line 1, in <module>
0.264   File "/usr/local/lib/python3.8/dist-packages/dawdreamer-0.8.0-py3.8-linux-aarch64.egg/dawdreamer/__init__.py", line 4, in <module>
0.264     from .dawdreamer import *
0.264 ImportError: /usr/local/lib/python3.8/dist-packages/dawdreamer-0.8.0-py3.8-linux-aarch64.egg/dawdreamer/dawdreamer.so: undefined symbol: _Z16deleteDSPFactoryP16llvm_dsp_factory

I have been tearing my hair out trying to figure out why the linked libfaustwithllvm has a bogus symbol.

Any idea how to fix this?

[Simon-L]

Hi @turian! Have you found a solution in the meantime?

A few things that could be checked:

• I note you're using make -f Make.llvm.static manually, any reason for that? If you build with make ... TARGETS=all.cmake and BACKENDS=all.cmake ... all it should be handled as part of the process.
• How is dawdreamer.so assembled? Can you check by running "ldd" on it see if it finds everything it needs.
• And look for deleteDSPFactory in the libraries that were built: nm <lib> | grep deleteDSPFactory, use nm -D for .so

What hardware did this container run on and what is the final target hardware?

I am using cmake-only commands for building faust, as of today this is how I build, tested for x86_64 and aarch64 (though not on ubuntu):

cd build
cmake -B faustdir -C ./backends/all.cmake -C ./targets/all.cmake \
        -DCMAKE_INSTALL_PREFIX=prefix.
cmake --build faustdir --parallel 16 --verbose
cmake --build faustdir --parallel 16 --verbose --target install

Likely unrelated, but you may try this fix that was added to the tests recently: b2eab61#diff-04dcd85c83a0f9b7f470b167ceb879b6ae8c4dd7418c3eec10efb1c9550970c2R14