Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(deps): update dependency ziglang/zig to v0.12.0 #56

Merged
merged 18 commits into from
May 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ ARG TARGET="x86_64-linux-gnu"
ARG VERSION="0.0.0"

# renovate: datasource=github-releases depName=ziglang/zig
ARG ZIG_VERSION='0.11.0'
ARG ZIG_VERSION='0.12.0'
# renovate: datasource=github-releases depName=mesonbuild/meson
ARG MESON_VERSION='1.4.0'
# renovate: datasource=github-releases depName=Kitware/CMake
Expand All @@ -32,12 +32,12 @@ RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selectio
RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt `
apt-get update && apt-get upgrade && apt-get install -y ca-certificates

# Add LLVM 16 repository
# Add LLVM 17 repository
ADD https://apt.llvm.org/llvm-snapshot.gpg.key /etc/apt/trusted.gpg.d/apt.llvm.org.asc

RUN chmod 644 /etc/apt/trusted.gpg.d/apt.llvm.org.asc

RUN echo "deb https://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-16 main" `
RUN echo "deb https://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-17 main" `
> /etc/apt/sources.list.d/llvm.list

# Install build dependencies
Expand All @@ -50,9 +50,9 @@ RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/
libtool `
python3 `
gettext `
llvm-16 `
llvm-17 `
autoconf `
clang-16 `
clang-17 `
pkg-config `
ninja-build `
libarchive-tools `
Expand Down Expand Up @@ -204,7 +204,7 @@ RUN --mount=type=cache,target=/root/.cache `
FROM layer-00 AS layer-10

COPY --from=layer-10-sse2neon "${PREFIX}/." "$PREFIX"
COPY --from=layer-10-compiler-rt "/usr/lib/llvm-16/lib/clang/16/." '/usr/lib/llvm-16/lib/clang/16'
COPY --from=layer-10-compiler-rt "/usr/lib/llvm-17/lib/clang/17/." '/usr/lib/llvm-17/lib/clang/17'

#--

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ It uses Zig 0.11 as a C/C++ toolchain to build the Windows and Linux targets:

It uses LLVM/Clang 16 with some tweaks from osxcross + Apple's cctools and linker to build the Darwin (macOS) targets:

- https://github.com/llvm/llvm-project/tree/llvmorg-16.0.6
- https://github.com/llvm/llvm-project/tree/llvmorg-17.0.6
- https://github.com/tpoechtrager/osxcross
- https://github.com/tpoechtrager/cctools-port

Expand Down
6 changes: 3 additions & 3 deletions scripts/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -95,15 +95,15 @@ case "$TARGET" in
FFLAGS="${FFLAGS} -fstack-check"

# https://github.com/tpoechtrager/osxcross/commit/3279f86
CFLAGS="${CFLAGS} -D__ENVIRONMENT_OS_VERSION_MIN_REQUIRED__=$(LC_ALL=C printf '%.2f' "11.0" | tr -d '.') -mmacos-version-min=${MACOSX_DEPLOYMENT_TARGET} -mmacosx-version-min=${MACOSX_DEPLOYMENT_TARGET}"
CFLAGS="${CFLAGS} -mmacos-version-min=${MACOSX_DEPLOYMENT_TARGET} -mmacosx-version-min=${MACOSX_DEPLOYMENT_TARGET}"
LDFLAGS="-fuse-ld=$(command -v "${APPLE_TARGET:?}-ld") -L${SDKROOT}/usr/lib -L${SDKROOT}/usr/lib/system -F${SDKROOT}/System/Library/Frameworks ${LDFLAGS}"
;;
*windows*)
# Zig doesn't support stack probing on Windows
# https://github.com/ziglang/zig/blob/b3462b7/src/target.zig#L326-L329
# https://github.com/ziglang/zig/blob/0.12.0/src/target.zig#L195-L198
FFLAGS="${FFLAGS} -fno-stack-check"
# https://github.com/strukturag/libheif/issues/357
CFLAGS="${CFLAGS} -D_GLIBCXX_ASSERTIONS -D__MINGW64__"
CFLAGS="${CFLAGS} -D_GLIBCXX_ASSERTIONS=1 -D__MINGW64__"
;;
esac
export CFLAGS="${CFLAGS} ${FFLAGS}"
Expand Down
88 changes: 56 additions & 32 deletions scripts/cc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,8 @@ set -euo pipefail
case "${TARGET:?TARGET envvar is required to be defined}" in
x86_64-linux-musl | aarch64-linux-musl) ;;
x86_64-linux-gnu* | aarch64-linux-gnu*)
# Set the glibc minimum version
# This is the lowest we can go at the moment
# https://github.com/ziglang/zig/issues/9412
TARGET="${TARGET%%.*}.2.18"
# Set the glibc minimum version, RHEL-7.9 and CentOS 7
TARGET="${TARGET%%.*}.2.17"
;;
x86_64-darwin-apple | x86_64-apple-darwin-macho)
SDKROOT="$MACOS_SDKROOT"
Expand All @@ -31,7 +29,7 @@ case "$(basename "$0")" in
case "$TARGET" in
*darwin*)
# Use clang instead of zig for darwin targets
CMD='clang-16'
CMD='clang-17'
;;
*) CMD='zig cc' ;;
esac
Expand All @@ -41,7 +39,7 @@ case "$(basename "$0")" in
case "$TARGET" in
*darwin*)
# Use clang instead of zig for darwin targets
CMD='clang++-16'
CMD='clang++-17'
;;
*) CMD='zig c++' ;;
esac
Expand Down Expand Up @@ -145,29 +143,42 @@ while [ "$#" -gt 0 ]; do
cpu_features+=("${_features[@]}")
true
else
if (case "$TARGET" in *darwin*) exit 0 ;; *) exit 1 ;; esac) then
if [ "$1" = '-arch=aarch64' ]; then
# macOS uses arm64 instead of aarch64
argv+=('-arch=arm64')
elif [ "$1" = '-arch' ] && [ "${2:-}" = 'aarch64' ]; then
argv+=('-arch' 'arm64')
shift 2
continue
elif (case "$1" in -DTARGET_OS_IPHONE*) exit 0 ;; *) exit 1 ;; esac) then
has_iphone=1
else
argv+=("$1")

# See https://github.com/apple-oss-distributions/libiconv/blob/a167071/xcodeconfig/libiconv.xcconfig
if [ "$1" = '-lcharset' ]; then
should_add_libcharset=-1
elif [ "$1" = '-liconv' ] && [ "$should_add_libcharset" -eq 0 ]; then
should_add_libcharset=1
case "$TARGET" in
*darwin*)
if [ "$1" = '-arch=aarch64' ]; then
# macOS uses arm64 instead of aarch64
argv+=('-arch=arm64')
elif [ "$1" = '-arch' ] && [ "${2:-}" = 'aarch64' ]; then
argv+=('-arch' 'arm64')
shift 2
continue
elif (case "$1" in -DTARGET_OS_IPHONE*) exit 0 ;; *) exit 1 ;; esac) then
has_iphone=1
else
argv+=("$1")

# See https://github.com/apple-oss-distributions/libiconv/blob/a167071/xcodeconfig/libiconv.xcconfig
if [ "$1" = '-lcharset' ]; then
should_add_libcharset=-1
elif [ "$1" = '-liconv' ] && [ "$should_add_libcharset" -eq 0 ]; then
should_add_libcharset=1
fi
fi
fi
else
argv+=("$1")
fi
;;
*windows*)
if (case "$1" in *.rlib | *libcompiler_builtins-*) exit 0 ;; *) exit 1 ;; esac) then
# compiler-builtins is duplicated with zig's compiler-rt
case "$CMD" in
clang*) argv+=("$1") ;;
esac
else
argv+=("$1")
fi
;;
*)
argv+=("$1")
;;
esac
fi

if [ "$1" = '-E' ]; then
Expand Down Expand Up @@ -222,6 +233,19 @@ while [ "$#" -gt 0 ]; do
argv+=(-v)
;;
esac
elif [ "$1" = '-Bdynamic' ]; then
case "$CMD" in
clang*)
l_args+=(-Bdynamic)
;;
*)
# https://github.com/ziglang/zig/pull/16058
# zig changes the linker behavior, -Bdynamic won't search *.a for mingw, but this may be fixed in the later version
# here is a workaround to replace the linker switch with -search_paths_first, which will search for *.dll,*lib first,
# then fallback to *.a
l_args+=(-search_paths_first)
;;
esac
elif [ "$1" = '-dynamic' ]; then
case "$CMD" in
clang*)
Expand Down Expand Up @@ -254,12 +278,12 @@ while [ "$#" -gt 0 ]; do
;;
esac
elif [ "$1" = '-dylib' ] \
|| [ "$1" = '--dynamicbase' ] \
|| [ "$1" = '--large-address-aware' ] \
|| [ "$1" = '--no-undefined-version' ] \
|| [ "$1" = '--disable-auto-image-base' ]; then
# zig doesn't support -dylib, -dynamic, --no-undefined-version, --large-address-aware and --disable-auto-image-base
# https://github.com/ziglang/zig/issues/16855
# https://github.com/ziglang/zig/pull/17326
# zig doesn't support -dylib, --dynamicbase, --large-address-aware and --disable-auto-image-base
# https://github.com/rust-cross/cargo-zigbuild/blob/841df510dff8c585690e85354998bbcb6695460f/src/zig.rs#L187-L190
# https://github.com/ziglang/zig/issues/10336
case "$CMD" in
clang*)
l_args+=("$1")
Expand Down
2 changes: 1 addition & 1 deletion scripts/rc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,4 @@ case "$_name" in
;;
esac

exec "/usr/bin/llvm-${_name}-16" "$@"
exec "/usr/bin/llvm-${_name}-17" "$@"
6 changes: 3 additions & 3 deletions scripts/tool-wrapper.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ fallback_llvm() {
if ! command -v "${_prefix}$(basename "$0")" 1>/dev/null 2>&1; then
_prefix='llvm-'
if [ "$0" = 'libtool' ]; then
_suffix='-darwin-16'
_suffix='-darwin-17'
else
_suffix='-16'
_suffix='-17'
fi
fi
}
Expand All @@ -26,7 +26,7 @@ case "${TARGET:?TARGET envvar is required to be defined}" in
;;
*)
_prefix='llvm-'
_suffix='-16'
_suffix='-17'
;;
esac
;;
Expand Down
9 changes: 0 additions & 9 deletions scripts/toolchain.sh
Original file line number Diff line number Diff line change
Expand Up @@ -188,12 +188,3 @@ EOF
ln -s "gcc_eh.pc.pc" "${PREFIX}/lib/pkgconfig/libgcc_eh.pc"
;;
esac

case "$TARGET" in
*windows-gnu)
# Work around LTO bugs when compiling C++ for windows targets
# https://github.com/ziglang/zig/issues/15958#issuecomment-1764915440
sed -i '/_free_locale))(_locale_t)/s/^/__attribute__((used)) /' "${SYSROOT}/lib/libc/mingw/misc/_free_locale.c"
sed -i '/_create_locale))(int, const char \*)/s/^/__attribute__((used)) /' "${SYSROOT}/lib/libc/mingw/misc/_create_locale.c"
;;
esac
4 changes: 2 additions & 2 deletions stages/00-apple/01-xar.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ esac

apt-get install libssl-dev libz-dev

export CC="clang-16"
export CXX="clang++-16"
export CC="clang-17"
export CXX="clang++-17"
export CFLAGS="-I${CCTOOLS}/include"
export LDFLAGS="-L${CCTOOLS}/lib"
export APPLE_TARGET='__BYPASS__'
Expand Down
4 changes: 2 additions & 2 deletions stages/00-apple/02-tapi.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ case "$TARGET" in
;;
esac

export CC="clang-16"
export CXX="clang++-16"
export CC="clang-17"
export CXX="clang++-17"
export CFLAGS="-I${CCTOOLS}/include"
export LDFLAGS="-L${CCTOOLS}/lib"
export APPLE_TARGET='__BYPASS__'
Expand Down
4 changes: 2 additions & 2 deletions stages/00-apple/03-dispatch.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ esac

apt-get install systemtap-sdt-dev libbsd-dev linux-libc-dev

export CC="clang-16"
export CXX="clang++-16"
export CC="clang-17"
export CXX="clang++-17"
export CFLAGS="-I${CCTOOLS}/include"
export LDFLAGS="-L${CCTOOLS}/lib"
export APPLE_TARGET='__BYPASS__'
Expand Down
8 changes: 4 additions & 4 deletions stages/00-apple/04-cctools.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ echo "APPLE_TARGET=$_target" >>/etc/environment

apt-get install uuid-dev libedit-dev

export CC="clang-16"
export CXX="clang++-16"
export CC="clang-17"
export CXX="clang++-17"
export CFLAGS="-I${CCTOOLS}/include"
export LDFLAGS="-L${CCTOOLS}/lib"
export APPLE_TARGET='__BYPASS__'
Expand All @@ -43,7 +43,7 @@ cd cctools/cctools
--with-libxar="$CCTOOLS" \
--with-libtapi="$CCTOOLS" \
--with-libdispatch="$CCTOOLS" \
--with-llvm-config=llvm-config-16 \
--with-llvm-config=llvm-config-17 \
--with-libblocksruntime="$CCTOOLS" \
--enable-xar-support \
--enable-lto-support \
Expand All @@ -56,4 +56,4 @@ make install
rm -r /srv/cctools

# Create symlinks for llvm-otool because cctools use it when calling its own otool
ln -fs "$(command -v llvm-otool-16)" /opt/cctools/bin/llvm-otool
ln -fs "$(command -v llvm-otool-17)" /opt/cctools/bin/llvm-otool
10 changes: 5 additions & 5 deletions stages/10-compiler-rt.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ case "$TARGET" in
esac

# LLVM install path
LLVM_PATH="/usr/lib/llvm-16"
LLVM_PATH="/usr/lib/llvm-17"

# Remove wrapper from PATH, because we need to call the original cmake
PATH="$(echo "${PATH}" | awk -v RS=: -v ORS=: '/\/wrapper^/ {next} {print}')"
Expand All @@ -19,9 +19,9 @@ echo "Download llvm compiler_rt..."

mkdir -p "${LLVM_PATH}/compiler_rt/build"

curl_tar 'https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.6/cmake-16.0.6.src.tar.xz' \
curl_tar 'https://github.com/llvm/llvm-project/releases/download/llvmorg-17.0.6/cmake-17.0.6.src.tar.xz' \
"${LLVM_PATH}/cmake" 1
curl_tar 'https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.6/compiler-rt-16.0.6.src.tar.xz' \
curl_tar 'https://github.com/llvm/llvm-project/releases/download/llvmorg-17.0.6/compiler-rt-17.0.6.src.tar.xz' \
"${LLVM_PATH}/compiler_rt" 1

# Link cmake files to where compiler_rt expect to find them
Expand All @@ -39,7 +39,7 @@ cmake \
-DLLVM_CMAKE_DIR="${LLVM_PATH}/cmake" \
-DDARWIN_osx_ARCHS="$(if [ "$_arch" == 'aarch64' ]; then echo 'arm64'; else echo "$_arch"; fi)" \
-DLLVM_MAIN_SRC_DIR="$LLVM_PATH" \
-DCMAKE_INSTALL_PREFIX="${LLVM_PATH}/lib/clang/16" \
-DCMAKE_INSTALL_PREFIX="${LLVM_PATH}/lib/clang/17" \
-DCMAKE_TOOLCHAIN_FILE='/srv/toolchain.cmake' \
-DDARWIN_macosx_SYSROOT="${MACOS_SDKROOT:?Missing macOS SDK path}" \
-DDARWIN_osx_BUILTIN_ARCHS="$(if [ "$_arch" == 'aarch64' ]; then echo 'arm64'; else echo "$_arch"; fi)" \
Expand All @@ -61,4 +61,4 @@ while IFS= read -r _lib; do
if [ "$_arch" == 'aarch64' ]; then
ln -s "${_lib_name}.a" "$(dirname "${_lib}")/${_lib_name}-arm64.a"
fi
done < <(find "${LLVM_PATH}/lib/clang/16/lib/darwin/" -name 'libclang_rt.*')
done < <(find "${LLVM_PATH}/lib/clang/17/lib/darwin/" -name 'libclang_rt.*')
2 changes: 1 addition & 1 deletion stages/50-x265.sh
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ ln -s ../10bit/libx265.a libx265_main10.a
mv libx265.a libx265_main.a

# Must use llvm ar due to mri-script
llvm-ar-16 -M <<EOF
llvm-ar-17 -M <<EOF
CREATE libx265.a
ADDLIB libx265_main.a
ADDLIB libx265_main10.a
Expand Down
6 changes: 4 additions & 2 deletions stages/99-ffmpeg.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ if [ "$(uname -m)" = "${TARGET%%-*}" ] && (case "$TARGET" in *linux* | x86_64-wi
# Unfortunatly that means we only suport cuda in the same architecture as the host system
# https://github.com/ziglang/zig/pull/10704#issuecomment-1023616464
env_specific_arg+=(
--nvcc="clang-16 -target ${TARGET}"
--nvcc="clang-17 -target ${TARGET}"
--enable-cuda-llvm
--enable-ffnvcodec
--disable-cuda-nvcc
Expand All @@ -72,6 +72,8 @@ case "$TARGET" in
--x86asmexe=false
--enable-vfp
--enable-neon
# Our baseline ARM platform for Linux and Windows is raspberry pi 3 which doesn't support i8mm
--disable-i8mm
)
;;
esac
Expand Down Expand Up @@ -171,7 +173,7 @@ if ! ./configure \
--cxx=c++ \
--strip=strip \
--ranlib=ranlib \
--host-cc=clang-16 \
--host-cc=clang-17 \
--windres="windres" \
--pkg-config=pkg-config \
--pkg-config-flags="--static" \
Expand Down
3 changes: 2 additions & 1 deletion stages/99-onnx.sh
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,8 @@ case "$TARGET" in
-Donnxruntime_USE_COREML=On
)
# Allow deprecated usage of ATOMIC_VAR_INIT by https://github.com/google/nsync
export CXXFLAGS="${CXXFLAGS} -D_LIBCPP_DISABLE_DEPRECATION_WARNINGS"
# Allow deprecated usage of this capture by https://gitlab.com/libeigen/eigen
export CXXFLAGS="${CXXFLAGS} -D_LIBCPP_DISABLE_DEPRECATION_WARNINGS -Wno-deprecated-this-capture"
;;
esac

Expand Down