Add static build with Nix (fixes #1295)

[monacoremo]

Building a static executable works, but there are two issues remaining that would be nice to get fixed:

• This currently relies on a nix/postgrest.nix that was pre-generated with cabal2nix - would be better to avoid that by using callCabal2nix as we do for the dynamic derivation, but I was not able to get that to work yet. Update: fixed with e6eb775
• This builds on GHC 8.6.5. Currently looking into how to get GHC 8.8.3 to work (see: PostgREST static build nh2/static-haskell-nix#92). Update: Also fixed, see comment below.

[monacoremo]

Note: Now that we have CI integration, this will be picked up and built through CircleCI - this will take ages, as there are no cached binaries for all the Musl based build that we defined here. This will become better once the PR is merged and builds are pushed to cachix.

@steve-chavez : If you have a halfway powerful machine, you could speed up the process by running nix-build -j 8 --cores 2 -A postgrestStatic | cachix push first, as you have the cachix signing key for postgrest. -j 8 means 8 parallel builds (where possible depending on the dependency tree), --cores 2 mean 2 processes per build, depending on what kind of build is running (e.g. make -j 2, cabal -j 2, ... ).

[monacoremo]

Regarding building on GHC 8.8.3: This previously failed due to NixOS/nixpkgs#85924 . A temporary fix is to revert a commit that changed what the build of GHC is bootstrapped with (see details in the linked issue). Until there is a proper fix, I pinned a version of Nixpkgs where that one commit is reverted. Not very pretty, but the building of static executables in Haskell is still a bit experimental.

[steve-chavez]

@monacoremo Building now! I've ran nix-build -j 8 --cores 2 -A postgrestStatic | cachix push postgrest on a desktop machine. Let's see how it goes..

[steve-chavez]

@monacoremo Got an error:

/nix/store/pxyxw7sy2hdwvgr1vyqxbamghmqf49v7-openssl-1.1.1d/etc/ssl\"" -DENGINESDIR="\"/nix/store/pxyxw7sy2hdwvgr1vyqxbamghmqf49v7-openssl-1.1.1d/lib/engines-1.1\"" -DNDEBUG  -MMD -MF ssl/statem/extensions.d.tmp -MT ssl/statem/extensions.o -c -o ssl/statem/extensions.o ssl/statem/extensions.c
checking for GNU gettext in libc... no
libtool: link: gcc -Wall -Wmissing-prototypes -Wc++-compat -Wpointer-arith -g -O2 -o tabs tabs.o  -L../src/.libs ./.libs/libfrtests.a -lm -lquadmath ../src/.libs/libmpfr.so -lgmp -Wl,-rpath -Wl,/tmp/nix-build-mpfr-4.0.2.drv-1/mpfr-4.0.2/src/.libs -Wl,-rpath -Wl,/nix/store/2jicgvwf60zfjvhj54hpicqsysdpf8a1-mpfr-4.0.2/lib
checking for iconv... yes
ok 366 - we_get_signals_mixed
FAIL: test/run-tests
======================================================
1 of 1 test failed
Please report to https://github.com/libuv/libuv/issues
======================================================
make[1]: *** [Makefile:5335: check-TESTS] Error 1
make[1]: Leaving directory '/tmp/nix-build-libuv-1.35.0.drv-0/source'
make: *** [Makefile:5595: check-am] Error 2
builder for '/nix/store/g8n9dq8dydm84arz0a6hvcs7wiq2glam-libuv-1.35.0.drv' failed with exit code 2
cannot build derivation '/nix/store/0qnw3s0g680pdgl7bs5g6wrpmcgnszwq-cmake-3.16.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/72ynv1qzciy5q4ihcjbjxxawzjnr84m3-libipt-2.0.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/8yswbbx2iil4ybdd6dck1i57smg6aw9l-gdb-9.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/fnhnzgb6jn8bgg9biic8s9m6amh549xr-python3.7-Cython-0.29.14.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9b8k415k25p1yqw58bp89v20ksk0zd2b-python3.7-PyStemmer-1.3.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/d6y827acyimh36w869h76s9lsnnc8rqa-python3.7-snowballstemmer-2.0.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/4lylp8yhg60pvfbhphj9kps5hyhfjdvb-python3.7-sphinx-2.3.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/kq490awjkjsqmc9r0mwdw0iimgzdq6dw-ghc-8.8.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/b31r1ll5wiz8k2wgsbar570hl1w3g7xx-Cabal-3.2.0.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/bcghscpwl7vm0nhzgv9z98ppb3jmv6pk-cabal2nix-2.15.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/22k2d41bk8rr9myb08z00qszvisxlgyg-cabal2nix-postgrest.drv': 1 dependencies couldn't be built
error: build of '/nix/store/22k2d41bk8rr9myb08z00qszvisxlgyg-cabal2nix-postgrest.drv' failed
(use '--show-trace' to show detailed location information)

NoInput "You need to specify store paths either as stdin or as a cli argument"

Did you saw this when building the static bin?

Looks like the last line comes from cachix: cachix/cachix#173. Though the libuv failure seems unrelated.

[steve-chavez]

These are the libuv tests that fail:

gcc  -I. -Icrypto/include -Iinclude -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MO
NT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_AS
M -DOPENSSLDIR="\"/nix/store/qlw2d1jwjijm1g70f0p4cbwqf9ccmw78-openssl-1.1.1d/etc/ssl\"" -DENGINESDIR="\"/nix/store/qlw2d1jwjijm1g70f0p4cbwqf9ccmw78-openssl-1.1.1d/lib/engines-1.1\"" -DNDEBUG
 -MMD -MF crypto/sm3/m_sm3.d.tmp -MT crypto/sm3/m_sm3.o -c -o crypto/sm3/m_sm3.o crypto/sm3/m_sm3.c                                                                                            
not ok 340 - udp_multicast_join                                                                                                                                                                
# timeout                                                                                                                                                                                      
# Output from process `udp_multicast_join`: (no output) 

gcc -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wno-cast-function-type -We
rror=implicit-function-declaration   -I. -I./Include -I/nix/store/86jphnvxfr49dj2wa2r96jyfxbs70ggw-zlib-1.2.11-dev/include -I/nix/store/q778gslzj5x14r6wzdyn9irkfrzd5016-bzip2-1.0.6.0.1-dev/in
clude -I/nix/store/a1i8lrkgx831w4gl3gs1x7v86ryfy6vm-expat-2.2.8-dev/include -I/nix/store/370awh2zpqjidxrp2m2s3jbicnvi99f3-xz-5.2.5-dev/include -I/nix/store/mxl1dylziyv4k8vmxnk1zkjzcr7k78a9-li
bffi-3.3-dev/include -I/nix/store/86jphnvxfr49dj2wa2r96jyfxbs70ggw-zlib-1.2.11-dev/include -I/nix/store/q778gslzj5x14r6wzdyn9irkfrzd5016-bzip2-1.0.6.0.1-dev/include -I/nix/store/a1i8lrkgx831w
4gl3gs1x7v86ryfy6vm-expat-2.2.8-dev/include -I/nix/store/370awh2zpqjidxrp2m2s3jbicnvi99f3-xz-5.2.5-dev/include -I/nix/store/mxl1dylziyv4k8vmxnk1zkjzcr7k78a9-libffi-3.3-dev/include -fPIC -DPy_
BUILD_CORE -I. -o Python/importdl.o ./Python/importdl.c                                                                                                                                        
not ok 341 - udp_multicast_join6                                                                                                                                                              
# timeout                                                                                                                                                                                      
# Output from process `udp_multicast_join6`: (no output)  

Seems related to libuv/libuv#2468.

Maybe this gives a hint on how to solve the issue: NixOS/nixpkgs#79331.
Building libuv independently and seeing if it fails.

[monacoremo]

@steve-chavez Hm, that's weird - as far as I remember the compile went through for me back then. I'll have access to my desktop again next week, will then try to recompile and see what the issue is!

[steve-chavez]

@monacoremo Seems it was a firewall issue. Got the hint from libuv/libuv#2827 (comment).

I did networking.firewall.enable = false on my NixOS machine and libuv now builds!

[steve-chavez]

The build was successful!

nix-build -j 8 --cores 2 -A postgrestStatic | cachix push postgrest                                                                           
All done.

ldd result/bin/postgrest                                                                                                                      
        not a dynamic executable

Also, the cachix storage for postgrest looks like this:

I'm guessing the build results got uploaded fine.

@monacoremo Rebase and I'll merge :)

[monacoremo]

@steve-chavez Awesome! I rebased and added a small tweak: The required fixes to Nixpkgs and static-haskell-nix are now included as patches, instead of pinning forked versions of the repositories. This should be easier to maintain.

Also with that tweak, nix-shell does not depend on the patched Nixpkgs anymore. Accordingly, the normal version of GHC that is available in the general binary cache will be used, which will be faster for users not building static executables.

Edit: Even though this results in the same static binary, it changed the hash of our Musl GHC and all derivations that build on it, so the CI will have to re-do a lot of work and take a very long time... :-) We might hit some maximum build time with CircleCI, in that case it would be best if you could build and push to cachix locally again.

Edit 2: Actually, not sure why it had to rebuild all the Musl stuff from the ground up... did it pick up the cachix cache at all? I'll try that locally also.

[steve-chavez]

@monacoremo Patches look better :]

I've done nix-build -j 8 --cores 2 | cachix push postgrest again. The build went fast and the cachix storage is now at 561.04 MB(+10 MB).

Edit 2: Actually, not sure why it had to rebuild all the Musl stuff from the ground up... did it pick up the cachix cache at all? I'll try that locally also.

I'm not sure, but the CircleCI job went for 5 hours 😮. Looks like it didn't pick up the cache.

[monacoremo]

@steve-chavez Hm, it's possible that some build dependencies were not pushed to the cache with the command above. I tried to add the static-haskell-nix cache and it picked up a few dependencies from there, so it seems to work in general.

Could you try to run nix-store -qR --include-outputs $(nix-instantiate -A postgrestStatic) | cachix push postgrest ?

nix-instantiate -A postgrestStatic prints the path of the derivation .drv file. nix-store -qR --include-outputs queries the Nix store (q) for all requisites (R, i.e. all dependencies and their dependencies) and also includes that paths of their outputs and all their dependencies. Those will then all be cached.

[steve-chavez]

@monacoremo I did:

nix-store -qR --include-outputs $(nix-instantiate -A postgrestStatic) | cachix push postgrest     
                                            
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
pushing /nix/store/w2jpd5dgvhzfril4qk2alxzdjwnnfpg6-musl-1.1.24
pushing /nix/store/5r43327i6brxn06xwm7681c2g2jprjwc-attr-2.4.48
pushing /nix/store/405xvq8mjw4ykb68rhncfx2246qkqzly-ncurses-6.2
pushing /nix/store/b3gczl0s0cjqgbw96xn73kbxm7kxw30c-coreutils-8.31
pushing /nix/store/nqnxz62s5la50yx4mrrqqmqsyp28j212-acl-2.2.53
pushing /nix/store/fagij14qgv3s4wlh6lmzds3r5nr5ylnb-binutils-2.31.1
pushing /nix/store/zpslyl3sq6vydqzj0ia7h3hs8hqwjr21-bash-4.4-p23
pushing /nix/store/2xc97afv2hmch8v5b5zzkqzsa2qpgz29-binutils-wrapper-2.31.1
pushing /nix/store/34hnf7lgff4l4frhh3lndmcsm6vkl3xc-gcc-9.3.0-lib
pushing /nix/store/bvh8d5hb5b24g2a076pydxqj9pa8wdxv-zlib-1.2.11
pushing /nix/store/d2w0lkhj99jif4cs0vigndyyws7mk72q-musl-1.1.24-dev
pushing /nix/store/vcqq3c09a45vv1ny1866idgsl7mcdami-expand-response-params
pushing /nix/store/v29drdkgvihdarslrgqwl3n36m81zjq0-gmp-6.2.0
pushing /nix/store/fw0y81sjkpi24bdv9c2plxzfbrgdhxqc-linux-headers-5.5
pushing /nix/store/f36v4zgaq88xdwi8bycbrrp00s4qlwrv-gmp-6.2.0-dev
pushing /nix/store/ggax7mmvda6wvc7kiszdgndh2xikkqkr-gcc-9.3.0
pushing /nix/store/l3a7r4f3riqdk4ffgfg303z0vx4i1vb8-libffi-3.3
pushing /nix/store/mf49l865c0vqyp48dz5wi71zx2b1d00x-musl-iconv-1.1.24
pushing /nix/store/my93gb68fija85z7v5k4ybc00i38bixr-ghc-8.8.3-doc
pushing /nix/store/mxl1dylziyv4k8vmxnk1zkjzcr7k78a9-libffi-3.3-dev
pushing /nix/store/pqyfd232n38mlvaph9w060jnp45mnyp0-perl-5.30.1
pushing /nix/store/dxxhnska8cz01lby8mj6dsp8xh23nywx-pcre-8.44
pushing /nix/store/jg8z2rbb17v57dch10zv3xcl8m8nnk0r-gnugrep-3.4
pushing /nix/store/9cwm9h789riv5qgfysbbs34pvh6j3clq-ghc-8.8.3
....

It took a while to finish, but it's done now.

Now the cachix storage shows 2.89GB(!). Do you think that now the CI build will be fast?

[monacoremo]

@steve-chavez Looking good!! :-)

[steve-chavez]

@monacoremo Great! Up next, I'll integrate the static bin with the release process. I'm thinking I could remove the centos7/ubuntu releases and prefer the new static bin. Those releases failed from time to time(package managers with outdated urls).

The ubuntui386 would remain though. Or maybe we can build the static bin for 32 bit somehow?

[monacoremo]

@steve-chavez In principle it would be possible to cross-compile a static 32 bit executable by using `pkgs.pkgsCross.musl32', but I ran into issues when I just tried that... Might be easier to stick with the Docker based build for now for 32 bit.