kfactory not installable on aarch64 due to build fails

[hpretl]

When installing gdsfactory 8.6.1 and thus kfactory 0.18.4, its dependency rectangle-packer causes issues on aarch64. This is may be a known issue (see Penlect/rectangle-packer#24), and I see a build fail on Ubuntu 22.04 LTS:

Building wheels for collected packages: rectangle-packer
  Building wheel for rectangle-packer (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [38 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-aarch64-3.10
      creating build/lib.linux-aarch64-3.10/rpack
      copying rpack/__init__.py -> build/lib.linux-aarch64-3.10/rpack
      running egg_info
      writing rectangle_packer.egg-info/PKG-INFO
      writing dependency_links to rectangle_packer.egg-info/dependency_links.txt
      writing top-level names to rectangle_packer.egg-info/top_level.txt
      reading manifest file 'rectangle_packer.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE.md'
      writing manifest file 'rectangle_packer.egg-info/SOURCES.txt'
      copying rpack/_core.pxd -> build/lib.linux-aarch64-3.10/rpack
      copying rpack/_core.pyx -> build/lib.linux-aarch64-3.10/rpack
      running build_ext
      cythoning rpack/_core.pyx to rpack/_core.c
      /usr/lib/python3/dist-packages/pythran/tables.py:4520: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
        if not hasattr(numpy, method):
      /usr/lib/python3/dist-packages/pythran/tables.py:4553: FutureWarning: In the future `np.bytes` will be defined as the corresponding NumPy scalar.
        obj = getattr(themodule, elem)

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          """

      # The following four functions are used as compare-functions for
      # sorting `Rectangle` structs by index, width, height and area.

      cdef int rectangle_index_cmp(const void *a, const void *b) noexcept nogil:
                                                                ^
      ------------------------------------------------------------

      rpack/_core.pyx:37:59: Syntax error in C variable declaration
      error: Command "aarch64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iinclude -I/usr/include/python3.10 -c rpack/_core.c -o build/temp.linux-aarch64-3.10/rpack/_core.o" failed with exit status 1

There is also another issue reported with Cython, see Penlect/rectangle-packer#22.

Either there is a solution for the upstream package, or you could consider removing the rectangle-packer dependency.

[hpretl]

@joamatab FYI.

[sebastian-goeldi]

Oh, I missed that there is no arm64 in their releases :(. I saw macosx and was like, ok cool, works on macosx.

I can check out whether there is an easy way to make it work on osx arm64

[hpretl]

That would be great! With gdsfactory 8.6.1 I can install with latest OpenLane2 finally, w/o dependency version clash :-)

[sebastian-goeldi]

@hpretl could you elaborate a bit on your build process? I just tried it on my underused OSX M2.

For both conda (more precicely miniforge/mamba but that's a technical details) and brew python3.12 it installs without a hiccup. I assume you are building a docker image or similar? So I suspect you are missing a library used to install.

[hpretl]

Indeed, I try to build a Docker container, base OS is Ubuntu 22.04 LTS.

We build in parallel for x86_64 and aarch64.

With the same scripts and everything, x86_64 works, and aarch64 fails on rectangle-packer with the above copied error log.

[hpretl]

@MrHighVoltage can you please take a look at this. Any ideas from your side?

[sebastian-goeldi]

Is the workflow/pipeline open sourced? I would be happy to take a look. I am using debian as a daily driver, so I can take a look with podman

[hpretl]

All is open source, we are building this here: https://github.com/iic-jku/IIC-OSIC-TOOLS

I tried to construct a minimum failing exampl, but failed, it was just working. Then I found a workaround, which could be a solution: I removed the package cython3 (which is installed via APT), and then the build worked fine. rectangle-packer installed, and no other package was failing.

So it seems it is a weird interaction of packages we have installed and Cython.

[sebastian-goeldi]

Argh, I should have read more carefully. Cython is indeed the culprit (you even linked against the issue detailing it...)

rpack requires <3, but the ubuntu 22.04 installs 3.X if I read correctly https://ubuntu.pkgs.org/22.04/ubuntu-universe-amd64/cython3_0.29.28-1ubuntu3_amd64.deb.html

[hpretl]

Thanks for confirming the issue with Cython. So I guess we should ask the rectangle-packer maintainer to upgrade to Cython3 then, as will be likely an issue for all current distros.

[sebastian-goeldi]

I see somewhat of a workaround (but generally I agree that the package should upgrade to cython3). As far as I understand it, cython will only be a build dependency. So what you can do is first install kfactory/gdsfactory and then install cython from apt.

I will try to see whether I can debug the issue

[sebastian-goeldi]

@hpretl I don't have a good solution for this atm. I tried everything, and can't really reproduce it. Would it be ok if I would implement this through a [all] optional dependency and then use a lazy import through PEP562? This would mean if someone were to try to access it, it would throw an import error telling them to install [all]

[hpretl]

I have moved on to gdsfactory 8.8.2 and kfactory 0.19.2, and with the removal of Cython it installed fine, including rectangle_packer. It is not a real solution, but works fine, so shall we leave it at that for now?

[sebastian-goeldi]

That sounds reasonable. Is there a minimal docker I can try against? As in is it just a cross compiled aarch64? If so, I can try to see whether it will work by adding Cython as a build dependency in pyproject.toml

[hpretl]

I have no minimal Docker, just the full-blown monster IIC-OSIC-TOOLS.