Support multi-arch images

[giqbal]

Thanks for putting this together 🙂

Tried to run this on an Apple silicon Mac and it doesn't play nicely as the image runs in emulation mode. When you adapt it for ARM it works great on Apple silicon. Any chance you could do a multi-arch build to support other platforms?

Happy to contribute with PR as well.

[tseaward]

Yes, thanks for this container, I find it very useful 👍.

I had the same issue. The only line I needed to change was the following:

databricksConnectDocker/Dockerfile

Line 30 in ac7b50c

RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && \

Replaced x86_64 with $(ARCH) where ARCH=aarch64.

Perhaps $(uname -m) would also work?

[giqbal]

@tseaward yes had to make the same change

[alexeyegorov]

Would you mind to do this as a MR? @tseaward @giqbal

[giqbal]

@alexeyegorov I don't have access to Apple Silicon machine atm. Will do next time I am near one.

[augusten]

Hi all! I created a PR in order to support ARM64 here

[alexeyegorov]

@augusten thanks for your PR. :)

basically, wanted to share with you all here @giqbal @tseaward , that DB connect will probably not be supported for a long time any more. But it will get a much better replacement called Spark Connect. It will be more lightweight client which will execute all the code on the cluster (rather then now executing python locally and pyspark on the cluster).

[augusten]

@alexeyegorov I added a comment in the PR, this might not be as straightforward as first thought with issues in building the image on a non-ARM machine. I will explore Spark Connect, thanks!