Easy GPU integration?

[hrstoyanov]

@tjake Thank you very much for picking up the torch for Java AI!

Watching your Devoxx 2024 session, it looks like you plan on incorporating GPUs as well.

The GPU sdks are very fragmented. You have CUDA (nvidia), ROCm(amd), Metal(apple) and Intel(?) all doing their own thing. This makes it very hard to make JLame work with all, and it is not cool to go with CUDA only, or wait for Babylon to provide all the plumbings.

Here is an alternative idea:

The web browser folks pushed the WebGPU standard for every browser and platform recently. Therefore, WebGPU is implemented for every GPU accelerator! So, if JLama can incorporate the .DLL/.SO native webgpu lib that the browsers link in , it will get accelerated tensor ops for free, using single api, on every OS+GPU combo!

This is already being done with the GPU.CPP project - take a look at it, but the wrapper is C++, which is hard to interface with Java (Panama does not support C++). However, JLama can do something similar and directly work with the webgpu shared libraries, which are C APIs - basically re-write the (single) gpu.hpp file into java/panama, and just bundle the pre-build Google's Dawn WebGPU implementation (also included into the gpu.cpp repo) as a shared library binary.

[tjake]

Very cool! Thanks for the pointers!

[hrstoyanov]

Np! I improved the initial post ...