Add Code Generation or Bridge for Java / JVM

[phdoerfler]

There is currently code generation for C and Python and there are a few inofficial bridges using the former to call futhark code from Haskell, Python, rust and Standard ML. However, there is no such convenient way to call futhark from a JVM language. Please add such support. I'd love to be able to call futhark code from, e.g., a Scala program. Thanks!

[athas]

A Scala backend is not realistic. We once toyed with the idea of having many backends (we even had one for C# for a while), but it ended up being too much work to maintain them. A bridge wouldn't be difficult. It could either use Scala's support for calling C code (which I know nothing about), or use the server protocol like futhark-server-haskell. The latter is much easier to implement, but not as efficient for many applications.

[phdoerfler]

I know of basically three viable approaches to calling native code from the JVM (which Scala runs on):

• JNA: https://github.com/java-native-access/jna (modern and convenient, this is similar to Python's ctypes)
• JNI: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/ (older and probably more cumbersome than JNA, perhaps a tiny bit faster but probably not worth it)
• SWIG: https://www.swig.org (generates binding code for a large number of languages based on C/C++ header files)

SWIG is a bit of a special case as it requires C/C++ code to call into.
I have used SWIG quite successfully in the past, it's a powerful tool that allows to call even object oriented C++ code from other languages quite nicely. Something that is normally quite cumbersome when hand writing a C interface for shared libraries.
If it is possible to generate wrapper code using SWIG, this would open the possibility to implement the idea you mentioned: Having many backends. This way you would only need to support the C backend and could let SWIG do the heavy lifting of dealing with other languages.

[athas]

I know some have used SWIG before. It probably works as well as you expect, and doesn't require any Futhark-specific code. The C API generated by Futhark is explicitly designed to be FFI-friendly (e.g. everything is a pointer or built-in primitive type). But the resulting API will be quite annoying to use in many languages, for example requiring manual memory management. The main advantage of something like futhark-pycffi is that it wraps the raw API in something that is (slightly) more idiomatic.

[FluxusMagna]

I'm not sure what facilities for abstraction java has, but when I wrote the Haskell bridge, the main things to consider were:

• memory management; pairing the pointers with their respective finalizers so that the GC can manage them.
• context configuration and generation; what made sense to me was to have a function taking a list of configuration options argument and then returning an initialized context.
• context application; almost all of the API functions take the context as one of their arguments, and it's kind of messy to pass it around explicitly all the time. I solved this by running everything in a reader monad, where the context is accessible, but this exact solution might not be a viable option in most languages.

To solve these things you'll probably need to write and/or generate some specialized code in the JVM language. Writing the first largely functional version of the haskell bridge did not take very long, so it's probably quite feasible.