[Android] OpenCL detection / namespace isolation

[brevilo]

Describe the bug
I'm trying to introduce OpenCL support for our Android apps. However, on my test device BOINC doesn't detect OpenCL properly, already failing at the namespace preparation:

Trying dlopen()
No android_get_exported_namespace()
No namespace

To me this means that __loader_android_get_exported_namespace returns NULL. Any idea why?

While at it (unrelated to the above):

1. Are you sure this path is correct? AFAIK, modern systems use /system/vendor/lib[64] instead.
2. Doesn't libOpenCL.so only apply to Adreno GPUs? AFAIK, Mali uses libGLES_mali.so and PowerVR libPVROCL.so for their OpenCL runtimes, no?

Steps To Reproduce

1. Use the system config below
2. Run BOINC
3. Check OpenCL/coproc detection (e.g. in coproc_info.xml or stderrgpudetect.txt)

System Information

• OS: Android 14 (LineageOS 21)
• BOINC Version: 8.0.3
• Arch: aarch64
• Platform: ARMv8a

[AenBleidd]

Hello @brevilo,

Thank you for the report.
The problem with the Android is that on different OS vendors you get different behavior.
I have tested this ob both Mali and Adreno on 2 generations of Samsung Galaxy and two generation of Xiaomi (don't recall what has what).
For me it was working fine.
Also, this doesn't mean that it works fine for you as well.
I'm not sure I will be able to reproduce this behavior since on all devices available to me I see no such issue.
I can try to think how to work on that with you to get this fixed.

To me this means that __loader_android_get_exported_namespace returns NULL. Any idea why?

Probably, your vendor doesn't support exported namespaces at all?
I'll check if that possible to ignore it and load library in some different way.

Are you sure this path is correct? AFAIK, modern systems use /system/vendor/lib[64] instead.

This is basically why we need exported namespace, because it gives us the correct 'virtual' path to the library that we can load.

Doesn't libOpenCL.so only apply to Adreno GPUs? AFAIK, Mali uses libGLES_mali.so and PowerVR libPVROCL.so for their OpenCL runtimes, no?

I'm not sure about libPVROCL.so since I have no device to test this, but on all the systems available to me libOpenCL.so was either a library or a symlink to the real library like libGLES_mali.so in your example.

[brevilo]

Probably, your vendor doesn't support exported namespaces at all?

Is this a (device) vendor or OS question? I thought the namespace isolation stuff came with project treble (Android 8) and should thus be a given on any newer Android release? Are "exported" namespaces something special and/or optional?

FYI, for my Adreno it's indeed (only) /system/vendor/lib[64]/libOpenCL.so, as expected.

[brevilo]

FYI, I recognize your approach to deal with the namespace / private API crap as being this one (see step 3 there). Are you aware of the changes in Android 11 shown there? Not sure how stable this offset-approach is but it might be worth a try.

[AenBleidd]

I don't have a definite answer to this this question.
Starting from some API (I really don't remember the number since it was several years ago), Google in their update restricted the direct access to the libraries, and introduced these namespaces to distinguish between the system libraries and custom vendor libraries. But different vendors who has applied this patch, configured namespaces differently.

Are you aware of the changes in Android 11 shown there?

Yes, I have seen them when I was working on this fix. For me this was not an issue on the devices I have tested (again, this might be vendor specific, and I was just lucky to not catch this issue with the removed function from the export table).

Not sure how stable this offset-approach is but it might be worth a try.

I'm pretty much sure this is not stable at all. For example in my case (as described above) there was no such an issue, so this is really vendor specific.

I need to think on possible solutions.

[brevilo]

This is the vendors' fault anyway since there is a solution, that's even part of the OpenCL CTS for Android. They just would have to adopt it but apparently can't be bothered 👎

[ahorek]

I tested the code on my Samsung S24 Ultra

1/ dlopen libOpenCL.so fails without export LD_LIBRARY_PATH="/system/vendor/lib64"

Trying dlopen()
dlopen works
opencl loaded
clGetPlatformIDs failed with error: -1001

2/ android_get_exported_namespace isn't available, but __loader_android_get_exported_namespace works

No android_get_exported_namespace()
Trying android_dlopen_ext()
opencl loaded
Number of OpenCL platforms: 1

btw, the native client detects the (Ardeno) GPU correctly, so the behavior in the app may differ (I'm testing it on Termux)

#include <dlfcn.h>
#include <stdio.h>
#include <android/dlext.h>
#include <string>
using std::vector;
using std::string;

void* opencl_lib = NULL;

void* (*p_android_dlopen_ext)(const char*, int, const android_dlextinfo*);
struct android_namespace_t* (*p_android_create_namespace)(const char*, const char*, const char*, uint64_t, const char*, struct android_namespace_t*);
struct android_namespace_t* (*p_android_get_exported_namespace)(const char*);

struct android_namespace_t* get_android_namespace() {
    p_android_get_exported_namespace = (struct android_namespace_t*(*)(const char*)) dlsym(RTLD_DEFAULT, "android_get_exported_namespace");
    if (!p_android_get_exported_namespace) {
        printf("No android_get_exported_namespace()\n");
    }
    if (!p_android_get_exported_namespace) {
        p_android_get_exported_namespace = (struct android_namespace_t*(*)(const char*)) dlsym(RTLD_DEFAULT, "__loader_android_get_exported_namespace");
        if (!p_android_get_exported_namespace) {
            printf("No __loader_android_get_exported_namespace()\n");

        }
    }
    if (p_android_get_exported_namespace) {
        return (*p_android_get_exported_namespace)("vndk");
    }

    p_android_create_namespace = (struct android_namespace_t*(*)(const char*, const char*, const char*, uint64_t, const char*, struct android_namespace_t*)) dlsym(RTLD_DEFAULT, "android_create_namespace");
    if (!p_android_create_namespace) {
        printf("No android_create_namespace()\n");
        return NULL;
    }
    string lib_path;
    if (sizeof(void*) == 8) {
        lib_path = "/system/lib64/";
    }
    else {
        lib_path = "/system/lib/";
    }
#define ANDROID_NAMESPACE_TYPE_ISOLATED 1
#define ANDROID_NAMESPACE_TYPE_SHARED 2
    return (*p_android_create_namespace)("trustme", lib_path.c_str(), lib_path.c_str(), ANDROID_NAMESPACE_TYPE_SHARED | ANDROID_NAMESPACE_TYPE_ISOLATED, "/system/:/data/:/vendor/", NULL);
}

void* android_dlopen(const char* filename) {
    char buf[256];
//    printf("Trying dlopen()\n");
//    void* handle = dlopen(filename, RTLD_NOW);
//    if (handle) {
//        printf("dlopen works\n");
//        return handle;
//    }

    p_android_dlopen_ext = (void*(*)(const char*, int, const android_dlextinfo*)) dlsym(RTLD_DEFAULT, "android_dlopen_ext");
    if (!p_android_dlopen_ext) {
        printf("No android_dlopen_ext()\n");
        return NULL;
    }

    struct android_namespace_t* ns = get_android_namespace();
    if (!ns) {
        printf("No namespace\n");
        return NULL;
    }

    const android_dlextinfo dlextinfo = {
        .flags = ANDROID_DLEXT_USE_NAMESPACE,
        .library_namespace = ns,
    };
    printf("Trying android_dlopen_ext()\n");
    return (*p_android_dlopen_ext)(filename, RTLD_NOW, &dlextinfo);
}

int main() {
    opencl_lib = android_dlopen("libOpenCL.so");
    if (!opencl_lib) {
        printf("Failed to open libOpenCL.so\n");
        return -1;
    }

    printf("opencl loaded\n");

    typedef int (*clGetPlatformIDs_t)(unsigned int, void*, unsigned int*);
    clGetPlatformIDs_t clGetPlatformIDs = (clGetPlatformIDs_t)dlsym(opencl_lib, "clGetPlatformIDs");

    if (!clGetPlatformIDs) {
        printf("Failed to locate clGetPlatformID\n");
        dlclose(opencl_lib);
        return -1;
    }

    unsigned int num_platforms = 0;
    int result = clGetPlatformIDs(0, NULL, &num_platforms);
    if (result == 0) {
        printf("Number of OpenCL platforms: %u\n", num_platforms);
    } else {
        fprintf(stderr, "clGetPlatformIDs failed with error: %d\n", result);
    }

    dlclose(opencl_lib);
    return 0;
}

[AenBleidd]

@ahorek, wanna make a proper PR?

[ahorek]

termux has an incorrect link to libOpenCL.so: /data/data/com.termux/files/usr/share/man/man7/libOpenCL.so.7.gz that's why the Android fallback doesn't work. However, it's specific to the terminal, so it doesn't have to be fixed. It won't happen in the real app.

attempt to load the library directly fails as expected
Error loading /system/vendor/lib64/libOpenCL.so: dlopen failed: library "/system/vendor/lib64/libOpenCL.so" needed or dlopened by "/data/data/com.termux/files/home/" is not accessible for the namespace "(default)"

but dlopen with namespaces works on both of my devices (Samsung S14 Ultra (Ardeno), Samsung S6 Edge (Mali))

@brevilo could you test the code on your device and check if you can load the library somehow?

does

export LD_LIBRARY_PATH="/system/vendor/lib64"
clinfo

work?

[brevilo]

Can do next week. Won't that run into the same isolation problem when run in the terminal?

[ahorek]

Direct dlopen("libOpenCL.so") part is commented to prevent loading the wrong termux library, but it should behave the same.

The code is extracted from boinc sources for easier experiments without external dependencies, you can simply compile it with:

pkg install clang or gcc (if necessary)
clang++ cltest.cpp -o cltest
./cltest

and you should get this output

No android_get_exported_namespace()
Trying android_dlopen_ext()
opencl loaded
Number of OpenCL platforms: 1

[AenBleidd]

export LD_LIBRARY_PATH="/system/vendor/lib64"

I'm not sure this is a portable solution

[brevilo]

Ah, sorry. I understood that you asked about clinfo only. That shouldn't work.

[brevilo]

export LD_LIBRARY_PATH="/system/vendor/lib64"

I'm not sure this is a portable solution

Not strictly. But it's the common path on virtually all modern devices. A discovery code could try the known paths and pick the one that returns the lib in search.

[ahorek]

Yes, I just wanted to check if clinfo works. The path can vary depending on the vendor, but this is the most commonly used one.