Polygot context can not re-evaluate python script that import pandas

[dangnguyenngochai]

Running the following example raise

package io.epsilo.web;
import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.Engine;
import org.graalvm.polyglot.Source;
import org.graalvm.polyglot.Value;

public class TestMultiPolygotContext {
    private static final String PYTHON_EXECUTABLE_PATH = "./work/graalpy_venv/bin/graalpy";
    private static final String LANGUAGE_ID = "python";
    private Engine engine;

    public TestMultiPolygotContext(){
        System.out.println("TestMultiPolygotContext - begin");
        engine = Engine.newBuilder(LANGUAGE_ID).build();
        Context.Builder builder = Context.newBuilder()
                .engine(engine)
                .allowAllAccess(true)
                .option("python.ForceImportSite", "true")
                .option("python.Executable", PYTHON_EXECUTABLE_PATH);

        try{
            System.out.println("Context 1");
            String pyScript = """
                    import pandas as pd
                                        
                    foodf = pd.DataFrame({"col1": [1,2,3], "col2": [4,5,6]})
                    print(foodf)
                    """;
            try(Context ctx1 = builder.build()){
                int attempt=0;
                while (attempt < 3) {
                    try {
                        ctx1.eval(Source.create(LANGUAGE_ID, pyScript));
                        break;
                    }
                    catch (Exception ex){
                        attempt++;
                        if (attempt == 3){
                            throw ex;
                        }
                    }
                }
            }catch(Exception ex){
                throw ex;
            }
            System.out.println("Context 1 - closed");
            Thread.sleep(5000);
            try( Context ctx2 = builder.build()){
                System.out.println("Context 2");
                int attempt=0;
                while (attempt < 3) {
                    try {
                        ctx2.eval(Source.create(LANGUAGE_ID, pyScript));
                        break;
                    }
                    catch (Exception ex){
                        attempt++;
                        if (attempt == 3){
                            throw ex;
                        }
                    }
                }
            }catch(Exception ex){
                throw ex;
            }
            System.out.println("Context 2 - closed");
        } catch (Exception ex) {
            ex.printStackTrace();
            System.err.println("Error: " + ex.getMessage());
        }
        System.out.println("TestMultiPolygotContext - end");
    }

    public static void main(String[] args) {
        System.out.println("TestMultiPolygotContext");
        TestMultiPolygotContext test = new TestMultiPolygotContext();
    }
}

[python::capi] WARNING: GraalPy option 'NativeModules' is set to true, but only one context in the process can use native modules, second and other contexts fallback to NativeModules=false and will use LLVM bitcode execution via GraalVM LLVM.
SystemError: no bitcode found for /Users/epsilo/Downloads/kwl/work/graalpy_venv/lib/python3.10/site-packages/numpy/core/_multiarray_umath.graalpy231-310-native-x86_64-darwin.so
	at <python> <module>(Unnamed:1-4:0-89)
	at org.graalvm.polyglot.Context.eval(Context.java:402)
	at io.epsilo.web.TestMultiPolygotContext.<init>(TestMultiPolygotContext.java:54)
	at io.epsilo.web.TestMultiPolygotContext.main(TestMultiPolygotContext.java:77)
Error: SystemError: no bitcode found for /Users/epsilo/Downloads/kwl/work/graalpy_venv/lib/python3.10/site-packages/numpy/core/_multiarray_umath.graalpy231-310-native-x86_64-darwin.so

System Info:

• GraalVM 21
• Graalpy 23.1

[msimacek]

Hi @dangnguyenngochai. It's a documented limitation of using native extensions. Native libraries have global state and there is no way to reset it for the new context. There are some experiments with copying and patching the library files to be able to load them a second time. Please follow #437 for that work. I'll close this issue as a duplicate of that one.