[BUG] ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory

[ss8319]

Describe the bug
When attempting to execute the simple_imaging_app example using the MONAI Deploy App SDK, the program fails to run due to a missing library error (libcudart.so.11.0: cannot open shared object file: No such file or directory). This issue occurs despite the environment being set up with a V100 GPU and CUDA 12.2.

Steps/Code to reproduce bug

Install the MONAI Deploy App SDK:

bash
pip install monai-deploy-app-sdk

Clone the MONAI Deploy App SDK repository:

bash

git clone https://github.com/Project-MONAI/monai-deploy-app-sdk.git
cd monai-deploy-app-sdk
Install the dependencies for the simple_imaging_app:

bash

pip install matplotlib Pillow scikit-image
Run the app with the following command:

bash

python examples/apps/simple_imaging_app/app.py -i examples/apps/simple_imaging_app/brain_mr_input.jpg -o output
Observe the error traceback:

Traceback (most recent call last):
Traceback (most recent call last):
File "examples/apps/simple_imaging_app/app.py", line 15, in
from gaussian_operator import GaussianOperator
File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/v100-monai/code/Users/monai-deploy-app-sdk/examples/apps/simple_imaging_app/gaussian_operator.py", line 14, in
from monai.deploy.core import ConditionType, Fragment, Operator, OperatorSpec
File "/home/azureuser/.local/lib/python3.8/site-packages/monai/deploy/init.py", line 23, in
from . import _version, conditions, core, exceptions, logger, resources, utils
File "/home/azureuser/.local/lib/python3.8/site-packages/monai/deploy/conditions/init.py", line 12, in
from holoscan.conditions import *
File "/home/azureuser/.local/lib/python3.8/site-packages/holoscan/init.py", line 17, in
from . import cli, core, gxf
File "/home/azureuser/.local/lib/python3.8/site-packages/holoscan/core/init.py", line 65, in
from ..graphs._graphs import FragmentGraph, OperatorGraph
File "/home/azureuser/.local/lib/python3.8/site-packages/holoscan/graphs/init.py", line 24, in
from ._graphs import FragmentFlowGraph, OperatorFlowGraph
ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory

Expected behavior
The application should execute successfully, processing the input image and saving the output without errors related to missing CUDA libraries.

Environment details (please complete the following information)

OS/Platform: Ubuntu 20.04.6 LTS (x86-64)
Python Version: Python 3.8
Method of MONAI Deploy App SDK install: pip
SDK Version: monai-deploy-app-sdk 0.6.0
GPU: NVIDIA V100
CUDA Version: CUDA 12.2
Additional context

[gigony]

Hi @ss8319 ,

Thank you for the report!

I see that you are using CUDA 12 and monai-deploy-app-sdk v0.6.0. I was able to reproduce the issue with v0.6.0 as well.

The latest version of MONAI Deploy App SDK is 2.0.0 (https://pypi.org/project/monai-deploy-app-sdk/2.0.0/).
Could you please install version >= 1.0.0?

pip install monai-deploy-app-sdk==2.0.0

python examples/apps/simple_imaging_app/app.py -i examples/apps/simple_imaging_app/input/brain_mr_input.jpg -o output

[ss8319]

Hi @gigony

Thanks for yr help!

My issue with the installation of 2.0 even after upgrading pip to the latest version. It seems like v0.6.0 is the only one I can install. I tried this on my laptop too and I have a similar issue.

azureuser@v100-monai:~/cloudfiles/code/Users/monai-deploy-app-sdk$ pip install monai-deploy-app-sdk==2.0
Defaulting to user installation because normal site-packages is not writeable
Collecting monai-deploy-app-sdk==2.0
  Using cached monai_deploy_app_sdk-2.0.0-py3-none-any.whl.metadata (7.6 kB)
Requirement already satisfied: numpy>=1.21.6 in /usr/local/lib/python3.8/dist-packages (from monai-deploy-app-sdk==2.0) (1.24.4)
INFO: pip is looking at multiple versions of monai-deploy-app-sdk to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement holoscan~=2.0 (from monai-deploy-app-sdk) (from versions: 0.0.0.post1, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.6.0)
ERROR: No matching distribution found for holoscan~=2.0

I think the issue is that holoscan is blocking the monai sdk installation. Tried uninstalling holoscan and reinstalling monai sdk via pip and get the same issue.

[MMelQin]

@ss8319 You have encountered an old known compatibility issue. With Ubuntu 20.04, you'd not be able to install MD App SDK 2.0 as it depends on Holoscan~=2.0 which requires Ubuntu 22.04. I'd suggest upgrade the OS to 22.04 so that you can have the latest packages. Also, there were discussion topics around CUDA version, Torch 2.0, MD App SDK v0.6 compatibility. I'll search up in the Discussions.

[ss8319]

Noted thanks!

Also, there were discussion topics around CUDA version, Torch 2.0, MD App SDK v0.6 compatibility. I'll search up in the Discussions.

Looking forward to this.

[ss8319]

I am actually also curious to know if MONAI deploy sdk is compatible with CPUs say a CPU from Intel. Do u have any setup instructions for this?

[MMelQin]

No support for CPU since v0.6.