failure of TensorRT 8.6 when running inferencing on GPU NVIDIA GeForce RTX 3070 with error [convBaseRunner.cpp::execute::295] Error Code 1: Cask (Cask convolution execution)

[sandeepgadhwal]

Description

I try to save a onnx model to trt engine, works well. But when loading it to python API it fails while in context.execute_v2(self.allocations)

Environment

TensorRT Version: 8.6.1 (nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-11.8_1.0-1_amd64.deb)

NVIDIA GPU: NVIDIA GeForce RTX 3070

NVIDIA Driver Version: 535.154.05

CUDA Version: 12.2

CUDNN Version:

Operating System:

Python Version (if applicable): 3.10

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link:

Steps To Reproduce

I converted the model using this command

/usr/src/tensorrt/bin/trtexec --onnx=model.onnx --saveEngine=engine.trt

I test the engine using this command

/usr/src/tensorrt/bin/trtexec --loadEngine=engine.trt --batch=1

all works fine

But when i try to load engine through python API it does not work and results in the following traceback.
a.txt

Commands or scripts:
Script i am using for inferencing:
tensorrt_test_fps.py.txt

.
Have you tried the latest release?: yes

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): yes works fine with onnxruntime with Tensorrt Execution backend.

[sandeepgadhwal]

Closing the issue because I made a silly mistake in code and now it is working.

[patrickhulce]

@sandeepgadhwal what was your silly mistake? asking for a...friend :)

[sandeepgadhwal]

Hi @patrickhulce,

I had allocated wrong size of memory. After allocating correct amount of memory it worked fine. I cross checked the number of bytes allocated.

[yaobaishen]

Hi @patrickhulce,

I had allocated wrong size of memory. After allocating correct amount of memory it worked fine. I cross checked the number of bytes allocated.

Hi @sandeepgadhwal , I have encountered the same error, how did you find out the "allocate wrong size of memory" issue? the context.execute_v2() API doesn't print any useful log to help debugging...

[yaobaishen]

To whom run into similar issue,

@sandeepgadhwal has mentioned it's related to the memory buffer you created.
In my case, I use pycuda to malloc the input/output buffer of the model, unfortunately, as the below link said, "pytorch-gpu and pycuda may have conflicts", so I switch to use CUDA python API instead of pycuda to malloc all the buffers, then the issue disappears.
please refer to:
https://forums.developer.nvidia.com/t/tensorrt-error-cuda-runtime-invalid-resource-handle-torchaudio-on-gpu-trt-engine-get-wrong-result/199750/5