How to run two YOLOv8 models in parallel on 3060

[xillee]

Description

When I run yolov8 model on 3060, only 7% of GPU is used. I want to run more the one model in parallel in order to reduce the inference duration.

Environment

TensorRT Version:
8.6
NVIDIA GPU:
3060
NVIDIA Driver Version:

CUDA Version:
12.1
CUDNN Version:
8.6

Operating System:

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link:

Steps To Reproduce

Commands or scripts:

Have you tried the latest release?:

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

[zerollzeng]

It just be too small to consume the power of GPU, e.g. consider the difference between mobilenet and resnet. You can increase the throughput(not inference time) by increase batch or use multi-thread inference.

[ttyio]

closing since no activity for more than 3 weeks per our policy. thanks!