Fix refinedet trt8

[fantes]

This PR addresses #1324

it also updates dependencies to TRT 8.x
BIG FAT WARNING :
TRT 8.0.1.x is subject to this bug : https://forums.developer.nvidia.com/t/build-engine-error-when-use-pointnet-like-structure-and-tensorrt-8-0-1-6/183569, which makes ssd models not working !!!

[beniz]

libnvinfer.so.8 instead

[fantes]

was already done, not pushed, sorry

[beniz]

@fantes I do confirme the docker does build with the latest NVidia+tensorrt container image, patch is below that could be integrated to this PR:

diff --git a/docker/gpu_tensorrt.Dockerfile b/docker/gpu_tensorrt.Dockerfile
index 59ce5317..03de8d62 100644
--- a/docker/gpu_tensorrt.Dockerfile
+++ b/docker/gpu_tensorrt.Dockerfile
@@ -1,5 +1,5 @@
 # syntax = docker/dockerfile:1.0-experimental
-FROM nvcr.io/nvidia/tensorrt:21.04-py3 AS build
+FROM nvcr.io/nvidia/tensorrt:21.07-py3 AS build
 
 ARG DEEPDETECT_RELEASE=OFF
 ARG DEEPDETECT_ARCH=gpu
@@ -110,7 +110,7 @@ RUN --mount=type=cache,target=/ccache/ mkdir build && cd build && ../build.sh
 RUN ./docker/get_libs.sh
 
 # Build final Docker image
-FROM nvcr.io/nvidia/tensorrt:21.04-py3 AS runtime
+FROM nvcr.io/nvidia/tensorrt:21.07-py3 AS runtime
 
 ARG DEEPDETECT_ARCH=gpu

[beniz]

This is the same test twice, no ?

[fantes]

right, fixed

[beniz]

At the moment, several models appear to be broken:

• Squeezenet SSD (and probably other SSDs as well), from the unit tests:

[2021-08-23 11:48:46.032] [imgserv] [error] [resources.cpp::~ScopedCudaEvent::438] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[2021-08-23 11:48:46.032] [imgserv] [error] [resources.cpp::~ScopedCudaEvent::438] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[2021-08-23 11:48:46.032] [imgserv] [error] [resources.cpp::~ScopedCudaEvent::438] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[2021-08-23 11:48:46.032] [imgserv] [error] /data1/beniz/code/deepdetect/build_trt/tensorrt-oss/src/tensorrt-oss/plugin/priorBoxPlugin/priorBoxPlugin.cpp (253) - Cuda Error in destroy: 700 (an illegal memory access was encountered)
terminate called after throwing an instance of 'nvinfer1::plugin::CudaError'
  what():  std::exception
Aborted (core dumped)

• refinedet in fp16:

[2021-08-23 11:51:40.440] [imgserv] [info] --------------- Timing Runner: detection_out (PluginV2)
[2021-08-23 11:51:40.453] [imgserv] [info] Deleting timing cache: 489 entries, 505 hits
[2021-08-23 11:51:40.466] [imgserv] [info] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 0, GPU 2745 (MiB)
[2021-08-23 11:51:40.467] [imgserv] [error] 2: [pluginV2Runner.cpp::execute::267] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed.)
[2021-08-23 11:51:40.467] [imgserv] [error] 2: [builder.cpp::buildSerializedNetwork::417] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed.)

• model saving ? can't find the .bs model after tensorrt has compiled the model.