(erroneous?) GPU not supported on datacenter RTX A4000 #6

flockman66 · 2024-05-19T14:08:55Z

The basic pipeline seems to fail with GPU not supported error when testing through the provided docker image

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+

ERROR

t@582fca7de7cd:/shared/gst-nvmaxine/build# Setting pipeline to PAUSED ...
0:00:00.056353274 331 0x6034deb18e00 WARN Pipeline is PREROLLING ...
0:00:00.056479573 331 0x6034ddfa2e40 WARN 0:00:00.056514235 331 0x6034ddfa2e40 WARN Redistribute latency...
0:00:00.131230869 331 0x6034ddfa2de0 ERROR 0:00:00.132698684 331 0x6034ddfa2de0 WARN 0:00:00.133006697 331 0x6034ddfa2de0 ERROR 0:00:00.134440667 331 0x6034ddfa2de0 WARN 0:00:00.134448681 331 0x6034ddfa2de0 WARN Redistribute latency...
0:00:00.135148417 331 0x6034ddfa2de0 ERROR 0:00:00.136604369 331 0x6034ddfa2de0 WARN 0:00:00.136922718 331 0x6034ddfa2de0 ERROR 0:00:00.138371278 331 0x6034ddfa2de0 WARN 0:00:00.138694887 331 0x6034ddfa2de0 ERROR 0:00:00.140130318 331 0x6034ddfa2de0 WARN 0:00:00.140454914 331 0x6034ddfa2de0 ERROR 0:00:00.141893895 331 0x6034ddfa2de0 WARN 0:00:00.141919942 331 0x6034ddfa2e40 WARN 0:00:00.141930157 331 0x6034ddfa2e40 WARN ERROR: from element /GstPipeline:pipeline0/ Additional debug info:
qtdemux.c(6619): gst_qtdemux_loop streaming stopped, reason not-negotiated ERROR: pipeline doesn't want to preroll.
Freeing pipeline ... gst-launch-1.0 filesrc location="/shared/Big_Buck_Bunny_720_10s_5MB.mp4" ! qtdemux ! avdec_h264 ! queue ! nvmaxinevideofx effect=SuperRes modeldir="/usr/local/VideoFX/lib/models" ! queue ! x264enc ! qtmux ! filesink location="example_video_output.mp4"
basesrc gstbasesrc.c:3600:gst_base_src_start_complete: pad not activated yet
qtdemux qtdemux_types.c:239:qtdemux_type_get: unknown QuickTime node type pasp
qtdemux qtdemux.c:3250:qtdemux_parse_trex: failed to find fragment defaults for stream 1
nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported
basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported
basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
GST_PADS gstpad.c:4231:gst_pad_peer_query:queue0:src could not send sticky events
nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported
basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported
basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported
basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported
basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
qtdemux qtdemux.c:6619:gst_qtdemux_loop: error: Internal data stream error.
qtdemux qtdemux.c:6619:gst_qtdemux_loop: error: streaming stopped, reason not-negotiated (-4)
GstQTDemux:qtdemux0: Internal data stream error.
(): /GstPipeline:pipeline0/GstQTDemux:qtdemux0:
(-4)
>

flockman66 · 2024-05-19T14:12:16Z

i've also tried this on my local machines 4080(mobile), with the same results, initially i thought it was due to it being a consumer part, so i tried it on our cloud gpu, but i get the exact same result

voidmainvoid95 · 2024-05-19T15:11:38Z

Hi, as stated here https://docs.nvidia.com/deeplearning/maxine/vfx-sdk-system-guide/index.html#vfx-sdk-system-guide-linux-intro

The Video Effects SDK is compatible with GPUs that are based on NVIDIA Turing™, NVIDIA Volta™, or the NVIDIA Ampere™ architecture.

Both RTX 4000 and RTX4080 are based on Ada Lovelace, so they are not supported. Watch out that you are not using an RTX A4000 (which is based on Ampere architecture) but an RTX 4000 SFF Ada, as nvidia-smi reports.

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.08 Driver Version: 535.161.08 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX 4000 SFF Ada ... Off | 00000000:01:00.0 Off | Off |
| 30% 32C P8 5W / 70W | 3MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+

I've a 2080ti and plugins runs fine using the provided Docker image. However the plugins fails to run using a ubuntu 22.04 image configured with the same dependencies and the same libraries version, saying that no model can run on my system.

Unfortunately, Nvidia is used to limiting libraries to certain distribution, versions of hardware and/or software. 😞

flockman66 · 2024-05-19T17:20:35Z

Ah crystal, thanks for replying, im new to nvidia ML, mostly used rocm before (came over to team green because its supposed to "just work"), feels strange that itd be better for me to get a 2000/3000 series though.

flockman66 · 2024-05-19T17:20:58Z

closing.

flockman66 · 2024-05-19T17:40:06Z

bafflingly it seems ADA is supported by the VFX SDK but only on windows , really hope thats just temporary, and not some bone-headed contractual idiocy between M$ and nvidia

voidmainvoid95 · 2024-05-20T07:22:24Z

Ah crystal, thanks for replying, im new to nvidia ML, mostly used rocm before (came over to team green because its supposed to "just work"), feels strange that itd be better for me to get a 2000/3000 series though.

In most of the cases your sentence is correct. A lot of ML frameworks and cuda based projects work on nvidia platform without any sort of problem. However a lot of them are open source projects, so you can always write a patch to make them run on hardware that is not officially supported. Maxine is a proprietary framework shipped in binary form, so if nvidia doesn't provided a compatible build for your system, you are screwed. Sometimes you can make a workaround (see #4 (comment)), sometimes not.

I'm sure that the next linux release will support your GPU as well, but I don't now when Nvidia will release it.

bafflingly it seems ADA is supported by the VFX SDK but only on windows , really hope thats just temporary, and not some bone-headed contractual idiocy between M$ and nvidia

The real reason, probably, is that on windows maxine sdk is shipped as a part of Nvidia Broadcast App (https://nvidia.com/it-it/geforce/broadcasting/broadcast-app/), so they need a wider support.

flockman66 · 2024-05-21T10:02:07Z

Good to know, we're doing a de-noising and up-scaling pipeline for real-time video so this pipeline is really intriguing to us/me, plenty of open source upscaling/denoising that uses cuda, but they are all offline-compute and very slow, i'll try and get a 3080 or data-center equivalent i really would like to try this out.

Thanks for the explanation :)

rocco-haro · 2024-09-30T17:12:11Z

@flockman66 @voidmainvoid95

Hey folks. I heard back from Nvidia. Supposedly it does support ADA on ubuntu 20 and 22. I think nvidia is notorious for not updating their documentation.

These latest releases are also blocked behind a "paywall"

From Nvidia support:

AR: https://registry.ngc.nvidia.com/orgs/ea-maxine/teams/ar-linux-sdk/collections/maxine_linux_ar_sdk_ea_collection

Video: https://registry.ngc.nvidia.com/orgs/ea-maxine/teams/vfx-linux-sdk/collections/maxine_linux_video_effects_sdk_collection

AFX: https://registry.ngc.nvidia.com/orgs/ea-maxine/teams/afx-linux-sdk/collections/maxine_linux_audio_effects_sdk_collection

 

Currently we support Ubuntu 18, 20 and 22 across all three SDK’s but we are considering dropping support for Ubuntu 18 from next release onwards

voidmainvoid95 · 2024-09-30T20:16:14Z

Hey @rocco-haro , thanks for the update!

Unfortunately, this is typical behavior from Nvidia 😡.

The fact that these latest releases are blocked behind a "paywall" makes it difficult for me to continue supporting the plugin. Even though I don't need the redistributable binary, I still require access to the libraries for development, which I can't get without a company license or affiliation.

flockman66 closed this as completed May 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(erroneous?) GPU not supported on datacenter RTX A4000 #6

(erroneous?) GPU not supported on datacenter RTX A4000 #6

flockman66 commented May 19, 2024

flockman66 commented May 19, 2024

voidmainvoid95 commented May 19, 2024 •

edited

Loading

flockman66 commented May 19, 2024 •

edited

Loading

flockman66 commented May 19, 2024

flockman66 commented May 19, 2024 •

edited

Loading

voidmainvoid95 commented May 20, 2024 •

edited

Loading

flockman66 commented May 21, 2024 •

edited

Loading

rocco-haro commented Sep 30, 2024

voidmainvoid95 commented Sep 30, 2024

(erroneous?) GPU not supported on datacenter RTX A4000 #6

(erroneous?) GPU not supported on datacenter RTX A4000 #6

Comments

flockman66 commented May 19, 2024

flockman66 commented May 19, 2024

voidmainvoid95 commented May 19, 2024 • edited Loading

flockman66 commented May 19, 2024 • edited Loading

flockman66 commented May 19, 2024

flockman66 commented May 19, 2024 • edited Loading

voidmainvoid95 commented May 20, 2024 • edited Loading

flockman66 commented May 21, 2024 • edited Loading

rocco-haro commented Sep 30, 2024

voidmainvoid95 commented Sep 30, 2024

voidmainvoid95 commented May 19, 2024 •

edited

Loading

flockman66 commented May 19, 2024 •

edited

Loading

flockman66 commented May 19, 2024 •

edited

Loading

voidmainvoid95 commented May 20, 2024 •

edited

Loading

flockman66 commented May 21, 2024 •

edited

Loading