Missing rtp packets in rtp-forwarder

[marlocarlo]

Hi,
When I send HD video from a peer to the rtp-forwarder and use ffmpeg to stream, I get this error :

[sdp @ 0x55a64b1aca00] max delay reached. need to consume packet
[sdp @ 0x55a64b1aca00] RTP: missed 43 packets
[sdp @ 0x55a64b1aca00] max delay reached. need to consume packet
[sdp @ 0x55a64b1aca00] RTP: missed 20 packets

In addition to that, the video is fully pixelated and garbled. Please advise.

Thanks.

[Sean-Der]

Copying the discussions we have had so far here.

Hi @marlocarlo

Do you know if the loss is between ffmpeg -> Pion or Pion -> Browser? Does this occur over the loopback/internet etc.. ?

Hi @Sean-Der ,

The issue is between pion and the browser across the internet. The browser is not opened inside the server. So it is between pion on the server and browser opening the webpage hosted there on my system.

When i stream with a much smaller resolution video like 260p,theres no problem but when i increase the resolution to 720p,theres packet loss for ffmpeg. It shows errors that rtp packets are missed and the streamed video is full of artifacts and pixels.

Thanks.

[Sean-Der]

I would try and determine if the issue is between Browser -> Pion or Pion -> ffmpeg

It seems unlikely that Pion -> ffmpeg is the issue. You should have a pretty reliable network.

From Browser -> Pion how are you forcing 720p? The browsers congestion controller should be handling that. Can you look at the remoteInboundRtpStats this will tell you if Pion is seeing loss on the inbound stream. If Pion isn't seeing any loss that you need to debug Pion -> ffmpeg.

I would check if you can increase ffmpeg buffer sizes and check if network/CPU is saturated?

[marlocarlo]

I just tried it on localhost. The packet loss is better but still there is packet loss. Parts of the video have glitches instead of the whole video. See a scrolling ticker called breaking news moving at the bottom of the video. See how it appears.

The correct video is supposed to show this :

On ffmpeg,I see this :

[h264 @ 0x55a14d47b9c0] concealing 960 DC, 960 AC, 960 MV errors in P frame [sdp @ 0x55a14d3b2600] RTP: dropping old packet received too late Last message repeated 14 times [sdp @ 0x55a14d3b2600] max delay reached. need to consume packetate=1387.5kbits/s dup=0 drop=5 speed= 1x [sdp @ 0x55a14d3b2600] RTP: missed 1 packets [sdp @ 0x55a14d3b2600] max delay reached. need to consume packetate=1386.9kbits/s dup=0 drop=5 speed= 1x [sdp @ 0x55a14d3b2600] RTP: missed 2 packets [sdp @ 0x55a14d3b2600] RTP: dropping old packet received too latete=1386.9kbits/s dup=0 drop=5 speed= 1x Last message repeated 1 times [sdp @ 0x55a14d3b2600] max delay reached. need to consume packetate=1386.6kbits/s dup=0 drop=5 speed= 1x [sdp @ 0x55a14d3b2600] RTP: missed 3 packets [sdp @ 0x55a14d3b2600] max delay reached. need to consume packet [sdp @ 0x55a14d3b2600] RTP: missed 18 packets [sdp @ 0x55a14d3b2600] max delay reached. need to consume packet [sdp @ 0x55a14d3b2600] RTP: missed 23 packets [sdp @ 0x55a14d3b2600] RTP: dropping old packet received too late Last message repeated 27 times [h264 @ 0x55a14d6d2640] mb_type 28 in I slice too large at 59 20 [h264 @ 0x55a14d6d2640] error while decoding MB 59 20 [h264 @ 0x55a14d6d2640] corrupted macroblock 55 35 (total_coeff=-1) [h264 @ 0x55a14d6d2640] error while decoding MB 55 35 [h264 @ 0x55a14d6d2640] Frame num change from 0 to 1 [h264 @ 0x55a14d6d2640] decode_slice_header error [h264 @ 0x55a14d6d2640] concealing 1744 DC, 1744 AC, 1744 MV errors in I frame [flv @ 0x55a14d45a2c0] Failed to update header with correct duration. [flv @ 0x55a14d45a2c0] Failed to update header with correct filesize.

Screenshot to give you a better idea :

CPU Usage is normal.

I think it's a pion->ffmpeg issue. because, when I stream from the webrtc client to another webrtc client in the browser, it works without these issues. The video is displayed there without packet loss even over the internet.

Please advise..

[marlocarlo]

From Browser -> Pion how are you forcing 720p?

I am using capturestream and streaming from the canvas so I have set the canvas size to 720p.

I'll check the remoteInboundRtpStats and post updates. Please give me some time....Ill get its results back asap. Meanwhile, please let me know if you have any other thoughts.

Thanks.

[marlocarlo]

Here's my result for remoteInboundRtpStats :

Between Browser to Pion :

Between Browser to Browser :

Please advise.

Thanks

[brucekim]

How was about stats of pli, nack, fir count?
As I commented on the thread in slack, I am suffering video stuttering. In my case I am using gstreamer instread of ffmpeg.

[marlocarlo]

@brucekim @Sean-Der Here's the stats between the browser and Pion :

@brucekim What command do you use for streaming using gstreamer? I can also give it a try and post my results.

[marlocarlo]

@Sean-Der Any thoughts based on my results?

[marlocarlo]

For comparison, here's the stats between two browsers using the same clients :

Browser 1

Browser 2

Please help me. Do you see any issues in the Browser to Pion stats above?

Thanks

[marlocarlo]

@Sean-Der I would appreciate any help on this issue. Any pointers or what I should do?

Thanks.

[brucekim]

I checked rtp-forwarder with gstreamer for receiving rtp packets and rendering it

• In my testing environemnt, sender (sending camera video at browser and rtp forwarder) and receiver (gstreamer) are on the same network. And then I have not seen any frame noise regardless of introducing jitterbuffer in gstreamer pipeline.
• Please see the gstreamer commands below. I hope this would be helpful for you.

withtout jitterbuffer

gst-launch-1.0 -v udpsrc port=4002 ! application/x-rtp, encoding-name=VP8,payload=96 ! rtpvp8depay ! decodebin ! autovideosink

with jitterbuffer

gst-launch-1.0 -v udpsrc port=4002 ! application/x-rtp, encoding-name=VP8,payload=96 ! rtpjitterbuffer ! rtpvp8depay ! decodebin ! autovideosink

My test sequence.

1. open chrome browser with url : https://jsfiddle.net/1qva2zd8/
2. copy sdp text at Browser base64 Session Description
3. open terminal. type following

export OFFER={sdp text copied at step 2}

4. change directory to pion/examples/rtp-forwarder in terminal
5. go build & run

echo $OFFER | ./rtp-forwarder

6. open another terminal
7. execute gstreamer commands above to receive rtp packet and then rendering it