Running out of 80GB RAM ONLY during second Animation

[rslosch]

I am using Runpod with an RTX 4090 with 16 vCPU 83 GB RAM.

I am using driving videos from 2-3min. The first animation always completes with 60-70% of Memory utilized.
When running the second animation directly after, it completes the animation but I get the following log when it's writing:
0:01:34pipe:: corrupt input packet in stream 0 [rawvideo @ 0x654c1c0] Invalid buffer size, packet size 65536 < expected frame_size 2359296 Error while decoding stream #0:0: Invalid argument

Runpod flashes to 100% memory utilization and then Liveportrait crashes. I am then able to run the first animation only when restarting the app.

Any suggestions for why this might be are appreciated.

[cleardusk]

In theory, the GPU memory should not overflow, as LivePortrait currently performs inference frame by frame. One situation that can occur is when the driving video is too long, causing the video to be converted into a CUDA tensor and exceeding the memory limit. This can be addressed by decoding the driver video frame by frame (needing modification of the inference : ) )