Transcoding stuck #10560

Goodwu · 2024-06-23T04:30:51Z

The bug

It seems like ffmpeg running into deadloop.
Ffmpeg running at 100% CPU usage for hours, and the output file size is only 48 bytes.
HW decoding and encoding switch are all ON.
Have accured two times, both have the same behavior.

The OS that Immich Server is running on

OMV7

Version of Immich Server

v1.106.4

Version of Immich Mobile App

v1.106.4

Platform with the issue

Server
Web
Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: quicksync # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - ${EXT_LOCATION}:/media/photos
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:d6c2911ac51b289db208767581a5d154544f2b2fe4914ea5056443f62dc6e900
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT SUM(checksum_failures) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres

EXT_LOCATION=/srv/dev-disk-by-uuid-2bf642cf-adb7-4ec8-9291-13bc77208fb0/store/WuStor_1/photo

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. HW decoding and encoding switch ON
2. More than 2000 video files
3. Transcoding stuck after 1 whole day
...

Relevant log output

No response

Additional information

No response

mertalev · 2024-06-23T04:46:35Z

Hmm, I agree this seems like a bug in FFmpeg. Have you tried restarting the container?

Goodwu · 2024-06-23T05:27:42Z

Hmm, I agree this seems like a bug in FFmpeg. Have you tried restarting the container?

No, I killed the ffmpeg with -9 and it retried with software transcoding. And it continue transcoding with HW accel normally.

dronnikovigor · 2024-06-25T03:47:14Z

Same bug here. Had to restart container. But after restart it again starts to transcode and stuck.
Had to terminate job.
I was using HW transcode

mertalev · 2024-06-26T00:28:28Z

Same bug here. Had to restart container. But after restart it again starts to transcode and stuck.

Had to terminate job.

I was using HW transcode

Which acceleration API were you using?

dronnikovigor · 2024-06-26T02:17:42Z

Which acceleration API were you using?

vaapi.
Disabled it for now.

Started encoding video 0885652b-e0c9-45ee-b837-32d0ea69c623 {"inputOptions":["-init_hw_device vaapi=accel:/dev/dri/card0","-filter_hw_device accel"],"outputOptions":["-c:v h264_vaapi","-c:a copy","-movflags faststart","-fps_mode passthrough","-map 0:1","-map 0:0","-g 256","-v verbose","-vf format=nv12,hwupload,scale_vaapi=1080:-2","-compression_level 7","-qp:v 23","-global_quality:v 23","-rc_mode 1"],"twoPass":false}

mertalev · 2024-06-26T04:53:52Z

Interesting, I wonder if it's specific to VAAPI then. I'll take a look at their issue tracker to see if this is a known issue.

mertalev · 2024-06-26T16:10:36Z

Can both of you clarify the following:

Was transcoding concurrency set above 1?
What kernel version does the server have?
What model is the processor (or GPU if it's a discrete GPU)?

The only relevant info I've seen online is this issue along with the kernel bug it links to. But since the issue is quite old, I'm not sure if this is really it.

mertalev · 2024-06-26T16:14:03Z

There's also this issue that seems relevant.

Goodwu · 2024-06-28T08:05:54Z

Can both of you clarify the following:

Was transcoding concurrency set above 1?

What kernel version does the server have?

What model is the processor (or GPU if it's a discrete GPU)?

The only relevant info I've seen online is this issue along with the kernel bug it links to. But since the issue is quite old, I'm not sure if this is really it.

The transcoding concurrency is 1
Linux b35022ea1bc4 6.8.4-3-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.4-3 (2024-05-02T11:55Z) x86_64 GNU/Linux
J3455

Goodwu · 2024-06-28T08:15:10Z

There's also this issue that seems relevant.

I have done more tests and it seems that it's all related to HDR videos.
I also open this issue under media-driver project.
When transcoding HDR video, it failed about 5 times and stuck 1 time in average.

dronnikovigor · 2024-07-04T14:49:23Z

Can both of you clarify the following:

Was transcoding concurrency set above 1?
What kernel version does the server have?
What model is the processor (or GPU if it's a discrete GPU)?

transcoding concurrency is 1
Linux 6.1.0-22-amd64 (omv 7)
AMD Ryzen 5 5600G

mertalev · 2024-07-04T21:16:06Z

It makes sense that it only fails for HDR videos since the command will only have the tonemap_opencl filter for HDR videos. Seems like that filter is where the issue lies, but I'm not sure what the root cause is.

Does disabling hardware decoding (and keeping hardware encoding enabled) avoid the issue?

mertalev · 2024-07-04T21:21:16Z

Also worth noting that we use bleeding edge versions for the relevant dependencies following this PR. It may be interesting to try different versions of these to see if it affects the behavior.

dronnikovigor · 2024-07-04T21:26:39Z

Does disabling hardware decoding (and keeping hardware encoding enabled) avoid the issue?

Can you advice how to set such config?

mertalev · 2024-07-04T22:03:39Z

In the transcoding settings, there is a setting for the hardware acceleration API, and a setting for hardware decoding below it. If hardware decoding is disabled but you set it to use an acceleration API, it will accelerate encoding only and handle decoding and tone-mapping on CPU.

dronnikovigor · 2024-07-04T22:48:44Z

It's said Applies only to NVENC and RKMPP. But i have VAAPI on my AMD

mertalev · 2024-07-05T02:36:57Z

Hmm, that's a good point: the common thread here isn't OpenCL.

We know that:

QSV also uses VAAPI
The VAAPI command uses software decoding and tone-mapping without OpenCL
@Goodwu tested QSV with hardware decoding and tone-mapping with OpenCL
Software transcoding doesn't cause FFmpeg to hang

That leaves QSV/VAAPI encoding as the most likely culprit. But that depends on whether the issue persists when hardware decoding for QSV is disabled. If it doesn't, it would poke a hole in this hypothesis.

mertalev · 2024-07-05T02:39:08Z

It's also possible that these are just two different issues that happen to both cause FFmpeg to hang.

TheNickOfTime · 2024-07-08T22:32:38Z

Coming here from the issue I opened (#10738) since this one originally seemed like a qsv issue when I was searching. A few points I can add to the convo here:

I have been using vaapi acceleration within immich for a while now - it only stopped working somewhat recently (unsure exactly when but within the past few months).
I think my issues are also related to HDR videos since my phone records HDR by default.
I also leverage vaapi transcoding for HDR content on the same device via Jellyfin (with no issues, just tested) using a similar config (passing /dev/dri to the container via docker compose).

dronnikovigor · 2024-09-21T23:53:36Z

Any news regarding this issue?

mertalev · 2024-09-22T03:23:04Z

If it's a bug in FFmpeg, it might be fixed once we update to 7.0. There's a draft PR for this upgrade that needs to go through testing.

zaggash · 2024-09-27T13:36:12Z

I have the same issue with small videos from animated photos.
ffmpeg started to transcode the video then it was running for days with no errors.

I disabled the Acceleration API to "disabled" and it used a lot of CPU for a while and it went through all the stuck videos.

VAAPI with AMD Ryzen 5600G
6.1.106-Unraid #1 SMP PREEMPT_DYNAMIC Wed Aug 21 23:36:07 PDT 2024 x86_64 AMD Ryzen 5 5600G with Radeon Graphics AuthenticAMD GNU/Linux
Always on latest Immich versions

mertalev · 2024-10-22T05:33:35Z

Is this still an issue as of 1.118.2?

vktr2b · 2024-10-22T07:51:52Z

It seems to have been solved, for me transcoding ran without any issues after updating to 1.118.2

mertalev · 2024-10-22T17:27:05Z

Nice! I’m marking this as closed, but feel free to chime in if there are still cases where this happens on 1.118.2 or later.

scherddel · 2024-10-22T18:15:44Z

@mertalev
I still experience a similar issue as described before. Using VAAPI and thousands of videos will be processed but for few it starts transcoding and gets stuck forever until I restart the container. (Using the newest release 1.118.2)

No Error message, just:

immich_server | [Nest] 6 - 10/22/2024, 6:09:12 PM LOG [Microservices:MediaService] Encoding video 1234dab1-5244-4745-a57c-765437c55f3f with VAAPI acceleration

And then it gets stuck and doesnt continue to process.

A restart doesnt resolve the issue it gets stuck on the same video which is recorded by the same mobile phone which has created videos that were transcoded without issues.

Is there any way to increase logginh on the ffmpeg level?

mertalev · 2024-10-22T18:44:45Z

Hmm, you can set the log level to debug in the admin settings and restart immich.

zaggash · 2024-10-22T19:19:50Z

Yup.
I changed the settings back to using VAAPI next to your message saying it is supposed to be fixed.
I launched the transoding at 4:53:21 PM
It processed about a hundred videos but now it is stuck for 4 hours on the same one.

scherddel · 2024-10-22T19:26:36Z

Ok i did some investigation:
Due to the debug level set to "debug" i was able to get the commands that immich-server is invoking to transcode the file. So I did a comparison between a file that worked and one that got stuck. (Basically run both commands manually in the immich-server container)

Please notice that I had to slightly adjust the command in order to make it work (remove -n 10 /usr/bin/ffmpeg) I am not sure why it is there in the first place but thats maybe another discussion.

This is the output of a file where everything worked:

root@b8a3d747695c:/usr/src/app# ffmpeg -init_hw_device vaapi=accel:/dev/dri/renderD128 -filter_hw_device accel -i upload/library/admin/2023/2023-07-23/IMG_3790.MOV -y -c:v h264_vaapi -c:a aac -mo
vflags faststart -fps_mode passthrough -map 0:0 -strict unofficial -map 0:1 -g 256 -v verbose -vf format=nv12,hwupload,scale_vaapi=720:-2 -compression_level 7 -qp:v 23 -global_quality:v 23 -rc_mo
de 1 upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/1e/ed/1eeda9a8-d580-4e38-992b-9b3cb02747fb.mp4
ffmpeg version 7.0.2-Jellyfin Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Routing option strict to both codec and muxer layer
[AVHWDeviceContext @ 0x567c306f8cc0] libva: VA-API version 1.22.0
[AVHWDeviceContext @ 0x567c306f8cc0] libva: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so
[AVHWDeviceContext @ 0x567c306f8cc0] libva: Found init function __vaDriverInit_1_22
amdgpu: os_same_file_description couldn't determine if two DRM fds reference the same file description.
If they do, bad things may happen!
[AVHWDeviceContext @ 0x567c306f8cc0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x567c306f8cc0] Initialised VAAPI connection: version 1.22
[AVHWDeviceContext @ 0x567c306f8cc0] VAAPI driver: Mesa Gallium driver 24.0.9 for AMD Radeon Graphics (radeonsi, raphael_mendocino, LLVM 16.0.6, DRM 3.57, 6.8.12-2-pve).
[AVHWDeviceContext @ 0x567c306f8cc0] Driver not found in known nonstandard list, using standard behaviour.
[aist#0:1/pcm_s16le @ 0x567c30940680] Guessed Channel Layout: mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'upload/library/admin/2023/2023-07-23/IMG_3790.MOV':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2023-07-23T14:22:11.000000Z
    com.apple.quicktime.location.accuracy.horizontal: 4.603016
    com.apple.quicktime.live-photo.auto: 1
    com.apple.quicktime.content.identifier: FF2F052D-A032-473D-9112-6E9F84AE340F
    com.apple.quicktime.live-photo.vitality-score: 1.000000
    com.apple.quicktime.live-photo.vitality-scoring-version: 0
    com.apple.quicktime.location.ISO6709: +49.6629+008.9982+238.676/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone SE (3rd generation)
    com.apple.quicktime.software: 16.5.1
    com.apple.quicktime.creationdate: 2023-07-23T16:22:08+0200
  Duration: 00:00:03.41, start: 0.000000, bitrate: 11303 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), yuvj420p(pc, smpte170m/smpte432/bt709, left), 1920x1440, 10538 kb/s, 28.74 fps, 29.58 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : HEVC
      Side data:
        displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](und): Audio: pcm_s16le (lpcm / 0x6D63706C), 44100 Hz, mono, s16, 705 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 40 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Metadata
  Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 504 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Metadata
  Stream #0:4[0x5](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Metadata
[out#0/mp4 @ 0x567c30900640] Adding streams from explicit maps...
[vost#0:0/h264_vaapi @ 0x567c309414c0] Created video stream from input stream 0:0
[aost#0:1/aac @ 0x567c30eb5040] Created audio stream from input stream 0:1
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_vaapi))
  Stream #0:1 -> #0:1 (pcm_s16le (native) -> aac (native))
[vost#0:0/h264_vaapi @ 0x567c309414c0] Starting thread...
[aost#0:1/aac @ 0x567c30eb5040] Starting thread...
[vf#0:0 @ 0x567c30944e00] Starting thread...
[af#0:1 @ 0x567c307d3800] Starting thread...
[vist#0:0/hevc @ 0x567c3078cc80] [dec:hevc @ 0x567c30942180] Starting thread...
[aist#0:1/pcm_s16le @ 0x567c30940680] [dec:pcm_s16le @ 0x567c30f41c80] Starting thread...
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x567c30900800] Starting thread...
Press [q] to stop, [?] for help
[graph_1_in_0:1 @ 0x70cde8002000] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:mono
[format_out_0_1 @ 0x70cde8002840] auto-inserting filter 'auto_aresample_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_1'
[auto_aresample_0 @ 0x70cde80035c0] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:fltp r:44100Hz
[graph 0 input from stream 0:0 @ 0x70cdf0002c80] w:1920 h:1440 pixfmt:yuvj420p tb:1/600 fr:355/12 sar:0/1 csp:smpte170m range:pc
[auto_scale_0 @ 0x70cdf0005740] w:iw h:ih flags:'' interl:0
[transpose @ 0x70cdf00032c0] auto-inserting filter 'auto_scale_0' between the filter 'graph 0 input from stream 0:0' and the filter 'transpose'
[swscaler @ 0x70cdf0013700] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x70cdf0005740] w:1920 h:1440 fmt:yuvj420p csp:unknown range:pc sar:0/1 -> w:1920 h:1440 fmt:nv12 csp:unknown range:unknown sar:0/1 flags:0x00000004
[transpose @ 0x70cdf00032c0] w:1920 h:1440 dir:1 -> w:1440 h:1920 rotation:clockwise vflip:0
[graph 0 input from stream 0:0 @ 0x70cdf0002c80] video frame properties congruent with link at pts_time: 0
[swscaler @ 0x70cdf0013680] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x70cdf0005740] w:1920 h:1440 fmt:yuvj420p csp:unknown range:pc sar:0/1 -> w:1920 h:1440 fmt:nv12 csp:unknown range:unknown sar:0/1 flags:0x00000004
[swscaler @ 0x70cdf0013680] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x70cdf0005740] w:1920 h:1440 fmt:yuvj420p csp:unknown range:pc sar:0/1 -> w:1920 h:1440 fmt:nv12 csp:unknown range:unknown sar:0/1 flags:0x00000004
[swscaler @ 0x70cdf0013680] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x70cdf0005740] w:1920 h:1440 fmt:yuvj420p csp:smpte170m range:pc sar:0/1 -> w:1920 h:1440 fmt:nv12 csp:unknown range:unknown sar:0/1 flags:0x00000004
[h264_vaapi @ 0x567c30946a00] Using input frames context (format vaapi) with h264_vaapi encoder.
[h264_vaapi @ 0x567c30946a00] Input surface format is nv12.
[h264_vaapi @ 0x567c30946a00] Using VAAPI profile VAProfileH264High (7).
[h264_vaapi @ 0x567c30946a00] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[h264_vaapi @ 0x567c30946a00] Using VAAPI render target format YUV420 (0x1).
[h264_vaapi @ 0x567c30946a00] RC mode: CQP.
[h264_vaapi @ 0x567c30946a00] Block Level bitrate control: OFF.
[h264_vaapi @ 0x567c30946a00] RC quality: 23.
[h264_vaapi @ 0x567c30946a00] RC framerate: 355/12 (29.58 fps).
[h264_vaapi @ 0x567c30946a00] Driver does not report any additional prediction constraints.
[h264_vaapi @ 0x567c30946a00] Using intra, P- and B-frames (supported references: 1 / 1).
[h264_vaapi @ 0x567c30946a00] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[h264_vaapi @ 0x567c30946a00] Using level 3.1.
Output #0, mp4, to 'upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/1e/ed/1eeda9a8-d580-4e38-992b-9b3cb02747fb.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    com.apple.quicktime.creationdate: 2023-07-23T16:22:08+0200
    com.apple.quicktime.location.accuracy.horizontal: 4.603016
    com.apple.quicktime.live-photo.auto: 1
    com.apple.quicktime.content.identifier: FF2F052D-A032-473D-9112-6E9F84AE340F
    com.apple.quicktime.live-photo.vitality-score: 1.000000
    com.apple.quicktime.live-photo.vitality-scoring-version: 0
    com.apple.quicktime.location.ISO6709: +49.6629+008.9982+238.676/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone SE (3rd generation)
    com.apple.quicktime.software: 16.5.1
    encoder         : Lavf61.1.100
  Stream #0:0(und): Video: h264 (High), 1 reference frame (avc1 / 0x31637661), vaapi(bt709, progressive, left), 720x960, q=2-31, 29.58 fps, 11360 tbn (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.3.100 h264_vaapi
      Side data:
        displaymatrix: rotation of -0.00 degrees
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, delay 1024, 69 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.3.100 aac
[out#0/mp4 @ 0x567c30900640] Starting thread...
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x567c30900800] EOF while reading input
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x567c30900800] Terminating thread with return code 0 (success)
[aist#0:1/pcm_s16le @ 0x567c30940680] [dec:pcm_s16le @ 0x567c30f41c80] Decoder thread received EOF packet
[aist#0:1/pcm_s16le @ 0x567c30940680] [dec:pcm_s16le @ 0x567c30f41c80] Decoder returned EOF, finishing
[aist#0:1/pcm_s16le @ 0x567c30940680] [dec:pcm_s16le @ 0x567c30f41c80] Terminating thread with return code 0 (success)
[af#0:1 @ 0x567c307d3800] Filtergraph returned EOF, finishing
[af#0:1 @ 0x567c307d3800] All consumers returned EOF
[af#0:1 @ 0x567c307d3800] Terminating thread with return code 0 (success)
[vist#0:0/hevc @ 0x567c3078cc80] [dec:hevc @ 0x567c30942180] Decoder thread received EOF packet
[vist#0:0/hevc @ 0x567c3078cc80] [dec:hevc @ 0x567c30942180] Decoder returned EOF, finishing
[vist#0:0/hevc @ 0x567c3078cc80] [dec:hevc @ 0x567c30942180] Terminating thread with return code 0 (success)
[vf#0:0 @ 0x567c30944e00] Filtergraph returned EOF, finishing
[vf#0:0 @ 0x567c30944e00] All consumers returned EOF
[aost#0:1/aac @ 0x567c30eb5040] Encoder thread received EOF
[aost#0:1/aac @ 0x567c30eb5040] Terminating thread with return code 0 (success)
[vost#0:0/h264_vaapi @ 0x567c309414c0] Encoder thread received EOF
[vf#0:0 @ 0x567c30944e00] Terminating thread with return code 0 (success)
[vost#0:0/h264_vaapi @ 0x567c309414c0] Terminating thread with return code 0 (success)
[out#0/mp4 @ 0x567c30900640] All streams finished
[out#0/mp4 @ 0x567c30900640] Terminating thread with return code 0 (success)
[mp4 @ 0x567c30940c80] Starting second pass: moving the moov atom to the beginning of the file
[AVIOContext @ 0x70cdf4006680] Statistics: 6970004 bytes read, 0 seeks
[AVIOContext @ 0x567c30f831c0] Statistics: 13944779 bytes written, 4 seeks, 57 writeouts
[out#0/mp4 @ 0x567c30900640] Output file #0 (upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/1e/ed/1eeda9a8-d580-4e38-992b-9b3cb02747fb.mp4):
[out#0/mp4 @ 0x567c30900640]   Output stream #0:0 (video): 98 frames encoded; 98 packets muxed (6940232 bytes); 
[out#0/mp4 @ 0x567c30900640]   Output stream #0:1 (audio): 147 frames encoded (150381 samples); 148 packets muxed (29724 bytes); 
[out#0/mp4 @ 0x567c30900640]   Total: 246 packets (6969956 bytes) muxed
[out#0/mp4 @ 0x567c30900640] video:6778KiB audio:29KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.069541%
frame=   98 fps=0.0 q=-0.0 Lsize=    6811KiB time=00:00:03.38 bitrate=16507.0kbits/s speed= 6.5x    
[aac @ 0x567c30eb54c0] Qavg: 126.082
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x567c30900800] Input file #0 (upload/library/admin/2023/2023-07-23/IMG_3790.MOV):
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x567c30900800]   Input stream #0:0 (video): 98 packets read (4492160 bytes); 98 frames decoded; 0 decode errors; 
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x567c30900800]   Input stream #0:1 (audio): 150 packets read (300762 bytes); 150 frames decoded; 0 decode errors (150381 samples); 
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x567c30900800]   Total: 248 packets (4792922 bytes) demuxed
[AVIOContext @ 0x567c3093d500] Statistics: 4853475 bytes read, 2 seeks

Thats the output of a file where it gets stuck and it does not go past the last line:

root@b8a3d747695c:/usr/src/app# ffmpeg -init_hw_device vaapi=accel:/dev/dri/renderD128 -filter_hw_device accel -i upload/library/admin/2023/2023-07-05/IMG_3630.MOV -y -c:v h264_vaapi -c:a aac -mo
vflags faststart -fps_mode passthrough -map 0:0 -strict unofficial -map 0:1 -g 256 -v verbose -vf format=nv12,hwupload,scale_vaapi=720:-2 -compression_level 7 -qp:v 23 -global_quality:v 23 -rc_mo
de 1 upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/12/6d/126d8e26-6db8-40fa-b1d5-7949948eec1c.mp4
ffmpeg version 7.0.2-Jellyfin Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Routing option strict to both codec and muxer layer
[AVHWDeviceContext @ 0x62e50db06cc0] libva: VA-API version 1.22.0
[AVHWDeviceContext @ 0x62e50db06cc0] libva: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so
[AVHWDeviceContext @ 0x62e50db06cc0] libva: Found init function __vaDriverInit_1_22
amdgpu: os_same_file_description couldn't determine if two DRM fds reference the same file description.
If they do, bad things may happen!
[AVHWDeviceContext @ 0x62e50db06cc0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x62e50db06cc0] Initialised VAAPI connection: version 1.22
[AVHWDeviceContext @ 0x62e50db06cc0] VAAPI driver: Mesa Gallium driver 24.0.9 for AMD Radeon Graphics (radeonsi, raphael_mendocino, LLVM 16.0.6, DRM 3.57, 6.8.12-2-pve).
[AVHWDeviceContext @ 0x62e50db06cc0] Driver not found in known nonstandard list, using standard behaviour.
[aist#0:1/pcm_s16le @ 0x62e50dcfd100] Guessed Channel Layout: mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'upload/library/admin/2023/2023-07-05/IMG_3630.MOV':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2023-07-05T16:58:07.000000Z
    com.apple.quicktime.location.accuracy.horizontal: 10.144191
    com.apple.quicktime.live-photo.auto: 1
    com.apple.quicktime.content.identifier: BEC6D08D-523E-4F5D-9089-2BDA9F6B2802
    com.apple.quicktime.live-photo.vitality-score: 1.000000
    com.apple.quicktime.live-photo.vitality-scoring-version: 0
    com.apple.quicktime.location.ISO6709: +35.3243+025.3871+006.141/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone SE (3rd generation)
    com.apple.quicktime.software: 16.5.1
    com.apple.quicktime.creationdate: 2023-07-05T19:58:04+0300
  Duration: 00:00:02.60, start: 0.000000, bitrate: 11557 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), yuvj420p(pc, smpte170m/smpte432/bt709, left), 1920x1440, 10647 kb/s, 28.46 fps, 600 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : HEVC
      Side data:
        displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](und): Audio: pcm_s16le (lpcm / 0x6D63706C), 44100 Hz, mono, s16, 705 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 38 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Metadata
  Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 504 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Metadata
  Stream #0:4[0x5](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Metadata
[out#0/mp4 @ 0x62e50dd56040] Adding streams from explicit maps...
[vost#0:0/h264_vaapi @ 0x62e50dd569c0] Created video stream from input stream 0:0
[aost#0:1/aac @ 0x62e50e3fa780] Created audio stream from input stream 0:1
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_vaapi))
  Stream #0:1 -> #0:1 (pcm_s16le (native) -> aac (native))
[vost#0:0/h264_vaapi @ 0x62e50dd569c0] Starting thread...
[aost#0:1/aac @ 0x62e50e3fa780] Starting thread...
[vf#0:0 @ 0x62e50dd54d40] Starting thread...
[af#0:1 @ 0x62e50dd07140] Starting thread...
[vist#0:0/hevc @ 0x62e50db9ac80] [dec:hevc @ 0x62e50dd07a00] Starting thread...
[aist#0:1/pcm_s16le @ 0x62e50dcfd100] [dec:pcm_s16le @ 0x62e50dd00180] Starting thread...
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x62e50dd09a00] Starting thread...
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0x710ce4002c80] w:1920 h:1440 pixfmt:yuvj420p tb:1/600 fr:15000/527 sar:0/1 csp:smpte170m range:pc
[auto_scale_0 @ 0x710ce4005740] w:iw h:ih flags:'' interl:0
[transpose @ 0x710ce40032c0] auto-inserting filter 'auto_scale_0' between the filter 'graph 0 input from stream 0:0' and the filter 'transpose'
[swscaler @ 0x710ce4013700] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x710ce4005740] w:1920 h:1440 fmt:yuvj420p csp:unknown range:pc sar:0/1 -> w:1920 h:1440 fmt:nv12 csp:unknown range:unknown sar:0/1 flags:0x00000004
[transpose @ 0x710ce40032c0] w:1920 h:1440 dir:1 -> w:1440 h:1920 rotation:clockwise vflip:0
[graph_1_in_0:1 @ 0x710ce0002000] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:mono
[format_out_0_1 @ 0x710ce0002840] auto-inserting filter 'auto_aresample_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_1'
[auto_aresample_0 @ 0x710ce00035c0] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:fltp r:44100Hz
[graph 0 input from stream 0:0 @ 0x710ce4002c80] video frame properties congruent with link at pts_time: 0
[swscaler @ 0x710ce4013680] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x710ce4005740] w:1920 h:1440 fmt:yuvj420p csp:unknown range:pc sar:0/1 -> w:1920 h:1440 fmt:nv12 csp:unknown range:unknown sar:0/1 flags:0x00000004
[swscaler @ 0x710ce4013680] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x710ce4005740] w:1920 h:1440 fmt:yuvj420p csp:unknown range:pc sar:0/1 -> w:1920 h:1440 fmt:nv12 csp:unknown range:unknown sar:0/1 flags:0x00000004
[swscaler @ 0x710ce4013680] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x710ce4005740] w:1920 h:1440 fmt:yuvj420p csp:smpte170m range:pc sar:0/1 -> w:1920 h:1440 fmt:nv12 csp:unknown range:unknown sar:0/1 flags:0x00000004
[h264_vaapi @ 0x62e50dceed80] Using input frames context (format vaapi) with h264_vaapi encoder.
[h264_vaapi @ 0x62e50dceed80] Input surface format is nv12.
[h264_vaapi @ 0x62e50dceed80] Using VAAPI profile VAProfileH264High (7).
[h264_vaapi @ 0x62e50dceed80] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[h264_vaapi @ 0x62e50dceed80] Using VAAPI render target format YUV420 (0x1).
[h264_vaapi @ 0x62e50dceed80] RC mode: CQP.
[h264_vaapi @ 0x62e50dceed80] Block Level bitrate control: OFF.
[h264_vaapi @ 0x62e50dceed80] RC quality: 23.
[h264_vaapi @ 0x62e50dceed80] RC framerate: 15000/527 (28.46 fps).
[h264_vaapi @ 0x62e50dceed80] Driver does not report any additional prediction constraints.
[h264_vaapi @ 0x62e50dceed80] Using intra, P- and B-frames (supported references: 1 / 1).
[h264_vaapi @ 0x62e50dceed80] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[h264_vaapi @ 0x62e50dceed80] Using level 3.1.
Output #0, mp4, to 'upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/12/6d/126d8e26-6db8-40fa-b1d5-7949948eec1c.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    com.apple.quicktime.creationdate: 2023-07-05T19:58:04+0300
    com.apple.quicktime.location.accuracy.horizontal: 10.144191
    com.apple.quicktime.live-photo.auto: 1
    com.apple.quicktime.content.identifier: BEC6D08D-523E-4F5D-9089-2BDA9F6B2802
    com.apple.quicktime.live-photo.vitality-score: 1.000000
    com.apple.quicktime.live-photo.vitality-scoring-version: 0
    com.apple.quicktime.location.ISO6709: +35.3243+025.3871+006.141/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone SE (3rd generation)
    com.apple.quicktime.software: 16.5.1
    encoder         : Lavf61.1.100
  Stream #0:0(und): Video: h264 (High), 1 reference frame (avc1 / 0x31637661), vaapi(bt709, progressive, left), 720x960, q=2-31, 28.46 fps, 15k tbn (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.3.100 h264_vaapi
      Side data:
        displaymatrix: rotation of -0.00 degrees
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, delay 1024, 69 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.3.100 aac
[out#0/mp4 @ 0x62e50dd56040] Starting thread...
frame=    0 fps=0.0 q=0.0 size=       0KiB time=N/A bitrate=N/A speed=N/A

Anything interesting that can be derived from that analysis? (Other than its most likely an issue with ffmpeg)

scherddel · 2024-10-22T20:14:43Z

@mertalev
Ok I think I have found the issue:

I ffprobe'd the two files to see if I see any major differences

File which got stuck:

root@b8a3d747695c:/usr/src/app# ffprobe upload/library/admin/2023/2023-07-05/IMG_3630.MOV
ffprobe version 7.0.2-Jellyfin Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'upload/library/admin/2023/2023-07-05/IMG_3630.MOV':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2023-07-05T16:58:07.000000Z
    com.apple.quicktime.location.accuracy.horizontal: 10.144191
    com.apple.quicktime.live-photo.auto: 1
    com.apple.quicktime.content.identifier: BEC6D08D-523E-4F5D-9089-2BDA9F6B2802
    com.apple.quicktime.live-photo.vitality-score: 1.000000
    com.apple.quicktime.live-photo.vitality-scoring-version: 0
    com.apple.quicktime.location.ISO6709: +35.3243+025.3871+006.141/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone SE (3rd generation)
    com.apple.quicktime.software: 16.5.1
    com.apple.quicktime.creationdate: 2023-07-05T19:58:04+0300
  Duration: 00:00:02.60, start: 0.000000, bitrate: 11557 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, smpte170m/smpte432/bt709), 1920x1440, 10647 kb/s, 28.46 fps, 600 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : HEVC
      Side data:
        displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](und): Audio: pcm_s16le (lpcm / 0x6D63706C), 44100 Hz, 1 channels, s16, 705 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 38 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Metadata
  Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 504 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Metadata
  Stream #0:4[0x5](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2023-07-05T16:58:07.000000Z
        handler_name    : Core Media Metadata
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 0 for input stream 3
Unsupported codec with id 0 for input stream 4

File that worked:

root@b8a3d747695c:/usr/src/app# ffprobe upload/library/admin/2023/2023-07-23/IMG_3790.MOV
ffprobe version 7.0.2-Jellyfin Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'upload/library/admin/2023/2023-07-23/IMG_3790.MOV':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2023-07-23T14:22:11.000000Z
    com.apple.quicktime.location.accuracy.horizontal: 4.603016
    com.apple.quicktime.live-photo.auto: 1
    com.apple.quicktime.content.identifier: FF2F052D-A032-473D-9112-6E9F84AE340F
    com.apple.quicktime.live-photo.vitality-score: 1.000000
    com.apple.quicktime.live-photo.vitality-scoring-version: 0
    com.apple.quicktime.location.ISO6709: +49.6629+008.9982+238.676/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone SE (3rd generation)
    com.apple.quicktime.software: 16.5.1
    com.apple.quicktime.creationdate: 2023-07-23T16:22:08+0200
  Duration: 00:00:03.41, start: 0.000000, bitrate: 11303 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, smpte170m/smpte432/bt709), 1920x1440, 10538 kb/s, 28.74 fps, 29.58 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : HEVC
      Side data:
        displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](und): Audio: pcm_s16le (lpcm / 0x6D63706C), 44100 Hz, 1 channels, s16, 705 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 40 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Metadata
  Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 504 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Metadata
  Stream #0:4[0x5](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2023-07-23T14:22:11.000000Z
        handler_name    : Core Media Metadata
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 0 for input stream 3
Unsupported codec with id 0 for input stream 4
root@b8a3d747695c:/usr/src/app#

The trained eye sees a difference in the tbr (time base rate, the combination of tbr and tbn affects how ffmpeg calculates timestamps)

FILE WHICH FAILED: Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, smpte170m/smpte432/bt709), 1920x1440, 10647 kb/s, 28.46 fps, 600 tbr, 600 tbn (default)
FILE WHICH WORKED: Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, smpte170m/smpte432/bt709), 1920x1440, 10538 kb/s, 28.74 fps, 29.58 tbr, 600 tbn (default)

Which led me to the stuipid idea to set the framerate in the ffmpeg to the standard frame rate of 29.97 as part of the ffmpeg command (I had to remove -fps_mode passthrough to resolve the conflict)

ffmpeg -init_hw_device vaapi=accel:/dev/dri/renderD128 -filter_hw_device accel -i upload/library/admin/2023/2023-07-05/IMG_3630.MOV -y -r 29.97 -c:v h264_vaapi -c:a aac -movflags faststart -map 0:0 -strict unofficial -map 0:1 -g 256 -v verbose -vf format=nv12,hwupload,scale_vaapi=720:-2 -compression_level 7 -qp:v 23 -global_quality:v 23 -rc_mode 1 upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/12/6d/126d8e26-6db8-40fa-b1d5-7949948eec1c.mp4

And the file got transcoded without any errors and with VAAPI enabled. IS THIS THE SOLUTION??? What does speak against setting -r 29.97 as a default setting and not preserve the initial frame rate?

mertalev · 2024-10-22T20:22:11Z

Huh, interesting! Could you try -fps_mode vfr instead of -fps_mode passthrough and see if it still gets stuck?

scherddel · 2024-10-22T20:24:14Z

Huh, interesting! Could you try -fps_mode vfr instead of -fps_mode passthrough and see if it still gets stuck?

-fps_mode vfr gets stuck as well. Same as before.

Also tried auto and cfr, both fail as well.

mertalev · 2024-10-22T20:42:22Z

Okay, some other things to try:
-fflags +genpts
-fflags +genpts+igndts
-copyts -avoid_negative_ts disabled -start_at_zero
-copyts -avoid_negative_ts make_zero

scherddel · 2024-10-22T20:54:41Z

Okay, some other things to try: -fflags +genpts -fflags +genpts+igndts -copyts -avoid_negative_ts disabled -start_at_zero -copyts -avoid_negative_ts make_zero

I tried all 4 options and all got stuck the same way. No luck.

Here the commands I tried

ffmpeg -init_hw_device vaapi=accel:/dev/dri/renderD128 \
-filter_hw_device accel \
-fflags +genpts \
-i upload/library/admin/2023/2023-07-05/IMG_3630.MOV -y \
-c:v h264_vaapi -c:a aac -movflags faststart -fps_mode passthrough \
-map 0:0 -strict unofficial -map 0:1 -g 256 -v verbose \
-vf format=nv12,hwupload,scale_vaapi=720:-2 -compression_level 7 \
-qp:v 23 -global_quality:v 23 -rc_mode 1 \
upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/12/6d/126d8e26-6db8-40fa-b1d5-7949948eec1c.mp4


ffmpeg -init_hw_device vaapi=accel:/dev/dri/renderD128 \
-filter_hw_device accel \
-fflags +genpts+igndts \
-i upload/library/admin/2023/2023-07-05/IMG_3630.MOV -y \
-c:v h264_vaapi -c:a aac -movflags faststart -fps_mode passthrough \
-map 0:0 -strict unofficial -map 0:1 -g 256 -v verbose \
-vf format=nv12,hwupload,scale_vaapi=720:-2 -compression_level 7 \
-qp:v 23 -global_quality:v 23 -rc_mode 1 \
upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/12/6d/126d8e26-6db8-40fa-b1d5-7949948eec1c.mp4


ffmpeg -init_hw_device vaapi=accel:/dev/dri/renderD128 \
-filter_hw_device accel \
-avoid_negative_ts disabled \
-i upload/library/admin/2023/2023-07-05/IMG_3630.MOV \
-copyts -start_at_zero -y \
-c:v h264_vaapi -c:a aac -movflags faststart -fps_mode passthrough \
-map 0:0 -strict unofficial -map 0:1 -g 256 -v verbose \
-vf format=nv12,hwupload,scale_vaapi=720:-2 -compression_level 7 \
-qp:v 23 -global_quality:v 23 -rc_mode 1 \
upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/12/6d/126d8e26-6db8-40fa-b1d5-7949948eec1c.mp4


ffmpeg -init_hw_device vaapi=accel:/dev/dri/renderD128 \
-filter_hw_device accel \
-avoid_negative_ts make_zero \
-i upload/library/admin/2023/2023-07-05/IMG_3630.MOV \
-copyts -y \
-c:v h264_vaapi -c:a aac -movflags faststart -fps_mode passthrough \
-map 0:0 -strict unofficial -map 0:1 -g 256 -v verbose \
-vf format=nv12,hwupload,scale_vaapi=720:-2 -compression_level 7 \
-qp:v 23 -global_quality:v 23 -rc_mode 1 \
upload/encoded-video/0d600514-c98d-427e-bbc0-1affa65573b9/12/6d/126d8e26-6db8-40fa-b1d5-7949948eec1c.mp4

mertalev · 2024-10-22T21:07:21Z

Is there anything useful in dmesg or journalctl?

scherddel · 2024-10-22T21:10:22Z

Nope. No error. No message. Nothing. Would have been too easy i guess :-D

mertalev · 2024-10-22T21:16:15Z

Haha I guess so. What if you change hwupload to hwupload=extra_hw_frames=64?

mertalev · 2024-10-22T21:22:55Z

Also, does it just immediately get stuck, or does it make any progress first?

scherddel · 2024-10-22T21:28:05Z

Haha I guess so. What if you change hwupload to hwupload=extra_hw_frames=64?

Did not work as well, sorry :-(

Also, does it just immediately get stuck, or does it make any progress first?

It shows immediately frame= 0 fps=0.0 q=0.0 size= 0KiB time=N/A bitrate=N/A speed=N/A without any change or progress (considering the longer output before that line as well)

mertalev · 2024-10-22T22:37:13Z

Well this is weird. What do you get if you set the FFmpeg verbosity to -v debug? Also, have you tested any other frame rate besides 29.97? I'd be interested to see if it works for some frame rates but not others, and if there's maybe a threshold above/below which it gets stuck again.

scherddel · 2024-10-23T07:16:11Z

-v debug does not provide any more information...

I tried setting -r to a couple of different values (1, 5, 10, 30, 50, 100, 500) and it worked each time without an issue.

mertalev · 2024-10-23T14:43:22Z

What happens if you set it to the original fps (I believe 28.46)?

scherddel · 2024-10-23T14:46:42Z

Works smoothly....

mertalev · 2024-10-24T01:44:18Z

That has to mean the FPS is variable, right? Or setting the FPS wouldn't have any effect. Can you output the individual frames with FFprobe to see if there's anything weird going on? Probably just around the start since it affects the transcoding immediately.

dronnikovigor mentioned this issue Jul 4, 2024

VAAPI hardware transcoding gets stuck indefinitely (with no errors) #10738

Closed

3 tasks

mertalev closed this as completed Oct 22, 2024

mertalev reopened this Oct 23, 2024

Transcoding stuck #10560

Transcoding stuck #10560

Comments

Goodwu commented Jun 23, 2024

The bug

The OS that Immich Server is running on

Version of Immich Server

Version of Immich Mobile App

Platform with the issue

Your docker-compose.yml content

Your .env content

Reproduction steps

Relevant log output

Additional information

mertalev commented Jun 23, 2024 • edited Loading

Goodwu commented Jun 23, 2024

dronnikovigor commented Jun 25, 2024

mertalev commented Jun 26, 2024

dronnikovigor commented Jun 26, 2024 • edited Loading

mertalev commented Jun 26, 2024

mertalev commented Jun 26, 2024

mertalev commented Jun 26, 2024 • edited Loading

Goodwu commented Jun 28, 2024

Goodwu commented Jun 28, 2024

dronnikovigor commented Jul 4, 2024

mertalev commented Jul 4, 2024

mertalev commented Jul 4, 2024

dronnikovigor commented Jul 4, 2024

mertalev commented Jul 4, 2024

dronnikovigor commented Jul 4, 2024

mertalev commented Jul 5, 2024

mertalev commented Jul 5, 2024

TheNickOfTime commented Jul 8, 2024

dronnikovigor commented Sep 21, 2024

mertalev commented Sep 22, 2024

zaggash commented Sep 27, 2024 • edited Loading

mertalev commented Oct 22, 2024

vktr2b commented Oct 22, 2024 • edited Loading

mertalev commented Oct 22, 2024

scherddel commented Oct 22, 2024 • edited Loading

mertalev commented Oct 22, 2024

zaggash commented Oct 22, 2024

scherddel commented Oct 22, 2024

scherddel commented Oct 22, 2024 • edited Loading

mertalev commented Oct 22, 2024

scherddel commented Oct 22, 2024 • edited Loading

mertalev commented Oct 22, 2024

scherddel commented Oct 22, 2024 • edited Loading

mertalev commented Oct 22, 2024

scherddel commented Oct 22, 2024

mertalev commented Oct 22, 2024

mertalev commented Oct 22, 2024

scherddel commented Oct 22, 2024

mertalev commented Oct 22, 2024 • edited Loading

scherddel commented Oct 23, 2024

mertalev commented Oct 23, 2024

scherddel commented Oct 23, 2024

mertalev commented Oct 24, 2024

mertalev commented Jun 23, 2024 •

edited

Loading

dronnikovigor commented Jun 26, 2024 •

edited

Loading

mertalev commented Jun 26, 2024 •

edited

Loading

zaggash commented Sep 27, 2024 •

edited

Loading

vktr2b commented Oct 22, 2024 •

edited

Loading

scherddel commented Oct 22, 2024 •

edited

Loading

scherddel commented Oct 22, 2024 •

edited

Loading

scherddel commented Oct 22, 2024 •

edited

Loading

scherddel commented Oct 22, 2024 •

edited

Loading

mertalev commented Oct 22, 2024 •

edited

Loading