Exitcode: 153 #210
Comments
|
|
|
Error happened on encoding01
|
|
I have no idea what happened. I cleaned up a little bit the filesystem but I'm not sure it's because of a disk space issue. Nothing useful in celery logs nor syslog... |
|
Its a proper Ffmpeg error, we'd have to debug the command for this specific video file. |
|
Here's a recent error: |
|
Error happened on encoding05 |
Exactly where may we find documentation of that exitcode? https://ffmpeg.org/documentation.html is of no help to me in my search. |
|
Wow I see we're using a very old version of ffmpeg, first thing would be to upgrade it: |
|
I'm trying to update the Gentoo Prefix, it hasn't been done since 2021 :'( |
|
OK it's way too complicated to update with so many skipped versions, and I managed to completely screw the gentoo installation. I'm reinstalling the system, 🔥 🔥 🔥 expect all new jobs to fail in the coming hours 🔥 🔥 🔥 until I complete this task. |
|
I've stopped the frontend so that new tasks cannot be created until I repair the backend. |
|
Gentoo has been reinstalled and ffmpeg updated. I have restarted the service but not tested anything yet. Can some people please tell me if it works? |
|
New version is built as follows: |
|
@Yannf @Jeff-G-commons I have managed to upload several videos so it's working again. Can you please try again the videos that caused the Error 153, to see if it's solved with the ffmepg update? |
|
OK, started again. Your task is pending... |
|
OK, I restarted one (Dixiana 1930 (colorized, 4k, 60FPS) 2), two more are Fragment of an Empire (1929, 1080p, silent) by Fridrikh Ermler (94%) and Storm Over Asia (1928, 1080p, silent) (89%) by Vsevolod Pudovkin in pass 2 (percentages increased since you asked, so they are still running and I am hesitant to abort them). I have no percentages to relay when in pass 1, just a horizontal barber pole. Also, the info in the column where the errors show is difficult to parse at a glance, and the columns are not labeled. |
|
Fragment of an Empire (1929, 1080p, silent) by Fridrikh Ermler had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/b6f1fff950a5fae9/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm.log /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm\nExitcode: 153')" |
|
Similarly, Storm Over Asia (1928, 1080p, silent) by Vsevolod Pudovkin had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/ccb74212f21d3a91/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm.log /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm\nExitcode: 153')" |
Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/8ecad562d8da07f2/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/8ecad562d8da07f2/dl.mp4.an.vp9.webm.log /srv/v2c/output/8ecad562d8da07f2/dl.mp4.an.vp9.webm\nExitcode: 153')" |
Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/a61d2fa98b38123c/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/a61d2fa98b38123c/dl.mp4.an.vp9.webm.log /srv/v2c/output/a61d2fa98b38123c/dl.mp4.an.vp9.webm\nExitcode: 153')" |
|
UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/5f1d66720427aa6c/dl.mkv -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -speed 4 -f webm -ss 0 -an -pass 1 -passlogfile /srv/v2c/output/5f1d66720427aa6c/dl.mkv.vp9.webm.log /dev/null\nExitcode: 8')" |
@BahYaje this ticket is only about the ExitCode 153, can you please create a new ticket for your issue with the link to the video files you're trying to upload? thanks. |
|
Volshebnoye Zerno - The Magic Seed (1942) by Fyodor Filippov
|
Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4.an.vp9.webm.log /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4.an.vp9.webm\nExitcode: 153')" |
Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/a461584cb01f01b0/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/a461584cb01f01b0/dl.mp4.an.vp9.webm.log /srv/v2c/output/a461584cb01f01b0/dl.mp4.an.vp9.webm\nExitcode: 153')" |
|
Dixiana 1930 (colorized, 4k, 60FPS) 2 had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/49fd5ad65ee6abef/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/49fd5ad65ee6abef/dl.mp4.vp9.webm.log /srv/v2c/output/49fd5ad65ee6abef/dl.mp4.vp9.webm\nExitcode: 153')" |
Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/3c85cf9ab7375c9e/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/3c85cf9ab7375c9e/dl.mp4.an.vp9.webm.log /srv/v2c/output/3c85cf9ab7375c9e/dl.mp4.an.vp9.webm\nExitcode: 153')" |
Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/08c147bc201c1eed/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/08c147bc201c1eed/dl.mp4.an.vp9.webm.log /srv/v2c/output/08c147bc201c1eed/dl.mp4.an.vp9.webm\nExitcode: 153')" |
Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/782c3b089253acde/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/782c3b089253acde/dl.mp4.vp9.webm.log /srv/v2c/output/782c3b089253acde/dl.mp4.vp9.webm\nExitcode: 153')" |
|
@Jeff-G-commons I have enough reported errors, thank you. No need to report every one of them. |
You're welcome. |
|
I managed today to convert manually the first reported file (https://archive.org/details/volshebnoye-zerno-1942-by-fyodor-filippov) on encoding06 instance: It took exactly 12 hours. Consumed only around 700Mb of memory during the process. And produced a file of 5 GB:
(the CPU spike is just another job that someone launched. The disk space event was me freeing some old unused files). I don't think it's an ffmpeg error, as it succeeded with this example. Error 153 might be the process killed by a SIGINT. video2commons launches jobs with some ulimit parameters that might explain why a SIGINT is sent by the system: cmd = 'ulimit -f ' + escape_shellarg(background_size_limit) + ';' + \
'ulimit -v ' + escape_shellarg(background_memory_limit) + ';' + \
'nice -n ' + escape_shellarg(background_priority) + ' ' + \
'timeout ' + escape_shellarg(background_time_limit) + ' ' + \
cmd + \
' 2>&1'The values are defined in globals.py: # The priority to be used with the nice transcode commands.
background_priority = 19
# The total amout of time a transcoding shell command can take:
background_time_limit = 3600 * 24 * 2 # 2 days
# Maximum amount of virtual memory available to transcoding processes in KB
# 2GB avconv
background_memory_limit = 8 * 1024 * 1024 # 8GB
# Maximum file size transcoding processes can create, in KB
background_size_limit = 10 * 1024 * 1024 # 10GBBut in this case we are way below the limits... So I still don't understand what's going on. |
|
@Yannf I am happy to report that I was able to upload https://archive.org/details/volshebnoye-zerno-1942-by-fyodor-filippov to https://commons.wikimedia.org/wiki/File:Volshebnoye_Zerno_-_The_Magic_Seed_%281942%2C_silent%29.webm . Please attend to the licensing. |
|
I now have three instances of 'An exception occurred: ConnectionError: b"Error while reading from video-redis-bookworm.video.eqiad1.wikimedia.cloud:6379 : (104, 'Connection reset by peer')"' with the same files. Should that be a separate issue? |
|
Grafana shows that there was a dip in Total RAM to 31.3 GiB 2024-09-17 11:20. |
|
Please try again, the outage lasted only for 10 minutes, I think it's simply a maintenance operation from WMCS team. |
|
Over a month later, are you any closer to understanding what's going on? |
|
I have no idea what causes this problem. Any help from someone more experienced than me in python/ffmpeg stack would be appreciated. |
I got
An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/b344e90a1e97be9a/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/b344e90a1e97be9a/dl.mp4.vp9.webm.log /srv/v2c/output/b344e90a1e97be9a/dl.mp4.vp9.webm\\nExitcode: 153')"while trying to upload https://archive.org/details/volshebnoye-zerno-1942-by-fyodor-filippovThe text was updated successfully, but these errors were encountered: