Troubles facing video archival

[vxbinaca]

Over the last 72 hours I've been triaging two or three issues that have either come up on Youtubes end, or with dependencies of yt-dlp.

• ffmpeg in Ubuntu is incredibly stale (ffmpeg version n4.4.4-6-gd5fa6e3a91), causing failure to mux on some formats. Upgrading to a ffmpeg beyond 5.1 - so not just 5.1 - seems to fix this. You'll have to compile it, upgrade your distro, or switch to a distro with fresher packages (gentoo, arch, possibly Debian Sid).

The symptom of this issue is a item that has no video in it, this is due to the mux failure leaving .part files that we have safeties in place to not shove junk into IA. Tubeup users should be monitoring the output of their rips anyway. Removing the safety will merely upload junk .part files to IA, where the problem won't be immediately noticed until it's too late.

• Youtube is beginning to roll out anti-adblock that will cause a 403 errors for getting metadata. Unless the heros at yt-dlp come up with a bypass, you're going to have to sign up for Youtube Premium and use that cookie to bypass the anti-adblock. If this route is chosen we'd need to implement Dont leak ip address #310 but actually test that it works instead of breaking everything.

I would recommend signing up for Premium on a burner account, using a debit card you don't mind not being able to use if you're banned. Do not use your main accounts cookie.

[mrpapersonic]

One thing that has annoyed me for ages about tubeup is that when dealing with playlists, instead of doing the obvious thing (that is, downloading and then uploading each video) it downloads each video and then uploads each video. It's such a waste of disk space that could easily be overcome by just doing the obvious, and FWIW I get around this issue myself by scripting yt-dlp and tubeup together...

One issue that is possible is when there is a private video within the playlist. Tubeup already errors out on these, but it would be diverging behavior from previous releases to error out after uploading part of a playlist. But... in any case private or unavailable videos in playlists ought to be considered warnings at best anyway and tubeup should just skip over them. It's better to warn out and preserve more videos than error out and risk losing the archival of some videos just because one video was private. Of course the main issue would be these getting lost in the console; a simple and effective solution is providing a summary at exit that users can evaluate to make sure everything went smoothly and printing the URLs of videos that couldn't be scraped.

[mrpapersonic]

Actually, with private videos it should (?) already be handled the way I described (as a warning...):

tubeup/tubeup/TubeUp.py

Lines 120 to 123 in 4ca730c

	def ydl_progress_each(entry):
	if not entry:
	self.logger.warning('Video "%s" is not available. Skipping.' % url)
	return

so maybe that can be disregarded. The part about providing a summary is still a good idea imo, and its what other downloaders like aria2c do as well.

[vxbinaca]

code something up and lets merge it

[mrpapersonic]

now if only I had the time... :)

[vxbinaca]

I understand business has picked up for me thankfully.