CUDA VRAM requirements and tweaking

[alexandervlpl]

I'm unable to run this on my GTX 1650 with 4GB VRAM. The part with the progress bar (which I guess is pure whisper) runs fine even with large and turbo models. But the next post-processing stage crashes every time, even with the medium model:

torch.OutOfMemoryError: CUDA out of memory

Is this expected? If so, what are the minimal VRAM requirements for whisper_timestamped? Is there anything I can do to get this running? Maybe there's a PYTORCH_CUDA_ALLOC_CONF that will work, or I can use CUDA only for the whisper part and CPU for the rest? I have plenty of RAM on this system.