Skip to content

Latest commit

 

History

History
167 lines (143 loc) · 8.57 KB

README.md

File metadata and controls

167 lines (143 loc) · 8.57 KB
Raw

REAL Video Enhancer

Visitors downloads_total pypresence license Discord Shield

Table of Contents

Introduction

REAL Video Enhancer is a redesigned and enhanced version of the original Rife ESRGAN App for Linux. This program offers convenient access to frame interpolation and upscaling functionalities on Windows, Linux and MacOS, and is an alternative to outdated software like Flowframes or enhancr.

Features:

  • NEW! Windows support. !!! NOTICE !!! The bin can be detected as a trojan. This is a false positive caused by pyinstaller.
  • Support for Ubuntu 20.04+ on Executable and Flatpak. (libxcb-cursor0 Required to launch on x11!)
  • Discord RPC support for Discord system package and Discord flatpak.
  • Scene change detection to preserve sharp transitions.
  • Preview that shows latest frame that has been rendered.
  • TensorRT and NCNN for efficient inference across many GPUs.
  • DEPRICATED MacOS support as of 2.0, this will not be returning due to changes made by apple in MacOS 15.1. Read more here.

Hardware/Software Requirements

Minimum Recommended
CPU Dual Core x64 bit Quad Core x64 bit
GPU Vulkan 1.3 capable device Nvidia RTX GPU (20 series and up)
RAM 8 GB 16 GB
Storage 1 GB free (NCNN install only) 10 GB free (TensorRT install)
Operating System Windows 10/11 64bit Any modern Linux distro (Ubuntu 20.04+)

Benchmarks:

Benchmarks done with 1920x1080 video, default settings.

RIFE NCNN

RX 6650 XT
rife-v4.6 31 fps
rife-v4.7 - v4.9 28 fps
rife-v4.10 - v4.15 23 fps
rife-v4.16-lite 31 fps
RTX 3080
rife-v4.6 81 fps
rife-v4.7 - v4.9 65 fps
rife-v4.10 - v4.15 55 fps
rife-v4.22 50 fps
rife-v4.22-lite 63 fps

RIFE TensorRT 10.3

RTX 3080
rife-v4.6 270 fps
rife-v4.7 - v4.9 204 fps
rife-v4.10 - v4.15 166 fps
rife-v4.22 140 fps
rife-v4.22-lite 192 fps

Cloning:

git clone https://github.com/TNTwise/REAL-Video-Enhancer

Building:

python3 build.py --build_exe

Canary Build:

https://github.com/TNTwise/REAL-Video-Enhancer/releases/tag/prerelease

Credits:

People:

Person For Link
NevermindNilas Some backend and reference code and working with me on many projects https://github.com/NevermindNilas/
Styler00dollar RIFE models (4.1-4.5, 4.7-4.12-lite), Sudo Shuffle Span and benchmarking https://github.com/styler00dollar
HolyWu TensorRT engine generation code, inference optimizations, and RIFE jagged lines fixes https://github.com/HolyWu/
Rick Astley Amazing music https://www.youtube.com/watch?v=dQw4w9WgXcQ

Software:

Software Used For Link
FFmpeg Multimedia framework for handling video, audio, and other media files https://ffmpeg.org/
PyTorch Neural Network Inference (CUDA/ROCm/TensorRT) https://pytorch.org/
NCNN Neural Network Inference (Vulkan) https://github.com/tencent/ncnn
RIFE Real-Time Intermediate Flow Estimation for Video Frame Interpolation https://github.com/hzwer/Practical-RIFE
rife-ncnn-vulkan Video frame interpolation implementation using NCNN and Vulkan https://github.com/nihui/rife-ncnn-vulkan
rife ncnn vulkan python Python bindings for RIFE NCNN Vulkan implementation https://github.com/media2x/rife-ncnn-vulkan-python
GMFSS GMFlow based Anime VFI https://github.com/98mxr/GMFSS_Fortuna
ncnn python Python bindings for NCNN Vulkan framework https://pypi.org/project/ncnn
Real-ESRGAN Upscaling https://github.com/xinntao/Real-ESRGAN
SPAN Upscaling https://github.com/hongyuanyu/SPAN
Spandrel CUDA upscaling model architecture support https://github.com/chaiNNer-org/spandrel
cx_Freeze Tool for creating standalone executables from Python scripts (Linux build) https://github.com/marcelotduarte/cx_Freeze
PyInstaller Tool for creating standalone executables from Python scripts (Windows/Mac builds) https://github.com/pyinstaller/pyinstaller
Feather Icons Open source icons library https://github.com/feathericons/feather
Python Standalone Builds Backend inference using portable python, helps when porting to different platforms. https://github.com/indygreg/python-build-standalone

Custom Models:

Model Author Link
4x-SPANkendata Crustaceous D 4x-SPANkendata
4x-ClearRealityV1 Kim2091 4x-ClearRealityV1
4x-Nomos8k-SPAN series Helaman 4x-Nomos8k-SPAN series
2x-OpenProteus SiroSky OpenProteus
2x-AnimeJaNai V3 Sharp The Database https://github.com/the-database/mpv-upscale-2x_animejanai

FAQ

General Application Usage

  • Q: What does this program attempt to accomplish?
    • A: Fast, efficient and easily accessable video interpolation (Ex: 24->48FPS) and video upscaling (Ex: 1920->3840)
  • Q: What backend should I use?
    • A:
      Modern Nvidia (20 series and up), TensorRT is recommended.
      Older Nvidia (10 and 16 series), CUDA is recommended.
      Oldest Nvidia (900 series and below), NCNN is recommended.
      Modern AMD Linux (6000 seies and up), ROCm is experimental.
      Other Cards (AMD/Intel) NCNN is the only backend currently working.
  • Q: Why is it failing to recognize installed backends?
    • A: REAL Video Enhancer uses PIP and portable python for inference, this can sometimes have issues installing. Please attempt reinstalling the app before creating an issue.

TensorRT related questions

  • Q: Why does it take so long to begin inference?
    • A: TensorRT uses advanced optimization at the beginning of inference based on your device, this is only done once per resolution of video inputed.
  • Q: Why does the optimization and inference fail?
    • A: The most common way an optimization can fail is Limited VRAM There is no fix to this except using CUDA or NCNN instead.

ROCm related questions

  • Q: Why am I getting (Insert Error here)?
    • A: ROCM is buggy, please take a look at ROCm Help.

NCNN related questions

  • Q: Why am I getting (Insert Vulkan Error here)?
    • A: This usually is an OOM (Out Of Memory) error, this can indicate a weak iGPU or very old GPU, I recommeding trying out the Colab Notebook instead.