Running Real-ESRGAN and NAFNet on Android: Engineering a 100% On-Device AI Video Upscaler

A technical breakdown of how OnDevice - Video Enhancer achieves deep-learning video super-resolution and motion deblurring locally on mobile GPUs and NPUs with zero cloud infrastructure.

Over the past decade, state-of-the-art super-resolution networks like Real-ESRGAN and image restoration networks like NAFNet (Nonlinear Activation Free Network) have set the gold standard in computer vision. However, deploying these architectures for consumer mobile applications has historically relied almost exclusively on cloud servers.

The rationale seemed obvious: video enhancement requires decoding hundreds or thousands of high-resolution video frames, running billions of floating-point operations per second (FLOPs), and re-encoding with hardware codecs like H.264 or HEVC without exceeding mobile thermal or memory envelopes.

Yet the cloud paradigm carries massive trade-offs:

This article examines how OnDevice - Video Enhancer ( com.nabil.ondevice.videoenhancer ) bypasses the cloud entirely, deploying optimized ONNX neural models directly onto Android devices running consumer mobile chipsets (Qualcomm Snapdragon, MediaTek Dimensity, Samsung Exynos, and Google Tensor).

The local pipeline is structured into four decoupled, memory-safe stages:

[ Input MP4/MOV Video ] │ ▼ ┌────────────────────────────────────────┐ │ 1. Native MediaCodec Decoder Pipeline │ │ Extracts raw RGB/YUV bitstream │ └────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ 2. Edge Neural Network Inference │ │ • Real-ESRGAN Compact 4x │ │ • OpenCV NAFNet Deblur 1x │ │ • MobileSR 2x (Fast Convolutional) │ │ • AnimeSharp 2x (Line-Art Restore) │ │ • MobileClear (Laplacian DSP) │ └────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ 3.

Foreground Service & Thermal Policy │ │ • Android Game Mode GPU Lock │ │ • Chunked In-Memory Frame Buffer │ └────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ 4. Hardware Encoder & Muxer │ │ • MediaMuxer + MediaCodec (H.264) │ │ • Audio Stream Preservation │ └────────────────────────────────────────┘ │ ▼ [ Output 2x / 4K Enhanced Video ] Enter fullscreen mode Exit fullscreen mode 1. The Model Suite: Balancing Quality, Compute, and Model Footprint Deploying deep learning models to edge devices requires matching the model complexity to the user's specific degradation profile and hardware tier.

Running deep convolutional networks on video frames introduces severe challenges that don't exist in desktop computing:

A naive implementation might load multiple frames into a queue as uncompressed 32-bit bitmaps. A single 1080p frame consumes ~8.3 MB of RAM; buffering 60 frames would easily breach Android's per-app memory limit (256MB–512MB on standard devices).

Solution: OnDevice implements a strict backpressure pipeline. The decoder extracts a single frame, copies the normalized tensor directly into the ONNX Runtime input tensor, executes inference, and pipes the output directly to the hardware encoder's input surface before releasing the buffer. Peak RAM usage remains constant regardless of whether the video is 10 seconds or 10 minutes long.

Long-running AI workloads cause mobile SoCs to quickly heat up, triggering Android's thermal daemon to throttle CPU/GPU frequencies down to 30% of their base clocks.

Solution: The app implements the Android Game Mode API ( android.game_mode_config ) and utilizes foreground wake locks. By signaling the OS that the application is performing an intensive, continuous compute task, the scheduler prevents erratic frequency downscaling, ensuring consistent FPS throughput across long export sessions.

Typical frame inference speeds across various mobile silicon tiers:

The success of OnDevice - Video Enhancer demonstrates that consumer mobile devices have reached the performance inflection point where desktop-class neural restoration models can run entirely locally.

By eliminating remote server infrastructure:

OnDevice - Video Enhancer is built using modern Kotlin, Jetpack Compose Material 3, and ONNX Runtime. The app is publicly available on the Google Play Store: