[REQ] MPEG (1/2/4) video decoding
- Dominant language
- C++
- Stars
- 474
- Forks
- 27
- PR merge metrics
- No merged PRs in 30d
Description
Hi there,
just discovered this interesting approach to JPEG decoding.
I've just asked GH-Copilot (Claude Haiku 4.5) if the technique is applicable to video decoding too:
> Based on my review of the Knusperli repository, here's my analysis of whether its approach can be applied to MPEG-encoded videos:
>
> ## Short Answer: **Yes, but with significant practical challenges.**
>
> ## How Knusperli Works (JPEG-focused)
>
> Knusperli reduces blocking artifacts in JPEG images by:
> 1. Treating quantized DCT coefficients as **intervals** rather than fixed values
> 2. Selecting values within those intervals that **minimize discontinuities at 8×8 block boundaries**
> 3. Maintaining mathematical equivalence (coefficients still round to the stored quantized values)
>
> ## Applicability to MPEG Video
>
> ### ✅ **Theoretically Compatible**
> - **MPEG-1/2/4 video codecs** also use DCT-based block coding (typically 8×8 blocks)
> - **Similar quantization scheme**: MPEG also quantizes DCT coefficients and discards precision
> - The **core algorithm** (interval-based coefficient selection to reduce blocking) could apply frame-by-frame
>
> ### ⚠️ **Major Practical Challenges**
>
> 1. **Temporal Coherence**: MPEG uses motion compensation and inter-frame prediction. Knusperli's block-boundary smoothing could introduce:
> - Temporal flickering between frames
> - Conflicts with motion vectors
> - Artifacts at frame boundaries in predictive coding
>
> 2. **Computational Complexity**:
> - Videos have **orders of magnitude more frames** than a single image
> - Real-time or near-real-time processing becomes critical
> - Knusperli's optimization per frame would need to be extremely efficient
>
> 3. **Chroma/Motion Components**:
> - MPEG's chroma subsampling (4:2:0) differs from standard JPEG handling
> - Motion prediction adds another layer of quantization that Knusperli doesn't address
>
> 4. **Modern Codec Differences**:
> - H.264/H.265/VP9/AV1 use more sophisticated entropy coding (not just Huffman)
> - Larger variable block sizes (beyond 8×8)
> - Would require substantial adaptation
>
> ### 🔄 **Potential Adaptations**
>
> 1. **Frame-by-frame processing** with temporal consistency constraints
> 2. **Motion-aware deblocking** that respects inter-frame predictions
> 3. **Selective application** to I-frames only (less risky temporally)
> 4. **Integration with existing post-processing** rather than standalone approach
>
> ## Recommendation
>
> This would be a worthwhile research project, but you'd need to:
> - Benchmark temporal stability across frame sequences
> - Design motion-aware constraints
> - Test on various MPEG versions/profiles
> - Consider modern codecs (H.264/HEVC) which already have built-in deblocking filters
>
> The technique is **not a drop-in replacement** but could inspire a **video-specific deblocking enhancement**.
Source: https://github.com/copilot/share/4a4f02b4-4920-84b5-a843-8a09e008080a
...it would be interesting to challenge Gemini for coding this...
Hope that inspires.
Contributor guide
Assessment
This issue has not been assessed yet.