aetaric / aetaric/checkrr

Migrating file-type detection from h2non/filetype to gabriel-vasile/mimetype?

未關閉
#164 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Go
星號
578
分支
17
PR 合併指標
30 天內沒有已合併 PR

描述

Context

h2non/filetype is checkrr's primary file-type detector (check/checkrr.go:401, :628). The library appears to be in maintenance hiatus — last release was January 2021, 16 open PRs sit unreviewed (the oldest from 2016), and the upstream issue referenced in checkrr's TODO at check/checkrr.go:122 and :812 (h2non/filetype#120) has remained open for years. The MPEG-TS workaround in checkrr (mpegtsMatcher, check/checkrr.go:813) exists precisely because the library has not accepted format additions.

A more recent and actively maintained alternative exists: gabriel-vasile/mimetype. Last release February 2026, regular updates, broader format coverage, validated against a ~50k-file corpus and libmagic for accuracy.

Why bring this up

Issue #107 (QuickTime files flagged as Unknown despite being valid media) is a symptom of h2non/filetype's coverage gaps. A targeted fix is possible — add quickTimeMatcher / m4vMatcher following the existing mpegtsMatcher precedent — and I'm happy to send that PR. But it would be the second downstream workaround for the same root cause, and likely not the last. Migrating the dependency removes the category of bug rather than patching individual format gaps.

Scope of the change

checkrr's usage of h2non/filetype is limited to one file (check/checkrr.go) and six call sites:


Line | Current call | mimetype equivalent
-- | -- | --
123-126 | filetype.AddType + matchers.Video[ts] = mpegtsMatcher | mimetype.Extend(...)
401 | filetype.IsVideo(buf) \\|\\| filetype.IsAudio(buf) | check detected MIME's category prefix
402 | filetype.IsAudio(buf) | check detected MIME starts with audio/
628 | filetype.IsImage(buf) \\|\\| filetype.IsDocument(buf) | check detected MIME category

No other package in checkrr imports filetype. The ffprobe code path, *arr integration, web server, notifications, and BoltDB layers are untouched.

Risks

The two libraries do not return identical results on every input. The main risk is behavior drift on borderline files — e.g. a corrupt header that one library classifies as application/octet-stream and the other as video/mp4. To address this, I would include in the PR:

  • A side-by-side comparison test against a curated corpus of known-good and known-bad sample headers.

  • Documentation in the PR description of any intentional or unintentional classification changes.

  • The MPEG-TS custom matcher reimplemented as a mimetype.Extend(...) registration so the workaround behavior is preserved.

Proposal

Before I invest the work, I wanted to check whether this kind of dependency swap is something you'd consider merging, or whether you'd prefer I send the smaller per-format matcher PRs for #107 and future similar reports. Either path is fine with me — your call on what's the right direction for the project.

If you're open to the migration, I'll open a draft PR with:

  1. The library swap with a thin internal helper for the four category checks

  2. Port of mpegtsMatcher to mimetype's extension API

  3. Test coverage for the QuickTime and M4V cases from #107

  4. A comparison-test artifact showing classification deltas vs the current library

    Happy to discuss tradeoffs before writing any code.

## Context

`h2non/filetype` is checkrr's primary file-type detector (`check/checkrr.go:401`, `:628`). Looks like the library appears to be in maintenance hiatus — last release was January 2021, 16 open PRs sit unreviewed (the oldest from 2016), and the upstream issue referenced in checkrr's `TODO` at `check/checkrr.go:122` and `:812` (h2non/filetype#120) has remained open for years. I'm guessing the MPEG-TS workaround in checkrr (`mpegtsMatcher`, `check/checkrr.go:813`) exists because the library has not accepted format additions.

A more recent and actively maintained alternative exists: [`[gabriel-vasile/mimetype](https://github.com/gabriel-vasile/mimetype)`](https://github.com/gabriel-vasile/mimetype). Last release February 2026, regular updates, broader format coverage, validated against a ~50k-file corpus and libmagic for accuracy.

## Why bring this up

Issue #107 (QuickTime files flagged as Unknown despite being valid media) is a symptom of `h2non/filetype`'s coverage gaps. A targeted fix is possible — add `quickTimeMatcher` / `m4vMatcher` following the existing `mpegtsMatcher` precedent — and I'm happy to send that PR. But it would be the second downstream workaround for the same root cause, and probably not the last. Migrating the dependency removes the category of bug rather than patching individual format gaps.

## Scope of the change

checkrr's usage of `h2non/filetype` is limited to one file (`check/checkrr.go`) and six call sites:

| Line | Current call | mimetype equivalent |
| ------- | --------------------------------------------------------- | ---------------------------------------- |
| 123-126 | `filetype.AddType` + `matchers.Video[ts] = mpegtsMatcher` | `mimetype.Extend(...)` |
| 401 | `filetype.IsVideo(buf) \|\| filetype.IsAudio(buf)` | check detected MIME's category prefix |
| 402 | `filetype.IsAudio(buf)` | check detected MIME starts with `audio/` |
| 628 | `filetype.IsImage(buf) \|\| filetype.IsDocument(buf)` | check detected MIME category |

No other package in checkrr imports filetype. The ffprobe code path, *arr integration, web server, notifications, and BoltDB layers are untouched.

## Risks

The two libraries do not return identical results on every input. The main risk is behavior drift on borderline files — e.g. a corrupt header that one library classifies as `application/octet-stream` and the other as `video/mp4`. This is how I want to address this: I would include in the PR:

- A side-by-side comparison test against a curated corpus of known-good and known-bad sample headers.
- Documentation in the PR description of any intentional or unintentional classification changes.
- The MPEG-TS custom matcher reimplemented as a `mimetype.Extend(...)` registration so the workaround behavior is preserved.

## Proposal

I'm very happy to do this but before I put in any work I wanted to check whether this kind of dependency swap is something you'd consider merging, or whether you'd prefer I send the smaller per-format matcher PRs for #107 and future similar reports. Either way is fine with me — your call on which direction you want to go.

If you're cool with the migration, I'll open a draft PR with:

1. The library swap with a thin internal helper for the four category checks

2. Port of `mpegtsMatcher` to mimetype's extension API

3. Test coverage for the QuickTime and M4V cases from #107

4. A comparison-test artifact showing classification deltas vs the current library

5. Anything else you might want to make sure the swap goes smoothly.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。