danielmiessler / danielmiessler/Fabric
YouTube visual OCR output uses index-based timestamps instead of real frame timing
- Dominant language
- Go
- Stars
- 43.9k
- Forks
- 4.3k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 8
Description
## Description
The YouTube visual extraction feature currently emits subtitle-like timestamps based on the OCR result index instead of the actual frame time.
That means the output can look temporally precise while actually pointing to the wrong time in the source video.
## Affected area
- `internal/tools/youtube/youtube.go`
- current upstream I validated locally: `9743e10273c45e1c377c10ec0ffaf0de8fefb3c8` (`v1.4.448`)
## Why this happens
`GrabVisual()` chooses frames using either:
- `fps=%d`, or
- `select='gt(scene,%f)'`
but later renders timestamps using:
```go
secs := i
```
So the nth OCR result is labeled as second `n`, regardless of when that frame actually occurred.
## Why this matters
This feature is meant to extract meaningful visual text from video. Incorrect timestamps reduce trust in the output and can mislead downstream prompts or users into attributing OCR text to the wrong moment in the source video.
## Local validation evidence
I reproduced this locally with a focused test that mocks `yt-dlp`, `ffmpeg`, and `tesseract`:
```bash
go test ./internal/tools/youtube -run 'TestGrabVisual_UsesFrameTimesFromFFmpeg' -v
```
In the repro, `GrabVisual(..., fps=2)` produces two OCR frames and the current upstream behavior labels the second frame by index instead of by the actual frame time.
I have a small fix prepared that preserves ffmpeg-derived frame times and includes regression coverage.
Contributor guide
Research direction
Start in internal/tools/youtube/youtube.go at GrabVisual and run go test ./internal/tools/youtube -run 'TestGrabVisual_UsesFrameTimesFromFFmpeg' -v. Trace how ffmpeg selects frames and how OCR results receive timestamps; done means the output uses the actual frame times and the regression test passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- audio-video-rtc
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100