Default `subs_then_webrtc` VAD aligns against forced-only embedded subtitle stubs, producing ±60 s offsets on in-sync subtitles

Summary

With the default --vad subs_then_webrtc, VideoSpeechTransformer.try_fit_using_embedded_subs uses an embedded text subtitle stream from the reference video as the alignment source whenever one exists. It picks the stream with the largest last-cue time (np.argmax(embedded_subs_times)) and never checks how many cues the stream has, whether it is flagged forced, or what language it is.

Most WEB-DL and many Blu-ray remuxes carry a forced-only English track with 1 to 60 cues (signs, foreign lines). Against such a track the FFT alignment has almost nothing to correlate, so ffsubsync returns a large offset pinned at or near --max-offset-seconds (60 s), often with a negative score, for a subtitle file that is in fact in sync. It then logs ...success!, writes the shifted file, and exits 0.

Reproduction

A feature-length WEB-DL mkv with two English audio streams and one embedded subrip English stream flagged forced, under 50 cues. The external English .srt has well over a thousand cues and plays in sync.

ffsubsync video.mkv -i video.en.srt -o out.srt --max-duration-seconds 1200
  score: -72067.320   offset seconds: -59.440   framerate scale factor: 1.042

ffsubsync video.mkv -i video.en.srt -o out.srt --max-duration-seconds 1200 --vad webrtc
  score: 33134.000    offset seconds: 0.030     framerate scale factor: 1.000

ffsubsync video.mkv -i video.en.srt -o out.srt --max-duration-seconds 1200 --reference-stream a:1
  score: 18426.000    offset seconds: 0.000     framerate scale factor: 1.000

Across a library of several thousand external subtitles, roughly three in four of the files the default mode flagged as badly out of sync re-measured as fine (within 1 s, or a clean 25/23.976 fps scale with a high positive score) once --vad webrtc was forced. Every video in that set had a forced-only embedded stream of a few dozen cues or fewer; videos with a full embedded track measured the same in both modes.

Version: 0.5.1 (master is identical at time of writing).

Why it matters

Batch tools built on ffsubsync (subtitle managers, media-server plugins) run it unattended with default flags and overwrite the input. With the default VAD, any library containing forced-only tracks gets its good subtitles shifted by up to 60 s, and because the output is deterministic the next run shifts them again. --skip-sync-on-low-quality is opt-in, and the exit code stays 0 when sync_was_successful is false, so callers get no signal.

Proposed fix

In speech_transformers.py:

1. Have _probe_embedded_subtitle_streams also request stream_disposition=forced and stream_tags=language, and skip streams with forced=1.
2. In try_fit_using_embedded_subs, pick the stream by cue count rather than by last-cue time, and log which stream was used and how many cues it had.
3. If the best remaining stream has fewer than a threshold of cues (I suggest 50, exposed as --min-embedded-subtitle-cues), raise so the caller falls back to the audio VAD, as it already does when no stream exists.

Optionally, when the subs-based alignment scores below zero, retry with the audio VAD before giving up. I have a patch for 1 to 3 and can open a PR if this approach is acceptable.

Start in speech_transformers.py, reading _probe_embedded_subtitle_streams and try_fit_using_embedded_subs, then reproduce the issue with the supplied ffsubsync commands. Trace how forced streams, cue counts, and low-quality alignment results reach the audio VAD fallback. Done means forced-only or too-short embedded tracks are not selected, the chosen stream is logged, and the reproduction avoids the large offset.