CCExtractor / CCExtractor/ccextractor
[BUG] Non-DVB caption PIDs (Teletext/ISDB/ATSC) are silently dropped when multiple exist in one program
- Dominant language
- C
- Stars
- 903
- Forks
- 589
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 10
Description
## Current State
When a program contains multiple caption-carrying PIDs of the same non-DVB codec (Teletext, ISDB, or ATSC), `get_best_stream()` picks exactly one, and every other same-type PID gets silently discarded — no error, no warning, no log line. The user has no way to know a second language track existed and was dropped, unless they already know to inspect the PMT themselves and manually force it via `--datapid`.
DVB does not have this problem: `general_loop.c`'s extra-PID loop (added in `ad4886e7`, #447) explicitly re-processes every other DVB PID so multi-language DVB output works correctly. No equivalent exists for Teletext/ISDB/ATSC.
## Root Cause
The actual gate is in the demuxer, not the loop:
- `src/lib_ccx/ts_info.c:40-47` (`ignore_other_stream()`) and the equivalent in `ignore_other_sib_stream()` (`ts_info.c:90-97`) mark every non-selected PID as `ignore = 1` **unless its codec is `CCX_CODEC_DVB`**.
- Once ignored, `src/lib_ccx/ts_functions.c:674` (`copy_payload_to_capbuf()`) returns early for that PID, so its payload is never buffered, and `ts_functions.c:969` tears down its decoder context entirely.
- By the time `general_loop.c:1321-1370`'s DVB-only extra-PID loop runs, non-DVB extra PIDs already have no data left to process regardless of that loop's codec check.
## Evidence
Reproduced on a real broadcast recording (single program, two Teletext PIDs, not synthetic):
**Sample:** `FILE_1_TICKET_3514_Input_stream_SD_mpeg2video_1_video_8_audios_10_subs.ts`
https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket3514/
Program 1002 carries two Teletext PIDs:
Stream #0:9 [0xd49](https://github.com/CCExtractor/ccextractor/issues/spa): dvb_teletext (Initial page — magazine, not subtitles)
Stream #0:10 [0xd4a](https://github.com/CCExtractor/ccextractor/issues/dut,nor,dan,swe,por): dvb_teletext (5 declared Subtitle page languages)
Default run:
$ ccextractor --tpages-all t3514.ts
VBI/teletext stream ID 3401 (0xd49) for SID 1002 (0x3ea)
VBI/teletext stream ID 3402 (0xd4a) for SID 1002 (0x3ea)
Teletext decoder: 0 packets processed <-- 0xd4a (5-language subtitle PID)
Teletext decoder: 1663 packets processed <-- 0xd49 (magazine PID, gets kept)
`get_best_stream()` selects purely by list order and has no concept of `teletext_type` — it kept the "Initial page" magazine PID and discarded the PID explicitly declaring five "Subtitle page" languages, with zero indication to the user that this happened.
Forcing the dropped PID recovers real, decodable data:
$ ccextractor --tpages-all --datapid 3402 t3514.ts
Teletext decoder: 831 packets processed
> In this particular sample the recovered languages happen to duplicate content already present via the kept PID, so no unique captions are lost *here* — but the silent-drop behavior itself is confirmed on real broadcast data, and is trivially reproducible with any file where the second PID carries content not present elsewhere. Confirmed independently on a synthetic split of `dvbteletext.ts` where the drop does cost real, non-duplicate caption lines.
## Proposed Fix (scoped)
This issue is scoped to the silent-drop / missing-warning problem only. Full multi-PID decoding support for Teletext/ISDB/ATSC is a separate, larger effort — it would require generalizing per-PID decoder allocation in `lib_ccx.c:352-374`, per-language encoder routing in `lib_ccx.c:452-466`, and storing ISO-639 codes into `cinfo->lang` for Teletext at `ts_tables.c:474`, none of which exist today for non-DVB codecs.
**For this issue:** add a warning in `ignore_other_stream()` / `ignore_other_sib_stream()` when a stream being ignored is a non-DVB caption codec (Teletext/ISDB/ATSC), naming the PID and pointing the user to `--datapid` to retrieve it manually. No decoding behavior changes.
## Status
Confirmed, reproducible on real broadcast data
Contributor guide
Assessment
This issue has not been assessed yet.