Leading BOM is counted as a column when the encoding is set explicitly
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 72/100
Research direction
Start in yaml_parser_scan_to_next_token() and trace how explicit YAML_UTF8_ENCODING bypasses yaml_parser_determine_encoding(). Run the BOM-prefixed mapping reproduction from the issue, then verify that the complete document parses and the first token remains at column 0 without breaking YAML_ANY_ENCODING behavior.
Written by the indexing model from the issue text.
Description
When the input encoding is set with yaml_parser_set_encoding(), yaml_parser_determine_encoding() (which strips a leading BOM during auto-detection) is skipped, so the BOM reaches the scanner and is handled in yaml_parser_scan_to_next_token():
/* Allow the BOM mark to start a line. */
if (parser->mark.column == 0 && IS_BOM(parser->buffer))
SKIP(parser);
SKIP() increments mark.column, so the first line's tokens start at column 1 instead of 0. A root-level block mapping then terminates at the first newline, and parsing fails with "did not find expected ".
Reproduction: parse "\xEF\xBB\xBFa: b\nc: d\n" with the encoding set to YAML_UTF8_ENCODING. Only the a: b pair is emitted before the error. The same input parses fully with YAML_ANY_ENCODING.
Per YAML 1.2 §5.2 the BOM is not content, so this skip should not advance mark.column.
Found via https://github.com/ruby/psych/issues/331 (Ruby's Psych always sets the encoding explicitly, so BOM-prefixed documents were silently truncated).
- Dominant language
- C
- Stars
- 1.2k
- Forks
- 370
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from yaml/libyaml
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 Under an hour Newbie friendliness 72/100
-
Difficulty 1/5 Under an hour Newbie friendliness 62/100
Similar issues
-
[adam] AdamNet network read doesn't cap to MAX_ADAM_PACKET_LEN, overflows client receive buffers Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
FujiNetWIFI/fujinet-firmware#1649 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
HarbourMasters/Shipwright#7229 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
riscv-software-src/riscv-isa-sim#2435 · 1 comment ·
-
bug Self Built Image SNAPSHOT Supported Device target/ramips
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100