SyntaxError for files with wrong encoding
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 597
- Forks
- 223
- Avg merge
- 11h 23m
- Merged PRs (30d)
- 3
Description
I have multiple files that just look fine with editors (gedit, sublime, vim) but produce the following error message when loaded by Psych:
Psych.safe_load(file)
/usr/local/lib/ruby/3.0.0/psych.rb:456:in `parse': (<unknown>): invalid trailing UTF-8 octet at line 1 column 1 (Psych::SyntaxError)
A file example can be found here (renamed as attaching a .yaml file is not supported). I am not that familiar with character encoding, but the file seems to get interpreted as UTF-8 while it actually isn't. isutf8 returns metal_copper (1).yaml: line 87, char 12, byte 1497: After a first byte between E1 and EC, expecting the 2nd byte between 80 and BF. on the file example.
I do not know how editor software today handle this topic, but it sure would be desirable to have Psych parse files in the way as shown in common editors. From googling the error message it seems I am not the only one who had problems with that and it seem to produce problem now and then for users/project that use Psych.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at Psych.safe_load and the parse call reported at psych.rb:456, then reproduce the failure with the attached example_file.txt and inspect its encoding. The issue does not name a test or precise acceptance condition; completion would require an agreed behavior for non-UTF-8 input and a regression check for that example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100