MacDownApp / MacDownApp/macdown
Can't open UTF-16 files with BOM
- Dominant language
- Rich Text Format
- Stars
- 9.8k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
Working with a new iPad and an app that defaults to saving files as UTF-16. MacDown wouldn't open one of these files, so I played around a little with the various UTF formats using BBEdit to create and save them. Here's what I found out:
MacDown can open (as reported by the bash `file` command):
- ASCII text, with no line terminators [equivalent to BBEdit's "UTF-8, no BOM"]
- UTF-8 Unicode (with BOM) text, with no line terminators
- UTF-16 without BOM
- Little-endian UTF-16 without BOM
Can't open:
- Little-endian UTF-16 Unicode text without BOM, with no line terminators
- Big-endian UTF-16 Unicode text without BOM, with no line terminators
In other words, it looks like the presence of a BOM prevents MacDown from opening a UTF-16 file.
This seems like a bug.
PS MacDown seems to strip the BOM when saving UTF-8 files that have them. It also seems to mess up UTF-16 files when saving them, at least when looked at with BBEdit or pico.
Contributor guide
Research direction
Reproduce the report with the UTF-16 variants described in the issue, including files with and without a BOM, and compare the results with the bash `file` command. Trace MacDown's file-opening and saving behavior for these encodings; done means the reported UTF-16 files open correctly and saving no longer corrupts their encoding.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos
- Domain
- desktop
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100