MacDownApp / MacDownApp/macdown

Can't open UTF-16 files with BOM

Open
#523 7 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rich Text Format
Stars
9.8k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

Working with a new iPad and an app that defaults to saving files as UTF-16. MacDown wouldn't open one of these files, so I played around a little with the various UTF formats using BBEdit to create and save them. Here's what I found out:

MacDown can open (as reported by the bash `file` command):
- ASCII text, with no line terminators [equivalent to BBEdit's "UTF-8, no BOM"]
- UTF-8 Unicode (with BOM) text, with no line terminators
- UTF-16 without BOM
- Little-endian UTF-16 without BOM

Can't open:
- Little-endian UTF-16 Unicode text without BOM, with no line terminators
- Big-endian UTF-16 Unicode text without BOM, with no line terminators

In other words, it looks like the presence of a BOM prevents MacDown from opening a UTF-16 file.

This seems like a bug.

PS MacDown seems to strip the BOM when saving UTF-8 files that have them. It also seems to mess up UTF-16 files when saving them, at least when looked at with BBEdit or pico.

Contributor guide

Open the contributing guide

Research direction

Reproduce the report with the UTF-16 variants described in the issue, including files with and without a BOM, and compare the results with the bash `file` command. Trace MacDown's file-opening and saving behavior for these encodings; done means the reported UTF-16 files open correctly and saving no longer corrupts their encoding.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos
Domain
desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.