ocaml / ocaml/dune

dune-project parser does not work if dune-project has a UTF-8 byte-order mark

Open
#9,396 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

windows
Dominant language
OCaml
Stars
1.9k
Forks
500
Avg merge
15h 21m
Merged PRs (30d)
277

Description

Edits

  1. Narrowed the issue to just supporting UTF-8 with BOM (ie. supporting the UTF-8 spec). Previously the original problem report was UTF-16 + BOM but as mentioned by @Alizter OCaml does not support UTF-16.

Expected Behavior

In PowerShell on Windows:

echo "(lang dune 3.12)" > dune-project
dune build

I would expect the project to build.

Actual Behavior

File "dune-project", line 1, characters 0-34:
1 | ��(lang dune 3.12)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: Invalid first line, expected: (lang <lang> <version>)

The reason? Windows conventionally has a byte-order mark in its Unicode files. The built-in PowerShell (or 5.x or less) inserts the BOMs; newer Command Prompts do not.

> with-dkml file dune-project
dune-project: Unicode text, UTF-16, little-endian text, with CRLF line terminators

> with-dkml od -xc dune-project
0000000    feff    0028    006c    0061    006e    0067    0020    0064
        377 376   (  \0   l  \0   a  \0   n  \0   g  \0      \0   d  \0
0000020    0075    006e    0065    0020    0033    002e    0031    0032
          u  \0   n  \0   e  \0      \0   3  \0   .  \0   1  \0   2  \0
0000040    0029    000d    000a
          )  \0  \r  \0  \n  \0
0000046

Specifications

  • Version of dune (output of dune --version): 3.12.1
  • Operating system (distribution and version): Windows 11 with PowerShell 5.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the PowerShell commands in the report on Windows, using the stated dune 3.12.1 behavior. Trace the dune-project parser from that failure and verify that a UTF-8 BOM is accepted while preserving the existing expected first-line validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
ocaml
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.