google / google/closure-compiler
Warn when producing invalid UTF-8 output files
- Dominant language
- JavaScript
- Stars
- 7.7k
- Forks
- 1.2k
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 6
Description
When running the compiler with `--charset UTF-8` one can potentially end up with not strictly valid output files.
The problem is that sometimes strings in input sources can be perfectly valid ASCII strings, but they encode invalid UTF-8 strings via unicode escape sequences. One such example can be found in Closure Library itself[1].
Closure Compiler when instructed to produce output in UTF-8 blindly assumes validity of such strings and outputs them as raw byte output (without checking). This can cause problems in some systems. For example in my case Google Chrome extension content script loading code is quite strict and uses UTF-8 validator to reject any non-valid scripts.

My proposal is to introduce a warning (which could be disabled on demand) to inform user about this edge situation. An alternative solution would be to leave string literals as-is, that means if they were defined with unicode escape sequences, output them without transformation even under UTF-8 output mode.
For more background info you can read my comment[2] in ClojureScript JIRA, where we discovered this issue after enabling UTF-8 output by default.
[1] https://github.com/google/closure-library/blob/d66b94513df131c9776cdf70ac476bbb1a62e5d0/closure/goog/i18n/bidi.js#L202
[2] http://dev.clojure.org/jira/browse/CLJS-1547?focusedCommentId=42617
Contributor guide
Assessment
This issue has not been assessed yet.