facebook / facebook/flow

Investigate what to do with malformed characters

Aperta
#3,814 0 commenti 1 reazione 1 assegnatario Rivendicata da @mroch Vedi su GitHub
parsing
Lingua principale
Rust
Stelle
22.3k
Fork
1.9k
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

`Sedlex.Utf8.from_string` will throw a `MalFormed` exception if given malformed unicode characters, like `\xa0`, which should be `\xc2\xa0` in UTF8 ([source](http://www.fileformat.info/info/unicode/char/00a0/index.htm)).

@bhamiltoncx made the following suggestion:

> There are three things Flow can do:
> 1) Strict (good default): raise an error when reading file from disk
> 2) Substitute: replace non-UTF-8 sequences with U+FFFD
> 3) Add option to specify an encoding and convert files from that encoding to Unicode when reading from disk

At the moment, we will just treat the malformed file as empty. The main problem with this is that it may be a `// @flow` file and in that case we would want to emit a parsing error.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.