eclipse-platform / eclipse-platform/eclipse.platform
content-type extension point "default-charset" is used for overwriting existing (e.g. BOM+UTF-16) files
- Vorherrschende Sprache
- Java
- Sterne
- 165
- Forks
- 174
- Ø Merge
- 2 T. 8 Std.
- Gemergte PRs (30 T.)
- 22
Beschreibung
Defining a [content-type](https://help.eclipse.org/latest/topic/org.eclipse.platform.doc.isv/reference/extension-points/org_eclipse_core_contenttype_contentTypes.html?cp=2_1_1_9#e.content-type) with a `default-charset` attribute will cause that charset to be used when _saving_ (modified) existing files, even if those files had some other character set, e.g. BOM+`UTF-16`.
This has multiple problems (the example is a file `foo.properties`, encoded with BOM and `UTF-16`, where its content type (`org.eclipse.core.runtime.properties`) has a `default-charset` attribute specifying `ISO-8859-1`):
1. Doing a no-op (e.g. deleting and re-inserting a space) modification and saving the file will change the encoding, and thus change the entire file contents. This is a problem in many ways, e.g. for version control systems.
2. Inserting a character, e.g. the euro symbol (U+20AC), which cannot be represented in the `default-charset` (`ISO-8859-1`), and trying to save the file, will bring up a dialog about "Save Problems" which will suggest to save in `UTF-8`. The (obviously?) right thing would be to save the file in its current encoding (BOM+`UTF-16`) which _can_ represent the new contents.
3. A variant of the above, if the file _already_ contains some character (such as the euro symbol) that cannot be encoding in the `default-charset`, then doing a no-op modification and attempting to save the file will give the error dialog and refuse to save.
I encountered this in my custom content type, but it is easy to reproduce using any of the common content types that has, e.g., `ISO-8859-1` as `default-charset` attribute:
1. In Eclipse, create a plain project (no natures).
2. Outside of Eclipse, create a file `foo.properties` in the project directory, and make its contents use BOM with `UTF-16`, e.g.:
```
$ echo 'abc' | iconv -f UTF-8 -t UTF-16 > foo.properties
$ file foo.properties
foo.properties: Unicode text, UTF-16, big-endian text
```
4. Back in Eclipse, refresh the project, and open the file from the Project Explorer.
Now all the issues above can easily be reproduced.
The documentation of the `default-charset` attribute is not very illuminating, but I had expected it to be used when _opening_ a file where the file contents itself did not provide a hint. When _saving_ the file, I would always expect the current encoding, however that was determined, to be preserved.
Beitragsleitfaden
Rechercherichtung
Reproduziere das Problem in Eclipse anhand der beschriebenen Schritte mit UTF-16 BOM und foo.properties und verfolge anschließend den content-type-Speicherpfad; der Payload nennt keine Quelldatei und keinen Test. Erledigt ist die Aufgabe, wenn beim Speichern einer unveränderten oder geänderten Datei ihre erkannte aktuelle Kodierung beibehalten wird, anstatt default-charset anzuwenden, auch wenn diese Kodierung neu eingefügte Zeichen unterstützt.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- java
- Bereich
- desktop, tooling
- Issue-Typ
- Bug
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 50/100