eclipse-platform / eclipse-platform/eclipse.platform
content-type extension point "default-charset" is used for overwriting existing (e.g. BOM+UTF-16) files
- Dominant language
- Java
- Stars
- 165
- Forks
- 174
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 22
Description
Defining a [content-type](https://help.eclipse.org/latest/topic/org.eclipse.platform.doc.isv/reference/extension-points/org_eclipse_core_contenttype_contentTypes.html?cp=2_1_1_9#e.content-type) with a `default-charset` attribute will cause that charset to be used when _saving_ (modified) existing files, even if those files had some other character set, e.g. BOM+`UTF-16`.
This has multiple problems (the example is a file `foo.properties`, encoded with BOM and `UTF-16`, where its content type (`org.eclipse.core.runtime.properties`) has a `default-charset` attribute specifying `ISO-8859-1`):
1. Doing a no-op (e.g. deleting and re-inserting a space) modification and saving the file will change the encoding, and thus change the entire file contents. This is a problem in many ways, e.g. for version control systems.
2. Inserting a character, e.g. the euro symbol (U+20AC), which cannot be represented in the `default-charset` (`ISO-8859-1`), and trying to save the file, will bring up a dialog about "Save Problems" which will suggest to save in `UTF-8`. The (obviously?) right thing would be to save the file in its current encoding (BOM+`UTF-16`) which _can_ represent the new contents.
3. A variant of the above, if the file _already_ contains some character (such as the euro symbol) that cannot be encoding in the `default-charset`, then doing a no-op modification and attempting to save the file will give the error dialog and refuse to save.
I encountered this in my custom content type, but it is easy to reproduce using any of the common content types that has, e.g., `ISO-8859-1` as `default-charset` attribute:
1. In Eclipse, create a plain project (no natures).
2. Outside of Eclipse, create a file `foo.properties` in the project directory, and make its contents use BOM with `UTF-16`, e.g.:
```
$ echo 'abc' | iconv -f UTF-8 -t UTF-16 > foo.properties
$ file foo.properties
foo.properties: Unicode text, UTF-16, big-endian text
```
4. Back in Eclipse, refresh the project, and open the file from the Project Explorer.
Now all the issues above can easily be reproduced.
The documentation of the `default-charset` attribute is not very illuminating, but I had expected it to be used when _opening_ a file where the file contents itself did not provide a hint. When _saving_ the file, I would always expect the current encoding, however that was determined, to be preserved.
Contributor guide
Assessment
This issue has not been assessed yet.