zopencommunity / zopencommunity/gitport
Using .gitattributes to encode file as IBM-1047 results in some characters being converted incorrectly
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 10
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
Description
The way I have it set up is that the file is stored as UTF-8 in git, and I'm using .gitattributes to set the working tree encoding to ibm-1047. When cloning the repo, git converts and tags the file as IBM-1047. The issue is that for some characters such as ® (the registered symbol), it appears as ▒ after the conversion.
Reproduce
- Create a new repository.
- Create
file.txtcontaining the ® character. - Create a
.gitattributesfile containing eitherfile.txt zos-working-tree-encoding=ibm-1047 git-encoding=iso8859-1orfile.txt working-tree-encoding=ibm-1047 - Clone the repository.
- Read
file.txtusing vim or cat. It will display ▒ instead of ®
Additional info
Upon examination of the file with a hex editor, it appears that it's converting the ® character from C2 AE to AF which results in it being unreadable. Whereas for it to be readable, it would have to be 62 AF.
This is consistent with the behavior when using iconv to convert from UTF-8 to IBM-1047.
Whereas converting from ISO8859-1 to IBM-1047 seems to result in the correct conversion.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproduced .gitattributes cases for file.txt and compare the UTF-8-to-IBM-1047 results with the ISO-8859-1-to-IBM-1047 results described in the issue. Check the relevant git encoding behavior and use the iconv comparison as a reference; done means cloning and reading file.txt preserves ® rather than displaying ▒.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100