Decoding issue with ingest (strings including diacritics)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
We upload Datasets in the .dta format used by Stata (version 14) encoded in UTF8. These files are then ingested by Dataverse which creates a .tab file. If downloaded in the original .dta format, strings including diacritics are displayed correctly. However, if downloaded as .tab file, diacritics are not displayed correctly anymore. We have included datafiles for reproducing this error.
Dataverse 4.18.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by downloading data.zip and reproducing the issue against Dataverse 4.18.1 with the included Stata version 14 UTF-8 .dta files. Trace the ingest path that creates the .tab file and compare its handling of strings with diacritics; done means the downloaded .tab representation displays those characters correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100