Ingest: line breaks in Excel cause ingest to fail
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
Hello, first time here - working with @adam3smith at QDR
Line breaks in Excel lead to a reading mismatch and cause ingest to fail:
Issue replicated on demo (https://demo.dataverse.org/dataset.xhtml?persistentId=doi%3A10.70122%2FFK2%2FOIWIG6) with two files that are identical except for the line break. The one with line break at text end fails to ingest, the other ingests successfully.
Converting the xlsx file to csv and opening it in a text editor shows that line breaks in xlsx create line breaks in the text where they should not (i.e. they create new rows), which is likely the root issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure with the two Excel files from the demo dataset and compare their converted text output. Trace the ingest path that reads Excel-derived rows and determine where line breaks at the end of a cell become new rows. Done means the affected file ingests successfully without changing legitimate line breaks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100