marklogic / marklogic/marklogic-contentpump
Save/export rows that failed ingest due to Delimited Text Ingest Fails on Unescaped Quotes
Open
@abika5 is already working on this.
Since Sep 27, 2021.
major
mlcp
new
task
- Dominant language
- Java
- Stars
- 37
- Forks
- 32
- Avg merge
- 1h 3m
- Merged PRs (30d)
- 12
Description
We're encountering a similar issue to https://github.com/marklogic/marklogic-contentpump/issues/68 for files that are tab delimited but with unescaped quotes:
Sample:
11:16:43.614 [pool-1-thread-1] WARN c.m.contentpump.DocumentMapper - Skipped record: () in file:/homes/local/projects/data-hub/data/omop/all/CONCEPT/CONCEPT.csv at line 1999360, reason: invalid char between encapsulated token and delimiter
02020201 "opt out" service Observation DOMAIN DOMAIN
It would be great if we could get the failed records in a separate file or in the log so we could examine quickly what went wrong during the ingest and see what kind of formatting error we have and fix it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.