ClickHouse / ClickHouse/ClickHouse
Importing gigabytes of CSV that contains non-standard CSV escaping value \"
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
**Use case**
Importing from Cassandra dump that have `Map` or string value that contains double quote from [dsbulk](https://docs.datastax.com/en/dsbulk/docs/reference/dsbulk-cmd.html) output, they are dumping to non standard CSV format (maybe to make it harder to move to another database).
they have output like this:
```
-- from Map
"{\"key\":\"value\"}"
-- from String cell value that have double quote inside it
"\"key\":\"value\""
```
**Describe the solution you'd like**
`SET format_csv_backslash_escaping=1`
will parse `\"` as normal `""` CSV (that inserted as `"` raw string or if targeting `Map(String, String)` it would parsed and inserted properly.
**Describe alternatives you've considered**
do text transformation before piping dsbulk output into gzip `tr '\\' '"'`, this one works but can only targeting `String`, cannot be inserted into `Map(String,String)` with error:
```
Column 6, name: locations, type: Map(String, String), parsed text: "{SD.42a0dfe9:src}"ERROR
```
**Additional context**
trying to move from Cassandra to Clickhouse
Contributor guide
Assessment
This issue has not been assessed yet.