[FEA] Support string column to struct column (and vice versa) conversions in `read_json`/`write_json`
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
https://github.com/rapidsai/cudf/pull/19307 and https://github.com/rapidsai/cudf/pull/19052 implemented cudf_polars features to convert of column of json-like strings to a struct column and vice versa. The current implementation involves converting/creating a column using a host buffer (a jsonl formatted string) when using `read_json` or `write_json`. It would be great to do this conversion entirely on device instead
1. Convert a string column with json-like strings with `read_json` to a struct column
2. Convert a struct column with `write_json` to a string column with json-like strings
**Describe the solution you'd like**
Sharing an ideal solution discussed with @wence- and @shrshi
> A better approach would be to modify the JSON tokenizer to take in the row offsets of the string column and reset the state of the automata every time we reach the last character of a row. We can process all rows using a single 'segmented' FST without the memory overhead of duplicating the contents of the string column
Contributor guide
Assessment
This issue has not been assessed yet.