[FEA] Attach name metadata to struct types in libcudf
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
libcudf's struct type is a sequence of types. [The Arrow specification additionally requires that fields have a name](https://arrow.apache.org/docs/format/Columnar.html#struct-layout):
> Each field must have a UTF8-encoded name, and these field names are part of the type metadata.
Since struct members are ordered in both libcudf and Arrow representations, the two are effectively equivalent for all algorithms, but the absence of struct field names in libcudf means that any conversion between the two is lossy. The result is that any round-trip conversion with pure libcudf will fail (#17061) and therefore every higher-level API built on top of libcudf must track this information on its own. That is particularly noticeable when using the Arrow C data interface for conversion, where we currently have a custom `column_metadata` type that users have to manually construct and pass to `to_arrow*` functions to fill this gap.
**Describe the solution you'd like**
We should add struct names to libcudf. Whether we add that information directly to the `data_type` or in some other way remains to be seen (keeping `data_type` as a trivially copyable type is quite valuable for maintaining identical host- and device-side type dispatch).
**Describe alternatives you've considered**
We could continue tracking the information in every implementation built on top of libcudf (pandas, polars, Velox, Spark) but not only is that duplicating work, each implementation will have to do this somewhat differently to match the constraints of each engine. Having the information tracked in libcudf itself would solve the problem uniformly for every engine.
**Additional context**
For flexibility or backwards compatibility we could decide that struct field names are optional. I'm not sure if that is a good idea though since I suspect that it will only cause more headaches in the long term over simply requiring a name.
Contributor guide
Assessment
This issue has not been assessed yet.