Add/improve tracing in the dataset writer
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
### Describe the enhancement requested
The telemetry code in the dataset writer currently does not trace the asynchronous tasks that are submitted to the I/O thread pool that perform the actual encoding, compression, and writing.
Follow-up to https://github.com/apache/arrow/pull/33738, where @westonpace noted: "Mentally, when I think of the dataset writer, I think there are two parts. The first part should be the trailing part of the fragment/pipeline that feeds the writer. In this first part we partition the batch, select the appropriate file queues, and deposit the batches into the queues. There is then a separate dedicated thread task to write each batch to the writer."
### Component(s)
C++
Contributor guide
Assessment
This issue has not been assessed yet.