apache / apache/arrow

[C++][IPC][Tensor] Incorrect SparseTensorIndexCSF IPC format

Open
#47,613 12 comments 0 reactions 0 assignees View on GitHub
Component: C++ Component: Documentation Component: Other Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

In the following schema file, it is assumed that there is a single type for all `indptr` buffers and another single type for all `indices` buffers. However, in the C++ implementation, each `indptr` and `indices` buffer can have its own type.

**Schema files**
[https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/format/SparseTensor.fbs#L160](https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/format/SparseTensor.fbs#L160)
[https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/format/SparseTensor.fbs#L179](https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/format/SparseTensor.fbs#L179)

**C++ implementation**
Note that each tensor is associated with its own type:
[https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/cpp/src/arrow/sparse\_tensor.h#L445-L447](https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/cpp/src/arrow/sparse_tensor.h#L445-L447)

In the C++ IPC implementation, however, the type of the first `indices` and `indptr` buffer is generalized to all:
[https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/cpp/src/arrow/ipc/metadata\_internal.cc#L1152-L1153](https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/cpp/src/arrow/ipc/metadata_internal.cc#L1152-L1153)
[https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/cpp/src/arrow/ipc/metadata\_internal.cc#L1158-L1159](https://github.com/apache/arrow/blob/d803afcc43f5d132506318fd9e162d33b2c3d4cd/cpp/src/arrow/ipc/metadata_internal.cc#L1158-L1159)

### Component(s)

C++, Documentation, Other

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.