apache / apache/arrow

[Python][C++] Segmentation fault on serialize custom_metadata of type string

Open
#47,816 3 comments 0 reactions 0 assignees View on GitHub
Component: C++ Component: FlightRPC Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

First of all thank you for this great library!

We are hitting a `Fatal Python error: Segmentation fault` if try to read a GET_STREAM with custom-metadata.

## Setup

1. Given a (Java) Arrow Server which on GET requests also sets custom metadata on the listener using `OutboundStreamListener::putMetadata(ArrowBuf metadata)`.
The custom-metadata is a basic String.

2. Given an pyArrow Flight client:
```
get = client.do_get(ticket)
get.read_all() # raises a Segmentation Fault
```

## Analysis

Looking at the stacktrace we could link it to the C++ method `Message::MessageImpl:Open()`.

We could verify, that even with only a single empty batch the issue occured.
In our Arrow Server implementation, we send a custom-metadata of type `String`.
The C++ Impl reads as if the custom metadata is expected to be a Map though? ([code)](https://github.com/apache/arrow/blob/be6dddf228f73dafc34dafbe1343dc6a6680eb97/cpp/src/arrow/ipc/message.cc#L105).

Omitting the String typed metadata resolves the issue.
We also know, the Java Arrow client implementation can also handle arbitrary custom metadata.

## Question

1. What format is allowed for custom metadata? Can it be also a simple string or should it always be Map?
2. If it should always be a Map, could the Python/C++ impl throw a better error message?

cc @DarthMax

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.