apache / apache/arrow

[C++] `DictionaryBuilder::InsertMemoValues` should not deduplicate values

Open
#47,151 4 comments 0 reactions 1 assignee Claimed by @egolearner View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Per #47134, duplicate values are tolerable in dictionaries, however the C++ API currently deduplicates memo values even when they are explicitly inserted by the user.

I believe this is the offending line: https://github.com/apache/arrow/blob/main/cpp/src/arrow/array/builder_dict.cc#L85

This becomes an issue when the user thinks they insert N values, but the dictionary ends up containing << N dictionary values. Since the user explicitly called `InsertMemoValues`, they might proceed by directly calling `AppendIndices` in the future - unaware that the dictionary contents do not directly map to the indices the user is appending

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.