apache / apache/arrow

parquet::schema::Node's field_id not being auto-generated

Open
#13,194 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

Hi,

We've been using auto-generated field_ids from `parquet::schema::Node` for querying nested types. When we updated to Arrow 7.0.0, we've noticed this field is not set anymore.

I've checked the discussion in the [PR](https://github.com/apache/arrow/pull/10289) and in the [JIRA ticket](https://issues.apache.org/jira/browse/PARQUET-1798), but I couldn't find the reasoning behind removing auto-generation of field_id. Is there a workaround which we can utilize to identify Nodes with the current implementation?

Thanks!

cc @pitrou and @westonpace

Contributor guide

Open the contributing guide

Research direction

Start at parquet::schema::Node and inspect how field_id is populated in the Arrow 7.0.0 implementation. Review the linked PR 10289 and JIRA ticket PARQUET-1798 for the rationale, then compare the current behavior with the reported auto-generation behavior; done means a confirmed explanation and documented workaround or agreed behavior change.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.