apache / apache/arrow

[C++][Dataset] Allow creating FileSystemDataset with FileInfoGenerator as a source

Open
#32,667 1 comment 0 reactions 0 assignees View on GitHub
Type: task
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

**Reporter**: [Pavel Solodovnikov](https://issues.apache.org/jira/browse/ARROW-17396) / @ManManson

**Note**: *This issue was originally created as [ARROW-17396](https://issues.apache.org/jira/browse/ARROW-17396). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by locating the C++ implementations and tests for FileSystemDataset creation, then trace how FileInfoGenerator is currently represented as a source. The issue is complete when a FileInfoGenerator can be accepted for creating a FileSystemDataset and the relevant behavior is covered by tests; the issue provides no file or test paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.