h2oai / h2oai/datatable

fread support for common big data formats

Open
#3,307 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.9k
Forks
164
Avg merge
7h 31m
Merged PRs (30d)
1

Description

This is a feature request to implement AVRO and ORC reads into datatable. The ORC read function would be akin to the Pandas implementation:

```python
df = pd.read_orc("my_orc_file")
```

To my knowledge, there is currently no fast way to read an avro file into Pandas. The fastest way I am familiar with is:

```python
import fastavro
with open(file, "rb") as f:
reader = fastavro.reader(f)
records = [r for r in reader]
df = pd.DataFrame.from_records(records)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.