fread support for common big data formats
Open
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 164
- Avg merge
- 7h 31m
- Merged PRs (30d)
- 1
Description
This is a feature request to implement AVRO and ORC reads into datatable. The ORC read function would be akin to the Pandas implementation:
```python
df = pd.read_orc("my_orc_file")
```
To my knowledge, there is currently no fast way to read an avro file into Pandas. The fastest way I am familiar with is:
```python
import fastavro
with open(file, "rb") as f:
reader = fastavro.reader(f)
records = [r for r in reader]
df = pd.DataFrame.from_records(records)
```
Contributor guide
Assessment
This issue has not been assessed yet.