dotnet / dotnet/machinelearning
Please Expose ParquetDataLoader add implement ParquetWriter
Open
area-Core
enhancement
Priority:2
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
Apache Parquet is a popular data format in the industry. It is used in ScikitLearn, Spark and a many other ML and big-data related software. Currently ParquetReader is an internal class that is not exposed to end users.
Feature Request:
- Please provide ParquetReader to ML.NET users
- Please implement ParquetWriter to ML.NET users.
- Update Parquet.NET used by the framework to the latest as it offers bug fixes and perf improvements.
This would significantly simplify integration with various other ML and ETL processes.
This would also provide a good industry-standard data interop between ML.NET and other ML and data tools, as an alternative to CSVs.
Contributor guide
Assessment
This issue has not been assessed yet.