snowflakedb / snowflakedb/snowpark-python
SNOW-826851: Improve DataFrameReader and DataFrameWriter API
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 341
- Forks
- 155
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 27
Description
What is the current behavior?
Currently the API for DataFrameReader and DataFrameWriter has several differences with the Spark DataFrame APIs. Also it is a little inconsistent as DataFrameReader has option()/options() and writer does not.
What is the desired behavior?
Provide an API that is more familiar with typical Spark reader/writer code,
How would this improve snowflake-snowpark-python?
that will make it easier for people transitioning or looking to move to snowpark. Also it might enable a more natural API for supporting additional formats
References, Other Background
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the existing DataFrameReader and DataFrameWriter APIs with the corresponding Spark DataFrame APIs, focusing on the option()/options() inconsistency and support for additional formats. Done means a more familiar, consistent reader/writer API for users transitioning to Snowpark, but the issue does not identify files, tests, or a precise API scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100