snowflakedb / snowflakedb/snowpark-python
SNOW-1650888: Add missing transform function for snowpark dataframe
@sfc-gh-yixie is already working on this.
Since Sep 9, 2024.
- Dominant language
- Python
- Stars
- 341
- Forks
- 155
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 27
Description
What is the current behavior?
transform function is not available for snowpark dataframe whereas its available in spark dataframe. Customers are using that function a lot and it would be better to add this method to this library.
What is the desired behavior?
Add transform funciton to snowpark dataframe class so its available when we migrate customer code to snowpark.
How would this improve snowflake-snowpark-python?
By adding this function, it will allow migrating customer code to snowpark directly without additional rewrite.
Increases SMA code compatibility.
References, Other Background
Sample code :
def transform(self, func: Callable[..., "DataFrame"], *args: Any, **kwargs: Any) -> "DataFrame":
result = func(self, *args, **kwargs)
return result
Equivalent link from spark lib - https://spark.apache.org/docs/latest/api/python/_modules/pyspark/sql/dataframe.html#DataFrame.transform
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.