snowflakedb / snowflakedb/snowpark-python

SNOW-1650888: Add missing transform function for snowpark dataframe

Open
#2,231 3 comments 0 reactions 1 assignee View on GitHub

@sfc-gh-yixie is already working on this.

Since Sep 9, 2024.

feature status-triage_done
Dominant language
Python
Stars
341
Forks
155
Avg merge
4d 16h
Merged PRs (30d)
27

Description

What is the current behavior?

transform function is not available for snowpark dataframe whereas its available in spark dataframe. Customers are using that function a lot and it would be better to add this method to this library.

What is the desired behavior?

Add transform funciton to snowpark dataframe class so its available when we migrate customer code to snowpark.

How would this improve snowflake-snowpark-python?

By adding this function, it will allow migrating customer code to snowpark directly without additional rewrite.
Increases SMA code compatibility.

References, Other Background

Sample code :

def transform(self, func: Callable[..., "DataFrame"], *args: Any, **kwargs: Any) -> "DataFrame":
	result = func(self, *args, **kwargs)
	return result

Equivalent link from spark lib - https://spark.apache.org/docs/latest/api/python/_modules/pyspark/sql/dataframe.html#DataFrame.transform

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.