apache / apache/spark-connect-go

Add Support for Additional DataFrame Functionality

Open
#58 10 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue
Dominant language
Go
Stars
252
Forks
53
PR merge metrics
No merged PRs in 30d

Description

The DataFrame still has lots of missing functions that need to be implemented. This is the umbrella item for that.

Most of them should be relatively mechanical, but some require some additional work.

|Method|Support|
|---|---|
|`agg`|:white_check_mark:|
|`alias`|:white_check_mark:|
|`approxQuantile`|:white_check_mark:|
|`cache`|:white_check_mark:|
|`checkpoint`|:x:|
|`coalesce`|:white_check_mark:|
|`colRegex`|:white_check_mark: (`OfDFWithRegex`)|
|`collect`|:white_check_mark: |
|`columns`|:white_check_mark:|
|`corr`|:white_check_mark:|
|`count`|:white_check_mark:|
|`cov`|:white_check_mark:|
|`createGlobalTempView`|:white_check_mark:|
|`createOrReplaceGlobalTempView`|:white_check_mark:|
|`createOrReplaceTempView`|:white_check_mark:|
|`createTempView`|:white_check_mark: |
|`crossJoin`|:white_check_mark:|
|`crosstab`|:white_check_mark:|
|`cube`|:white_check_mark:|
|`describe`|:white_check_mark:|
|`distinct`|:white_check_mark:|
|`drop`|:white_check_mark:|
|`dropDuplicates`|:white_check_mark:|
|`dropDuplicatesWithinWatermark`|:x:|
|`dropna`|:white_check_mark:|
|`dtypes`|:x:|
|`exceptAll`|:white_check_mark:|
|`executionInfo`|:x:|
|`explain`|:white_check_mark:|
|`fillna`|:white_check_mark:|
|`filter`|:white_check_mark: |
|`first`|:white_check_mark:|
|`foreach`|:heavy_minus_sign: (needs native UDFs)|
|`foreachPartition`|:heavy_minus_sign: (needs native UDFs)|
|`freqItems`|:white_check_mark:|
|`groupBy`|:white_check_mark: (some restrictions apply)|
|`head`|:white_check_mark:|
|`hint`|:x:|
|`inputFiles`|:x:|
|`intersect`|:white_check_mark:|
|`intersectAll`|:white_check_mark:|
|`isEmpty`|:white_check_mark:|
|`isLocal`|:x:|
|`isStreaming`|:x:|
|`is_cached`|:x:|
|`join`|:white_check_mark:|
|`limit`|:white_check_mark:|
|`localCheckpoint`|:x:|
|`mapInArrow`|:heavy_minus_sign: (needs native UDFs)|
|`mapInPandas`|:heavy_minus_sign: (needs native UDFs)|
|`melt`|:white_check_mark:|
|`mergeInto`|:x:|
|`na`|:white_check_mark:|
|`observe`|:x:|
|`offset`|:white_check_mark:|
|`orderBy`|:white_check_mark:|
|`pandas_api`|:heavy_minus_sign:|
|`persist`|:white_check_mark:|
|`printSchema`|:x:|
|`randomSplit`|:white_check_mark:|
|`rdd`|:heavy_minus_sign:|
|`registerTempTable`|:x:|
|`repartition`|:white_check_mark: |
|`repartitionByRange`|:white_check_mark: |
|`replace`|:white_check_mark:|
|`rollup`|:white_check_mark:|
|`sameSemantics`|:white_check_mark:|
|`sample`|:white_check_mark:|
|`sampleBy`|:x:|
|`schema`|:white_check_mark:|
|`select`|:white_check_mark:|
|`selectExpr`|:white_check_mark:|
|`semanticHash`|:white_check_mark:|
|`show`|:white_check_mark: |
|`sort`|:white_check_mark:|
|`sortWithinPartitions`|:white_check_mark:|
|`sparkSession`|:x:|
|`stat`|:white_check_mark:|
|`storageLevel`|:white_check_mark:|
|`subtract`|:white_check_mark:|
|`summary`|:white_check_mark:|
|`tail`|:white_check_mark:|
|`take`|:white_check_mark:|
|`to`|:x:|
|`toArrow`|:white_check_mark:|
|`toDF`|:x:|
|`toJSON`|:x:|
|`toLocalIterator`|:x:|
|`toPandas`|:heavy_minus_sign:|
|`transform`|:x:|
|`union`|:white_check_mark:|
|`unionAll`|:white_check_mark:|
|`unionByName`|:white_check_mark:|
|`unpersist`|:white_check_mark:|
|`unpivot`|:white_check_mark:|
|`where`|:white_check_mark:|
|`withColumn`|:white_check_mark:|
|`withColumnRenamed`|:white_check_mark:|
|`withColumns`|:white_check_mark:|
|`withColumnsRenamed`|:white_check_mark:|
|`withMetadata`|:white_check_mark:|
|`withWatermark`|:white_check_mark:|
|`write`|:white_check_mark: |
|`writeStream`|:x:|
|`writeTo`|:x:|

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.