holdenk / holdenk/spark-testing-base

Nice to have more types in DataFrameSuiteBaseLike.approxEquals

Open
#73 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.6k
Forks
357
PR merge metrics
No merged PRs in 30d

Description

At present the following test involving an Array will fail.

val arr1 = Array("a","b")
val arr2 = Array("a","b")
val r1 = Row(arr1)
val r2 = Row(arr2)

```
assert(DataFrameSuiteBase.approxEquals(r1,r2, 0.0))
```

This is because the comparsion would use arr1 != arr2 , which is true.
The comparison should use !arr1.sameElements(arr2)

Maybe DataFrameSuiteBaseLike.approxEquals should also cater for Array[T], where T is String, Float, Long, Double.

Shing

Contributor guide

Open the contributing guide

Research direction

Start in DataFrameSuiteBaseLike.approxEquals, the entry point named in the issue, and inspect how Row values are compared. Reproduce the Array("a", "b") example, then cover the requested Array element types and verify that equivalent arrays compare equal in the relevant test suite.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.