apache / apache/datafusion-comet

Improve `checkSparkAnswer` methods to allow for validation of expected results

Open
#2,692 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

### What is the problem the feature request solves?

When writing tests using `checkSparkAnswerAndOperator`, the testing API doesn't provide a convenient way to verify that the query actually produces the ~correct~ expected results, or any results at all.

I would like to improve the API to make it easy to improve our tests to verify results above and beyond just making sure that Comet and Spark produced the same results.

### Describe the potential solution

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

No file or test path is provided; begin by locating the checkSparkAnswer and checkSparkAnswerAndOperator definitions and their current call sites in the testing API. Done means tests can conveniently validate expected query results, including whether results exist, in addition to comparing Comet and Spark.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.