apache / apache/auron

EPIC: Enhance Auron's Correctness Testing

Open
#1,745 5 comments 0 reactions 0 assignees View on GitHub
spark test
Dominant language
Rust
Stars
1.8k
Forks
241
Avg merge
2d 12h
Merged PRs (30d)
21

Description

## Background

Auron is a great Native Engine. But we also see some correctness related issues recently:

* https://github.com/apache/auron/issues/1674
* https://github.com/apache/auron/issues/1716
* https://github.com/apache/auron/issues/1680

Apache Gluten(https://github.com/apache/incubator-gluten) faces the similar issue, Gluten has built a great mechanism that runs all the Unit Test bundled with Spark(https://github.com/apache/incubator-gluten/tree/main/gluten-ut), but with native engine enabled. Doing this Gluten can check whether Gluten produces the same result as Vanilla Spark.

I think Auron can copy Gluten's idea to do the similar thing, to enhance the user's confidence to use Auron in production environment.

## Sub Tasks

Adding all the tests of Spark involves a lot work, so we divide them into sub tasks, feel free to take one:

- [x] #1812
- [ ] #1813
- [x] #1814
- [ ] #1815
- [x] #1816
- [x] #1817
- [x] #1818

## Correctness Issues We Found

- [x] #1724
- [ ] #1807
- [ ] #1810
- [ ] #1811
- [ ] #1821

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.