apache / apache/datafusion

[Epic] A collection of Substrait conversion issues

Open
#16,248 3 comments 0 reactions 0 assignees View on GitHub
enhancement EPIC
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

This is a follow up on https://github.com/apache/datafusion/issues/15069. A `--substrait-round-trip` flag was added to the sqllogictest runner, which runs the sqllogictest suite with one additional step: when a DataFusion logical plan is produced, it's converted to Substrait and from Substrait back to DataFusion logical before executing it.

This new sqllogictest execution mode has revealed the following issues:

- https://github.com/apache/datafusion/issues/16271
- https://github.com/apache/datafusion/issues/16272
- https://github.com/apache/datafusion/issues/16273
- https://github.com/apache/datafusion/issues/16274
- https://github.com/apache/datafusion/issues/16275
- https://github.com/apache/datafusion/issues/16276
- https://github.com/apache/datafusion/issues/16278
- https://github.com/apache/datafusion/issues/16279
- https://github.com/apache/datafusion/issues/16280
- https://github.com/apache/datafusion/issues/16281
- https://github.com/apache/datafusion/issues/16282
- https://github.com/apache/datafusion/issues/16283
- https://github.com/apache/datafusion/issues/16285
- https://github.com/apache/datafusion/issues/16287
- https://github.com/apache/datafusion/issues/16293
- https://github.com/apache/datafusion/issues/16294
- https://github.com/apache/datafusion/issues/16295
- https://github.com/apache/datafusion/issues/16296
- https://github.com/apache/datafusion/issues/16297
- https://github.com/apache/datafusion/issues/16298

### Describe the solution you'd like

A fix to the mentioned issues so that the sqllogictest in Substrait round-trip mode can run in the CI

Contributor guide

Open the contributing guide

Research direction

Start with the --substrait-round-trip mode in the sqllogictest runner and read the linked child issues to identify an individual conversion failure. Done means the listed Substrait round-trip cases pass and this mode can run in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
backend, testing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.