Improve regeneration of sqlite expected test suite
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Thanks to some great work from @Omega359 as part of each commit to main DataFusion runs many thousand queries from the sqlite test suite ❤
This is documented here:
https://github.com/apache/datafusion/blob/d47f7fb00f8ce175885ce6a59b8361f4ef8a93e1/datafusion/sqllogictest/README.md?plain=1#L218-L252
When expected output changes (for example, error messages) we currently use a script https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/regenerate_sqlite_files.sh that:
1. It relies on a fork of sqlogictest
2. It relies on a modified driver program
This is problematic because
1. As the code in datafusion is updated (for example to new sqlogictest versions) the modified driver program may not work with the new sqlogictest version
2. The modified driver program may not work with new versions of sqllogictest
This happened with https://github.com/apache/datafusion/pull/14824 which made it hard to update the expected output
### Describe the solution you'd like
I would like to make sure that `regenerate_sqlite_files.sh` will always work and will not bitrot over time
@Omega359 says:
> That is exactly what I was thinking and hopefully will fix tonight. I think a decent short-term fix is to 'lock' the sqllogictest-rs dependency version and add a comment that any update to it will require a full run of the regenerate script before committing.
>
> Long term ideally would be to improve my changes to my fork of sqllogictest-rs such that they would be suitable to submit a PR to that project. That is not an insignificant amount of work to be honest and I'm a bit thin on time for the next month or two.
### Describe alternatives you've considered
_No response_
### Additional context
See last time we had to update the scripts based on changes:
- https://github.com/apache/datafusion/pull/14901
- https://github.com/apache/datafusion/pull/14824
Contributor guide
Research direction
Start with datafusion/sqllogictest/regenerate_sqlite_files.sh and the SQLite test-suite regeneration section in datafusion/sqllogictest/README.md. Review the changes and dependency context in PRs 14901 and 14824, then determine how the script can remain compatible with the sqllogictest version used by DataFusion. Done means regeneration works reliably without depending on a modified driver that can break after dependency updates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, shell, sqlite
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100