apache / apache/datafusion-sqlparser-rs
Time travel support
- Dominant language
- Rust
- Stars
- 3.5k
- Forks
- 772
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 17
Description
Several databases and data lakes support time travel which allows historical versions of a table to be queried. Regrettably, syntax and capabilities do not seem to be terribly consistent across the various implementations.
## Querying Historical Times
This option lets a database be queried as it existed at a certain point in time. (*Question:* Is this "last version prior to the given time"? or is it an error if no snapshot exists with that exact time?)
### [Databricks](https://docs.databricks.com/gcp/en/delta/history#delta-time-travel-syntax)
```
SELECT * FROM people10m TIMESTAMP AS OF '2018-10-18T22:15:12.013Z';
```
### [Snowflake](https://docs.snowflake.com/en/sql-reference/constructs/at-before)
```
SELECT * FROM my_table AT(TIMESTAMP => 'Wed, 26 Jun 2024 09:20:00 -0700'::TIMESTAMP_LTZ);
```
### [Cockroach](https://www.cockroachlabs.com/blog/time-travel-queries-select-witty_subtitle-the_future/)
```
SELECT * FROM t AS OF SYSTEM TIME '2016-06-15 12:45:00'
```
### [DuckLake](https://ducklake.select/docs/stable/duckdb/usage/time_travel.html)
```
SELECT * FROM tbl AT (TIMESTAMP => now() - INTERVAL '1 week');
```
## Querying Specific versions
This options lets a specific "version" of a database to be queried. What exactly this means seems to depend on the database implementation in question.
### Databricks
```
SELECT * FROM people10m VERSION AS OF 123;
```
### Snowflake
Note: unclear if this is the same thing (specific versions) or different
```
SELECT * FROM my_table BEFORE(STATEMENT => '8e5d0ca9-005e-44e6-b858-a8f5b37c5726');
```
### Cockroach
Unsupported
### DuckLake
```
SELECT * FROM tbl AT (VERSION => 3);
```
Contributor guide
No contributing guide indexed for this repository
Research direction
No repository files, tests, or entry points are identified in the issue. Start by comparing the Databricks, Snowflake, Cockroach, and DuckLake syntax and resolve whether historical-time and version queries should share semantics or remain dialect-specific. Done should include an agreed scope for the supported syntax and behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100