matrixorigin / matrixorigin/matrixone
[Feature Request]: support native ASOF JOIN with key and tolerance for time-series workloads
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Summary
MatrixOne has no native `ASOF JOIN` (nearest preceding temporal join). This is a core time-series operation used to attach the latest known state, calibration, quote, metadata, or configuration row to each event.
The semantics can be emulated with an inequality join plus `ROW_NUMBER`, but that plan materializes all qualifying historical candidates per event and sorts them. It is not an equivalent execution capability for high-frequency time-series workloads.
## Environment
- Repository: `matrixorigin/matrixone`
- Branch: `main`
- Commit: `c3ba9c7df14388956cb38d78681c6c35b75c42b8`
- Host: Debian x86_64, `10.222.1.55`
- SQL endpoint: `127.0.0.1:6001`
- Server version: `8.0.30-MatrixOne-v`
## Example workload
```sql
CREATE TABLE readings (
event_ts TIMESTAMP(6),
device VARCHAR(20),
value DOUBLE
);
CREATE TABLE device_config (
device VARCHAR(20),
effective_ts TIMESTAMP(6),
firmware VARCHAR(20)
);
INSERT INTO readings VALUES
('2026-01-01 00:02:10', 'd1', 20),
('2026-01-01 00:04:20', 'd1', 40);
INSERT INTO device_config VALUES
('d1', '2026-01-01 00:00:00', 'v1'),
('d1', '2026-01-01 00:03:00', 'v2');
```
Desired operation: for every reading, match the latest configuration row for the same device whose `effective_ts <= event_ts`.
## Native syntax is rejected
```sql
SELECT r.device, r.event_ts, r.value, c.firmware
FROM readings r
ASOF JOIN device_config c
ON r.device = c.device
AND r.event_ts >= c.effective_ts
ORDER BY r.device, r.event_ts;
```
Observed:
```text
ERROR 1064 (HY000): SQL parser error ... near "ASOF JOIN device_config"
```
A correlated `OUTER APPLY (... ORDER BY effective_ts DESC LIMIT 1)` is also not available:
```text
ERROR 20101 (HY000): internal error: must apply a table function
```
## Current relational emulation
```sql
SELECT device, event_ts, value, firmware
FROM (
SELECT r.device, r.event_ts, r.value, c.firmware,
ROW_NUMBER() OVER (
PARTITION BY r.device, r.event_ts
ORDER BY c.effective_ts DESC
) AS rn
FROM readings r
LEFT JOIN device_config c
ON r.device = c.device
AND c.effective_ts <= r.event_ts
) x
WHERE rn = 1
ORDER BY device, event_ts;
```
This returns the correct small-data result:
```text
d1 2026-01-01 00:02:10 20 v1
d1 2026-01-01 00:04:20 40 v2
```
A tolerance can be emulated by adding another predicate:
```sql
AND c.effective_ts >= DATE_SUB(r.event_ts, INTERVAL 2 MINUTE)
```
However, the generic plan must join each event to every qualifying historical row and then rank/sort the candidates.
## Expected capability
Support a native temporal predecessor join with:
- equality keys such as `device`, `symbol`, or a composite series key;
- left timestamp and right effective timestamp;
- inclusive predecessor semantics (`right_ts <= left_ts`);
- optional strict predecessor semantics (`right_ts < left_ts`);
- optional maximum lookback/tolerance;
- deterministic tie handling for equal right-side timestamps;
- `TIMESTAMP(6)` precision and session-time-zone correctness;
- efficient execution over time-ordered or time-clustered inputs without candidate explosion.
Illustrative syntax:
```sql
SELECT ...
FROM readings r
ASOF LEFT JOIN device_config c
ON r.device = c.device
AND r.event_ts >= c.effective_ts
TOLERANCE INTERVAL 2 MINUTE;
```
The exact syntax is open for design.
## Professional time-series comparison
Native ASOF/predecessor joins are first-class operations in professional time-series systems for prevailing-value enrichment. QuestDB, for example, supports keyed `ASOF JOIN` with an optional `TOLERANCE` and uses the ordered time dimension for execution.
## Impact
Without a native predecessor join, common workloads such as trades-to-quotes, sensor-to-calibration, event-to-current-configuration, and telemetry-to-deployment-version either:
- create a potentially large many-to-many intermediate result;
- require a full per-event rank/sort;
- or move stateful temporal enrichment out of the database.
The generic SQL workaround proves semantic expressibility, but not production-grade time-series execution completeness.
Contributor guide
Assessment
This issue has not been assessed yet.