matrixorigin / matrixorigin/matrixone
[Subtask]: Establish an evidence-based JSON/VARIANT baseline for agent observability workloads
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
### Parent Issue
Parent issue: #23360
### Detail of Subtask
MatrixOne already exposes a `JSON` SQL type backed by a binary `ByteJson` representation, JSON path functions/operators, `UNNEST`, and JSON-capable full-text indexing. However, issue #23360 proposes JSON shredding and accelerated querying without a reproducible workload baseline that shows which semantic or physical gaps require that design.
This subtask establishes an evidence-based JSON/VARIANT capability baseline for semi-structured analytics and agent-observability workloads. It is a validation and decision task. It must not assume that adding a `VARIANT` alias, implementing shredding, or changing the type system is the correct solution before the evidence exists.
#### Claims and invariants
1. **Ingestion integrity**: every accepted NDJSON document is queryable with its required values and types preserved; every rejected document has an explicit error. Silent row loss, truncation, or type corruption negates the claim.
2. **Semi-structured semantics**: missing paths, JSON `null`, SQL `NULL`, mixed scalar types, nested objects, arrays, and schema evolution behave according to a frozen SQL contract. Conflating those states or changing results because object keys are reordered negates the claim.
3. **Query correctness**: filtering, projection, aggregation, unnesting, trace lookup, and text retrieval match an independent expected-result corpus. A query merely completing without an exact/differential oracle does not satisfy the claim.
4. **Evidence completeness**: unsupported queries, timeouts, OOMs, rejected rows, and invalid runs remain visible. Skipping them while reporting a green aggregate result negates the claim.
5. **Mixed workload recoverability**: under bounded concurrent ingestion and query load, acknowledged source records are neither lost nor duplicated, and a restart/retry converges to the same result as the fault-free reference run.
#### Public reachability
All externally visible claims must be exercised through the MySQL protocol and supported public SQL paths:
```text
NDJSON / INSERT / LOAD DATA
-> MatrixOne JSON column
-> JSON path extraction / UNNEST / aggregation / full-text retrieval
-> typed result rows and persisted side effects
```
Package-level `ByteJson` tests are supporting white-box evidence only; they cannot replace the public-path witness.
#### Required work
- [ ] Freeze the MatrixOne commit/image, SQL surface, dataset manifest, query corpus, expected results, topology, and resource fingerprint.
- [ ] Produce a capability matrix for JSON ingestion, path semantics, type fidelity, schema evolution, aggregation, unnesting, and full-text retrieval.
- [ ] Add a small MatrixOne-owned deterministic fixture covering missing/NULL states, mixed types, deep nesting, arrays, large values, key reordering, and path evolution.
- [ ] Map every scored JSONBench and agent-observability query to MatrixOne SQL, after code and dataset licenses are approved.
- [ ] Validate every mapped query against an independent exact, differential, or metamorphic oracle.
- [ ] Record unsupported syntax/capabilities explicitly; do not rewrite dynamic payload fields into predeclared top-level columns merely to make the workload pass.
- [ ] Collect cold/hot runtimes, load acceptance counts, storage footprint, peak RSS, CPU, scan volume, spill, timeout, and OOM classifications.
- [ ] Run a bounded ingestion-plus-query recovery test with retry and service restart.
- [ ] Use plans/profiles to identify whether slow cases are dominated by parsing, full scans, JSON path evaluation, text retrieval, aggregation, spill, or result materialization.
- [ ] Feed the resulting gap analysis back to #23360.
#### Decision output
The final report must distinguish these possible conclusions instead of collapsing them into `VARIANT supported=true`:
- Existing `JSON` is sufficient for the target workload; documentation and best practices are the remaining gaps.
- Existing `JSON` is semantically sufficient, but explicit generated/materialized paths or path indexes are needed.
- Automatic shredding/subcolumnization from #23360 is justified by measured hot-path and scale evidence.
- A genuinely distinct VARIANT logical type is required because the needed value/type semantics cannot be represented by the existing JSON contract.
### Describe implementation you've considered
Start with the smallest black-box correctness fixture and a benchmark adapter. Do not modify production type/storage code in this subtask. If a gap is confirmed, open or update a separate implementation issue with the smallest counterexample, owning boundary, expected hot-path cost, and dedicated regression matrix.
Suggested execution stages:
1. 10K MatrixOne-owned fixture for adapter and semantic debugging.
2. 1M public benchmark tier for complete query correctness.
3. 10M same-host, host-exclusive runs for scale and profile evidence.
4. Distributed 6-24 hour ingestion/query recovery validation.
### Acceptance Criteria
- [ ] Every declared query is `passed`, `unsupported`, `timeout`, `oom`, or `infra_invalid`; no query disappears from the denominator.
- [ ] Passed queries have independent result evidence, not only exit code or runtime.
- [ ] Missing path, JSON null, SQL NULL, type drift, array/object, malformed input, large value, and schema-evolution counterexamples are covered.
- [ ] Benchmark/config/data/MO revisions and environment fingerprints are immutable and recorded.
- [ ] At least three valid same-host runs are retained for performance claims; diagnostic/shared-host results are not promoted to formal evidence.
- [ ] Mixed ingestion/query restart and replay produce the same business result as the no-fault reference.
- [ ] The report states what current JSON proves, what remains unknown, and which measured gaps (if any) justify #23360.
- [ ] User-facing documentation gaps and support-version boundaries are listed.
### Additional information
Related public references:
- #23360
- Apache Doris VARIANT: https://doris.apache.org/docs/dev/key-features/variant-data-type/
- Snowflake semi-structured types: https://docs.snowflake.com/en/sql-reference/data-types-semistructured
- JSONBench: https://github.com/ClickHouse/JSONBench
- AgentLogsBench: https://github.com/velodb/agentlogsbench
License review is a prerequisite for copying benchmark code/data or publishing derived adapters/results. A public repository without an explicit license must not be treated as implicitly reusable.
Contributor guide
Assessment
This issue has not been assessed yet.