pingcap / pingcap/tiflash

Replace the JSON_VALID workaround with general short-circuit evaluation

Open
#11,086 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/enhancement
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Enhancement

#11036 fixed queries combining JSON_VALID with JSON operations in filter conditions. It detects a preceding JSON_VALID guard and allows the matching string-to-JSON cast to substitute JSON null for invalid input, so those rows can later be filtered out.

This is a targeted workaround rather than short-circuit evaluation: the guarded expression subtree still executes. Its scope is limited to recognized filter patterns, leaving cases such as projection expressions, OR conditions, and invalid dynamic JSON paths uncovered.

For example:

SELECT id FROM t
WHERE JSON_VALID(doc)
  AND JSON_EXTRACT(doc, json_path) IS NOT NULL;

If a row contains both invalid JSON and an invalid JSON path, tolerating the JSON cast error is insufficient: subsequent path evaluation can still fail. When JSON_VALID(doc) is false, the entire right-hand expression should be skipped for that row.

Replace this workaround with general short-circuit evaluation for AND, OR, and two_value_and, adapting ClickHouse's deferred expression evaluation approach to TiFlash. Support filters and projections, preserve TiDB NULL semantics, and retain errors from expressions that must execute.

Evaluation should follow the expression order supplied by TiDB.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the workaround described in #11036 and ClickHouse's deferred expression evaluation approach. Trace how TiFlash evaluates AND, OR, and two_value_and in filters and projections, then verify expression order, TiDB NULL semantics, invalid JSON-path behavior, and retained errors. Done means guarded right-hand expressions are skipped where appropriate without changing expressions that must execute.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.