opensearch-project / opensearch-project/sql

[BUG] plugins.calcite.all_join_types.allowed guardrail is inactive on the Analytics Engine path

Open
#5,734 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

analytic-engine bug PPL
Dominant language
Java
Stars
176
Forks
229
Avg merge
2d 21h
Merged PRs (30d)
43

Description

What is the bug?

On the Analytics Engine (unified query) path, the high-cost-join guardrail plugins.calcite.all_join_types.allowed is inactive. High-cost join types that the default (non-AE) pipeline rejects are silently accepted on the AE route.

Root cause

AstBuilder.validateJoinType gates on a non-null read:

https://github.com/opensearch-project/sql/blob/main/ppl/src/main/java/org/opensearch/sql/ppl/parser/AstBuilder.java#L442-L452

private void validateJoinType(Join.JoinType joinType) {
  Object config = settings.getSettingValue(Key.CALCITE_SUPPORT_ALL_JOIN_TYPES);
  if (config != null && !((Boolean) config)) {
    if (Join.highCostJoinTypes().contains(joinType)) {
      throw new SemanticCheckException(...);
    }
  }
}

CALCITE_SUPPORT_ALL_JOIN_TYPES is neither seeded into UnifiedQueryContext.Builder's default settings map nor forwarded by RestUnifiedQueryAction.applyClusterOverrides(). UnifiedQueryContext's Settings implementation resolves an unmapped key to settings.get(key), i.e. null — so config != null is false and the whole check is skipped.

Verified directly against a default AE context:

plugins.calcite.all_join_types.allowed -> null

The cluster-side default is false (OpenSearchSettings.CALCITE_SUPPORT_ALL_JOIN_TYPES_SETTING), meaning high-cost joins are meant to be rejected unless an operator opts in. On the AE path they are always permitted, and setting the flag to false explicitly has no effect either.

How can one reproduce the bug?
  1. Route a query to the Analytics Engine (composite/pluggable-dataformat index, or cluster.pluggable.dataformat=composite).
  2. Leave plugins.calcite.all_join_types.allowed at its default false, or set it explicitly to false.
  3. Run a PPL query using one of Join.highCostJoinTypes().

Expected: SemanticCheckException — "Join type X is performance sensitive. Set plugins.calcite.all_join_types.allowed to true to enable it."
Actual: the query plans and executes.

What is the expected behavior?

The AE path should honor plugins.calcite.all_join_types.allowed the same way the default pipeline does.

Suggested fix

Add Key.CALCITE_SUPPORT_ALL_JOIN_TYPES to RestUnifiedQueryAction.FORWARDED_CLUSTER_SETTINGS.

Note this is a user-visible tightening: queries with high-cost joins that run on AE today would start being rejected unless the operator opts in. That is the correct behavior (it matches the default engine), but it warrants its own PR and release note rather than being folded into an unrelated fix — which is why it was split out of #5611.

Do you have any additional context?

Found while fixing the same class of defect in #5611, which forwards plugins.query.size_limit, the plugins.ppl.pattern.* family, and plugins.ppl.values.max.limit to the AE path. That PR adds a drift guard (everySeededPlanningSettingIsClassified) covering settings the builder seeds; CALCITE_SUPPORT_ALL_JOIN_TYPES is not seeded at all, so it falls outside that guard and needs to be handled explicitly.

Related: #5735 (PPL subsearch maxout settings do not apply on the AE path).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in RestUnifiedQueryAction.applyClusterOverrides() and inspect how UnifiedQueryContext.Builder settings reach AstBuilder.validateJoinType in the Analytics Engine path. Verify coverage for the default and explicitly false plugins.calcite.all_join_types.allowed cases, with high-cost joins rejected on AE as they are on the default pipeline.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.