opensearch-project / opensearch-project/sql

[BUG] Query anonymization doesn't work in fallback log

Open
#2,803 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug catch-all acknowledged SQL
Dominant language
Java
Stars
176
Forks
229
Avg merge
2d 21h
Merged PRs (30d)
43

Description

What is the bug?
Query anonymization is not working in the fallback log

How can one reproduce the bug?
Write a simple query and add ?format=json to trigger falling back to old engine.

POST /_plugins/_sql?format=json
{
  "query" : """
  SELECT min(DistanceMiles), avg(DistanceMiles), max(DistanceMiles)
    FROM opensearch_dashboards_sample_data_flights
      GROUP BY Carrier
  """
}

Or write a query which is unsupported in V2, e.g JOIN

POST /_plugins/_sql
{
  "query" : """
  SELECT s.FlightNum, s.OriginCityName, d.DestCityName
    FROM opensearch_dashboards_sample_data_flights s
      JOIN opensearch_dashboards_sample_data_flights d
        ON s.DestCityName = d.OriginCityName
  """
}

See this line in the logs

Request SQLQueryRequest(jsonContent={"query":"\n SELECT min(DistanceMiles),avg(DistanceMiles),max(DistanceMiles) FROM opensearch_dashboards_sample_data_flights GROUP BY Carrier\n "}, query=
SELECT min(DistanceMiles),avg(DistanceMiles),max(DistanceMiles) FROM opensearch_dashboards_sample_data_flights GROUP BY Carrier
, path=/_plugins/_sql, format=json, params={pretty=true, format=json}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine

Request SQLQueryRequest(jsonContent={"query":"\n SELECT s.FlightNum, s.OriginCityName, d.DestCityName FROM opensearch_dashboards_sample_data_flights s JOIN opensearch_dashboards_sample_data_flights d ON s.DestCityName = d.OriginCityName\n "}, query=
SELECT s.FlightNum, s.OriginCityName, d.DestCityName FROM opensearch_dashboards_sample_data_flights s JOIN opensearch_dashboards_sample_data_flights d ON s.DestCityName = d.OriginCityName
, path=/_plugins/_sql, format=jdbc, params={pretty=true}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine

What is the expected behavior?
Logs should show

Request SQLQueryRequest(query=
SELECT MIN ( identifier ), AVG ( identifier ), MAX ( identifier ) FROM table GROUP BY identifier
, path=/_plugins/_sql, format=json, params={pretty=true, format=json}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine

Request SQLQueryRequest(query=
SELECT identifier, identifier, identifier FROM table s JOIN table d ON identifier = identifier
, path=/_plugins/_sql, format=jdbc, params={pretty=true}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine

What is your host/environment?

  • Version 2.15
  • Plugins SQL

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the documented POST /_plugins/_sql requests, using format=json or an unsupported JOIN to trigger the fallback log. Trace the fallback log's SQLQueryRequest formatting and query anonymization path; done means the log shows anonymized tables, identifiers, and functions rather than the original names and values.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.