opensearch-project / opensearch-project/sql
[BUG] Query anonymization doesn't work in fallback log
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
What is the bug?
Query anonymization is not working in the fallback log
How can one reproduce the bug?
Write a simple query and add ?format=json to trigger falling back to old engine.
POST /_plugins/_sql?format=json
{
"query" : """
SELECT min(DistanceMiles), avg(DistanceMiles), max(DistanceMiles)
FROM opensearch_dashboards_sample_data_flights
GROUP BY Carrier
"""
}
Or write a query which is unsupported in V2, e.g JOIN
POST /_plugins/_sql
{
"query" : """
SELECT s.FlightNum, s.OriginCityName, d.DestCityName
FROM opensearch_dashboards_sample_data_flights s
JOIN opensearch_dashboards_sample_data_flights d
ON s.DestCityName = d.OriginCityName
"""
}
See this line in the logs
Request SQLQueryRequest(jsonContent={"query":"\n SELECT min(DistanceMiles),avg(DistanceMiles),max(DistanceMiles) FROM opensearch_dashboards_sample_data_flights GROUP BY Carrier\n "}, query=
SELECT min(DistanceMiles),avg(DistanceMiles),max(DistanceMiles) FROM opensearch_dashboards_sample_data_flights GROUP BY Carrier
, path=/_plugins/_sql, format=json, params={pretty=true, format=json}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engineRequest SQLQueryRequest(jsonContent={"query":"\n SELECT s.FlightNum, s.OriginCityName, d.DestCityName FROM opensearch_dashboards_sample_data_flights s JOIN opensearch_dashboards_sample_data_flights d ON s.DestCityName = d.OriginCityName\n "}, query=
SELECT s.FlightNum, s.OriginCityName, d.DestCityName FROM opensearch_dashboards_sample_data_flights s JOIN opensearch_dashboards_sample_data_flights d ON s.DestCityName = d.OriginCityName
, path=/_plugins/_sql, format=jdbc, params={pretty=true}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine
What is the expected behavior?
Logs should show
Request SQLQueryRequest(query=
SELECT MIN ( identifier ), AVG ( identifier ), MAX ( identifier ) FROM table GROUP BY identifier
, path=/_plugins/_sql, format=json, params={pretty=true, format=json}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engineRequest SQLQueryRequest(query=
SELECT identifier, identifier, identifier FROM table s JOIN table d ON identifier = identifier
, path=/_plugins/_sql, format=jdbc, params={pretty=true}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine
What is your host/environment?
- Version 2.15
- Plugins SQL
Do you have any screenshots?
If applicable, add screenshots to help explain your problem.
Do you have any additional context?
Add any other context about the problem.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the documented POST /_plugins/_sql requests, using format=json or an unsupported JOIN to trigger the fallback log. Trace the fallback log's SQLQueryRequest formatting and query anonymization path; done means the log shows anonymized tables, identifiers, and functions rather than the original names and values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100