opensearch-project / opensearch-project/sql

[BUG] Duplicate fields returned when disable_objects set

Open
#5,746 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug PPL
Dominant language
Java
Stars
176
Forks
229
Avg merge
2d 21h
Merged PRs (30d)
43

Description

What is the bug?

Fields are duplicated in search results when objects have disable_objects set to true.

How can one reproduce the bug?

  1. Create the index with disable_objects: true, index a document and query using the PPL API
PUT test-disable-objects
{
  "mappings": {
    "properties": {
        "attributes": {
          "disable_objects": true,
          "type": "object"
        }
      }
  }
}

POST test-disable-objects/_doc
{
  "attributes": {
    "log.file.path": "/var/log/app.log"
  }
}

POST _plugins/_ppl
{
  "query" : "source=test-disable-objects"
}
  1. This returns both an attributes struct field and a log.file.path string field:
{
  "schema": [
    {
      "name": "attributes",
      "type": "struct"
    },
    {
      "name": "attributes.log.file.path",
      "type": "string"
    }
  ],
  "datarows": [
    [
      {
        "log": {
          "file": {
            "path": "/var/log/app.log"
          }
        }
      },
      "/var/log/app.log"
    ]
  ],
  "total": 1,
  "size": 1
}

What is the expected behavior?

Fields should not be duplicated

What is your host/environment?

Opensearch 3.8.0

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?

This causes fields to be duplicated in the Opensearch Dashboards Explore Logs app:

Image

Interestingly, there is no attributes.logtag in that screenshot, which suggests the bug only presents itself when there are multiple layers of nesting.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the documented reproduction through the _plugins/_ppl API with an index using disable_objects: true, then trace how nested fields are assembled into the schema and data rows. Done means the attributes struct and nested attributes.log.file.path are not returned as duplicate fields, including the multi-level nesting case.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend-api-design, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.