opensearch-project / opensearch-project/sql

[BUG] "DATE_NANOS fieldName types are currently not supported." for join operation

Open
#3,422 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

error-experience field types support PPL
Dominant language
Java
Stars
176
Forks
229
Avg merge
2d 21h
Merged PRs (30d)
43

Description

What is the bug?
According to the supported types page it says that date_nanos type is supported. However, when I write a SELECT where I refer to a field with type "timestamp" I receive an exception in return saying that DATE_NANOS is not supported, like so

OpenSearchException({'error': {'reason': 'There was internal problem at backend', 'details': 'DATE_NANOS fieldName types are currently not supported.', 'type': 'IllegalArgumentException'}, 'status': 500})

The index in question was produced by Data Prepper writing trace information to OpenSearch. The query was a join that looked like this:

SELECT L.body,T.startTime FROM otel-v1-apm-span-000010 T JOIN mylogs L ON L.spanId=T.spanId;

, where "mylogs" is an index with log entries also written by Data Prepper, which has matching spanId and traceId information.

This does NOT happen for a simple non-join query like:

SELECT startTime FROM otel-v1-apm-span-000010;

, so strangely it seems like it is the join that triggers this exception to happen.

How can one reproduce the bug?
This can probably be done in simpler manners, but my method was to let Data Prepper write one index with trace information and one index with log information, and after that try a join based on spanId for instance, where you select the startTime from the trace as one of the fields.

What is the expected behavior?
It should just return the startTime, similar to how it is returned on a direct non-join query.

What is your host/environment?

  • OS: ubuntu linux
  • Version 24.04
  • Plugins OpenSearch SQL, OpenSearch Dashboard, OpenSearch

Do you have any screenshots?

Do you have any additional context?
Opensearch, Opensearch Dashboard and Data Prepper were installed with helm in kubernetes.
Versions were:
$ helm list -n tracestudy
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
data-prepper-release tracestudy 1 2025-03-12 08:07:10.940609 +0100 CET deployed data-prepper-0.3.1 2.8.0
opensearch-dashboard tracestudy 1 2025-03-03 09:18:38.3799 +0100 CET deployed opensearch-dashboards-2.28.0 2.19.1
opensearch-deployment tracestudy 1 2025-03-03 08:44:44.309739 +0100 CET deployed opensearch-2.32.0 2.19.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source files or tests are named. First reproduce the JOIN query selecting startTime alongside the direct non-join query, then trace the SQL JOIN handling for DATE_NANOS fields. Done means the join returns startTime without the unsupported-type exception and existing behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.