apache / apache/pinot

Getting list of distinct non singleFieldValue field using select / distinct

Open
#6,307 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Hello,
I am trying to find which values are stored for a specific field.
Works okay with singleField ones, but does not with multiples.

With below schema
```json
{
"schemaName": "metric",
"dimensionFieldSpecs": [
{
"name": "name",
"dataType": "STRING"
},
{
"name": "tags",
"dataType": "STRING",
"singleValueField": false,
"defaultNullValue": "null"
}
],
"metricFieldSpecs": [
{
"name": "value",
"dataType": "LONG",
"defaultNullValue": -1
}
],
"dateTimeFieldSpecs": [
{
"name": "timestampInEpoch",
"dataType": "LONG",
"format" : "1:MILLISECONDS:EPOCH",
"granularity": "1:MILLISECONDS"
}
]
}
```

```
select name from metric group by name
```
gives me list of name field in the table

However,
```
select tags from metric group by tags
```
returns errorCode 200
```json
[
{
"errorCode": 200,
"message": "QueryExecutionError:\njava.lang.RuntimeException: Caught exception while building data table.\n\tat
org.apache.pinot.core.operator.blocks.InstanceResponseBlock.(InstanceResponseBlock.java:46)\n\tat
org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:37)\n\tat
org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:26)\n\tat
org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)\n\tat
org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:48)\n\tat
org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:294)\n\tat
org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:215)\n\tat
org.apache.pinot.core.query.executor.QueryExecutor.processQuery(QueryExecutor.java:60)\n\tat
org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:155)\n\tat
org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:139)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat
shaded.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)\n\tat
shaded.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)"
}
]
```

```
select COUNT(tags) from metric group by tags
```
gives out count of individual tag - I guess

Is there any workaround for this?
Thank you

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the two GROUP BY queries against the schema in the issue and trace the failure from InstanceResponseBlock.java through InstanceResponseOperator.java and ServerQueryExecutorV1Impl.java. Compare the successful single-value query with the multivalue tags query; done means selecting and grouping tags no longer returns errorCode 200 and the resulting distinct values are clearly defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.