apache / apache/pinot

Choose which fields get flattened by complexType

Open
#10,549 1 comment 0 reactions 1 assignee Claimed by @jonas-grgt View on GitHub
feature help wanted
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Say we have the following event structure:

```json
{
"...",
"auditFields": {
"a": "b",
"c": "d"
},
"results": [
{
"name": "result1",
"value": 33.2
},
{
"name": "result2",
"value": 45.6
}
]
}
```

We want to create the following columns:

```
results.name
results.value
auditFields
```

At the moment I don't think that's possible as auditFields will be flattened into:

```
audiFields.a
auditFields.c
```

By this line of code - https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java#L176

Can we make the columns that get flattened configurable, like the fields to unnest can be configured?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.