apache / apache/druid

Ineffective "segmentGranularity" with customized "duration granularity"

Open
#9,894 7 comments 0 reactions 0 assignees View on GitHub
Area - Streaming Ingestion Bug
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version

0.17.0

### Description

Ingest data using kafka indexing service, I set param "segmentGranularity" with: `{"type": "duration", "duration": 86400000, "origin": "1970-01-01T18:19:00.000Z"}`, I hopped that segment time interval from "2020-05-19T18:19:00.000Z" to "2020-05-20T18:19:00.000Z"

But segment intervals seemed to ignore the `origin` param:

![image](https://user-images.githubusercontent.com/17106104/82325363-b7f8a700-9a0d-11ea-9ec0-cf5ddc273ce0.png)

![image](https://user-images.githubusercontent.com/17106104/82325330-a7e0c780-9a0d-11ea-809a-8e7f51727ad2.png)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Kafka indexing service configuration from the report with duration segmentGranularity and its origin value. Trace how the duration and origin are handled when segment intervals are created, then verify that the resulting intervals begin at the requested origin and span the configured duration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.