line / line/decaton

Cleanup CompactionProcessor metrics upon subscription close

Open
#97 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
373
Forks
61
PR merge metrics
No merged PRs in 30d

Description

refs: https://github.com/line/decaton/pull/95#issuecomment-797296705

- In principle, we close every Decaton's internal metrics by implementing https://github.com/line/decaton/blob/master/processor/src/main/java/com/linecorp/decaton/processor/metrics/Metrics.java#L64
- But `CompactionProcessor` is an exception
* Since there's no way to pass `subscriptionId` upon `CompactionProcessor` instantiation, `CompactionProcessor`'s metrics are registered every time per task, so `AbstractMetrics` isn't useable
- Though we confirmed it doesn't cause much overhead
- Possible fix for this will be like below:
* Make `CompactionProcessor` constructor to receive subscriptionId
- Or It'd better to pass information which is available on the processor scope (partition, threadId) in addition to subscriptionId
* Add `ProcessorsBuilder#thenProcess` overload that receives `supplier` of type `Function`

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with processor/src/main/java/com/linecorp/decaton/processor/metrics/Metrics.java at the referenced lifecycle implementation, then inspect CompactionProcessor and ProcessorsBuilder#thenProcess. Determine which scope information must be passed so CompactionProcessor metrics can be closed on subscription close; done means the API and processor behavior consistently clean up those metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka
Domain
backend, distributed-systems, observability
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.