Communication整理
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
Communication作为作业/任务(组)的状态及统计组件,概念很不清晰,我大概整理了一下,
1, 分类
Communication可分为summary和snapshot,其中summary是速率统计,以一个时间段的起始和结束snapshot统计,summary可进一步分两类,阶段(作业未完成)的summary和最终summary
datax目前只有作业有summary
Communication也可分为作业/任务/任务组,但Communication并不感知
2,reporter
报告器也是概念混乱的组件,字面上理解是上报,即任务的快照上报到任务组,任务组上报到作业。但报告器自身还有记录的功能
我建议,reporter分离出Communication输出功能,支持不同的输出,如,数据库,redis,elasticsearch,log等;输出的策略分summary和snapshot,如,snapshot不需要持久,放在redis即可;中段 summary存入到redis,最终可持久到elasticsearch
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files or tests; start by locating the existing Communication and reporter components and tracing job, task, and task-group reporting. Map how summary and snapshot data are handled, then define the separation and output strategies clearly enough that the resulting behavior and completion criteria can be reviewed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, java, redis
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100