killme2008 / killme2008/Metamorphosis

metaSpout过大数据不准确

Open
#100 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.3k
Forks
672
PR merge metrics
No merged PRs in 30d

Description

@killme2008
我们一个分析日志的应用,会统计日志里面的某个字段的出现次数,最后将结果存入数据库。数据分区开了10个。
1、在设置metaSpout的时候设置了48个。测试数据发送了100条,结果数据比实际数据多出一部分,多次测试,数据不一样但都不准确
2、其他不变条件不变,减少metaSpout的数量(测试了10,11),结果准确。

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the log-counting case with 10, 11, and 48 metaSpout instances against 10 data partitions, using the 100-message test described in the issue. Trace the counting and database-write path to identify why larger metaSpout counts produce varying totals; done means repeated runs match the 100-message input exactly.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.