alibaba / alibaba/DataX

同步starrock数据到es 速度限制问题

Open
#2,324 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

起了100个channel,同步速度最大也才1w3/s 有什么办法可以提升这个性能吗

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue mentions a StarRocks-to-Elasticsearch sync using 100 channels, but names no files, tests, or entry points. Start by reproducing the reported throughput of about 13,000 records per second and identify the limiting component. Done should mean a documented, measurable performance improvement with the same workload.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, java
Domain
data-engineering, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.