apache / apache/hugegraph-toolchain
[Bug] 边shard不均导致backup很慢
- Dominant language
- Java
- Stars
- 121
- Forks
- 125
- Avg merge
- 10d 10h
- Merged PRs (30d)
- 3
Description
### Bug Type (问题类型)
performance (性能下降)
### Before submit
- [X] 我已经确认现有的 [Issues](https://github.com/apache/hugegraph/issues) 与 [FAQ](https://hugegraph.apache.org/docs/guides/faq/) 中没有相同 / 重复问题 (I have confirmed and searched that there are no similar problems in the historical issue and documents)
### Environment (环境信息)
- Server Version: 1.0.0 (Apache Release Version)
- Backend: RocksDB x nodes, HDD or SSD
- OS: xx CPUs, xx G RAM, Ubuntu 2x.x / CentOS 7.x
- Data Size: xx vertices, xx edges
### Expected & Actual behavior (期望与实际表现)
执行边的备份,发现非常慢(点备份很快),命令为:
`bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph backup -t edge --directory ./backup_test`
通过源码发现问题应该是边shard不均匀,导致大部分的线程执行完各自的小分片之后退出,最后只会有1~2个线程在执行较大的分片。虽然配置了多线程,但最后只有一个线程在执行备份,其他线程均退出。

也打印了线程池的活跃情况,一开始全部活跃,到最后只有2个活跃。(忽略线程数与备份生成的文件数不一致的情况,修改过线程数)


### Vertex/Edge example (问题点 / 边数据举例)
_No response_
### Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the reported `bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph backup -t edge --directory ./backup_test` command and compare edge and vertex backup behavior. Inspect the backup path and shard workload using the thread-activity evidence in the issue; done means edge backup no longer leaves most threads idle and completes faster.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- cli, databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100