mongodb增量用query reader数据时,当数据量很大时报错Sort exceeded memory limit
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
datax版本:
http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz
mongodb增量用query reader数据时,当数据量很大时报错报错详情:
com.mongodb.MongoQueryException: Query failed with error code 292 and error message 'PlanExecutor error during aggregation :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.'
请问mongodbreader中有地方设置allowDiskUse:true吗?或者有其他方式解决分页排序问题不?
我在源码中未找到相关配置。
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by inspecting the MongoDBReader entry point and its query or pagination handling; the issue does not name specific source files or tests. Reproduce the large-result query and trace where sorting or aggregation is configured. Done means the reader can process the reported large dataset without the memory-limit failure, with a test or documented verification if the repository provides one.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mongodb
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100