infiniflow / infiniflow/ragflow
[Feature Request]: I hope to implement a batch parsing queue
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 91k
- Forks
- 10.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 705
Description
Self Checks
- I have searched for existing issues search for existing issues, including closed ones.
- I confirm that I am using English to submit this report (Language Policy).
- Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
- Please do not modify this template :) and fill in all the required fields.
Is your feature request related to a problem?
Describe the feature you'd like
I hope to implement a batch parsing queue with a maximum concurrency of 10. Any tasks exceeding this limit should enter a waiting queue for sequential processing. This is necessary because some APIs restrict concurrent requests, and parsing takes time (e.g., when running overnight). A queuing mechanism is therefore essential.
Describe implementation you've considered
No response
Documentation, adoption, use case
Additional information
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by locating how parsing tasks are submitted and how concurrent parsing is handled, then determine where a maximum concurrency of 10 and a waiting queue belong. Done should mean excess tasks wait and are processed sequentially without exceeding the concurrency limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100