class Xhgui_Profiles method "paginate" return so large data
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 1.7k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
Now I saw my mongo db in results collection have more than 400W record.
when the paginate find record for the url list, it return so large data and cost more times.
the url: /url/view?url=* is always give the error:

1: the most fields in the document is not to need. I give a parameter to the find method made it return the need fields then the find speed is better.

2: the other resaon
the method "getPercentileForUrl", it use "aggregate", it also take so many times.
but now I don't konw how to Optimize it, I think should write a crontab script excute everty day, call this function and then save the result in another collection, then find the result from the new collection, it will passed the aggregate's Calculate time, there just select time.
or use ajax call getPercentileForUrl to graph.
any other idea?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the Xhgui_Profiles paginate method and its getPercentileForUrl aggregation for the /url/view?url=* entry point. Reproduce the slow query against a large MongoDB results collection and determine a bounded or precomputed result; done means the URL view avoids excessive data retrieval and aggregation time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100