perftools / perftools/xhgui

class Xhgui_Profiles method "paginate" return so large data

Open
#93 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
JavaScript
Stars
1.7k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

Now I saw my mongo db in results collection have more than 400W record.
when the paginate find record for the url list, it return so large data and cost more times.

the url: /url/view?url=* is always give the error:
qq20140126181913

1: the most fields in the document is not to need. I give a parameter to the find method made it return the need fields then the find speed is better.
qq20140126181318
2: the other resaon
the method "getPercentileForUrl", it use "aggregate", it also take so many times.
but now I don't konw how to Optimize it, I think should write a crontab script excute everty day, call this function and then save the result in another collection, then find the result from the new collection, it will passed the aggregate's Calculate time, there just select time.
or use ajax call getPercentileForUrl to graph.
any other idea?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the Xhgui_Profiles paginate method and its getPercentileForUrl aggregation for the /url/view?url=* entry point. Reproduce the slow query against a large MongoDB results collection and determine a bounded or precomputed result; done means the URL view avoids excessive data retrieval and aggregation time.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.