Large query to /executions fails in mongodb
@Kami is already working on this.
Since Dec 14, 2016.
- Dominant language
- Python
- Stars
- 6.5k
- Forks
- 787
- PR merge metrics
- No merged PRs in 30d
Description
The indexes added in 2.1 helped with a similar error but now I'm getting the following using production data that did not reproduce using 2k test executions.
My use case is that I need to query all the parent executions (status, action name, arguments, start/stop time) in the past 24 hours where the action name matches a substring. This allows me to throttle/deny additional requests from users based on a set of policies.
2016-12-13 22:03:54,425 140362311938768 INFO hooks [-] e02fb208-4202-4fee-bd77-1e4d7fe356d8 - GET /executions with filters={u'limit': u'2000', u'exclude_attributes': u'result,trigger_instance', u'timestamp_gt': u'2016-12-12T22:03:54.403458Z', u'parent': u'null'} (remote_addr='127.0.0.1',method='GET',filters={u'exclude_attributes': u'result,trigger_instance', u'limit': u'2000', u'parent': u'null', u'timestamp_gt': u'2016-12-12T22:03:54.403458Z'},request_id='e02fb208-4202-4fee-bd77-1e4d7fe356d8',path='/executions')
[..]
2016-12-13 22:03:54,430 140362311938768 INFO resource [-] GET all /executions with filters={'start_timestamp__gt': datetime.datetime(2016, 12, 12, 22, 3, 54, 403458, tzinfo=tzutc()), 'order_by': ['+start_timestamp', 'action.ref'], 'parent': u'null'} (offset=0,limit=2000,filters={'start_timestamp__gt': datetime.datetime(2016, 12, 12, 22, 3, 54, 403458, tzinfo=tzutc()), 'order_by': ['+start_timestamp', 'action.ref'], 'parent': u'null'},sort=[])
2016-12-13 22:03:54,578 140362311938768 ERROR hooks [-] API call failed: errmsg: "Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit."
Traceback (most recent call last):
File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/pecan/core.py", line 631, in __call__
self.invoke_controller(controller, args, kwargs, state)
File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/pecan/core.py", line 531, in invoke_controller
result = controller(*args, **kwargs)
File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/rbac/decorators.py", line 61, in func_wrapper
return func(*args, **kwargs)
File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/models/api/base.py", line 284, in callfunction
raise e
OperationFailure: errmsg: "Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit." (_exception_data={'_OperationFailure__code': 96, '_OperationFailure__details': {u'ok': 0.0, u'code': 96, u'waitedMS': 0L, u'errmsg': u'errmsg: "Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit."'}},_exception_class='OperationFailure',_exception_message='errmsg: "Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit."')
Is it worth increasing internalQueryExecMaxBlockingStortBytes or is there a missing index that would help here? Or is there an index that is not being updated often enough?
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.