StackStorm / StackStorm/st2

MongoDB can only handle up to 8-byte ints

Open
#3,365 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
6.5k
Forks
787
PR merge metrics
No merged PRs in 30d

Description

Hi,
I have an issue with stackstorm.
I configured a webhook and http request should trigger StackStorm action, but there is some parse error.
Seems like its not mongo error, maybe some internal client error.
OverflowError: MongoDB can only handle up to 8-byte ints

  • I replaced some names and ips with "*"

The whole log
From the log: st2rulesengine.log

2017-04-19 09:41:38,818 140175459950832 ERROR consumers [-] StagedQueueConsumer failed to process message: {'trace_context': <st2common.models.api.trace.TraceContext object at 0x7f7d2455a590>, 'trigger': {'uid': u'trigger:core:940aea90-0f2e-4257-b7df-10b0b6994567:39ad46681f6d890b5585141e7a3c4252', 'parameters': {u'url': u'******_one_cluster_node_down_PD'}, 'name': u'940aea90-0f2e-4257-b7df-10b0b6994567', 'type': u'core.st2.webhook', 'id': '58dd19eea18bd533a505d984', 'pack': u'core'}, 'payload': {'body': {'status': 'firing', 'groupLabels': {'cluster': 'none', 'alertname': '******_one_cluster_node_down_PD_ST', 'instance': '*******-******.******.******'}, 'groupKey': 11168737559087072609L, 'commonAnnotations': {'description': '[StackStorm] ****** node is down in cluster ******-*** at datacenter ****** https://***********', 'summary': '[StackStorm] One main ****** cluster node is down ********* at datacenter ******'}, 'alerts': [{'status': 'firing', 'labels': {'datacenter': '******', 'hook_url': '***********', 'target': '*********', 'cluster': 'none', 'environment': 'none', 'instance': '*****-******.******.******', 'job': 'node-******-******', 'servicename': '******-******', 'monitor_source': '******.nydc1', 'alertname': '******_one_cluster_node_down_PD_ST', 'monitor_owner': '******', 'owner': '******-******', 'action': 'none'}, 'endsAt': '0001-01-01T00:00:00Z', 'generatorURL': '*************', 'startsAt': '2017-04-19T08:48:38.59-04:00', 'annotations': {'description': '[StackStorm] ****** node is down in cluster ******-****** at datacenter ****** https://********.******/dashboard/db/ops-data-******-overview?from=now-5m&to=now&var-instance=*******-******.******.******', 'summary': '[StackStorm] One main ****** cluster node is down ******-****** at datacenter ******'}}], 'version': '3', 'receiver': 'webhook_router', 'externalURL': 'http://alert-40005-prod-******.******.******:9093', 'commonLabels': {'datacenter': '******', 'hook_url': 'https://www*********/api/v1/webhooks/******_one_cluster_node_down_PD?st2-api-key=******', 'target': '******', 'cluster': 'none', 'environment': 'none', 'instance': '*****', 'job': 'node-******-******', 'servicename': '******-******', 'monitor_source': '******.nydc1', 'alertname': '******_one_cluster_node_down_PD_ST', 'monitor_owner': '******', 'owner': '******-******', 'action': 'none'}}, 'headers': {'X-Request-Id': '327fb4aa-4b1c-4e99-97b2-a6e89703e85e', 'Accept-Encoding': 'gzip', 'X-Forwarded-For': '************', 'Content-Length': '2610', 'User-Agent': 'Go-http-client/1.1', 'Host': '******.******,*******.******', 'X-Real-Ip': '************', 'Content-Type': 'application/json'}}}
Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/transport/consumers.py", line 85, in process
    response = self._handler.pre_ack_process(body)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2reactor/rules/worker.py", line 56, in pre_ack_process
    raise_on_no_trigger=True)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2reactor/container/utils.py", line 80, in create_trigger_instance
    return TriggerInstance.add_or_update(trigger_instance)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/persistence/base.py", line 173, in add_or_update
    model_object = cls._get_impl().add_or_update(model_object)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/models/db/__init__.py", line 313, in add_or_update
    instance.save()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/mongoengine/document.py", line 340, in save
    object_id = collection.save(doc, **write_concern)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/pymongo/collection.py", line 2445, in save
    check_keys, manipulate, write_concern)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/pymongo/collection.py", line 562, in _insert
    check_keys, manipulate, write_concern, op_id, bypass_doc_val)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/pymongo/collection.py", line 543, in _insert_one
    check_keys=check_keys)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/pymongo/pool.py", line 424, in command
    self._raise_connection_failure(error)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/pymongo/pool.py", line 552, in _raise_connection_failure
    raise error
**OverflowError: MongoDB can only handle up to 8-byte ints**

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with st2reactor/rules/worker.py and st2reactor/container/utils.py, then follow TriggerInstance.add_or_update through st2common/persistence/base.py and st2common/models/db/init.py. Use the webhook log and its payload to trace the value reaching MongoDB; done means the webhook action is processed without the reported OverflowError.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.