jorgebastida / jorgebastida/awslogs

JSONDecodeError in apparently well formatted JSON

Open
#256 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5k
Forks
342
PR merge metrics
No merged PRs in 30d

Description

Hi, I'm getting the following `JSONDecodeError` but as far as I'm concerned this JSON is not ill formatted, therefore I shouldn't be getting the error I'm getting and shouldn't be connected with #246 or #248, am I missing something?

This is the error reported by `awslogs`:

```Version: 0.11.0
Python: 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0]
boto3 version: 1.7.47
Platform: Linux-4.15.0-52-generic-x86_64-with-Ubuntu-18.04-bionic
Config: {'aws_access_key_id': 'SENSITIVE', 'aws_secret_access_key': 'SENSITIVE', 'aws_session_token': 'SENSITIVE', 'aws_profile': 'SENSITIVE', 'aws_region': None, 'log_group_name': '/aws/elasticbeanstalk/predict-api-prod/var/log/eb-docker/containers/eb-current-app/stdouterr.log', 'log_stream_name': 'ALL', 'filter_pattern': None, 'watch': True, 'watch_interval': 1, 'output_group_enabled': True, 'output_stream_enabled': True, 'output_timestamp_enabled': False, 'output_ingestion_time_enabled': False, 'start': '5m', 'end': None, 'color': 'auto', 'query': 'EVENT', 'func': 'list_logs'}
Args: ['/home/user/.local/bin/awslogs', 'get', '/aws/elasticbeanstalk/myebs-env/var/log/eb-docker/containers/eb-current-app/stdouterr.log', 'ALL', '--watch', '--timestamps', '--query=EVENT']

Traceback (most recent call last):
File "/home/user/.local/lib/python3.6/site-packages/awslogs/bin.py", line 178, in main
getattr(logs, options.func)()
File "/home/user/.local/lib/python3.6/site-packages/awslogs/core.py", line 223, in list_logs
consumer()
File "/home/user/.local/lib/python3.6/site-packages/awslogs/core.py", line 206, in consumer
parsed = json.loads(event['message'])
File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 37 column 25 (char 1486)

```
The [`structlog`](http://www.structlog.org/en/stable/) log that reproduces this issue is:

```
{
"CALLER": "api.resources.post+880",
"EVENT": "Sending retrieved score to LP",
"HAPPENED_AT": "[+0000] 02/07/2019 09:56:16:059991",
"LEVEL": "INFO",
"application_scores": [
{
"applicationId": "1312558q184849",
"pred_app_score": 5.0
},
{
"applicationId": "1312558q184850",
"pred_app_score": 5.0
},
{
"applicationId": "1312558q184851",
"pred_app_score": 5.0
},
{
"applicationId": "1312558q184852",
"pred_app_score": 5.0
}
],
"bucket": 2.0,
"bucket_label": "HIGH",
"bucket_reason": "Based on prediction score.",
"candidate_id": 1312558,
"job_id": 44105,
"lp_url": "https://dtest.launchpadrecruitsapp.com/predict/scores/receive",
"model_id": 485,
"score": 5.0
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at awslogs/core.py, where consumer() passes event['message'] to json.loads(), and compare that input with the supplied structlog payload and traceback. Reproduce the failure with the reported command and determine why this message reaches the parser; done means the cause and expected handling of this input are established.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.