jorgebastida / jorgebastida/awslogs
Missing data
- Dominant language
- Python
- Stars
- 5k
- Forks
- 342
- PR merge metrics
- No merged PRs in 30d
Description
I was using AwsLogs to request a large volume of data over an extended period and when I repeated the exact same request the resulting logfiles were different. I used something along the lines of the following command:
awslogs get -G --timestamp _log-group_ "_log-stream_-*" --filter-pattern="_node-id_." -s "2021-02-19 11:00" -e "2021-03-08 11:59:59" | grep "_node-id_.[RT]X" > "_node-id_.log"
In one instance all data from 2021-02-26 till 2021-03-02 was missing from one of the logging steps, in another case only 5 minutes of data was missing.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the issue with the reported awslogs get command over the same date range, stream pattern, and filter pattern, comparing repeated outputs. Trace the command's CloudWatch log retrieval path and identify where records may be skipped; done means the missing-data behavior is explained and covered by a regression test or a documented reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cli, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100