awslabs / awslabs/amazon-redshift-utils
SimpleReplay: extract.py gives confusing errors when the selected time range has no data in redshift audit bucket
- Dominant language
- Python
- Stars
- 2.8k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
extract.py gives rather confusing error messages when trying fetch a time range from the Redshift audit log bucket that has no data. Especially when it can take hours until the data arrives to the bucket.
```
python3 extract.py extract.yaml
[INFO] 2021-04-06 11:12:24 Retrieving logs from s3://my-redshift-audit-bucket/
[INFO] 2021-04-06 11:14:55 Exporting 0 transactions (0 queries) to my-workload/Extraction_2021-04-06T11:12:24.210817+00:00
[INFO] 2021-04-06 11:14:55 Generating 0 missing connections.
[INFO] 2021-04-06 11:14:55 Exporting a total of 0 connections to my-workload/Extraction_2021-04-06T11:12:24.210817+00:00
Traceback (most recent call last):
File "extract.py", line 1153, in
config_file["workload_location"] + "/" + extraction_name,
File "extract.py", line 661, in save_logs
connections_dict = connection_time_replacement([connection.__dict__ for connection in sorted_connections])
File "extract.py", line 506, in connection_time_replacement
min_init_time = sorted_connections[0]['session_initiation_time']
IndexError: list index out of range
```
Contributor guide
Research direction
Start with extract.py and reproduce the issue using the documented `python3 extract.py extract.yaml` command against a selected range with no audit data. Trace `save_logs` into `connection_time_replacement`, where the traceback shows the empty connection list failing. Done means an empty range produces a clear, useful message instead of an IndexError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- data, databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100