awslabs / awslabs/amazon-redshift-utils

SimpleReplay: extract.py gives confusing errors when the selected time range has no data in redshift audit bucket

Open
#546 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.8k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

extract.py gives rather confusing error messages when trying fetch a time range from the Redshift audit log bucket that has no data. Especially when it can take hours until the data arrives to the bucket.

```
python3 extract.py extract.yaml
[INFO] 2021-04-06 11:12:24 Retrieving logs from s3://my-redshift-audit-bucket/
[INFO] 2021-04-06 11:14:55 Exporting 0 transactions (0 queries) to my-workload/Extraction_2021-04-06T11:12:24.210817+00:00
[INFO] 2021-04-06 11:14:55 Generating 0 missing connections.
[INFO] 2021-04-06 11:14:55 Exporting a total of 0 connections to my-workload/Extraction_2021-04-06T11:12:24.210817+00:00
Traceback (most recent call last):
File "extract.py", line 1153, in
config_file["workload_location"] + "/" + extraction_name,
File "extract.py", line 661, in save_logs
connections_dict = connection_time_replacement([connection.__dict__ for connection in sorted_connections])
File "extract.py", line 506, in connection_time_replacement
min_init_time = sorted_connections[0]['session_initiation_time']
IndexError: list index out of range
```

Contributor guide

Open the contributing guide

Research direction

Start with extract.py and reproduce the issue using the documented `python3 extract.py extract.yaml` command against a selected range with no audit data. Trace `save_logs` into `connection_time_replacement`, where the traceback shows the empty connection list failing. Done means an empty range produces a clear, useful message instead of an IndexError.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
data, databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.