awslabs / awslabs/amazon-redshift-utils

replay_target_cluster_status_check failing and output getting truncated

Open
#587 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.8k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

The replay_target_cluster_status_check runs successfully several times and then fails. Both the step function logs, and the Lambda logs are getting truncated, so it is difficult to see the issue. Seems like there may be some limit to the amount of data that gets pushed. I have attached a couple logs and screen shots to show you what we are seeing.

[error.log](https://github.com/awslabs/amazon-redshift-utils/files/7476048/error.log)
![cloudwatchlambda](https://user-images.githubusercontent.com/66700407/140357232-0c7ccd18-a36a-4bcb-b3a3-58ad6bfdb793.jpg)
![stepfunctionexception](https://user-images.githubusercontent.com/66700407/140357262-36833c13-5f45-4192-9b19-4f14fc1b4b78.jpg)
[lambdaerror.log](https://github.com/awslabs/amazon-redshift-utils/files/7476050/lambdaerror.log)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the attached error.log and lambdaerror.log alongside the replay_target_cluster_status_check Step Functions and Lambda logs. Trace the repeated successful runs through the eventual failure and determine the relevant data-size limit; done means the check no longer fails after repeated runs and its logs are no longer truncated.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.