danilop / danilop/SampleBatchProcessing

Alarm sounding before message deleted

Đang mở
#3 3 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Shell
Star
48
Fork
8
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

I believe this issue is more of a lack of understanding on my part than clarity in the docs. I've configured a batch processing setup similar to this. Here's what has been happening:

1. Message is sent to the queue.
2. Alarm sounds and starts instance.
3. Get_Jobs.py pulls down the message a t=0.
4. The message changes from "Messages Available" to "Messages In Flight".
5. Get_Jobs.py begins processing the message.
6. At t=2, the message is still in "Messages In Flight" and is currently being processed on the EC2 instance but the alarm sounds and kills the EC2 instance.

It looks like the alarm is sounding 2 minutes after the message is no longer available. This would mean any job would need to take less than 2 minutes to complete or it's at risk of being killed by the alarm.

What do I need to change? Ideally, a message is pulled from the queue and added to an instance. That instance isn't killed until two minutes after the message has been deleted, not two minutes after the message is retrieved.

For context, the messages can take anywhere from 4 to 6 hours to process - I've set the visibilitytimeout to 8 hours. Does this mean the cloudwatch alarm has to wait the same amount as the VisibilityTimeout?

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.