DataTalksClub / DataTalksClub/course-management-platform

[ALERT] cmp-prod-alb-target-5xx

Open
#236 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

infra
Dominant language
Python
Stars
85
Forks
37
PR merge metrics
No merged PRs in 30d

Description

This issue was opened automatically from a CloudWatch alarm notification.

Alarm

Field Value
Name cmp-prod-alb-target-5xx
State ALARM
Changed at 2026-08-31T14:52:54.040+0000
Description ALB target 5xx responses for CMP prod.
Reason Threshold Crossed: 1 datapoint [1.0 (31/08/26 14:45:00)] was greater than or equal to the threshold (1.0).
AWS account 387546586013
Region eu-west-1

Open this alarm in CloudWatch

Metric

Field Value
Namespace AWS/ApplicationELB
Metric HTTPCode_Target_5XX_Count
Dimensions TargetGroup=targetgroup/course-management-prod/164455bdbcc646d8, LoadBalancer=app/course-management-shared/bd279e4cc4f97601
Statistic SUM
Period 300 seconds
Evaluation periods 1
Threshold GreaterThanOrEqualToThreshold 1.0

Investigation

  • Check application and load-balancer logs around the state-change time.
  • Check the current service version at /api/health/.
  • Record the cause and resolution here before closing the issue.
Application logs
Field Value
Log group /ecs/course-management-prod
Suggested window 2026-08-31T14:42:54.040000+00:00 to 2026-08-31T14:57:54.040000+00:00
Investigator role arn:aws:iam::387546586013:role/cmp-alert-investigator

Open CloudWatch Logs Insights, select /ecs/course-management-prod, or use the
cmp-alert-investigator AWS CLI profile described in the infrastructure README:

aws --profile cmp-alert-investigator logs filter-log-events \
  --region eu-west-1 \
  --log-group-name /ecs/course-management-prod \
  --start-time 1788187374040 \
  --end-time 1788188274040 \
  --limit 100

Review log output for sensitive information before copying any excerpt into
this public issue.

Raw CloudWatch notification
{
  "AWSAccountId": "387546586013",
  "ActionExecutedAfterMuteWindow": false,
  "AlarmActions": [
    "arn:aws:sns:eu-west-1:387546586013:course-management-operator-alerts"
  ],
  "AlarmArn": "arn:aws:cloudwatch:eu-west-1:387546586013:alarm:cmp-prod-alb-target-5xx",
  "AlarmConfigurationUpdatedTimestamp": "2026-07-08T22:06:44.752+0000",
  "AlarmDescription": "ALB target 5xx responses for CMP prod.",
  "AlarmName": "cmp-prod-alb-target-5xx",
  "InsufficientDataActions": [],
  "NewStateReason": "Threshold Crossed: 1 datapoint [1.0 (31/08/26 14:45:00)] was greater than or equal to the threshold (1.0).",
  "NewStateValue": "ALARM",
  "OKActions": [],
  "OldStateValue": "OK",
  "Region": "EU (Ireland)",
  "StateChangeTime": "2026-08-31T14:52:54.040+0000",
  "Trigger": {
    "ComparisonOperator": "GreaterThanOrEqualToThreshold",
    "Dimensions": [
      {
        "name": "TargetGroup",
        "value": "targetgroup/course-management-prod/164455bdbcc646d8"
      },
      {
        "name": "LoadBalancer",
        "value": "app/course-management-shared/bd279e4cc4f97601"
      }
    ],
    "EvaluateLowSampleCountPercentile": "",
    "EvaluationPeriods": 1,
    "MetricName": "HTTPCode_Target_5XX_Count",
    "Namespace": "AWS/ApplicationELB",
    "Period": 300,
    "Statistic": "SUM",
    "StatisticType": "Statistic",
    "Threshold": 1.0,
    "TreatMissingData": "notBreaching",
    "Unit": null
  }
}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the application and load-balancer logs for the suggested window, using the cmp-alert-investigator AWS CLI profile described in the infrastructure README. Check the current service version at /api/health/ and review the CloudWatch output for sensitive information. Done means the cause and resolution are recorded here before closing the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
backend, cloud, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.