aws / aws/amazon-ssm-agent

Session manger dropping websocket connection after idling for around 5 minutes on mac

Open
#496 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
1.2k
Forks
357
PR merge metrics
No merged PRs in 30d

Description

Hi

Having issues with ssm dropping connections after being idle for a few minutes. maybe worth noting we use a VPN and shared IP of multiple people; although linux users have no issues

- Mac Ventura 13.1 (22C65) an
- aws --version aws-cli/2.9.13 Python/3.11.1 Darwin/22.2.0 source/arm64 prompt/off
- session-manager-plugin --version 1.2.398.0

command
```
aws ssm start-session \
--region=$AWS_REGION \
--target "$BASTION_INSTANCE_ID" \
--document-name AWS-StartPortForwardingSessionToRemoteHost \
--parameters "{\"portNumber\":[\"$REMOTE_PORT\"],\"localPortNumber\":[\"$LOCAL_PORT\"],\"host\":[\"$DB_HOST\"]}"
```

debug logs
```
2023-01-11 10:26:11 DEBUG An error happened when receiving the message. Retried times: 1, Error: websocket: close 1006 (abnormal closure): unexpected EOF, Messagetype: -1
2023-01-11 10:26:11 DEBUG An error happened when receiving the message. Retried times: 2, Error: websocket: close 1006 (abnormal closure): unexpected EOF, Messagetype: -1
2023-01-11 10:26:11 DEBUG An error happened when receiving the message. Retried times: 3, Error: websocket: close 1006 (abnormal closure): unexpected EOF, Messagetype: -1
2023-01-11 10:26:11 DEBUG An error happened when receiving the message. Retried times: 4, Error: websocket: close 1006 (abnormal closure): unexpected EOF, Messagetype: -1
2023-01-11 10:26:11 ERROR Reach the retry limit 5 for receive messages.
2023-01-11 10:26:11 ERROR Trying to reconnect the session: wss://ssmmessages.eu-west-1.amazonaws.com/v1/data-channel/notareal@email.com-0012f2843d51b1de6?role=publish_subscribe&cell-number=AAEAAXxVmmlUO9az0SCRpKGhs5fz5YUIwUv6w9W4xkT7g43PAAAAAGO+jlKhUFIv9ZcQq52bZU0pumK0xlG6doxUlG8jYWTDAHr9fg== with seq num: 5
2023-01-11 10:26:11 DEBUG Resume Session input parameters: {
SessionId: "notareal@email.com-0012f2843d51b1de6"
}
2023-01-11 10:26:21 ERROR Resume Session failed: SerializationError: failed to unmarshal response error
status code: 404, request id: 9b8badfa-4e5f-4f48-b9db-c84772b7a3af
caused by: UnmarshalError: failed decoding error message
00000000 3c 55 6e 6b 6e 6f 77 6e 4f 70 65 72 61 74 69 6f |.|

caused by: invalid character '<' looking for beginning of value
```

Contributor guide

Open the contributing guide

Research direction

No source file or test is named. Start by tracing the Go session data-channel WebSocket receive and reconnect path using the supplied aws ssm start-session command and DEBUG logs. Done means the macOS idle disconnect and subsequent resume failure are reproduced, diagnosed, and covered by an appropriate regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go, macos
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.