[JENKINS-76151] Continuous attempts to connect to the stopped instance

Open

#1,979 opened on Sep 26, 2025 4 comments 0 reactions 0 assignees View on GitHub

component:ec2-plugin imported-jira-issue priority:major resolution:unresolved

Dominant language: Java
Stars: 292
Forks: 709
Avg merge: 3d 16h
Merged PRs (30d): 4

Description

My usual flow is to launch EC2 instance, run some jobs, and then stop it after idle timeout, not terminate. This allows to run next builds faster because all the dependencies are already downloaded. So when next job appears instance is launched, build is run, and then stopped again.

With the latest plugin (2034.v0a_11fb_792b_ee) instance is launched, job is run and instance is stopped after idle timeout, but then it changes its status to "launching..." and stays like this forever, new jobs are not accepted. In EC2 instance stays in Stopped state. Maybe this change in last build has something to do with this behavior - #1142

Logs from jenkins:

Sep 25, 2025 12:21:02 PM INFO hudson.plugins.ec2.EC2RetentionStrategy internalCheck
Idle timeout of EC2 (EC2 Nodes) - Ubuntu 24.04 with 66Gb disk and 16Gb swap on it (i-0aa5d718ff507ba1c) after 1 idle minutes, instance statusRUNNING
Sep 25, 2025 12:21:02 PM INFO hudson.plugins.ec2.EC2AbstractSlave idleTimeout
EC2 instance idle time expired: i-0aa5d718ff507ba1c
Sep 25, 2025 12:21:03 PM INFO hudson.plugins.ec2.EC2AbstractSlave stop
EC2 instance stop request sent for i-0aa5d718ff507ba1c
Sep 25, 2025 12:22:03 PM WARNING hudson.plugins.ec2.EC2RetentionStrategy attemptReconnectIfOffline
EC2Computer EC2 (EC2 Nodes) - Ubuntu 24.04 with 66Gb disk and 16Gb swap on it (i-0aa5d718ff507ba1c) is offline
Sep 25, 2025 12:22:03 PM WARNING hudson.plugins.ec2.EC2RetentionStrategy attemptReconnectIfOffline
Attempting to reconnect EC2Computer EC2 (EC2 Nodes) - Ubuntu 24.04 with 66Gb disk and 16Gb swap on it (i-0aa5d718ff507ba1c)
Sep 25, 2025 12:22:03 PM INFO hudson.plugins.ec2.EC2Cloud log
Launching instance: i-0aa5d718ff507ba1c
Sep 25, 2025 12:22:03 PM INFO hudson.plugins.ec2.EC2Cloud log
bootstrap()
Sep 25, 2025 12:22:03 PM INFO hudson.plugins.ec2.EC2Cloud log
Getting keypair...
Sep 25, 2025 12:22:03 PM INFO hudson.plugins.ec2.EC2Cloud log
Using private key devops-key-pair (SHA-1 fingerprint xxx)
Sep 25, 2025 12:22:03 PM INFO hudson.plugins.ec2.EC2Cloud log
Authenticating as ubuntu
Sep 25, 2025 12:22:03 PM INFO hudson.plugins.ec2.EC2Cloud log
Connecting to 10.131.100.147 on port 22, with timeout 10000.
Sep 25, 2025 12:22:17 PM INFO hudson.plugins.ec2.EC2Cloud log
Failed to connect via ssh: DefaultConnectFuture[ubuntu@/10.131.100.147:22]: Failed (NoRouteToHostException) to execute: No route to host
Sep 25, 2025 12:22:17 PM INFO hudson.plugins.ec2.EC2Cloud log
Waiting for SSH to come up. Sleeping 5.
Sep 25, 2025 12:22:22 PM INFO hudson.plugins.ec2.EC2Cloud log
Connecting to 10.131.100.147 on port 22, with timeout 10000.
Sep 25, 2025 12:22:25 PM INFO hudson.plugins.ec2.EC2Cloud log
Failed to connect via ssh: DefaultConnectFuture[ubuntu@/10.131.100.147:22]: Failed (NoRouteToHostException) to execute: No route to host
Sep 25, 2025 12:22:25 PM INFO hudson.plugins.ec2.EC2Cloud log
Waiting for SSH to come up. Sleeping 5.
Sep 25, 2025 12:22:30 PM INFO hudson.plugins.ec2.EC2Cloud log
Connecting to 10.131.100.147 on port 22, with timeout 10000.
Sep 25, 2025 12:22:33 PM INFO hudson.plugins.ec2.EC2Cloud log
Failed to connect via ssh: DefaultConnectFuture[ubuntu@/10.131.100.147:22]: Failed (NoRouteToHostException) to execute: No route to host
Sep 25, 2025 12:22:33 PM INFO hudson.plugins.ec2.EC2Cloud log
Waiting for SSH to come up. Sleeping 5.
Sep 25, 2025 12:22:38 PM INFO hudson.plugins.ec2.EC2Cloud log
Connecting to 10.131.100.147 on port 22, with timeout 10000.
Sep 25, 2025 12:22:42 PM INFO hudson.plugins.ec2.EC2Cloud log
Failed to connect via ssh: DefaultConnectFuture[ubuntu@/10.131.100.147:22]: Failed (NoRouteToHostException) to execute: No route to host
Sep 25, 2025 12:22:42 PM INFO hudson.plugins.ec2.EC2Cloud log
Waiting for SSH to come up. Sleeping 5.

Continues like this forever...

---
Originally reported by shohou, imported from: Continuous attempts to connect to the stopped instance

assignee: jgarciacloudbees
status: In Review
priority: Major
component(s): ec2-plugin
resolution: Unresolved
votes: 5
watchers: 9
imported: 2025-12-06

Raw content of original issue

Logs from jenkins:

Continues like this forever...

environment

```
Jenkins 2.516.3

Amazon EC2 Plugin 2034.v0a_11fb_792b_ee
```

Contributor guide

Open the contributing guide

Research direction

Start by tracing EC2RetentionStrategy.attemptReconnectIfOffline and the EC2AbstractSlave.stop and idleTimeout flow shown in the logs, then inspect EC2Cloud.launch and bootstrap. Reproduce the stopped-instance sequence and verify that a stopped instance does not enter an endless SSH reconnect loop and that new jobs are accepted afterward.

Written by the indexing model from the issue text.

Assessment

Tech stack: aws, java
Domain: cloud, devops
Issue type: Bug
Difficulty: 4/5
Estimated time: 3-5 days
Activity status: Stale
Clarity: Mostly clear
Newbie friendliness: 25/100

[JENKINS-76151] Continuous attempts to connect to the stopped instance

Description

Contributor guide

Research direction

Assessment

Get new issues in your inbox