jenkinsci / jenkinsci/ec2-plugin
[JENKINS-69540] Windows Agents getting disconnected after jenkins restart
- Dominant language
- Java
- Stars
- 292
- Forks
- 709
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 4
Description
EC2 plugin version - 2.0.1
Windows Agents are hanging at the following after Jenkins restart
Sep 07, 2022 12:08:47 AM hudson.plugins.ec2.EC2Cloud
INFO: Copying remoting.jar to: c:/windows/temp
We tried debugging, but logs didn't indicate anything. This happens 100% of the time and only to windows agents.
After deleting the Agent and recreating new one, it is able to connect to the master. Please note that Windows agents are hanging after Jenkins restart and they cannot reattach.
We haven't noticed same issue happening for Linux agents.
Agent config:
EC2 (EC2 Agent) - WinAgent (i-xxxxxxxxx)
C:\jenkins
1
EXCLUSIVE
-1
180
windows-buildnode
i-xxxxxxxxx
c:/windows/temp
admin
WinAgent
java
false
180
false
jenkins_slave_type
demand_WinAgent
Name
Jenkins-WinAgent
jenkins_server_url
https://jenkins-master.example.com/jenkins/
ec2-EC2 Agent
22
-1
Default
true
false
1
2147483647
Logs from the Agent:
Sep 07, 2022 12:10:53 AM hudson.remoting.SynchronousCommandTransport$ReaderThread run
INFO: I/O error in channel channel
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2681)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862)
at java.io.ObjectInputStream.(ObjectInputStream.java:358)
at hudson.remoting.ObjectInputStreamEx.(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Any help will be highly appreciated.
Thanks in advance!
---
Originally reported by todori438, imported from: Windows Agents getting disconnected after jenkins restart
Raw content of original issue
EC2 plugin version - 2.0.1
Windows Agents are hanging at the following after Jenkins restart
Sep 07, 2022 12:08:47 AM hudson.plugins.ec2.EC2Cloud
INFO: Copying remoting.jar to: c:/windows/tempWe tried debugging, but logs didn't indicate anything. This happens 100% of the time and only to windows agents.
After deleting the Agent and recreating new one, it is able to connect to the master. Please note that Windows agents are hanging after Jenkins restart and they cannot reattach.
We haven't noticed same issue happening for Linux agents.
Agent config:
<hudson.plugins.ec2.EC2Slave plugin="ec2@2.0.1">
<name>EC2 (EC2 Agent) - WinAgent (i-xxxxxxxxx)</name>
<remoteFS>C:\jenkins</remoteFS>
<numExecutors>1</numExecutors>
<mode>EXCLUSIVE</mode>
<retentionStrategy class="hudson.plugins.ec2.EC2RetentionStrategy">
<nextCheckAfter>-1</nextCheckAfter>
<idleTerminationMinutes>180</idleTerminationMinutes>
</retentionStrategy>
<launcher class="hudson.plugins.ec2.ssh.EC2UnixLauncher"/>
<label>windows-buildnode</label>
<nodeProperties/>
<instanceId>i-xxxxxxxxx</instanceId>
<initScript/>
<tmpDir>c:/windows/temp</tmpDir>
<remoteAdmin>admin</remoteAdmin>
<templateDescription>WinAgent</templateDescription>
<javaPath>java</javaPath>
<jvmopts/>
<stopOnTerminate>false</stopOnTerminate>
<idleTerminationMinutes>180</idleTerminationMinutes>
<isConnected>false</isConnected>
<tags class="linked-list">
<hudson.plugins.ec2.EC2Tag>
<name>jenkins_slave_type</name>
<value>demand_WinAgent</value>
</hudson.plugins.ec2.EC2Tag>
<hudson.plugins.ec2.EC2Tag>
<name>Name</name>
<value>Jenkins-WinAgent</value>
</hudson.plugins.ec2.EC2Tag>
<hudson.plugins.ec2.EC2Tag>
<name>jenkins_server_url</name>
<value>https://jenkins-master.example.com/jenkins/</value>
</hudson.plugins.ec2.EC2Tag>
</tags>
<cloudName>ec2-EC2 Agent</cloudName>
<amiType class="hudson.plugins.ec2.UnixData">
<rootCommandPrefix/>
<slaveCommandPrefix/>
<slaveCommandSuffix/>
<sshPort>22</sshPort>
<bootDelay/>
</amiType>
<maxTotalUses>-1</maxTotalUses>
<tenancy>Default</tenancy>
<metadataEndpointEnabled>true</metadataEndpointEnabled>
<metadataTokensRequired>false</metadataTokensRequired>
<metadataHopsLimit>1</metadataHopsLimit>
<launchTimeout>2147483647</launchTimeout>
</hudson.plugins.ec2.EC2Slave>
Logs from the Agent:
Sep 07, 2022 12:10:53 AM hudson.remoting.SynchronousCommandTransport$ReaderThread run
INFO: I/O error in channel channel
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2681)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)Any help will be highly appreciated.
Thanks in advance!
Contributor guide
Research direction
Start by reproducing the restart sequence for the Windows EC2 agent and inspect the EC2Cloud log at “Copying remoting.jar to: c:/windows/temp” alongside the hudson.remoting channel error. Compare the Windows behavior with the reported Linux behavior. Done means Windows agents reconnect after a Jenkins restart without deleting and recreating the agent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100