jenkinsci / jenkinsci/remoting

[JENKINS-5703] Exception leaves zombie processes for slaves started by command on master

Open
#985 7 comments 0 reactions 0 assignees View on GitHub
component:remoting imported-jira-issue priority:major resolution:unresolved
Dominant language
Java
Stars
262
Forks
288
Avg merge
8h 35m
Merged PRs (30d)
4

Description

I have several slaves that are started via the "Launch slave via execution of command on the Master." The command is a bash script that acquires Kerberos credentials and then ssh's over to the slave. Periodically, something happens that kills all the connections launched this way. Regardless, the original bash process is not collected by Hudson, and is left as a zombie or defunct process.

The relevant excerpt from the hudson log appears to be:

```
Feb 17, 2010 12:43:14 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel
java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:852)

```

I wonder if the fix is something as simple as adding a proc.destroy() to CommandLauncher.java?:



@​Override

public void onClosed(Channel channel, IOException cause) {
if (cause != null) {
cause.printStackTrace(
listener.error(hudson.model.Messages.Slave_Terminated(getTimestamp())));
}
ProcessTree.get().killAll(proc, cookie);
proc.destroy();
}

---
Originally reported by jsiirola, imported from: Exception leaves zombie processes for slaves started by command on master


  • status: Open
  • priority: Major
  • component(s): remoting
  • resolution: Unresolved
  • votes: 8
  • watchers: 8
  • imported: 2025-11-25

Raw content of original issue

I have several slaves that are started via the "Launch slave via execution of command on the Master." The command is a bash script that acquires Kerberos credentials and then ssh's over to the slave. Periodically, something happens that kills all the connections launched this way. Regardless, the original bash process is not collected by Hudson, and is left as a zombie or defunct process.

The relevant excerpt from the hudson log appears to be:



Feb 17, 2010 12:43:14 PM hudson.remoting.Channel$ReaderThread run

SEVERE: I/O error in channel <host>
java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:852)

I wonder if the fix is something as simple as adding a proc.destroy() to CommandLauncher.java?:



@Override

public void onClosed(Channel channel, IOException cause) {
if (cause != null) {
cause.printStackTrace(
listener.error(hudson.model.Messages.Slave_Terminated(getTimestamp())));
}
ProcessTree.get().killAll(proc, cookie);
proc.destroy();
}

1 attachment

- [javapath.png](https://issues.jenkins.io/secure/attachment/30781/javapath.png)
> ![javapath.png](https://issues.jenkins.io/secure/attachment/30781/javapath.png)

Contributor guide

Open the contributing guide

Research direction

Start with CommandLauncher.java and its onClosed(Channel, IOException) entry point, then compare the EOFException sequence in the remoting log with the process cleanup shown in the issue. Investigate the existing ProcessTree cleanup and proc lifecycle; done means a channel failure no longer leaves the command-launch process as a zombie or defunct process.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.