Remote builds infrequently hang without build ID
- Dominant language
- HTML
- Stars
- 155
- Forks
- 346
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 36
Description
**Summary**
In some cases (approx 2%) attempts to launch remote jobs via Parameterized Remote Trigger results in a hang.
**Key Symptoms**
1. The AQA_Test_Pipeline_JCK job displays many instances of "Waiting for 240 seconds until next poll."
2. The AQA_Test_Pipeline_JCK job makes no reference to "Remote build started", or a build ID, for a given test target.
3. The AQA_Test_Pipeline_JCK job runs forever (one instance ran for 150+ hours), or until terminated by the user.
**Links**
- https://ci.adoptium.net/job/AQA_Test_Pipeline_JCK/435/consoleFull
- Note: Only the sanity target had this issue. Extended and Special ran without issue.
- https://www.jenkins.io/doc/pipeline/steps/Parameterized-Remote-Trigger/
- [Existing issues for parameterized-remote-trigger-plugin](https://issues.jenkins.io/browse/JENKINS-22231?jql=project%20%3D%20%2210172%22%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22In%20Review%22%2C%20%22Fixed%20but%20Unreleased%22)%20AND%20component%20%3D%20parameterized-remote-trigger-plugin)
- Note: No hang issues that resemble what we're seeing.
- https://github.com/jenkinsci/parameterized-remote-trigger-plugin/blob/master/src/main/java/org/jenkinsci/plugins/ParameterizedRemoteTrigger/RemoteBuildConfiguration.java
**Short-term workaround**
I propose to put a timeout block around the triggerRemoteJob call in aqaTestPipeline.groovy. Say, 4 days?
If timeout then kill and retry.
**Long-term fix**
Some form of upstream fix into parameterized-remote-trigger-plugin. Bob suggests that one cause for this problem could be in [QueueItemData.java](https://github.com/jenkinsci/parameterized-remote-trigger-plugin/blob/master/src/main/java/org/jenkinsci/plugins/ParameterizedRemoteTrigger/remoteJob/QueueItemData.java):
This is a well-known Jenkins core race. When a build transitions from BUILDABLE → LEFT (executor assigned), the queue item is garbage-collected from Jenkins' internal queue after a short TTL (~5 minutes). There is a window — measurable under load or slow agent handshakes — where the item's status is LEFT but the executable block hasn't been flushed to the API yet. In QueueItemData.update():
```
if (isLeft()) {
JSONObject remoteJobInfo = queueResponse.getJSONObject("executable");
if (!(remoteJobInfo.isNullObject())) {
buildNumber = remoteJobInfo.getInt("number");
buildURL = new URL(remoteJobInfo.getString("url"));
}
if (buildNumber != 0 && buildURL != null) status = QueueItemStatus.EXECUTED;
// ↑ if executable is null/absent, status stays LEFT, but RemoteBuildInfo never advances
}
```
Contributor guide
Research direction
Start with the triggerRemoteJob call in aqaTestPipeline.groovy and the linked AQA_Test_Pipeline_JCK console output; compare the polling behavior when no build ID appears. Then read QueueItemData.java and the Parameterized Remote Trigger documentation, focusing on the LEFT state and missing executable data. Done means the remote job cannot hang indefinitely when a build ID is unavailable, with the affected pipeline behavior covered by verification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy, java
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100