[JENKINS-46453] matrix job with dynamic axis aborting builds

When building a matrix job with a large amount of dynamic axis, some of the axis builds will suddenly abort, the log output shows

```
11:10:23 kernel-single-defconfig-builder » multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y,builder appears to be cancelled
11:10:23 kernel-single-defconfig-builder » multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y,builder completed with result ABORTED
```

It only seems to happen to builds that are a "new" axis or havent been built recently (possibly old one cleaned out by discarding old builds), clicking on the hyperlink in the log gives a 404, it's like the new axis doesn't get setup correctly. I had a look on the filesystem and the aborted jobs have no data in the jobs axis-label directories:

 

```
root@​machine:/var/lib/jenkins/jobs/kernel-single-defconfig-builder/configurations/axis-defconfig# ls -alh *RANDOMIZE*/axis-label/
defconfig+CONFIG_RANDOMIZE_BASE=y/axis-label/:
total 12K
drwxr-xr-x 3 jenkins jenkins 4.0K Jul 19 13:54 .
drwxr-xr-x 3 jenkins jenkins 4.0K Mar 27 15:21 ..
drwxr-xr-x 3 jenkins jenkins 4.0K Aug 25 11:08 builder
multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y/axis-label/:
total 8.0K
drwxr-xr-x 2 jenkins jenkins 4.0K Aug 24 18:03 .
drwxr-xr-x 3 jenkins jenkins 4.0K Aug 24 18:03 ..
multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y+CONFIG_RANDOMIZE_BASE=y/axis-label/:
total 8.0K
drwxr-xr-x 2 jenkins jenkins 4.0K Aug 24 18:03 .
drwxr-xr-x 3 jenkins jenkins 4.0K Aug 24 18:03 ..
omap2plus_defconfig+CONFIG_RANDOMIZE_BASE=y/axis-label/:
total 8.0K
drwxr-xr-x 2 jenkins jenkins 4.0K Aug 24 18:03 .
drwxr-xr-x 3 jenkins jenkins 4.0K Aug 24 18:03 ..

```

 

The label used is 'builder' and the first axis that contains this directory built fine but the others were aborted.

Seems to be a repeat of Jenkins-13972 that was closed as fixed without really any code changes or resolution.

I also found this google presentation which would seem to discuss the same issue:

https://docs.google.com/presentation/d/1ybtB-Bhkb4c3dhb5ZMArr4prtEZ-pjLqH9Vk7yhdZTg/edit#slide=id.g2c21d8fdc_00

 

Note that I have a 'staging' setup of the same plugins and jenkins version doing the same build which does not show this problem, but only has 1 builder so they are not executed concurrently.

 

---
Originally reported by mattface, imported from: matrix job with dynamic axis aborting builds

• assignee: kohsuke
  
• status: Open
  
• priority: Major
  
• component(s): matrix-project-plugin
  
• resolution: Unresolved
  
• votes: 1
  
• watchers: 4
  
• imported: 2025-12-06
  

Raw content of original issue

When building a matrix job with a large amount of dynamic axis, some of the axis builds will suddenly abort, the log output shows



11:10:23 kernel-single-defconfig-builder » multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y,builder appears to be cancelled

11:10:23 kernel-single-defconfig-builder » multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y,builder completed with result ABORTED



It only seems to happen to builds that are a "new" axis or havent been built recently (possibly old one cleaned out by discarding old builds), clicking on the hyperlink in the log gives a 404, it's like the new axis doesn't get setup correctly. I had a look on the filesystem and the aborted jobs have no data in the jobs axis-label directories:


 



root@machine:/var/lib/jenkins/jobs/kernel-single-defconfig-builder/configurations/axis-defconfig# ls -alh *RANDOMIZE*/axis-label/

 defconfig+CONFIG_RANDOMIZE_BASE=y/axis-label/:

 total 12K

 drwxr-xr-x 3 jenkins jenkins 4.0K Jul 19 13:54 .

 drwxr-xr-x 3 jenkins jenkins 4.0K Mar 27 15:21 ..

 drwxr-xr-x 3 jenkins jenkins 4.0K Aug 25 11:08 builder

multi_v7_defconfig+CONFIG_RANDOMIZE_BASE=y/axis-label/:

 total 8.0K

 drwxr-xr-x 2 jenkins jenkins 4.0K Aug 24 18:03 .

 drwxr-xr-x 3 jenkins jenkins 4.0K Aug 24 18:03 ..

multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y+CONFIG_RANDOMIZE_BASE=y/axis-label/:

 total 8.0K

 drwxr-xr-x 2 jenkins jenkins 4.0K Aug 24 18:03 .

 drwxr-xr-x 3 jenkins jenkins 4.0K Aug 24 18:03 ..

omap2plus_defconfig+CONFIG_RANDOMIZE_BASE=y/axis-label/:

 total 8.0K

 drwxr-xr-x 2 jenkins jenkins 4.0K Aug 24 18:03 .

 drwxr-xr-x 3 jenkins jenkins 4.0K Aug 24 18:03 ..





 


The label used is 'builder' and the first axis that contains this directory built fine but the others were aborted.


Seems to be a repeat of Jenkins-13972 that was closed as fixed without really any code changes or resolution.

 I also found this google presentation which would seem to discuss the same issue:

 https://docs.google.com/presentation/d/1ybtB-Bhkb4c3dhb5ZMArr4prtEZ-pjLqH9Vk7yhdZTg/edit#slide=id.g2c21d8fdc_00


 


Note that I have a 'staging' setup of the same plugins and jenkins version doing the same build which does not show this problem, but only has 1 builder so they are not executed concurrently.


 

environment

```
jenkins 2.72

matrix project plugin 1.11

installed from debian package - running directly

java-8-openjdk

13 executors labelled for the failing job, label is 'builder'
```

Start by reviewing dynamic-axis handling in the matrix-project-plugin and the axis-label directories described in the report. Reproduce a matrix job with many dynamic axes and concurrent builder executors using the stated environment details; done means newly created or recently cleaned axes no longer abort and their build links resolve instead of returning 404.

Written by the indexing model from the issue text.