jenkinsci / jenkinsci/branch-api-plugin
[JENKINS-59970] Launching two nodes with similar workspace deletes in use workspaces
- Dominant language
- Java
- Stars
- 39
- Forks
- 154
- PR merge metrics
- No merged PRs in 30d
Description
We run Jenkins with configuration as code in two environments:
- Dev - For testing changes to our configuration and jobs
- Prod - Used by all developers
Some jobs are multibranch jobs, and most are normal pipeline jobs.
We recently started seeing the following message pop up in the multibranch job:
ERROR: missing workspace /localdata/jenkins_workspace/workspace/ on
And we have now tracked this down to being caused by a job running in prod while at the same time someone launching Jenkins locally in their Dev environment.
After placing an immutable file in the workspace we got the following output in the Jenkins boot:
WARNING j.b.WorkspaceLocatorImpl$Collector#onOnline: could not delete workspace /localdata/jenkins_workspace/workspace/
So apparently there does not seem to be any sort of check to see if any of the workspaces it decides to delete/clean are actually in use by anyone else.
Given that we are using two separate Jenkins nodes, and they both connect to the same workspace I admit we might be out of scope here, but I wanted to ask if this is something that can be avoided? does the {{workspace/workspaces.txt }}file have any purpose here?
Having these two almost identical dev/prod environments allows us a good workflow of testing changes before pushing to production.
---
Originally reported by martinn_graphcore, imported from: Launching two nodes with similar workspace deletes in use workspaces
Raw content of original issue
We run Jenkins with configuration as code in two environments:
- Dev - For testing changes to our configuration and jobs
- Prod - Used by all developers
Some jobs are multibranch jobs, and most are normal pipeline jobs.
We recently started seeing the following message pop up in the multibranch job:
ERROR: missing workspace /localdata/jenkins_workspace/workspace/<job_name> on <node_name>
And we have now tracked this down to being caused by a job running in prod while at the same time someone launching Jenkins locally in their Dev environment.
After placing an immutable file in the workspace we got the following output in the Jenkins boot:
WARNING j.b.WorkspaceLocatorImpl$Collector#onOnline: could not delete workspace /localdata/jenkins_workspace/workspace/<job_name>
So apparently there does not seem to be any sort of check to see if any of the workspaces it decides to delete/clean are actually in use by anyone else.
Given that we are using two separate Jenkins nodes, and they both connect to the same workspace I admit we might be out of scope here, but I wanted to ask if this is something that can be avoided? does the {{workspace/workspaces.txt }}file have any purpose here?
Having these two almost identical dev/prod environments allows us a good workflow of testing changes before pushing to production.
environment
```
Jenkins 2.190.1
branch-api:2.5.4
```
Contributor guide
Research direction
Start by tracing WorkspaceLocatorImpl$Collector#onOnline and the workspace/workspaces.txt behavior in branch-api-plugin. Reproduce the collision with two Jenkins environments sharing a workspace, and determine whether shared workspaces are supported. Done requires an agreed behavior that prevents loss of an in-use workspace, with regression coverage if a code change is appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100