[JENKINS-65277] git plugin left lock in caches repo, blocked all jobs using that cache

The git plugin left a config.lock file in one of the caches folders The folder was two weeks old and was showing indications that it might have benefited from a git gc.

Because the config.lock file was left in that folder, many jobs failed (hundreds). They could not acquire a configuration lock on that repository. The plugin needs to either be less concerned about acquiring locks when reading the repository configuration or it needs to be more aggressive in cleaning locks and possibly detecting repositories that need to be garbage collected.

Might consider an Admin Monitor to report cached repositories that would benefit from garbage collection along with an operation that would perform garbage collection on repositories that were detected as needing cleanup.

The directory contents looked like this when the lock was held for 7 or more hours:

```
-rw-r--r-- 1 jenkins jenkins 0 Mar 22 11:24 FETCH_HEAD
-rw-r--r-- 1 jenkins jenkins 164 Mar 22 11:24 config
-rw-r--r-- 1 jenkins jenkins 0 Mar 22 11:24 config.lock
drwxr-xr-x 8 jenkins jenkins 4096 Mar 31 19:11 .
jenkins@​bfe5372c3e46:~/caches/git-c3acd33583448548d14667df83b93ee1$ date
Wed 31 Mar 2021 07:12:05 PM MDT

```

The build log using command line git reported:

```
15:04:49 > git config remote.origin.url https://github.com/MarkEWaite/jenkins-bugs.git # timeout=11
15:04:49 hudson.plugins.git.GitException: Command "git config remote.origin.url https://github.com/MarkEWaite/jenkins-bugs.git" returned status code 255:
15:04:49 stdout:
15:04:49 stderr: error: could not lock config file .git/config: File exists
15:04:49
15:04:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608)
15:04:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2538)
15:04:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2534)
15:04:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1920)
15:04:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1932)
15:04:49 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1546)
15:04:49 at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:161)
15:04:49 at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:357)
15:04:49 at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:330)
15:04:49 at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:396)
15:04:49 at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582)
15:04:49 at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:100)
15:04:49 at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)
15:04:49 at hudson.model.ResourceController.execute(ResourceController.java:97)
15:04:49 at hudson.model.Executor.run(Executor.java:429)
15:04:49 Finished: FAILURE

```

The build log using JGit reported:

```
5:03:01 Started by remote host Started from Groovy Console by Mark Waite with note: Groovy console started
15:03:01 Setting origin to https://github.com/MarkEWaite/jenkins-bugs.git
15:03:06 org.eclipse.jgit.errors.LockFailedException: Cannot lock /var/jenkins_home/caches/git-c3acd33583448548d14667df83b93ee1/.git/config. Ensure that no other process has an open file handle on the lock file /var/jenkins_home/caches/git-c3acd33583448548d14667df83b93ee1/.git/config.lock, then you may delete the lock file and retry.
15:03:06 at org.eclipse.jgit.storage.file.FileBasedConfig.save(FileBasedConfig.java:220)
15:03:06 at org.jenkinsci.plugins.gitclient.JGitAPIImpl.setRemoteUrl(JGitAPIImpl.java:945)
15:03:06 Caused: hudson.plugins.git.GitException
15:03:06 at org.jenkinsci.plugins.gitclient.JGitAPIImpl.setRemoteUrl(JGitAPIImpl.java:947)
15:03:06 at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:357)
15:03:06 at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:330)
15:03:06 at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:396)
15:03:06 at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582)
15:03:06 at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:100)
15:03:06 at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)
15:03:06 at hudson.model.ResourceController.execute(ResourceController.java:97)
15:03:06 at hudson.model.Executor.run(Executor.java:429)
15:03:06 Finished: FAILURE

```

---
Originally reported by markewaite, imported from: git plugin left lock in caches repo, blocked all jobs using that cache

• status: Open
  
• priority: Major
  
• component(s): git-plugin
  
• resolution: Unresolved
  
• votes: 0
  
• watchers: 2
  
• imported: 2025-12-02
  

Raw content of original issue

The git plugin left a config.lock file in one of the caches folders  The folder was two weeks old and was showing indications that it might have benefited from a git gc.


Because the config.lock file was left in that folder, many jobs failed (hundreds).  They could not acquire a configuration lock on that repository.  The plugin needs to either be less concerned about acquiring locks when reading the repository configuration or it needs to be more aggressive in cleaning locks and possibly detecting repositories that need to be garbage collected.


Might consider an Admin Monitor to report cached repositories that would benefit from garbage collection along with an operation that would perform garbage collection on repositories that were detected as needing cleanup.


The directory contents looked like this when the lock was held for 7 or more hours:




-rw-r--r--   1 jenkins jenkins      0 Mar 22 11:24 FETCH_HEAD

-rw-r--r--   1 jenkins jenkins    164 Mar 22 11:24 config

-rw-r--r--   1 jenkins jenkins      0 Mar 22 11:24 config.lock

drwxr-xr-x   8 jenkins jenkins   4096 Mar 31 19:11 .

jenkins@bfe5372c3e46:~/caches/git-c3acd33583448548d14667df83b93ee1$ date

Wed 31 Mar 2021 07:12:05 PM MDT





The build log using command line git reported:




15:04:49   > git config remote.origin.url https://github.com/MarkEWaite/jenkins-bugs.git # timeout=11

15:04:49  hudson.plugins.git.GitException: Command "git config remote.origin.url https://github.com/MarkEWaite/jenkins-bugs.git" returned status code 255:

15:04:49  stdout: 

15:04:49  stderr: error: could not lock config file .git/config: File exists

15:04:49  

15:04:49  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608)

15:04:49  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2538)

15:04:49  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2534)

15:04:49  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1920)

15:04:49  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1932)

15:04:49  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1546)

15:04:49  	at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:161)

15:04:49  	at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:357)

15:04:49  	at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:330)

15:04:49  	at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:396)

15:04:49  	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582)

15:04:49  	at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:100)

15:04:49  	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)

15:04:49  	at hudson.model.ResourceController.execute(ResourceController.java:97)

15:04:49  	at hudson.model.Executor.run(Executor.java:429)

15:04:49  Finished: FAILURE





The build log using JGit reported:




5:03:01  Started by remote host Started from Groovy Console by Mark Waite with note: Groovy console started

15:03:01  Setting origin to https://github.com/MarkEWaite/jenkins-bugs.git

15:03:06  org.eclipse.jgit.errors.LockFailedException: Cannot lock /var/jenkins_home/caches/git-c3acd33583448548d14667df83b93ee1/.git/config. Ensure that no other process has an open file handle on the lock file /var/jenkins_home/caches/git-c3acd33583448548d14667df83b93ee1/.git/config.lock, then you may delete the lock file and retry.

15:03:06  	at org.eclipse.jgit.storage.file.FileBasedConfig.save(FileBasedConfig.java:220)

15:03:06  	at org.jenkinsci.plugins.gitclient.JGitAPIImpl.setRemoteUrl(JGitAPIImpl.java:945)

15:03:06  Caused: hudson.plugins.git.GitException

15:03:06  	at org.jenkinsci.plugins.gitclient.JGitAPIImpl.setRemoteUrl(JGitAPIImpl.java:947)

15:03:06  	at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:357)

15:03:06  	at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:330)

15:03:06  	at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:396)

15:03:06  	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582)

15:03:06  	at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:100)

15:03:06  	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)

15:03:06  	at hudson.model.ResourceController.execute(ResourceController.java:97)

15:03:06  	at hudson.model.Executor.run(Executor.java:429)

15:03:06  Finished: FAILURE

• environment: Jenkins 2.277.1 with git plugin 4.7.0 and other plugins as described in my docker-lfs repo dated 31 Mar 2021