jenkinsci / jenkinsci/lockable-resources-plugin

Re-define ephemeral resources used by revived builds when Jenkins controller is restarted

Open
#588 1 comment 0 reactions 0 assignees View on GitHub
Triage
Dominant language
Java
Stars
99
Forks
205
Avg merge
2d 13h
Merged PRs (30d)
8

Description

### What feature do you want to see added?

Porting from Gitter discussion:

Had a curious question in a dayjob discussion: lockable resources can be ephemeral, and pipelines aim to be restartable (does not make physical sense for us with some SUT types, but I agree the goal is worthy where it works - so gotta love to suffer programming around CPS and restartability of the server presumed by some code regardless of job performance-vs-persistence settings ;) ).

Ephemeral resources are not saved into configs and do not impact storage I/O in particular, so can be used en-masse.

Question from the audience: when a job that had an ephemeral resource locked (and notably defined by the act of locking, for the duration of the lock, and only known in-memory) was interrupted by a Jenkins controller restart - whether graceful or not, does the serialization of jobs in flight (something that happens regularly or done at least once to handle a graceful restart) consider to save currently known ephemeral lock definitions, so as to re-define and re-assign them when Jenkins starts up again and remembers the job mid-flight?

> ...there is also a request form to keep ephemeral resources on living (do not remove them). Because there are many cases, when you don't want to or can't create the resources as provider (some external configs ...) but it makes sense to keep them after creation.

To clarify: I think ephemeral resources do have their place in the ecosystem specifically as described in some comment about them: to allow rapid-fire locking without a bottleneck of updating and saving the Jenkins server configuration for every change (and I/O latency or storage wear involved in that). There are ways in scripted pipelines to use the `LockableResourcesManager` directly to define and `save()` the resources, if needed to be done persistently and programmatically (with proper permissions I believe).

For example of ephemeral resource use I've practiced, on some executors which share filesystem cached resources which assume one writer at a time (git repo cache, maven .m2 location before maven 3.9.x, etc.) I use ephemeral resources to gate the access to those locations. If there's a Jenkins restart, processes which might write there and conflict also die, so there's zero need to remember that some build "owned" that directory most recently or incur the performance loss associated with such remembering.

What the question above entailed was rather to discover and define the ephemeral resources and their owners "from scratch" if they were saved as part of a job state - and if we do reload such jobs (are persistent). So it is not about saving the ephemeral resources as such, but about saving their use in a running job's state (if we do) and then applying that knowledge upon restart.

### Upstream changes

_No response_

### Are you interested in contributing this feature?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.