Add an API for generating content directly in the tool cache
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 5.9k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Description
Describe the enhancement
@actions/tool-cache currently supports populating the tool cache with
cacheFile() and cacheDir(). Both APIs copy existing content into the cache:
cacheFile()copies through@actions/io, which uses Node.js
fs.copyFile().cacheDir()recursively copies each child through@actions/io.
Some callers need to generate content directly at its final cache location.
Copying is either unnecessarily expensive or changes important filesystem
properties. Examples include:
- sparse filesystem images, where
fs.copyFile()can materialize holes; - large generated SDK or tool directories;
- files produced using reflinks or hardlinks;
- transformations whose output is already being written once and should not be
written a second time merely to enter the tool cache.
Please add a callback-based API that lets the caller populate a toolkit-managed
cache destination directly, while the toolkit continues to own the cache layout,
completion marker, cleanup, version normalization, and architecture handling.
One possible shape is:
export async function cacheGeneratedTool(
tool: string,
version: string,
producer: (destination: string) => Promise<void>,
arch?: string
): Promise<string>
Expected semantics:
- Normalize
versionand resolvearchconsistently withcacheFile()and
cacheDir(). - Remove an incomplete destination and its stale
.completemarker. - Create the destination directory in the tool cache.
- Invoke
producer(destination). - Write the existing
.completemarker only after the producer resolves. - If the producer rejects, remove the incomplete destination and do not leave a
completion marker. - Return the completed cache directory.
- Do not copy, move, archive, or otherwise reinterpret files created by the
producer. - Preserve the tool cache's current concurrency contract; this proposal does
not require introducing cross-process locking or shared-cache coordination.
The existing cacheFile() and cacheDir() APIs should remain unchanged.
Exposing the private path and completion-marker functions directly would make it
easy for callers to forget cleanup or mark an incomplete cache entry as complete,
so a callback API appears safer.
Whether an existing completed entry should be checked by this API or by the
caller can follow the package's current conventions. For example:
let toolPath = tc.find(tool, version, arch)
if (!toolPath) {
toolPath = await tc.cacheGeneratedTool(
tool,
version,
async destination => {
await generateToolDirectlyInto(destination)
await validateGeneratedTool(destination)
},
arch
)
}
Code snippet
A sparse-image consumer could use the API without asking the toolkit to
understand sparse files or any particular compression format:
const cached = tc.find('example-rootfs', version, arch)
if (cached) {
return path.join(cached, 'base.ext4')
}
const cacheDir = await tc.cacheGeneratedTool(
'example-rootfs',
version,
async destination => {
const output = path.join(destination, 'base.ext4')
await exec.exec('zstd', ['--sparse', '-dqf', compressed, '-o', output])
await validateImage(output)
},
arch
)
return path.join(cacheDir, 'base.ext4')
The producer writes the output once, directly in the toolkit-managed cache
directory. Validation completes before the toolkit writes .complete.
Additional information
The motivating measurement used a 2 GiB logical sparse file with approximately
197 MiB physically allocated on ext4:
| Operation | Wall time | Resulting allocation |
|---|---|---|
Node.js fs.promises.copyFile() |
2.97 seconds | 2.147 GB |
cp --sparse=always |
0.17 seconds | 206.6 MB |
The exact timings depend on the host, but the semantic issue is independent of
throughput: copying through fs.copyFile() can turn a sparse cache artifact into
a fully allocated file.
The package already has the necessary internal lifecycle:
_createToolPath() prepares the destination and removes stale state, while
_completeToolPath() writes the marker consumed by find(). The proposed API
would expose that lifecycle safely without exposing those private functions or
requiring every specialized action to reproduce the tool-cache directory and
marker conventions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the tool-cache implementation around cacheFile(), cacheDir(), _createToolPath(), and _completeToolPath(), then inspect find() to understand version, architecture, and completion-marker conventions. The API is done when it creates the managed destination, invokes the producer directly, marks completion only after success, cleans up on rejection, and returns the completed cache directory without changing the existing APIs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 57/100