google / google/osdfir-infrastructure

Github action hitting max storage space causing tests to fail

Open
#216 3 comments 0 reactions 0 assignees View on GitHub
process
Dominant language
Go Template
Stars
213
Forks
30
Avg merge
14h 33m
Merged PRs (30d)
5

Description

### Name and Version

charts/*

### What environment are you using?

None

### What steps will reproduce the bug?

1. When creating a new PR
2. Github action lint-test runs https://github.com/google/osdfir-infrastructure/blob/main/.github/workflows/lint-test.yaml

Running some additional commands, we can see we have about 70% disk usage
```
Run df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 72G 50G 22G 70% /
tmpfs 7.9G 84K 7.9G 1% /dev/shm
tmpfs 3.2G 1.1M 3.2G 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sdb16 881M 60M 760M 8% /boot
/dev/sdb15 105M 6.2M 99M 6% /boot/efi
/dev/sda1 74G 4.1G 66G 6% /mnt
tmpfs 1.6G 12K 1.6G 1% /run/user/1001
```

Doing some more tests, we can see after the run it reaches up to 99% so likely its been capping passed this causing it to fail. It's so strange as this was working last week, they must've added more things to github actions and we have even more limited space to work with

### Are you using any custom parameters or values?

_No response_

### What is the expected behavior?

For the tests to pass

### What do you see instead?

3. Fails due to no disk space
```

lint-testSystem.IO.IOException: No space left on device : '/home/runner/actions-runner/cached/_diag/Worker_20251007-173706-utc.log' at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset) at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder) at System.Diagnostics.TextWriterTraceListener.Flush() at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id) at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message) at GitHub.Runner.Worker.Worker.RunAsync(String pipeIn, String pipeOut) at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args) System.IO.IOException: No space left on device : '/home/runner/actions-runner/cached/_diag/Worker_20251007-173706-utc.log' at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset) at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder) at System.Diagnostics.TextWriterTraceListener.Flush() at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id) at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message) at GitHub.Runner.Common.Tracing.Error(Exception exception) at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args) Unhandled exception. System.IO.IOException: No space left on device : '/home/runner/actions-runner/cached/_diag/Worker_20251007-173706-utc.log' at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset) at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder) at System.Diagnostics.TextWriterTraceListener.Flush() at System.Diagnostics.TraceSource.Flush() at GitHub.Runner.Common.Tracing.Dispose(Boolean disposing) at GitHub.Runner.Common.Tracing.Dispose() at GitHub.Runner.Common.TraceManager.Dispose(Boolean disposing) at GitHub.Runner.Common.TraceManager.Dispose() at GitHub.Runner.Common.HostContext.Dispose(Boolean disposing) at GitHub.Runner.Common.HostContext.Dispose() at GitHub.Runner.Worker.Program.Main(String[] args)
--

[lint-test](https://github.com/google/osdfir-infrastructure/actions/runs/18321129125/job/52174180975)
System.IO.IOException: No space left on device : '/home/runner/actions-runner/cached/_diag/Worker_20251007-173706-utc.log'
at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.Diagnostics.TextWriterTraceListener.Flush()
at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
at GitHub.Runner.Worker.Worker.RunAsync(String pipeIn, String pipeOut)
at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
System.IO.IOException: No space left on device : '/home/runner/actions-runner/cached/_diag/Worker_20251007-173706-utc.log'
at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.Diagnostics.TextWriterTraceListener.Flush()
at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
at GitHub.Runner.Common.Tracing.Error(Exception exception)
at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
Unhandled exception. System.IO.IOException: No space left on device : '/home/runner/actions-runner/cached/_diag/Worker_20251007-173706-utc.log'
at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.Diagnostics.TextWriterTraceListener.Flush()
at System.Diagnostics.TraceSource.Flush()
at GitHub.Runner.Common.Tracing.Dispose(Boolean disposing)
at GitHub.Runner.Common.Tracing.Dispose()
at GitHub.Runner.Common.TraceManager.Dispose(Boolean disposing)
at GitHub.Runner.Common.TraceManager.Dispose()
at GitHub.Runner.Common.HostContext.Dispose(Boolean disposing)
at GitHub.Runner.Common.HostContext.Dispose()
at GitHub.Runner.Worker.Program.Main(String[] args)
```

### Additional information

_No response_

Contributor guide

Open the contributing guide

Research direction

Inspect .github/workflows/lint-test.yaml and the linked lint-test run; begin by reproducing the disk-growth report with df -h during a pull request check. Confirm the workflow completes without the runner's “No space left on device” error and that the charts/* tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, helm
Domain
ci-cd, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.