Long log lines slow down the runner
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 1.4k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 24
Description
Describe the bug
Outputting very long log lines seems to cause the runner to slow down significantly.
A step that takes less than a second to actually execute can easily take 10s of minutes or even hours to be considered complete, often hitting timeouts.
https://github.com/hamishforbes/actions-test
This repo has a super simple python script that outputs the current timestamp and "a" repeated for various lengths.
Turn on timestamps and you can see a discrepancy.
Once a log message gets up to about 6k characters it looks like the runner starts to take a few seconds to process.
At 100k characters this can be 10 minutes or so
Running the script locally on a macbook pro, even with 100k characters, completes within a couple hundred ms.
Runner timestamp | Python timestamp
Tue, 30 Mar 2021 01:34:19 GMT | 2021-03-30 01:34:16.831361 - Message length: 10240 - aaaaa{snip}
Tue, 30 Mar 2021 01:34:30 GMT | 2021-03-30 01:34:16.831413 - Message length: 20480 - aaaaa{snip}
To Reproduce
Copy script and workflow from above repo :)
Expected behavior
I would expect the runner to be able to handle long log lines even if truncating them.
At the very least it shouldn't delay how long the step takes to run, the real world workflow that caused this was completing its test suite in < 10 mins but the step would never finish in actions because it would hit timeouts
Runner Version and Platform
Github hosted but tested with the latest self-hosted runner as well
Seems to affect both steps executed in docker and those directly on ubuntu-latest, although slightly less severe.
https://github.com/hamishforbes/actions-test/tree/no_docker
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproduction script and workflow in the linked actions-test repository, including the no_docker variant, and compare runner timestamps with the Python timestamps at increasing message lengths. Confirm the delay for long log lines and verify that handling or truncating them no longer makes the step exceed its actual execution time or hit a timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, python
- Domain
- ci-cd, devtools, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100