actions / actions/runner

Incorrect self-hosted runners disappeared message

Open
#2,967 8 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C#
Stars
6.3k
Forks
1.4k
Avg merge
1d 16h
Merged PRs (30d)
24

Description

Describe the bug

I have a runner running in a virtualised mac - which is generally working very well.

Occasionally the VM instance is going to sleep (or the host that it's running on is going to sleep), and the run.sh script is losing its connection or timing out and attempting to retry.

It doesn't manage to retry when the session is restored. If I ctrl-c the script and launch it again, I get the error about the self-hosted runner having disappeared.

This error is spurious in this case. The runner is still registered, and if I restart the VM and re-run the script it works again.

I think the real reason is some sort of token / auth timeout, which is incorrectly being reported.

To Reproduce

Install runner in a VM (eg VirtualBuddy using macOS 13.6)
Run the runner and leave it for a while.
When it times out, try to restart the run.sh script.

Expected behavior

Ideally the run.sh script should successfully reconnect without needing to be restarted.
In the absence of that, quitting and re-running the script should successfully connect.
In the absence of that (!), a more informative message should be output, explaining the real problem.

Runner Version and Platform

runner: 2.311.0
virtualized host: macos 13.6
host: macos 13.4 (also tested on 13.6)

What's not working?

Runner is failing to reconnect, and is reporting that it's been deleted when it hasn't.

Job Log Output

runner@Runners-Virtual-Machine actions-runner % ./run.sh

√ Connected to GitHub

Current runner version: '2.311.0'
2023-11-02 07:18:17Z: Listening for Jobs
2023-11-02 08:17:44Z: Running job: Deploy Client
2023-11-02 08:18:38Z: Job Deploy Client completed with result: Failed
2023-11-02 09:59:39Z: Running job: Deploy Client
2023-11-02 10:04:05Z: Job Deploy Client completed with result: Failed
2023-11-02 10:06:03Z: Running job: Deploy Client
2023-11-02 10:07:54Z: Runner connect error: The HTTP request timed out after 00:01:00.. Retrying until reconnected.
2023-11-02 10:08:35Z: Job Deploy Client completed with result: Abandoned
^CExiting...
An error occurred: The token expired on 11/02/2023 10:42:58. Current server time is 11/02/2023 15:00:30.
Runner listener exit with retryable error, re-launch runner in 5 seconds.
Restarting runner...

√ Connected to GitHub

Failed to create a session. The runner registration has been deleted from the server, please re-configure. Runner registrations are automatically deleted for runners that have not connected to the service recently.
Runner listener exit with terminated error, stop the service, no retry needed.
Exiting runner...
runner@Runners-Virtual-Machine actions-runner % ./run.sh

√ Connected to GitHub

Failed to create a session. The runner registration has been deleted from the server, please re-configure. Runner registrations are automatically deleted for runners that have not connected to the service recently.
Runner listener exit with terminated error, stop the service, no retry needed.
Exiting runner...
runner@Runners-Virtual-Machine actions-runner % 

Runner and Worker's Diagnostic Logs

Runner_20231102-071814-utc.log
Worker_20231102-081745-utc.log

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the run.sh entry point and the attached Runner_20231102-071814-utc.log and Worker_20231102-081745-utc.log, focusing on the timeout, expired-token, and session-creation messages. Done means the runner reconnects after the virtualized macOS host sleeps, or reports the actual failure instead of claiming the registration was deleted.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, github-actions, macos
Domain
ci-cd, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.