Agent stuck finishing a build

One of my agents got 'stuck' doing this every second or so:
PUT https://agent.buildkite.com/v3/jobs//finish: 401 Invalid access token (Attempt 2484030/∞ Retrying in 1s

Another agent didn't have a current job, but was also stuck trying to do a heartbeat and receiving the same 401.

This is quite a lot of retries and it can't really recover now it has lost it authorisation. The agent should probably attempt a full reconnection at this point.