Azure / Azure/azure-linux-extensions

Stale configchunks make the agent fail

Open
#1,995 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
333
Forks
278
Avg merge
2d 9h
Merged PRs (30d)
4

Description

If one creates a VM image from a machine that had AMA installed (1.33.1) with a particular data collection rule for sending text logs to Log Analytics,

then the resulting new instances from the source image will silently fail to publish logs going forward. This is due to stale config chunks sitting in `/etc/opt/microsoft/azuremonitoragent/config-cache/configchunks` which have `tokenEndpointUri` that contains the name of the source VM image.

None of the log files under `/var/opt/microsoft/azuremonitoragent/log` indicate this error, unless the repeated, but cryptic:

```
2024-11-11T13:50:30.1561510Z: [/__w/1/s/external/WindowsAgent/src/shared/mcsmanager/lib/src/RefreshGigToken.cpp:243,RefreshGigToken]
2024-11-11T13:50:30.1767310Z: [/__w/1/s/external/WindowsAgent/src/shared/mcsmanager/lib/src/RefreshGigToken.cpp:243,RefreshGigToken]
2024-11-11T13:51:30.2037210Z: [/__w/1/s/external/WindowsAgent/src/shared/mcsmanager/lib/src/RefreshGigToken.cpp:243,RefreshGigToken]
```

messages are connected to this problem (I don't know, these log lines are not very meaningful to me).

I would have expected that AMA over time refreshes the `config-cache` or cleans it up upon reboot, but apparently the only way to work this around now is 1) stopping the service 2) manually cleaning the config chunk json files 3) restarting the service.

I think AMA should be able to recover from this scenario on it's own.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.