aws / aws/amazon-cloudwatch-agent

Question/Feature request: how can I prepend an environment name to configured log groups at runtime

Open
#300 5 comments 6 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
550
Forks
271
Avg merge
1d 21h
Merged PRs (30d)
13

Description

**Scenario**
We have implemented a 'golden image' pipeline using AWS ImageBuilder that bakes in our base cloudwatch agent configuration (core logs files and system metrics),

e.g.
```
...
"logs": {
"log_stream_name": "${aws:InstanceId}",
"force_flush_interval" : 60,
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/syslog",
"log_group_name": "syslog",
"timestamp_format": "%b %d %H:%M:%S"
},
...
```

This ends up with a massive log group with streams of every instance we're running in the region. We want to be able to separate those out into say '/production/syslog' and '/staging/syslog'...

**Work Around**
Our current work-around is to run `sed` over the base config included in the golden image early in the boot order...

e.g. `sed -E 's/"log_group_name": "(.*)"/"log_group_name": "\/ENV_NAME\/TIER\/\1"/g' /path/to/amazon-cloudwatch-agent.json`

```
"logs": {
"log_stream_name": "${aws:InstanceId}",
"force_flush_interval" : 60,
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/syslog",
"log_group_name": "/ENV_NAME/TIER/syslog",
"timestamp_format": "%b %d %H:%M:%S"
},
{
"file_path": "/var/log/auth",
"log_group_name": "/ENV_NAME/TIER/auth",
"timestamp_format": "%b %d %H:%M:%S"
},
{
"file_path": "/var/log/audit/audit.log",
"log_group_name": "/ENV_NAME/TIER/audit"
},
{
"file_path": "/var/log/aide/aide.log",
"log_group_name": "/ENV_NAME/TIER/aide"
}
]
}
}
```

**Desired Behaviour**
It would be great if we could just declare arbitrary environment variables that are picked up by and agent and substituted in at runtime. e.g. `"log_group_name": "/${env.ENV_NAME}/${env.TIER}/aide"`

Contributor guide

Open the contributing guide

Research direction

No source files or tests are identified in the issue. Start by tracing how the agent loads and expands its configuration, then determine where runtime environment-variable substitution for log_group_name would be handled. Done means configured environment values produce the intended separated log-group names.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
cloud, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.