aws / aws/amazon-cloudwatch-agent
Allow for variable substitution in the logs configuration beyond the prebuilt variables
- Dominant language
- Go
- Stars
- 550
- Forks
- 271
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 13
Description
**Is your feature request related to a problem? Please describe.**
I want to simplify my log configuration for different environments (production, staging, etc) so I can share a single cloudwatch agent config and swap out the little pieces I need per environment through existing variable substitution mechanisms. But the variable substitution facility is pre-defined and fixed preventing me from simplifying my configs.
**Describe the solution you'd like**
In the logs configuration section you can use {instanceId} or other variables to substitute in values from the environment, but these are fixed values controlled by the agent. They are not extensible in any way. These fixed values don't cover things like environment variables, EC2 metadata, or tags which could expand what is possible to cover the situation where I want to differ values by environment.
I want the ability to substitute in either environment variables, tags, or EC2 metadata into these because I want to define things log groups or log streams that differ per environment so that I don't have to write multiple configs or roll your own bash scripts to accomplish this.
So for example I want to define a couple of log groups for my different environments then I can share the log configuration if I was able to do something like this:
```
"log_group_name": "myApp.{env:ENV_VAR}.sys"
```
Where ENV_VAR is an environment variable in linux I could substitute into that expression for the log group. Then my log configuration could be shared between production and staging without needing to copy them.
Or possibly using EC2 metadata that I can add properties too like so:
```
"log_stream_name": "myApp.{meta:region}.cron"
```
Other features like AWS Backup use tags to signify which assets to backup. Using tags allows the user to define this in one place and have it be applicable across lots of AWS services:
```
"log_stream_name": "myApp.{tag:Environment}.secure"
```
The tag Environment have a value of "production", "staging", "testing", etc. Tags are like cloud environment variables.
**Describe alternatives you've considered**
The alternative is a complex architecture of bash scripts, and deployment code to modify and carefully deploy copies to parameter store using aws cli. The other option was to write a gradle script to "build" the config files at deployment time. Since committing these configs to git makes sense for long term management then treating them like yet another build also makes sense if I'm willing to accept the additional overhead.
**Additional context**
If I had this feature I wouldn't have to build devops scripts and it would simplify my management of our dev ops which if I'm truthful devops == me.
Contributor guide
Research direction
No files or tests are named. Start by locating the logs configuration variable-substitution entry point and reviewing how the existing prebuilt variables are defined and resolved. Done should support a documented extensible syntax for the requested environment, metadata, or tag values while preserving current substitutions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go
- Domain
- cloud, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100