aws / aws/amazon-cloudwatch-agent
[Feature Request] Implement additional glob functionality for collect_list.file_path
- Dominant language
- Go
- Stars
- 550
- Forks
- 271
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 13
Description
In the [CloudWatch Agent configuration documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html#CloudWatch-Agent-Configuration-File-Logssection), the `file_path` section is described:
> file_path – Specifies the path of the log file to upload to CloudWatch Logs. Standard Unix glob matching rules are accepted, with the addition of ** as a super asterisk. For example, specifying /var/log/**.log causes all .log files in the /var/log directory tree to be collected. **For more examples, see [Glob Library](https://github.com/gobwas/glob).**
It links to the `gobwas/glob` library which describes a few different glob patterns that do not appear to be implemented in the `amazon-cloudwatch-agent` package.
From their documentation:
```
// create glob with pattern-alternatives list
g = glob.MustCompile("{cat,bat,[fr]at}")
g.Match("cat") // true
g.Match("bat") // true
g.Match("fat") // true
g.Match("rat") // true
g.Match("at") // false
g.Match("zat") // false
```
It would be great add this functionality so that these pattern-alternative lists could be used to describe file paths. I tested this and confirmed that it does not appear to be working, see the example from my config file below:
```json
...
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/pattern_test/{test1,test2}/**",
"publish_multi_logs": true,
"log_group_name": "pattern_test"
},
...
```
I also tried:
```json
"file_path": "{/var/log/pattern_test/test4/**,/var/log/pattern_test/test5/**}"
```
One use case I can think of would be if you want a handful of directories within a file path to be captured, this would be a cleaner way to specify that instead of having multiple elements added to the collect_list with each directory being described individually.
Contributor guide
Research direction
Start by tracing how collect_list.file_path is parsed and matched, then compare the current behavior with the gobwas/glob pattern-alternative examples linked in the issue. Done means the supplied patterns using brace alternatives match the intended log directories and existing glob behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100