aws / aws/amazon-cloudwatch-agent

[Feature Request] Implement additional glob functionality for collect_list.file_path

Open
#328 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
550
Forks
271
Avg merge
1d 21h
Merged PRs (30d)
13

Description

In the [CloudWatch Agent configuration documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html#CloudWatch-Agent-Configuration-File-Logssection), the `file_path` section is described:
> file_path – Specifies the path of the log file to upload to CloudWatch Logs. Standard Unix glob matching rules are accepted, with the addition of ** as a super asterisk. For example, specifying /var/log/**.log causes all .log files in the /var/log directory tree to be collected. **For more examples, see [Glob Library](https://github.com/gobwas/glob).**

It links to the `gobwas/glob` library which describes a few different glob patterns that do not appear to be implemented in the `amazon-cloudwatch-agent` package.

From their documentation:
```
// create glob with pattern-alternatives list
g = glob.MustCompile("{cat,bat,[fr]at}")
g.Match("cat") // true
g.Match("bat") // true
g.Match("fat") // true
g.Match("rat") // true
g.Match("at") // false
g.Match("zat") // false
```

It would be great add this functionality so that these pattern-alternative lists could be used to describe file paths. I tested this and confirmed that it does not appear to be working, see the example from my config file below:

```json
...
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/pattern_test/{test1,test2}/**",
"publish_multi_logs": true,
"log_group_name": "pattern_test"
},
...
```

I also tried:

```json
"file_path": "{/var/log/pattern_test/test4/**,/var/log/pattern_test/test5/**}"
```

One use case I can think of would be if you want a handful of directories within a file path to be captured, this would be a cleaner way to specify that instead of having multiple elements added to the collect_list with each directory being described individually.

Contributor guide

Open the contributing guide

Research direction

Start by tracing how collect_list.file_path is parsed and matched, then compare the current behavior with the gobwas/glob pattern-alternative examples linked in the issue. Done means the supplied patterns using brace alternatives match the intended log directories and existing glob behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.