elastic / elastic/elastic-serverless-forwarder
Allow unquoted Secret Manager ARNs in config.yaml
- Dominant language
- Python
- Stars
- 38
- Forks
- 50
- PR merge metrics
- No merged PRs in 30d
Description
### Version: 1.8.0
### Steps to Reproduce:
- Given the following `config.yaml` with unquoted Secret Manager ARNs:
```
inputs:
- type: cloudwatch-logs
id: arn:aws:logs:REGION:ACCOUNT:log-group:LOG_GROUP_NAME:*
outputs:
- type: elasticsearch
args:
api_key: arn:aws:secretsmanager:REGION:ACCOUNT:secret:SECRET_NAME_1
cloud_id: arn:aws:secretsmanager:REGION:ACCOUNT:secret:SECRET_NAME_2
es_datastream_name: datastream
```
- When using a deployed ESF forwarder that uses this config.yaml, the following exception is raised in the lambda logs:
```
[ERROR] ConfigFileException: Invalid arn format: arn:aws:secretsmanager:REGION:ACCOUNT:secret:SECRET_NAME_1
cloud_id: arn:aws:secretsmanager:REGION:ACCOUNT:secret:SECRET_NAME_2
es_datastream_name: datastream
Traceback (most recent call last):
File "/var/task/main_aws.py", line 17, in handler
return lambda_handler(lambda_event, lambda_context)
File "/var/task/handlers/aws/utils.py", line 63, in wrapper
return func(lambda_event, lambda_context)
File "/var/task/handlers/aws/utils.py", line 98, in wrapper
raise e
File "/var/task/handlers/aws/utils.py", line 82, in wrapper
return func(lambda_event, lambda_context)
File "/var/task/handlers/aws/handler.py", line 75, in lambda_handler
raise ConfigFileException(e)
```
This happens because the Secrets Manager ARN regex defined at [share/secretsmanager.py](https://github.com/elastic/elastic-serverless-forwarder/blob/main/share/secretsmanager.py#L41) does not terminate on line endings, only terminating on single- and double-quotes using the following pattern:
`re_pattern = r"arn:aws:secretsmanager:(?:[^:]+)?:(?:[^:]+)?:secret:(?:[^\"']+)?"`
adding the new line character in the final capturing group should allow unquoted ARNs to parse nicely:
`re_pattern = r"arn:aws:secretsmanager:(?:[^:]+)?:(?:[^:]+)?:secret:(?:[^\"'\n]+)?"`
We're currently defining our `config.yaml` programmatically (using the AWS CDK to retrieve ARNs and then build the config file), so while admittedly this isn't much of a problem - we could just update this to wrap the ARNs in quotes - the regex would be useful in our case as Java requires a not-so-clean addition of code such as `"\"" + secret.getSecretArn() + "\""` which would be nice to avoid.
Appreciate this is a small problem, and interested to hear any pro-quotation arguments against this suggestion (e.g. protect against special characters) - my understanding is that the YAML is parsed as a string, so this doesn't have an impact on the parser. I'm happy to contribute a PR for the change with a test case if accepted.
Contributor guide
Assessment
This issue has not been assessed yet.