elastic / elastic/elastic-serverless-forwarder

Allow unquoted Secret Manager ARNs in config.yaml

Open
#401 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
38
Forks
50
PR merge metrics
No merged PRs in 30d

Description

### Version: 1.8.0
### Steps to Reproduce:
- Given the following `config.yaml` with unquoted Secret Manager ARNs:
```
inputs:
- type: cloudwatch-logs
id: arn:aws:logs:REGION:ACCOUNT:log-group:LOG_GROUP_NAME:*
outputs:
- type: elasticsearch
args:
api_key: arn:aws:secretsmanager:REGION:ACCOUNT:secret:SECRET_NAME_1
cloud_id: arn:aws:secretsmanager:REGION:ACCOUNT:secret:SECRET_NAME_2
es_datastream_name: datastream
```
- When using a deployed ESF forwarder that uses this config.yaml, the following exception is raised in the lambda logs:
```
[ERROR] ConfigFileException: Invalid arn format: arn:aws:secretsmanager:REGION:ACCOUNT:secret:SECRET_NAME_1
cloud_id: arn:aws:secretsmanager:REGION:ACCOUNT:secret:SECRET_NAME_2
es_datastream_name: datastream

Traceback (most recent call last):
  File "/var/task/main_aws.py", line 17, in handler
    return lambda_handler(lambda_event, lambda_context)
  File "/var/task/handlers/aws/utils.py", line 63, in wrapper
    return func(lambda_event, lambda_context)
  File "/var/task/handlers/aws/utils.py", line 98, in wrapper
    raise e
  File "/var/task/handlers/aws/utils.py", line 82, in wrapper
    return func(lambda_event, lambda_context)
  File "/var/task/handlers/aws/handler.py", line 75, in lambda_handler
    raise ConfigFileException(e)
```

This happens because the Secrets Manager ARN regex defined at [share/secretsmanager.py](https://github.com/elastic/elastic-serverless-forwarder/blob/main/share/secretsmanager.py#L41) does not terminate on line endings, only terminating on single- and double-quotes using the following pattern:
`re_pattern = r"arn:aws:secretsmanager:(?:[^:]+)?:(?:[^:]+)?:secret:(?:[^\"']+)?"`
adding the new line character in the final capturing group should allow unquoted ARNs to parse nicely:
`re_pattern = r"arn:aws:secretsmanager:(?:[^:]+)?:(?:[^:]+)?:secret:(?:[^\"'\n]+)?"`

We're currently defining our `config.yaml` programmatically (using the AWS CDK to retrieve ARNs and then build the config file), so while admittedly this isn't much of a problem - we could just update this to wrap the ARNs in quotes - the regex would be useful in our case as Java requires a not-so-clean addition of code such as `"\"" + secret.getSecretArn() + "\""` which would be nice to avoid.

Appreciate this is a small problem, and interested to hear any pro-quotation arguments against this suggestion (e.g. protect against special characters) - my understanding is that the YAML is parsed as a string, so this doesn't have an impact on the parser. I'm happy to contribute a PR for the change with a test case if accepted.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.