elastic / elastic/integrations
[Amazon S3]: Grok Parsing Error with Valid S3 Object Key Characters
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
### Integration Name
AWS [aws]
### Dataset Name
aws.s3access
### Integration Version
v2.45.2
### Agent Version
8.17.4
### Agent Output Type
elasticsearch
### Elasticsearch Version
8.17.4
### OS Version and Architecture
elastic-agent-wolfi:8.17.4
### Software/API Version
_No response_
### Error Message
```
Processor 'grok' with tag '' failed with message 'Provided Grok expressions do not match field value: [28bdbd9102b76c3da07c44e094b16b382f90f94be71626f4d9589b1309d2d7ef my-bucket [31/Mar/2025:22:33:24 +0000] 18.68.18.10 97cb88763cd5f6a666a34ac23fec99f53137a309c78d86459aa421f2c82e82877e77c206a23ad9e41ecc07e3c586e292 MD9TCAF28XVJT7H7 REST.GET.OBJECT path/thumbnails/75*75/image-1.jpeg "GET /path/thumbnails/75*75/image-1.jpeg HTTP/1.1" 200 - 8642 8642 103 102 "-" "Amazon CloudFront" - 79o5Ho+xysFrip1ZRvYqOd2rhlOcCxkbPMPJ88s7YaM/k797fr+9ZCXsLvKs3yW7nhOta+g6yAI= SigV4 TLS_AES_128_GCM_SHA256 AuthHeader my-bucket.s3.amazonaws.com TLSv1.3 - -]'
```
### Event Original
```
28bdbd9102b76c3da07c44e094b16b382f90f94be71626f4d9589b1309d2d7ef my-bucket [02/Apr/2025:23:10:13 +0000] 64.252.71.106 97cb88763cd5f6a666a34ac23fec99f53137a309c78d86459aa421f2c82e82877e77c206a23ad9e41ecc07e3c586e292 0FMAC0C5SEXE32V4 REST.GET.OBJECT path/thumbnails/75*75/image-1.jpeg "GET /path/thumbnails/75*75/image-1.jpeg HTTP/1.1" 200 - 8208 8208 69 68 "-" "Amazon CloudFront" - bOe41Qirrpdy0aueV7z6WG1YGLVKBg64L0ghi4go4PRTOrxJxEStT196gst2/CURe0Lar7wQE0YEDWfjiiqhWZ7AO4Dgf0ileq6xldSGrTk= SigV4 TLS_AES_128_GCM_SHA256 AuthHeader my-bucket.s3.amazonaws.com TLSv1.3 - -
```
### What did you do?
I created a Case [01850523] with Support. The finding was that there is a problem with parsing the bucket key because of the `*` character in the name. We didn't really have any other option around this besides editing the first Grok processor step in the `logs-aws.s3access-*` pipeline by updating the pattern definition for "S3KEY" to allow for the `*` character as follows:
```
"S3KEY": """[a-zA-Z0-9\/_\.\*\-%+]+"""
```

### What did you see?
After updating the processor, the bucket access logs were successfully being parsed - the solution worked!
### What did you expect to see?
The `*` should have been a supported character all along because it is included in AWS's Safe Characters for S3 Object names: https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines
I would expect that all characters in the Safe Characters list are supported.

### Anything else?
I filed this as a Bug instead of a Feature Enhancement because the `*` is a supported character by S3.
Contributor guide
Assessment
This issue has not been assessed yet.