elastic / elastic/integrations
[Apache Tomcat] Access events with unparseable request targets are marked as pipeline errors
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 182
Description
### Integration Name
Apache Tomcat [packages/apache_tomcat]
### Dataset Name
apache_tomcat.access
### Integration Version
1.13.1 (also present on current 1.13.x)
### Agent Version
N/A (ingest pipeline)
### Agent Output Type
elasticsearch
### Elasticsearch Version
N/A (ingest pipeline)
### OS Version and Architecture
N/A (ingest pipeline)
### Software/API Version
Apache Tomcat access logs (standard AccessLogValve combined-style output)
### Error Message
```
Processor uri_parts with tag fail-uri_parts_processor in pipeline logs-apache_tomcat.access-1.13.1 failed with message: unable to parse URI [/demo/%u002e%u002e/fake-file.txt]
```
```
Processor uri_parts with tag fail-uri_parts_processor in pipeline logs-apache_tomcat.access-1.13.1 failed with message: unable to parse URI [%notvalid/example.properties]
```
### Event Original
```
192.0.2.1 - - [15/Jan/2024:10:11:12 +0000] "GET /demo/%u002e%u002e/fake-file.txt HTTP/1.1" 400 512
198.51.100.2 - - [15/Jan/2024:10:11:13 +0000] "GET %notvalid/example.properties HTTP/1.1" 400 256
```
### What did you do?
Ingest Tomcat access logs with the default `apache_tomcat.access` pipeline. Some clients send syntactically invalid request targets (malformed percent-encoding). Tomcat logs those lines and typically answers with HTTP 4xx. The log line itself is valid combined-style access output.
### What did you see?
The line is parsed (source IP, method, status, `url.original`). `uri_parts` then fails on `url.original`. The processor `on_failure` appends to `error.message`, and the last pipeline processor sets `event.kind: pipeline_error` whenever `error.message` is set.
Result: a correctly ingested access event is labelled as an ingest failure. The event is not dropped. Derived `url.*` fields are missing; `url.original` is kept.
Ingested documents from the examples above:
```json
{
"@timestamp": "2024-01-15T10:11:12.000Z",
"apache_tomcat": {
"access": {
"http": {
"ident": "-",
"useragent": "-"
}
}
},
"destination": {
"bytes": 512
},
"ecs": {
"version": "8.11.0"
},
"error": {
"message": [
"Processor uri_parts with tag fail-uri_parts_processor in pipeline logs-apache_tomcat.access-1.13.1 failed with message: unable to parse URI [/demo/%u002e%u002e/fake-file.txt]"
]
},
"event": {
"category": ["web"],
"kind": "pipeline_error",
"module": "apache_tomcat",
"original": "192.0.2.1 - - [15/Jan/2024:10:11:12 +0000] \"GET /demo/%u002e%u002e/fake-file.txt HTTP/1.1\" 400 512",
"outcome": "failure",
"type": ["access"]
},
"http": {
"request": {
"method": "GET"
},
"response": {
"status_code": 400
},
"version": "1.1"
},
"related": {
"ip": ["192.0.2.1"]
},
"source": {
"ip": "192.0.2.1"
},
"url": {
"original": "/demo/%u002e%u002e/fake-file.txt"
}
}
```
```json
{
"@timestamp": "2024-01-15T10:11:13.000Z",
"apache_tomcat": {
"access": {
"http": {
"ident": "-",
"useragent": "-"
}
}
},
"destination": {
"bytes": 256
},
"ecs": {
"version": "8.11.0"
},
"error": {
"message": [
"Processor uri_parts with tag fail-uri_parts_processor in pipeline logs-apache_tomcat.access-1.13.1 failed with message: unable to parse URI [%notvalid/example.properties]"
]
},
"event": {
"category": ["web"],
"kind": "pipeline_error",
"module": "apache_tomcat",
"original": "198.51.100.2 - - [15/Jan/2024:10:11:13 +0000] \"GET %notvalid/example.properties HTTP/1.1\" 400 256",
"outcome": "failure",
"type": ["access"]
},
"http": {
"request": {
"method": "GET"
},
"response": {
"status_code": 400
},
"version": "1.1"
},
"related": {
"ip": ["198.51.100.2"]
},
"source": {
"ip": "198.51.100.2"
},
"url": {
"original": "%notvalid/example.properties"
}
}
```
### What did you expect to see?
The event should stay `event.kind: event` with no `error.message`. A failed URL split is expected for invalid request targets and should not be treated as a pipeline error. This matches Apache HTTP Server, Nginx, and IIS access pipelines, which use `ignore_failure: true` on `uri_parts`.
### Anything else?
Proposed fix: set `ignore_failure: true` on the access `uri_parts` processor instead of writing `error.message`. No custom URI parsing; invalid targets keep `url.original` only.
Contributor guide
Research direction
Locate the apache_tomcat.access ingest pipeline and its uri_parts processor, then compare its failure handling with the Apache HTTP Server, Nginx, and IIS access pipelines. Verify that malformed request targets retain url.original without producing error.message or event.kind: pipeline_error, while valid access-log fields remain parsed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- apache, handlebars
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100