elastic / elastic/integrations
[tcp]: Line Delimiter input text field does not work for control characters like \r
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
### Integration Name
Custom TCP Logs [tcp]
### Dataset Name
tcp.generic
### Integration Version
1.19.1
### Agent Version
8.15.0
### Agent Output Type
elasticsearch
### Elasticsearch Version
8.15.0
### OS Version and Architecture
Linux
### Software/API Version
_No response_
### Error Message
From Filebeat log output:
`{"log.level":"debug","@timestamp":"2024-08-07T20:12:49.340Z","message":"Client error","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"tcp-default","type":"tcp"},"log":{"source":"tcp-default"},"log.logger":"tcp","log.origin":{"file.line":202,"file.name":"streaming/listener.go","function":"github.com/elastic/beats/v7/filebeat/inputsource/common/streaming.(*Listener).handleConnection"},"remote_address":"127.0.0.1:57109",
"error":{"message":"bufio.Scanner: token too long"},"ecs.version":"1.6.0","service.name":"filebeat","address":"localhost:2107","ecs.version":"1.6.0"}`
This is an indicator that the delimiter is not working correctly and it is over running the bufio scanner.
### Event Original
_No response_
### What did you do?
Configured the "Line Delimiter" option with `\r` with the intention of using a carriage return as the delimiter.
```
POST kbn:/api/fleet/package_policies
{
"policy_ids": [
"6a93bbd0-6234-49f3-b0d4-50a0c762458a"
],
"package": {
"name": "tcp",
"version": "1.19.1"
},
"name": "tcp-1",
"description": "",
"namespace": "",
"inputs": {
"tcp-tcp": {
"enabled": true,
"streams": {
"tcp.generic": {
"enabled": true,
"vars": {
"listen_address": "localhost",
"listen_port": 2107,
"data_stream.dataset": "tcp.generic",
"line_delimiter": "\\r",
"tags": [],
"syslog_options": "field: message\n#format: auto\n#timezone: Local\n",
"preserve_original_event": false,
"ssl": "#certificate: |\n# -----BEGIN CERTIFICATE-----\n# ...\n# -----END CERTIFICATE-----\n#key: |\n# -----BEGIN PRIVATE KEY-----\n# ...\n# -----END PRIVATE KEY-----\n",
"custom": ""
}
}
}
}
}
}
```
### What did you see?
The package policy was generated with `"line_delimiter": "\\r",` which is not a carriage return, but instead a `\` followed by `r` (or 0x5c72).
### What did you expect to see?
I expected to see the package policy request generated with `"line_delimiter": "\r",` which would be 0x0d.
### Anything else?
The workaround is to use the custom configuration YAML option to specify the `line_delimiter` instead of using the "Line Delimiter" input text field.
```yaml
line_delimiter: "\r"
```
Contributor guide
Assessment
This issue has not been assessed yet.