elastic / elastic/integrations

[tcp]: Line Delimiter input text field does not work for control characters like \r

Open
#10,817 6 comments 2 reactions 0 assignees View on GitHub
bug Integration:tcp Team:Security-Deployment and Devices
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

### Integration Name

Custom TCP Logs [tcp]

### Dataset Name

tcp.generic

### Integration Version

1.19.1

### Agent Version

8.15.0

### Agent Output Type

elasticsearch

### Elasticsearch Version

8.15.0

### OS Version and Architecture

Linux

### Software/API Version

_No response_

### Error Message

From Filebeat log output:

`{"log.level":"debug","@timestamp":"2024-08-07T20:12:49.340Z","message":"Client error","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"tcp-default","type":"tcp"},"log":{"source":"tcp-default"},"log.logger":"tcp","log.origin":{"file.line":202,"file.name":"streaming/listener.go","function":"github.com/elastic/beats/v7/filebeat/inputsource/common/streaming.(*Listener).handleConnection"},"remote_address":"127.0.0.1:57109",
"error":{"message":"bufio.Scanner: token too long"},"ecs.version":"1.6.0","service.name":"filebeat","address":"localhost:2107","ecs.version":"1.6.0"}`

This is an indicator that the delimiter is not working correctly and it is over running the bufio scanner.

### Event Original

_No response_

### What did you do?

Configured the "Line Delimiter" option with `\r` with the intention of using a carriage return as the delimiter.

Screenshot 2024-08-19 at 22 26 18

```
POST kbn:/api/fleet/package_policies
{
"policy_ids": [
"6a93bbd0-6234-49f3-b0d4-50a0c762458a"
],
"package": {
"name": "tcp",
"version": "1.19.1"
},
"name": "tcp-1",
"description": "",
"namespace": "",
"inputs": {
"tcp-tcp": {
"enabled": true,
"streams": {
"tcp.generic": {
"enabled": true,
"vars": {
"listen_address": "localhost",
"listen_port": 2107,
"data_stream.dataset": "tcp.generic",
"line_delimiter": "\\r",
"tags": [],
"syslog_options": "field: message\n#format: auto\n#timezone: Local\n",
"preserve_original_event": false,
"ssl": "#certificate: |\n# -----BEGIN CERTIFICATE-----\n# ...\n# -----END CERTIFICATE-----\n#key: |\n# -----BEGIN PRIVATE KEY-----\n# ...\n# -----END PRIVATE KEY-----\n",
"custom": ""
}
}
}
}
}
}
```

### What did you see?

The package policy was generated with `"line_delimiter": "\\r",` which is not a carriage return, but instead a `\` followed by `r` (or 0x5c72).

### What did you expect to see?

I expected to see the package policy request generated with `"line_delimiter": "\r",` which would be 0x0d.

### Anything else?

The workaround is to use the custom configuration YAML option to specify the `line_delimiter` instead of using the "Line Delimiter" input text field.

```yaml
line_delimiter: "\r"
```
356237312-4e8d855d-9e77-4d8e-8d74-b0afb95b14bf

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.