elastic / elastic/logstash

input UTF-16LE file creates garbled output

Open
#15,977 0 comments 0 reactions 0 assignees View on GitHub
bug status:needs-triage
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

**Logstash information**:

1. Logstash version : 8.12
2. Logstash installation source : zip
3. How is Logstash being run Via command line

**OS version** : windows11

***input file***: `test.sql: text/plain; charset=utf-16le`

**this file encode UTF-16LE, With this config, the output is garbled.**

***config***

```conf
input {
file {
path => ["/test.sql"]
tags => ["test"]
start_position => "beginning"
codec => plain {
charset => "UTF-16LE"
}
}
}

filter {
}

output {
stdout {
codec => rubydebug
}
}
```
***debug log***

```log
Received line {:path=>"/test.sql", :text=>"\xFF\xFEU\x00S\x00E\x00 \x00[\x00B\x00i\x00g\x00D\x00a\x00t\x00a\x00]\x00"}
config LogStash::Codecs::Plain/@charset = "UTF-16LE"
config LogStash::Codecs::Plain/@id = "f0a2ff23-f6f4-4a06-bf48-4e5cd4e3f1a1"
config LogStash::Codecs::Plain/@enable_metric = true
inputs/LogStash::Inputs::File: adding tag {"tag"=>"test"}
Received line {:path=>"/test.sql", :text=>"\x00G\x00O\x00"}
inputs/LogStash::Inputs::File: adding tag {"tag"=>"test"}
Received line {:path=>"/test.sql", :text=>"\x00/\x00*\x00*\x00*\x00*\x00*\x00*\x00 \x00O\x00b\x00j\x00e\x00c\x00t\x00:\x00 \x00 \x00T\
inputs/LogStash::Inputs::File: adding tag {"tag"=>"test"}
Received line {:path=>"/test.sql", :text=>"\x00S\x00E\x00T\x00 \x00A\x00N\x00S\x00I\x00_\x00N\x00U\x00L\x00L\x00S\x00 \x00O\x00N\x00"}
```

**I've noticed that the charset configuration(`Codecs::Plain/@charset = "UTF-16LE"`) doesn't output until after the first row of data,
this reason cause garbled , or is there a problem with config**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.