elastic / elastic/logstash

UTF-8 strictness is Event field setter

Open
#6,035 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

This is a followup on #5349

I am a bit perplexed on the encoding validation upon setting an Event field string value with non UTF-8 encoding. I wrote the following spec

``` ruby
it "should fail on non UTF-8 encoding" do
e = LogStash::Event.new
s1 = "\xE0 Montr\xE9al".force_encoding("ISO-8859-1")
expect(s1.encoding.name).to eq("ISO-8859-1")
expect(s1.valid_encoding?).to eq(true)
e.set("test", s1)
s2 = e.get("test")

# it currently fails on this line below
expect(s2.encoding.name).to eq("UTF-8")
expect(s2.valid_encoding?).to eq(true)
end
```

This tests fails a the `expect(s2.encoding.name).to eq("UTF-8")` line with

```
1) LogStash::Event []= should fail on non UTF-8 encoding
Failure/Error: expect(s2.encoding.name).to eq("UTF-8")

expected: "UTF-8"
got: "ISO-8859-1"
```

Should we enforce string UTF-8 encoding at the Event field setter? It could be done in the Event class or in the BiValue transformations.

Thoughts?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.