elastic / elastic/logstash

Fieldref StringInterpolation and Fixnum/Float values

Open
#5,114 19 comments 0 reactions 0 assignees View on GitHub
breaking change discuss enhancement
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

In 2.3 we have introduced a change to deal with reference values when applying string interpolation for fieldref (see #4592). This change broke backward compatibility with existing configuration that were using the `add_field` function to copy `Fixnum` and `Float` values (#4961) into new fields.

Before 2.3, the string interpolation wasn't doing pessimistic conversion of the value into a string and was keeping the original type.

**Example:**

``` ruby
e = LogStash::Event.new({ "data" => 0.9)

# LS < 2.3
e.sprintf("%{data}") => 0.9

# LS > 2.3
e.sprintf("%{data}") => "0.9"
```

Since we are explicitly using **quoted string** to use the fieldref reference, I think its fair to expect it to behave like the [Kernel.sprintf](http://ruby-doc.org/core-2.2.0/Kernel.html#method-i-sprintf) method and always return a string. (The underlying method is actually called sprintf)

I think for 5.0 we should make `sprintf` always return a string and this is already the case in the current Java Event implementation. The downside of this, people will actually need to manually convert the value using the mutate filter if they really need a specific type.

```
mutate
{
convert => { "src_ip_coords" => "float" }
}
```

In #4961 @colinsurprenant also suggested to support non-quoted string to keep the original value, this could be something to investigate.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the changes described in #4592 and #4961, then locate the current fieldref string-interpolation and sprintf implementations, including the Java Event implementation mentioned here. Determine whether sprintf should always return a string or whether unquoted references should preserve types; done means the behavior and compatibility direction are decided and covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, ruby
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.