elastic / elastic/beats

Review numeric conversions in the `convert` ingest processor

Open
#46,340 5 comments 0 reactions 0 assignees View on GitHub
needs_team Stalled
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

The Elasticsearch team have an [open issue](https://github.com/elastic/elasticsearch/issues/133153) to review the behaviour of the ES `convert` processor for numeric conversions. This was opened in response to a user request to change the behaviour. The first steps were to investigate, document, and add test coverage for the current behaviour. We are now ready to look at whether to make any changes. One factor we are concerning is to what extent the behaviour of the ES processor aligns with the Beats one ([and the logstash mutation](https://github.com/logstash-plugins/logstash-filter-mutate/issues/173)): all other things being equal, we think that the user is better served if there are fewer inconsistencies across the products.

To establish the current behaviour, it would help to run the list of test cases we have built out for the ES processor against the Beats processor, and identify any cases where the behaviour differs. These cases are described in this CSV: [convert_list.csv](https://github.com/user-attachments/files/22080087/convert_list.csv). The first three columns give the type of the input value (as a Java type, in our case), the input value itself, and the target `type` of the conversion; for the cases where the processor successfully converts the value, the fourth and fifth columns give the output type and value; for the cases where the processor throws an exception, the fourth column says `THROWS` and the fifth is empty.

Thanks for your help.

Contributor guide

Open the contributing guide

Research direction

Start with the attached convert_list.csv and locate the Beats ingest processor's numeric-conversion implementation and tests. Run the listed cases against Beats, compare its results with the documented Elasticsearch outcomes, and record every differing case so the current behavior is established.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, go
Domain
backend, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.