elastic / elastic/eland

Add heuristic for detecting string dtype -> text in Elasticsearch instead of always keyword

Open
#286 0 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted topic:dataframe
Dominant language
Python
Stars
693
Forks
112
PR merge metrics
No merged PRs in 30d

Description

When calling `pandas_to_eland()` it'd be good to start creating `text` fields if data is:

- Longer than 255 characters (longer than default keyword)
- Contains 5+ spaces (ML magic number)
- Has a high cardinality of values compared to entries (This one is hard to detect)

Contributor guide

Open the contributing guide

Research direction

Start by locating pandas_to_eland() and inspect its current string-to-Elasticsearch mapping behavior. Define and test how string length, whitespace, and cardinality should influence text versus keyword selection, including the unspecified high-cardinality case. Done means the behavior is covered and pandas data produces the intended Elasticsearch mapping.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, pandas, python
Domain
data, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.