elastic / elastic/eland

Support aggregations on scripted fields

Open
#267 3 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted topic:dataframe
Dominant language
Python
Stars
693
Forks
112
PR merge metrics
No merged PRs in 30d

Description

The following code exposed a issues with scripted fields:

Assigning a new field based on concat of 2 fields failed:
```
> ed_flights['new_field'] = ed_flights['DestCountry'] + ed_flights['OriginCountry']
E TypeError: 'DataFrame' object does not support item assignment
```

Aggs on scripted fields:
```
s = ed_flights['DestCountry'] + ed_flights['OriginCountry']
print(s.nunique())
```
This returns 0 rather than the results of:

```
{
"aggs": {
"dc_DestCountry_OriginCountry": {
"cardinality": {
"script": {
"source": "doc['DestCountry'].value+doc['OriginCountry'].value"
}
}
}
}
}
```
This is probably due to aggs not works on scripted fields unless script is in the agg. i.e.

```
GET flights/_search
{
"script_fields": {
"dc_DestCountry_OriginCountry": {
"script": {
"source": "doc['DestCountry'].value+doc['OriginCountry'].value"
}
}
},
"aggs": {
"dc_DestCountry_OriginCountry": {
"cardinality": {
"field": "dc_DestCountry_OriginCountry"
}
}
}
}
```

returns 0 results.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the DataFrame item-assignment and scripted-field aggregation examples in the issue, then trace the DataFrame field and aggregation paths that handle them. Done means assigning a concatenated field works and aggregations such as nunique return the scripted-field results described in the examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, pandas, python
Domain
data, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.