Support aggregations on scripted fields
- Dominant language
- Python
- Stars
- 693
- Forks
- 112
- PR merge metrics
- No merged PRs in 30d
Description
The following code exposed a issues with scripted fields:
Assigning a new field based on concat of 2 fields failed:
```
> ed_flights['new_field'] = ed_flights['DestCountry'] + ed_flights['OriginCountry']
E TypeError: 'DataFrame' object does not support item assignment
```
Aggs on scripted fields:
```
s = ed_flights['DestCountry'] + ed_flights['OriginCountry']
print(s.nunique())
```
This returns 0 rather than the results of:
```
{
"aggs": {
"dc_DestCountry_OriginCountry": {
"cardinality": {
"script": {
"source": "doc['DestCountry'].value+doc['OriginCountry'].value"
}
}
}
}
}
```
This is probably due to aggs not works on scripted fields unless script is in the agg. i.e.
```
GET flights/_search
{
"script_fields": {
"dc_DestCountry_OriginCountry": {
"script": {
"source": "doc['DestCountry'].value+doc['OriginCountry'].value"
}
}
},
"aggs": {
"dc_DestCountry_OriginCountry": {
"cardinality": {
"field": "dc_DestCountry_OriginCountry"
}
}
}
}
```
returns 0 results.
Contributor guide
Research direction
Start by reproducing the DataFrame item-assignment and scripted-field aggregation examples in the issue, then trace the DataFrame field and aggregation paths that handle them. Done means assigning a concatenated field works and aggregations such as nunique return the scripted-field results described in the examples.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, pandas, python
- Domain
- data, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100