elastic / elastic/eland

Difference between pandas.DataFrame.min() and eland.DataFrame.min()

Open
#121 3 comments 0 reactions 0 assignees View on GitHub
help wanted topic:dataframe
Dominant language
Python
Stars
693
Forks
112
PR merge metrics
No merged PRs in 30d

Description

```
>>> import eland as ed
>>> edf = ed.DataFrame('localhost', 'flights')
>>> pdf = ed.eland_to_pandas(edf)
>>> pdf
AvgTicketPrice Cancelled Carrier Dest DestAirportID ... OriginLocation OriginRegion OriginWeather dayOfWeek timestamp
0 841.265642 False Kibana Airlines Sydney Kingsford Smith International Airport SYD ... {'lon': '8.570556', 'lat': '50.033333'} DE-HE Sunny 0 2018-01-01 00:00:00
1 882.982662 False Logstash Airways Venice Marco Polo Airport VE05 ... {'lon': '18.60169983', 'lat': '-33.96480179'} SE-BD Clear 0 2018-01-01 18:27:00
2 190.636904 False Logstash Airways Venice Marco Polo Airport VE05 ... {'lon': '12.3519', 'lat': '45.505299'} IT-34 Rain 0 2018-01-01 17:11:14
3 181.694216 True Kibana Airlines Treviso-Sant'Angelo Airport TV01 ... {'lon': '14.2908', 'lat': '40.886002'} IT-72 Thunder & Lightning 0 2018-01-01 10:33:28
4 730.041778 False Kibana Airlines Xi'an Xianyang International Airport XIY ... {'lon': '-99.072098', 'lat': '19.4363'} MX-DIF Damaging Wind 0 2018-01-01 05:13:00
... ... ... ... ... ... ... ... ... ... ... ...
13054 1080.446279 False Logstash Airways Xi'an Xianyang International Airport XIY ... {'lon': '10.3927', 'lat': '43.683899'} IT-52 Sunny 6 2018-02-11 20:42:25
13055 646.612941 False Logstash Airways Zurich Airport ZRH ... {'lon': '-97.23989868', 'lat': '49.90999985'} CA-MB Rain 6 2018-02-11 01:41:57
13056 997.751876 False Logstash Airways Ukrainka Air Base XHBU ... {'lon': '-99.072098', 'lat': '19.4363'} MX-DIF Sunny 6 2018-02-11 04:09:27
13057 1102.814465 False JetBeats Ministro Pistarini International Airport EZE ... {'lon': '135.4380035', 'lat': '34.78549957'} SE-BD Hail 6 2018-02-11 08:28:21
13058 858.144337 False JetBeats Washington Dulles International Airport IAD ... {'lon': '138.531006', 'lat': '-34.945'} SE-BD Rain 6 2018-02-11 14:54:34

[13059 rows x 27 columns]
>>> pdf.min()
AvgTicketPrice 100.021
Cancelled False
Carrier ES-Air
Dest Abu Dhabi International Airport
DestAirportID ABQ
DestCityName Abu Dhabi
DestCountry AE
DestRegion AR-B
DestWeather Clear
DistanceKilometers 0
DistanceMiles 0
FlightDelay False
FlightDelayMin 0
FlightDelayType Carrier Delay
FlightNum 00882F6
FlightTimeHour 0
FlightTimeMin 0
Origin Abu Dhabi International Airport
OriginAirportID ABQ
OriginCityName Abu Dhabi
OriginCountry AE
OriginRegion AR-B
OriginWeather Clear
dayOfWeek 0
timestamp 2018-01-01 00:00:00
dtype: object
>>> edf.min()
AvgTicketPrice 100.020531
Cancelled 0.000000
DistanceKilometers 0.000000
DistanceMiles 0.000000
FlightDelay 0.000000
FlightDelayMin 0.000000
FlightTimeHour 0.000000
FlightTimeMin 0.000000
dayOfWeek 0.000000
dtype: float64
>>>
```

Elasticsearch only supports 'min' on numeric + bool + timestamp fields.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Python examples comparing pandas.DataFrame.min() with eland.DataFrame.min() on the flights index. Read the min behavior for Elasticsearch numeric, boolean, and timestamp fields, then determine the project's intended handling of the discrepancy. Done means the expected behavior is resolved and covered by an appropriate test or documented decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, pandas, python
Domain
data, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.