mars-project / mars-project/mars
[BUG] Behavior of creating DataFrame from dicts not consistent with pandas
- Dominant language
- Python
- Stars
- 2.7k
- Forks
- 325
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
When creating Mars DataFrame from dicts with Series and supply an extra index, pandas will reindex the series using supplied index object, while Mars merely replaces the whole index.
All DataFrame / Series initializers with extra indices need to be inspected.
**To Reproduce**
```python
import numpy as np
import pandas as pd
import mars.dataframe as md
import mars.tensor as mt
raw_dict = {
'C': np.random.choice(['u', 'v', 'w'], size=(100,)),
'A': pd.Series(np.random.rand(100)),
'B': np.random.randint(0, 10, size=(100,)),
}
idx_array = np.arange(0, 100)
np.random.shuffle(idx_array)
raw_idx = pd.Index(idx_array, name='test_index')
r = md.DataFrame(raw_dict, columns=list('ABC'), index=md.Index(raw_idx)).execute()
# assertion error here
pd.testing.assert_frame_equal(r, pd.DataFrame(raw_dict, columns=list('ABC'), index=raw_idx))
```
Contributor guide
Research direction
Start by running the provided reproduction and comparing Mars DataFrame construction with pandas when a Series and extra index are supplied. Inspect the DataFrame and Series initializers for similar extra-index handling; done when the assertion passes and related initializers match pandas behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100