Method to check that two Dask Series have the same index
- Dominant language
- Python
- Stars
- 89
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
Hi - I discussed this a bit with @phofl , as I'm aiming to have zero-cost-abstraction around Dask DataFrame in [Narwhals](https://github.com/narwhals-dev/narwhals)
One thing I'd like to check is how to check that two Dask Series have the same index. Or, rather, that concatenating them would not result in any index alignment
Patrick pointed me to `dask_expr._expr.are_co_aligned`, which seemed to work great for me until I tried using `__getitem__`. Here's an example:
```python
In [21]: df = dd.from_pandas(pd.DataFrame({'a': [1,2,3], 'b': [4,5,6]}))
In [22]: dask_expr._expr.are_co_aligned(df._expr, df['a'][[1,2,0]]._expr)
Out[22]: True
```
This isn't quite what I was expecting - if I compute `df['a'][[1,2,0]]`, then the index has indeed been shuffled with respect to `df`
Is this a bug in `are_co_aligned`? If not, is there another way to check that index alignment does _not_ happen?
Thanks 🙏
Contributor guide
Research direction
Reproduce the example using dask_expr._expr.are_co_aligned, df['a'][[1,2,0]], and a computed result. Read the implementation and related alignment tests to determine whether the reported result is expected, then define and verify a supported way to detect when concatenation will avoid index alignment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100