Group by with nunique reduction on bcolz throws NotImplementedError
- Dominant language
- Python
- Stars
- 3.2k
- Forks
- 389
- PR merge metrics
- No merged PRs in 30d
Description
I converted some 2M json records to a bcolz file using odo in the hopes of experimenting with blaze's query features. But unfortunately I am still new to the blaze world so I don't know if this is an error or to be expected for a bcolz backend. The `nunique` function works fine on its own but not when placed within a `by` expression.
If it helps, I can reproduce this with fake data but I thought I'd put this out there to know for sure this is either unexpected or expected behavior.
I am using Python 3.5.2, blaze 0.11.3, bcolz 1.1.0.
```
>>> from blaze import *
>>> d = data('data.bcolz')
>>> nunique(d.username)
932
>>> by(d.app_ver, n=count(d.username))
app_ver n
0 1_0 18109
1 1_0.0 32594
2 1_0.1 11931
3 1_0.4 64228
4 1_0.5 2167670
5 1_1.0 102524
6 1_1.1 36819
7 1_2 21000
8 1_2.0 18
9 1_2.2 37139
>>> by(d.app_ver, n=nunique(d.username))
Traceback (most recent call last):
....
NotImplementedError: Don't know how to compute:
type(expr):
expr: by(_.app_ver, n=nunique(_.birth_date)).head(11)
data: {<`_` symbol; dshape='2830325 * { app_ver: string[10, 'U32'], username...'>: .c object at 0x7fd5804e28e8>}
```
I am using an Anaconda environment and the blaze channel to install the current release:
```{yaml}
name: blaze
channels:
- blaze
dependencies:
- python=3.5.2
- blaze=0.11.3
- pip
- pip:
- bcolz
````
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the documented Python 3.5.2, blaze 0.11.3, and bcolz 1.1.0 example with data('data.bcolz'), comparing nunique(d.username) with by(d.app_ver, n=nunique(d.username)). Trace the by and nunique entry points into the bcolz backend. Done means grouped nunique has a defined, tested result or the backend's unsupported behavior is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100