Odo discover chokes on mongoDB DBRef types
- Dominant language
- Python
- Stars
- 1k
- Forks
- 131
- PR merge metrics
- No merged PRs in 30d
Description
Odo is choking on a mongoDB DBRef types
not sure if this is by design but I wanted to make you aware of it.
The following code shows the issue:
_(connection actuals are hidden and printed uri changed to show masked actuals)_
``` python
from blaze import odo
from odo import resource, discover
collections = []
mongo_template = u"mongodb://{0}:{1}@{2}:{3}/{4}::{5}"
db_uri_template = mongo_template.split("::")[0]
db_uri = db_uri_template.format(username,password,host,port,db)# defined elsewhere
collections = resource(db_uri).collection_names()
uri=mongo_template.format(username,password,host,port,db,collections[6])
print('URI:')
print(uri)
print()
entity_state=resource(uri)
print('mongo record:')
print(entity_state.find_one())
ds=discover(entity_state)
```
yields the following output:
``` python
URI:
mongodb://user:pass@host:27017/db_name::entityState
mongo record:
{
'values': [],
'entity': DBRef('entityLifecycle', ObjectId('5525dd21a4a4b2a53ea03b53')),
'entityType': 'ORG',
'name': 'HistoricalValidation',
'_id': ObjectId('5525dd22a4a4b2a53ea03b61'),
'lastModifiedDate': datetime.datetime(2015, 4, 9, 2, 0, 2, 17000),
'createdDate': datetime.datetime(2015, 4, 9, 2, 0, 2, 17000),
'EntityId': 'e6bdf2cc-10ca-4107-9435-3b34162c8fff',
'timestamp': datetime.datetime(2015, 4, 9, 2, 0, 0, 1000)
}
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
in ()
8 print('mongo record:')
9 print(entity_state.find_one())
---> 10 ds=discover(entity_state)
C:\Anaconda3\envs\oz\lib\site-packages\multipledispatch\dispatcher.py in __call__(self, *args, **kwargs)
162 self._cache[types] = func
163 try:
--> 164 return func(*args, **kwargs)
165
166 except MDNotImplementedError:
C:\Anaconda3\envs\oz\lib\site-packages\odo\backends\mongo.py in discover_pymongo_collection(coll, n)
26 del item[col]
27
---> 28 ds = discover(items)
29
30 if isdimension(ds[0]):
C:\Anaconda3\envs\oz\lib\site-packages\multipledispatch\dispatcher.py in __call__(self, *args, **kwargs)
162 self._cache[types] = func
163 try:
--> 164 return func(*args, **kwargs)
165
166 except MDNotImplementedError:
C:\Anaconda3\envs\oz\lib\site-packages\datashape\discovery.py in discover(seq)
220 try:
221 types = [unite([discover(data) for data in column]).subshape[0]
--> 222 for column in columns]
223 return len(seq) * Record(list(zip(keys, types)))
224 except AttributeError:
C:\Anaconda3\envs\oz\lib\site-packages\datashape\discovery.py in (.0)
220 try:
221 types = [unite([discover(data) for data in column]).subshape[0]
--> 222 for column in columns]
223 return len(seq) * Record(list(zip(keys, types)))
224 except AttributeError:
C:\Anaconda3\envs\oz\lib\site-packages\datashape\discovery.py in (.0)
219 columns = [[item.get(key) for item in seq] for key in keys]
220 try:
--> 221 types = [unite([discover(data) for data in column]).subshape[0]
222 for column in columns]
223 return len(seq) * Record(list(zip(keys, types)))
C:\Anaconda3\envs\oz\lib\site-packages\multipledispatch\dispatcher.py in __call__(self, *args, **kwargs)
162 self._cache[types] = func
163 try:
--> 164 return func(*args, **kwargs)
165
166 except MDNotImplementedError:
C:\Anaconda3\envs\oz\lib\site-packages\datashape\discovery.py in discover(obj, **kwargs)
64 )
65 return from_numpy(obj.shape, obj.dtype)
---> 66 raise NotImplementedError("Don't know how to discover type %r" % type_name)
67
68
NotImplementedError: Don't know how to discover type 'DBRef'
```
I'm using the following semi-relevant packages:
```
# packages in environment at C:\Anaconda3\envs\oz:
#
blaze 0.9.1 py35_0
datashape 0.5.1 py35_0
multipledispatch 0.4.8 py35_0
numpy 1.10.4 py35_0
odo 0.4.2 py35_0
pymongo 3.0.3 py35_0
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at odo/backends/mongo.py, where discover_pymongo_collection passes collection items to discovery, then inspect datashape/discovery.py and the reported DBRef failure. Reproduce with a MongoDB collection containing a DBRef and verify that discover completes without the NotImplementedError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100