ipython / ipython/ipykernel

Dealing with non-contiguous memoryviews as binary buffers

Aperta
#245 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
734
Fork
412
Merge medio
1g 5h
PR unite (30g)
8

Descrizione

While testing the latest ipywidgets support for passing numpy arrays as binary buffers with metadata, I inadvertedly tried to assign a numpy slice to a `traittypes.Array` trait, and this custom function (lifted from pythreejs) then creates a `memoryview` of a non-contiguous `ndarray`:
```
def array_to_json(value, widget):
return {
'shape': value.shape,
'dtype': str(value.dtype),
'buffer': memoryview(value)
}
```
which shortly after fails inside zmq:
```
ERROR:tornado.general:Uncaught exception, closing connection.
Traceback (most recent call last):
File "/home/martinal/envs/visenv-dev/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "/home/martinal/envs/visenv-dev/lib/python3.5/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/martinal/envs/visenv-dev/lib/python3.5/site-packages/ipykernel/iostream.py", line 105, in _handle_event
event_f()
File "/home/martinal/envs/visenv-dev/lib/python3.5/site-packages/ipykernel/iostream.py", line 199, in
self.schedule(lambda : self._really_send(*args, **kwargs))
File "/home/martinal/envs/visenv-dev/lib/python3.5/site-packages/ipykernel/iostream.py", line 207, in _really_send
self.socket.send_multipart(msg, *args, **kwargs)
File "/home/martinal/envs/visenv-dev/lib/python3.5/site-packages/zmq/sugar/socket.py", line 366, in send_multipart
self.send(msg, SNDMORE|flags, copy=copy, track=track)
File "zmq/backend/cython/socket.pyx", line 636, in zmq.backend.cython.socket.Socket.send (zmq/backend/cython/socket.c:7305)
File "zmq/backend/cython/socket.pyx", line 683, in zmq.backend.cython.socket.Socket.send (zmq/backend/cython/socket.c:7048)
File "zmq/backend/cython/socket.pyx", line 188, in zmq.backend.cython.socket._send_copy (zmq/backend/cython/socket.c:2815)
File "zmq/utils/buffers.pxd", line 200, in zmq.utils.buffers.asbuffer_r (zmq/backend/cython/socket.c:8876)
File "zmq/utils/buffers.pxd", line 159, in zmq.utils.buffers.asbuffer (zmq/backend/cython/socket.c:8295)
BufferError: memoryview: underlying buffer is not contiguous
ERROR:tornado.general:Uncaught exception, closing connection.
```

This could be dealt with a number of places:

1) by the user (not very user friendly)
2) by the widget developer (but this is a fairly generic problem)
3) in traittypes.Array.validate, by calling np.ascontiguousarray instead of np.asarray
4) or, as @jasongrout suggests:
> Perhaps the underlying python/zmq interface should use something like PyMemoryView_GetContiguous to get a contiguous buffer.

@minrk any opinions on the matter?

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.