Error preprocessing Imagenet for GoogleNet: AttributeError: 'NDArray' object has no attribute '__array_interface__'
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 9.8k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
Hello all, I am trying to evaluate GoogleNet model. I have successfully evaluated other models with the Imagenet dataset. However, GoogleNet requires a different preprocessing, which I cannot reproduce successfully.
# Preprocessing function for ImageNet models using numpy
def transform(img, label):
'''
Preprocessing required on the images for inference with mxnet gluon
The function takes loaded image and returns processed tensor
'''
img = np.array(Image.fromarray(img).resize((224, 224))).astype(np.float32)
img[:, :, 0] -= 123.68
img[:, :, 1] -= 116.779
img[:, :, 2] -= 103.939
img[:,:,[0,1,2]] = img[:,:,[2,1,0]]
img = img.transpose((2, 0, 1))
img = np.expand_dims(img, axis=0)
return img,label
##### Evaluate the entire dataset #############
def evaluate(data_dir, order):
print("Evaluate")
ctx = [mx.cpu()]
# batch size (set to 1 for cpu)
batch_size = 1
# number of preprocessing workers
num_workers = multiprocessing.cpu_count()
val_data = gluon.data.DataLoader(
imagenet.classification.ImageNet(data_dir, train=False).transform(transform),
batch_size=batch_size, shuffle=False, num_workers=num_workers)
ort_session_cpu = ort.InferenceSession(onnx_model.SerializeToString())
# Compute evaluations
print("----Running ONNXRuntime----")
num_batches = int(50000/batch_size)
print('[0 / %d] batches done'%(num_batches))
# Loop over batches
for i, batch in enumerate(val_data):
data = gluon.utils.split_and_load(batch[0], ctx_list=ctx, batch_axis=0)
label = gluon.utils.split_and_load(batch[1], ctx_list=ctx, batch_axis=0)
print(label)
# Perform forward pass
ort_inputs_cpu = {ort_session_cpu.get_inputs()[0].name: data[0].asnumpy()}
print("before run")
outputs=ort_session_cpu.run(None, ort_inputs_cpu)
....
When I execute the test, I obtain the following error:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "../miniconda3/envs/ONNX_env/lib/python3.11/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
^^^^^^^^^^^^^^^^^^^
File "../miniconda3/envs/ONNX_env/lib/python3.11/site-packages/mxnet/gluon/data/dataloader.py", line 400, in _worker_fn
batch = batchify_fn([_worker_dataset[i] for i in samples])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "../miniconda3/envs/ONNX_env/lib/python3.11/site-packages/mxnet/gluon/data/dataloader.py", line 400, in <listcomp>
batch = batchify_fn([_worker_dataset[i] for i in samples])
~~~~~~~~~~~~~~~^^^
File "../miniconda3/envs/ONNX_env/lib/python3.11/site-packages/mxnet/gluon/data/dataset.py", line 124, in __getitem__
return self._fn(*item)
^^^^^^^^^^^^^^^
File "../TODOgooglenet_inf_get_params.py", line 51, in transform
img = np.array(Image.fromarray(img).resize((224, 224))).astype(np.float32)
^^^^^^^^^^^^^^^^^^^^
File "../miniconda3/envs/ONNX_env/lib/python3.11/site-packages/PIL/Image.py", line 2982, in fromarray
arr = obj.__array_interface__
^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NDArray' object has no attribute '__array_interface__'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "../TODOgooglenet_inf_get_params.py", line 324, in <module>
evaluate(dataset, 'B')
File "../TODOgooglenet_inf_get_params.py", line 165, in evaluate
for i, batch in enumerate(val_data):
File "../miniconda3/envs/ONNX_env/lib/python3.11/site-packages/mxnet/gluon/data/dataloader.py", line 451, in __next__
batch = pickle.loads(ret.get()) if self._dataset is None else ret.get()
^^^^^^^^^
File "../miniconda3/envs/ONNX_env/lib/python3.11/multiprocessing/pool.py", line 774, in get
raise self._value
AttributeError: 'NDArray' object has no attribute '__array_interface__'
Any idea on this?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at TODOgooglenet_inf_get_params.py line 51 and inspect the object returned by ImageNet's transform before the PIL Image.fromarray call. Reproduce the failure through the DataLoader with the shown preprocessing and check whether the input type is compatible with PIL. Done means the ImageNet preprocessing completes without the AttributeError and evaluation can proceed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100