NVIDIA / NVIDIA/TensorRT

Error return type mismatch in onnx_helper.py

Open
#4,612 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Module:ONNX
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

In line 86 in quickstart/IntroNotebooks/onnx_helper.py, in the method predict err gets assigned the result of cudart.cudaMemcpyAsync which as per the documentation should be of type cudart.cudaError_t.
The same goes for cudart.cudaStreamSynchronize in line 107

However the actual calls return a tuple int the form (err, ) which leads to the following error being thrown when the method is called

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[5], line 1
----> 1 predictions = trt_model.predict(dummy_input_batch)

File [/data/gsg/onnx_helper.py:90](http://192.168.2.115:8888/lab/tree/data/gsg/data/gsg/onnx_helper.py#line=89), in ONNXClassifierWrapper.predict(self, batch)
     86 err = cudart.cudaMemcpyAsync(
     87     self.d_input, batch.ctypes.data, batch.nbytes, cudart.cudaMemcpyKind.cudaMemcpyHostToDevice, self.stream
     88 )
     89 if err != cudart.cudaError_t.cudaSuccess:
---> 90     raise RuntimeError(f"Failed to copy input to device: {cudart.cudaGetErrorString(err)}")
     92 # execute model
     93 self.context.execute_async_v3(self.stream)

File [/usr/local/lib/python3.10/dist-packages/cuda/cudart.pyx:13554](http://192.168.2.115:8888/lab/tree/data/gsg/usr/local/lib/python3.10/dist-packages/cuda/cudart.pyx#line=13553), in cuda.cudart.cudaGetErrorString()

AttributeError: 'tuple' object has no attribute 'value'

This can be remedied by checking whether err is a tuple and proceeding accordingly. Another variation may be to directly unpack the returned tuple.

The code is run under jetson-containers run $(autotag l4t-ml) with L4T 36.4.4 on Jetson Orin Nano 8gb Developer Kit

!pip list | grep cuda
cuda-python                  12.6.0+0.gb9f40f6.dirty   /usr/local/lib/python3.10/dist-packages pip
jax-cuda12-pjrt              0.6.0.dev20250414         /usr/local/lib/python3.10/dist-packages pip
jax-cuda12-plugin            0.6.0.dev20250414         /usr/local/lib/python3.10/dist-packages pip
pycuda                       2024.1.2                  /usr/local/lib/python3.10/dist-packages pip

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in quickstart/IntroNotebooks/onnx_helper.py at the predict method, especially lines 86 and 107, and reproduce the failure by calling predict with the notebook's dummy_input_batch. Check the return values from cudaMemcpyAsync and cudaStreamSynchronize against the documented types. Done means predict handles the returned values without the tuple-related AttributeError on the reported Jetson environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.