NVIDIA / NVIDIA/DALI

DataNode to integer

Open
#3,418 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted
Dominant language
C++
Stars
5.8k
Forks
678
Avg merge
3d 1h
Merged PRs (30d)
27

Description

I'm trying to implement somehow more complicated pipeline using Dali and Triton, I want for it to look like this:

  • image goes to preprocessing Dali pipeline, which results in preprocessed image
  • preprocessed image goes to neural net hosted on Triton, this neural net produces output in such format [number_of_bboxes,x,y,width,height,score,label,x,y,width,height...] lets call it predictions. In config.pbtxt it has size of 6001,1,1 (1000 is max number of detections)
  • this output and also preprocessed image should both go to another Dali pipe which should prepare batch of cropped bounding boxes from this image (using output from neural network), and those crops in one batch should go to another neural network
  • then whole pipeline should return both predictions of first neural network on whole image and all predictions on batched crops from second neural network.

Is it possible to do this in such way? Right now I'm stuck at trying to get number of bboxes from first neural network predictions so I can iterate over those bboxes. It seems that it is impossible to get DataNode as int, and use it in for loop.
Code for the second Dali element looks like this:

@dali.pipeline_def(batch_size=1, num_threads=64, device_id=0)
def pipe():
    detections = dali.fn.external_source(
        device="gpu", name="DETECTIONS", batch=False)
    image = dali.fn.external_source(
        device="gpu", name="preprocess_output", batch=False)
    batch = []
    number = dali.fn.cast(detections[0], dtype=dali.types.INT64)
    for i in range(0, number, 6):
        x = detections[i+1]
        y = detections[i+2]
        width = detections[i+3]
        height = detections[i+4]
        batch.append(dali.fn.resize(
            image[y:y+height, x:x+width], resize_x=640,
            resize_y=640, dtype=dali.types.FLOAT))
    batch_result = dali.fn.expand_dims(batch[0], axes=[0])
    for i in range(1, len(batch)):
        dali.fn.cat(batch_result, batch[i], axis=0)
    return batch_result

pipe().serialize(filename="/home/somewhere/model.dali")

Error that I'm getting is:

Traceback (most recent call last):
  File "1_test.py", line 26, in <module>
    pipe().serialize(filename="/home/somewhere/model.dali")
  File "/home/user/.local/lib/python3.8/site-packages/nvidia/dali/pipeline.py", line 1346, in create_pipeline
    pipe_outputs = func(*args, **fn_kwargs)
  File "1_test.py", line 12, in pipe
    for i in range(0, number, 6):
TypeError: 'DataNode' object cannot be interpreted as an integer

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pipeline definition in pipe(), especially the DataNode value used by range() and the final serialize() call. Review how DALI handles DataNode values and dynamic detection counts, then determine whether this pipeline can produce crops from the runtime count and return both prediction outputs; done means the supported approach or limitation is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.