NVIDIA / NVIDIA/DALI

Use random index using input shape for ElementExtract

Open
#3,201 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C++
Stars
5.8k
Forks
678
Avg merge
3d 1h
Merged PRs (30d)
27

Description

Hi,

I'm trying to extract random N number of elements from varying shape data.

  • As the first step, I tested to extract first 3 elements using ElementExtract and it works.
    x = dali_ops.ElementExtract(device='gpu', element_map=[0,1,2])(x.gpu())

  • Now I want to extract random N numbers similar to this way
    random_idx = np.random.choice(x.shape[0], size=512, replace=False)
    x = dali_ops.ElementExtract(device='gpu', element_map=random_idx)(x.gpu())

  • To get x.shape[0] in dali way, I tried two operators Shapes and PeekImageShape but both returns DataNode rather than integer. So I cannot use the DataNode to generate random numbers. Is there any other way to do this?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ElementExtract, Shapes, and PeekImageShape operator documentation and entry points to understand how shape values and element maps are handled. Determine whether the requested shape-dependent random extraction is supported; done should be a documented, reproducible way to achieve it or a clearly scoped change proposal.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.