How to read data with multiple ndarrays linked to one label?
Open
Nobody has claimed this yet.
question
- Dominant language
- C++
- Stars
- 5.8k
- Forks
- 678
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 27
Description
When an image such as jpeg is linked to the label,I think the pipeline is like this
class DALIPipeline(Pipeline):
def __init__(self):
self.input=nvidia.dali.ops.readers.File(file_root='.', file_list='dali.txt')
self.decode = nvidia.dali.ops.decoders.Image(device = "mixed", output_type = types.DALIImageType.RGB)
def define_graph(self):
images, labels = self.input(name="Reader")
images = self.decode(images)
but, I don't know how to cope with the case that image1.npy &image2.npy are linked to a label(0 or 1 or 2...)
I have train.csv which has like below
target,file_path,extra_file_path
1,~~/id0.npy,~~~/id0.npy
0,~~/id1.npy,~~~/id1.npy
0,~~/id2.npy,~~~/id2.npy
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the shown DALIPipeline.define_graph entry point and the train.csv and dali.txt input formats. Check the documented reader and ndarray-loading entry points, then define what a supported example for multiple .npy files sharing one label should demonstrate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data-engineering, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100