NVIDIA / NVIDIA/DALI

[Question] How optimally load sequence of 16 bit images?

Open
#2,535 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
C++
Stars
5.8k
Forks
678
Avg merge
3d 1h
Merged PRs (30d)
27

Description

Hi, I'am seeing for optimal way of loading stack of PNG uint16 images with fixed sequence length 1 < N <= 8 and GT image as label.
I thinking about few cases:

  1. N images saved in dir and GT in other dir
  2. Save all images as tfrecord with corresponding labels for stack and GT

For 1 case it looks like SequenceReader should be used, but I am not see an option to load GT image - should I just load a sequence with GT and take a split from this or maybe another way?

For 2 case I met the next problem - I can customize TFRecord reader, but can't decode that due to stack of images, what the right way to customly decode this sequence?
Currently I use next code based from tensorflow RecordDataset and want to adapt this with DALI:

labeled_tfrec_format = {
'n_imgs': tf.io.FixedLenFeature([], tf.int64),
'height': tf.io.FixedLenFeature([], tf.int64),
'width': tf.io.FixedLenFeature([], tf.int64),
'channels': tf.io.FixedLenFeature([], tf.int64),
'image_raw': tf.io.FixedLenFeature([], tf.string),
'gt_raw': tf.io.FixedLenFeature([], tf.string)}
def read_tfrecord(example):
        example = tf.io.parse_single_example(example, labeled_tfrec_format)
        image = tf.io.decode_raw(example['image_raw'], tf.uint16)
        image = tf.reshape(image, (example['N'], example['H'], example['W'], example['channels']))
        gt = tf.io.decode_raw(example['gt_raw'], tf.uint16)
        gt = tf.reshape(gt, (example['H'], example['W'], example['channels']))
        return image, gt
ds = tf.data.TFRecordDataset(some_pathes)
ds = ds.map(read_tfrecord)
ds = ds.map(..)  # some reshapes, crops, etc...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the SequenceReader and TFRecord reader paths mentioned in the issue, then compare them with the provided Python parsing example. Determine whether loading a variable-length image sequence with a separate GT image is already supported or needs a documented capability; done means a clear recommended approach for both storage cases and decoding uint16 stacks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.