NVIDIA / NVIDIA/DALI

Error when implementing probabilistic augmentations

Open
#5,544 16 comments 0 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

lack_of_repro question
Dominant language
C++
Stars
5.8k
Forks
678
Avg merge
3d 1h
Merged PRs (30d)
27

Description

Describe the question.

I have tested with DALI 1.38 and DALI 1.39 on CUDA 12.2

I am trying to specify configurations for various augmentation operations in a hydra configuration file and then I want to construct those augmentations inside a pipeline. For instance, following is an example of a configuration file for random_horizontal_flip

Screenshot 2024-06-30 221624

The corresponding implementation is as below :

Screenshot 2024-06-30 221733

In a similar fashion I have configurations for other augmentations as well.

In my main code, I construct a pipeline as follows ( explanation follows )

Screenshot 2024-06-30 221952

So, as you can see following is the sequence of operations

  1. You pass a set of hydra configuration files to a class MyDataset .
  2. The corresponding functions are then built as partial functions using hydra.utils.instantiate
  3. Then they are used for augmentation inside the pipeline using self._transform_data(....)

I have chosen this mechanism as it allows me to choose and modify all aspects of data augmentation using Hydra.

However, when running I often get error messages which tend to suggest that something is wrong with dali_fn.random.uniform(...) because whenever I use two augmentations ( e.g :- resize_exact and horizontal_flip ) it will always give me weird errors which will disappear when :

a) Either I don't use any random.uniform function

b) Or I use only one augmentation

what is going wrong in my implementation ? Any hints ? Shouldn't NVIDIA DALI consider random.uniform nodes defined in different functions as different nodes ?

Check for duplicates
  • I have searched the open bugs/issues and have found no duplicates for this bug report

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.