EscVM / EscVM/Efficient-CapsNet

Performance issues in /utils (by P3)

Open
#12 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
273
Forks
60
PR merge metrics
No merged PRs in 30d

Description

Hello! I've found a performance issue in /utils: `batch()` should be called before `map()`, which could make your program more efficient. Here is [the tensorflow document](https://tensorflow.google.cn/guide/data_performance?hl=zh_cn#vectorized_mapping) to support it.

Detailed description is listed below:

- /pre_process_mnist.py: `dataset_train.batch(batch_size)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L126) should be called before `dataset_train.map(image_rotate_random,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L116), `dataset_train.map(image_shift_rand,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L118), `dataset_train.map(image_squish_random,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L120), `dataset_train.map(image_erase_random,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L122) and `dataset_train.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L124).
- /pre_process_mnist.py: `dataset_test.batch(batch_size)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L133) should be called before `dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L131).
- /pre_process_smallnorb.py: `dataset_train.batch(batch_size)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L83) should be called before `dataset_train.map(random_patches,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L75), `dataset_train.map(random_brightness,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L77), `dataset_train.map(random_contrast,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L79) and `dataset_train.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L81).
- /pre_process_smallnorb.py: `dataset_test.batch(1)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L90) should be called before `dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L88).

Besides, you need to check the function called in `map()`(e.g., `generator` called in `dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`) whether to be affected or not to make the changed code work properly. For example, if `generator` needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.