EscVM / EscVM/Efficient-CapsNet
Performance issues in /utils (by P3)
- Dominant language
- Python
- Stars
- 273
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
Hello! I've found a performance issue in /utils: `batch()` should be called before `map()`, which could make your program more efficient. Here is [the tensorflow document](https://tensorflow.google.cn/guide/data_performance?hl=zh_cn#vectorized_mapping) to support it.
Detailed description is listed below:
- /pre_process_mnist.py: `dataset_train.batch(batch_size)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L126) should be called before `dataset_train.map(image_rotate_random,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L116), `dataset_train.map(image_shift_rand,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L118), `dataset_train.map(image_squish_random,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L120), `dataset_train.map(image_erase_random,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L122) and `dataset_train.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L124).
- /pre_process_mnist.py: `dataset_test.batch(batch_size)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L133) should be called before `dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_mnist.py#L131).
- /pre_process_smallnorb.py: `dataset_train.batch(batch_size)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L83) should be called before `dataset_train.map(random_patches,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L75), `dataset_train.map(random_brightness,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L77), `dataset_train.map(random_contrast,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L79) and `dataset_train.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L81).
- /pre_process_smallnorb.py: `dataset_test.batch(1)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L90) should be called before `dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`[(here)](https://github.com/EscVM/Efficient-CapsNet/blob/4b337f5bf79d70a56e4dec8b113fe54b44cfe963/utils/pre_process_smallnorb.py#L88).
Besides, you need to check the function called in `map()`(e.g., `generator` called in `dataset_test.map(generator,num_parallel_calls=PARALLEL_INPUT_CALLS)`) whether to be affected or not to make the changed code work properly. For example, if `generator` needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).
Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.