tensorflow / tensorflow/models
Accuracy of MobileNetV3-minimalistic model is different from given checkpoint
@marksandler2 is already working on this.
Since May 29, 2020.
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
System information
- What is the top-level directory of the model you are using: /research/slim/nets/mobilenet
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
- TensorFlow installed from (source or binary): source
- TensorFlow version (use command below): 1.15.0
- Bazel version (if compiling from source): 0.24.1
- CUDA/cuDNN version: CUDA 10.0 CuDNN 7
- GPU model and memory: 5 Titan XP / 12G
- Exact command to reproduce:
python research/slim/train_image_classifier.py \
--train_dir={TRAIN_DIR} \
--dataset_name=imagenet \
--dataset_split_name=train \
--dataset_dir={DATASET_DIR} \
--model_name=v3_large_min \
--learning_rate=0.1 \
--preprocessing_name="inception_v2" \
--label_smoothing=0.1 \
--moving_average_decay=0.9999 \
--batch_size=64 \
--num_readers=16 \
--num_preprocessing_threads=16 \
--num_clones=5 \
--learning_rate_decay_factor=0.01 \
--num_epoches_per_decay=3.0 \
--weight_decay=0.00001
Custom code
-
nets_factory.py
Since there is no keys for MobilenetV3 in mapping dictionary in nets_factory.py, I added 'v3_large_min' in networks_map and arg_scopes_map (I think it should be added in official repo) -
mobilenet.py
According to paper, I change default value of bn_decay in training_scope to 0.99
Describe the problem
Since I want to train MobileNetV3 Large minimalistic (float) model, I measured accuracy from pre-trained checkpoint (it was step 342500). It performs 72.292% in Top 1 Accuracy. (I use eval_image_classifier.py)
However, when I train and evaluate this model from scratch, it seems to have 61% peak accuracy.
(even more than 342500 steps)
As you can see in command, I use hyperparameter according to MobileNetV3 paper.
Did I miss something for training (e.g. other hyperparameter)? If then, can you share them?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.