tensorflow / tensorflow/models
ERROR: NotFoundError - File under path not being found
@pkulzc is already working on this.
Since Jun 23, 2020.
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to train a whole model based on the COCO dataset using this scripts provided but reducing the number of classes to only 6.
I run the download_and_preprocess_coco.sh script which downloads the dataset and calls the create_coco_tf_record.py script which creates the TFRecords from the dataset previously downloaded. After that steps (successfully achieved) I try to run the retrain_detection_model.sh as it is described in the tutorial, but modifying the labels .pdtxt file in order to take into account only 6 clases and modifying the pipeline.config file in order to achieve the same (with a v2 net and training the whole model option).
The first error that came out was:
RuntimeError: Did not find any input files matching the glob pattern [u'/tensorflow/models/research/tmp/mscoco/coco_train.record-00001-of-00010']
When I do have a file under: /tensorflow/models/research/tmp/mscoco/ which contains files of the following format:
coco_testdev.record-00000-of-00100
coco_train.record-00024-of-00100
coco_val.record-00001-of-00010
Being the first set of 5 numbers after the record part numbers that go from 00000 to 00099.
So I do have those files that the error reports I do not have, and I have the PATH specified in the pipeline.config file.
I managed to move on a bit by skipping the use of the glob library in the dataset_builder.py script under the route research/object_detection/builders/. It is not working as it should, so by just removing the use of it the script runs a bit ahead, but it still throws and error:
NotFoundError (see above for traceback): /tensorflow/models/research/tmp/mscoco/coco_train.record-00001-of-00010; No such file or directory
[[node IteratorGetNext (defined at object_detection/model_main.py:105) = IteratorGetNext[output_shapes=[[128], [128,300,300,3], [128,2], [128,3], [128,100], [128,100,4], [128,100,2], [128,100,2], [128,100], [128,100], [128,100], [128]],
output_types=[DT_INT32, DT_FLOAT, DT_INT32, DT_INT32, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT32, DT_BOOL, DT_FLOAT, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorV2)]]
I have not figured out how to move on from here.
I paste my pipeline.config file:
model {
ssd {
num_classes: 2
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
}
}
feature_extractor {
type: "ssd_mobilenet_v2"
depth_multiplier: 1.0
min_depth: 16
conv_hyperparams {
regularizer {
l2_regularizer {
weight: 3.99999989895e-05
}
}
initializer {
random_normal_initializer {
mean: 0.0
stddev: 0.00999999977648
}
}
activation: RELU_6
batch_norm {
decay: 0.97000002861
center: true
scale: true
epsilon: 0.0010000000475
}
}
override_base_feature_extractor_hyperparams: true
}
box_coder {
faster_rcnn_box_coder {
y_scale: 10.0
x_scale: 10.0
height_scale: 5.0
width_scale: 5.0
}
}
matcher {
argmax_matcher {
matched_threshold: 0.5
unmatched_threshold: 0.5
ignore_thresholds: false
negatives_lower_than_unmatched: true
force_match_for_each_row: true
use_matmul_gather: true
}
}
similarity_calculator {
iou_similarity {
}
}
box_predictor {
convolutional_box_predictor {
conv_hyperparams {
regularizer {
l2_regularizer {
weight: 3.99999989895e-05
}
}
initializer {
random_normal_initializer {
mean: 0.0
stddev: 0.00999999977648
}
}
activation: RELU_6
batch_norm {
decay: 0.97000002861
center: true
scale: true
epsilon: 0.0010000000475
}
}
min_depth: 0
max_depth: 0
num_layers_before_predictor: 0
use_dropout: false
dropout_keep_probability: 0.800000011921
kernel_size: 1
box_code_size: 4
apply_sigmoid_to_scores: false
class_prediction_bias_init: -4.59999990463
}
}
anchor_generator {
ssd_anchor_generator {
num_layers: 6
min_scale: 0.20000000298
max_scale: 0.949999988079
aspect_ratios: 1.0
aspect_ratios: 2.0
aspect_ratios: 0.5
aspect_ratios: 3.0
aspect_ratios: 0.333299994469
}
}
post_processing {
batch_non_max_suppression {
score_threshold: 0.300000011921
iou_threshold: 0.600000023842
max_detections_per_class: 100
max_total_detections: 100
}
score_converter: SIGMOID
}
normalize_loss_by_num_matches: true
loss {
localization_loss {
weighted_smooth_l1 {
}
}
classification_loss {
weighted_sigmoid_focal {
gamma: 2.0
alpha: 0.75
}
}
classification_weight: 1.0
localization_weight: 1.0
}
encode_background_as_zeros: true
normalize_loc_loss_by_codesize: true
inplace_batchnorm_update: true
freeze_batchnorm: false
}
}
train_config {
batch_size: 128
data_augmentation_options {
random_horizontal_flip {
}
}
data_augmentation_options {
ssd_random_crop {
}
}
sync_replicas: true
optimizer {
momentum_optimizer {
learning_rate {
cosine_decay_learning_rate {
learning_rate_base: 0.20000000298
total_steps: 50000
warmup_learning_rate: 0.0599999986589
warmup_steps: 2000
}
}
momentum_optimizer_value: 0.899999976158
}
use_moving_average: false
}
fine_tune_checkpoint: "/tensorflow/models/research/learn_human_car/ckpt/model.ckpt"
from_detection_checkpoint: true
load_all_detection_checkpoint_vars: true
num_steps: 50000
startup_delay_steps: 0.0
replicas_to_aggregate: 8
max_number_of_boxes: 100
unpad_groundtruth_tensors: false
}
train_input_reader {
label_map_path: "/tensorflow/models/research/object_detection/data/mscoco_label_map.pbtxt"
tf_record_input_reader {
input_path: "/tensorflow/models/research/tmp/mscoco/coco_train.record-00001-of-00010"
}
}
eval_config {
num_examples: 8000
metrics_set: "coco_detection_metrics"
use_moving_averages: false
}
eval_input_reader {
label_map_path: "/tensorflow/models/research/object_detection/data/mscoco_label_map.pbtxt"
shuffle: false
num_readers: 1
tf_record_input_reader {
input_path: "/tensorflow/models/research/tmp/mscoco/coco_val.record-?????-of-00010"
}
}
graph_rewriter {
quantization {
delay: 48000
weight_bits: 8
activation_bits: 8
}
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.