tensorflow / tensorflow/models
Add additional channel to pre-trained resnet-50 model
@jaeyounkim is already working on this.
Since May 15, 2020.
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
What is the top-level directory of the model you are using:
from nets.resnet_v1 import bottleneck, resnet_arg_scope
Have I written custom code:
Yes, I am trying to use pre-trained Resnet50 model. However, I intend to modify the first convolution layer to be able to take a 4-channel tensor(image+binary mask), where the network weights (including the first convolutional layer 3 channels) are initialized from the pre-trained model, except for the newly added channel which are initialized randomly.
Adding this feature to existing Resnet models would be really helpful for segmentation tasks.
OS Platform and Distribution:
Ubuntu 16.04
TensorFlow installed from:
Anaconda
TensorFlow version:
1.12.0
Bazel version:
N/A
CUDA/cuDNN version:
CUDA - 10.0
GPU model and memory:
Nvidia Titan Xp 12GB
Exact command to reproduce:
[I intend to add extra channel to the first convolutional layer, while still using the pre-trained Imagenet weights on the other channels of the first convolutional layer]
with tf.contrib.slim.arg_scope(resnet_arg_scope(batch_norm_decay=0.9, weight_decay=0.0)):
with tf.variable_scope('resnet_v1_50', values=[image]) as sc:
end_points_collection = sc.name + '_end_points'
with slim.arg_scope([slim.conv2d, bottleneck],
outputs_collections=end_points_collection):
with slim.arg_scope([slim.batch_norm], is_training=is_training):
net = image
net = conv2d_same(net, 64, 7, stride=2, scope='conv1')
net = slim.max_pool2d(net, [3, 3], stride=2, scope='pool1')
with tf.variable_scope('block1', values=[net]) as sc_block:
with tf.variable_scope('unit_1', values=[net]):
net = bottleneck(net, depth=4*64, depth_bottleneck=64, stride=1)
with tf.variable_scope('unit_2', values=[net]):
net = bottleneck(net, depth=4*64, depth_bottleneck=64, stride=1)
with tf.variable_scope('unit_3', values=[net]):
net = bottleneck(net, depth=4*64, depth_bottleneck=64, stride=2)
with tf.variable_scope('block2', values=[net]) as sc_block:
with tf.variable_scope('unit_1', values=[net]):
net = bottleneck(net, depth=4*128, depth_bottleneck=128, stride=1)
with tf.variable_scope('unit_2', values=[net]):
net = bottleneck(net, depth=4*128, depth_bottleneck=128, stride=1)
with tf.variable_scope('unit_3', values=[net]):
net = bottleneck(net, depth=4*128, depth_bottleneck=128, stride=1)
with tf.variable_scope('unit_4', values=[net]):
net = bottleneck(net, depth=4*128, depth_bottleneck=128, stride=2)
with tf.variable_scope('block3', values=[net]) as sc_block:
with tf.variable_scope('unit_1', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_2', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_3', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_4', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_5', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_6', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=2)
net = nonlocal_dot(net, depth=512, embed=True, softmax=True, maxpool=2, scope='nonlocal3')
with tf.variable_scope('block4', values=[net]) as sc_block:
with tf.variable_scope('unit_1', values=[net]):
net = bottleneck(net, depth=4*512, depth_bottleneck=512, stride=1)
with tf.variable_scope('unit_2', values=[net]):
net = bottleneck(net, depth=4*512, depth_bottleneck=512, stride=1)
with tf.variable_scope('unit_3', values=[net]):
net = bottleneck(net, depth=4*512, depth_bottleneck=512, stride=2)
net = tf.reduce_mean(net, [1, 2], name='pool5', keep_dims=False)
endpts = slim.utils.convert_collection_to_dict(
end_points_collection)
endpts['model_output'] = endpts['global_pool'] = net
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.