apple / apple/coremltools

How to print model layers shape

Open
#879 5 comments 0 reactions 0 assignees View on GitHub
NN backend only question
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## ❓Question
How to print model layers shape

Use `neural_network.NeuralNetworkBuilder` built a model spec.
when use spec new `MLModel` produce an error:
"compiler error: Espresso exception: "Invalid blob shape": generic_elementwise_kernel: cannot broadcast [1, 14280, 2, 1, 1] and [14280, 2, 1, 1, 1]".

print spec just some layer connection. no shape info.

I'm using coremltools build a model decode bbox like sdd-caffe does.

## System Information
- Mac OS
- coremltools 4.0b2

## Code
```python
fd_mbox_priorbox = np.load('fd_mbox_priorbox.npy')
num_prior_ = int(fd_mbox_priorbox.shape[2] / 4)

# MLMultiArray inputs of neural networks must have 1 or 3 dimensions.
# We only have 2, so add an unused dimension of size one at the back.
input_features = [("scores", datatypes.Array(num_classes + 1, num_prior_, 1)),
("boxes", datatypes.Array(4, num_prior_, 1))]

# The outputs of the decoder model should match the inputs of the next
# model in the pipeline, NonMaximumSuppression. This expects the number
# of bounding boxes in the first dimension.
output_features = [
# ("decoded_bbox_out", datatypes.Array(4, num_prior_, 1)),
("confidence", datatypes.Array(num_prior_, num_classes + 1, 1)),
("coordinates", datatypes.Array(num_prior_, 4, 1))
]

builder = neural_network.NeuralNetworkBuilder(input_features,
output_features,
disable_rank5_shape_mapping=True
)

# decode_bbox_center_x = prior_variance[0] * bbox.xmin() * prior_width + prior_center_x;
# decode_bbox_center_y = prior_variance[1] * bbox.ymin() * prior_height + prior_center_y;
# decode_bbox_width = exp(prior_variance[2] * bbox.xmax()) * prior_width;
# decode_bbox_height = exp(prior_variance[3] * bbox.ymax()) * prior_height;

fd_mbox_priorbox = fd_mbox_priorbox.reshape([4, -1])
prior_bbox = fd_mbox_priorbox[:, :num_prior_]
prior_variances = fd_mbox_priorbox[:, num_prior_:]

xmin, ymin, xmax, ymax = prior_bbox
prior_width = xmax - xmin
prior_height = ymax - ymin
prior_center_x = (xmin + xmax) / 2.
prior_center_y = (ymin + ymax) / 2.

prior_wh = np.stack([prior_width, prior_height])
prior_xy = np.stack([prior_center_x, prior_center_y])

prior_variances = np.expand_dims(prior_variances, axis=-1)
prior_wh = np.expand_dims(prior_wh, axis=-1)
prior_xy = np.expand_dims(prior_xy, axis=-1)

builder.add_load_constant('prior_wh',
output_name='prior_wh_out',
constant_value=prior_wh,
shape=prior_wh.shape
)

builder.add_load_constant('prior_xy',
output_name='prior_xy_out',
constant_value=prior_xy,
shape=prior_xy.shape)

builder.add_load_constant('prior_variances',
output_name='prior_variances_out',
constant_value=prior_variances,
shape=prior_variances.shape)
print(prior_variances.shape)
print(prior_wh.shape)
print(prior_xy.shape)

builder.add_split('split_variances',
input_name='prior_variances_out',
output_names=['split_variances_xy', 'split_variances_wh'], # (2,num_prior_,1) , (2,num_prior_,1)
# num_splits=2,
# axis=-3
)

builder.add_split('split_box',
input_name='boxes',
output_names=['split_box_min_out', 'split_box_max_out'], # (2,num_prior_,1) , (2,num_prior_,1)
# num_splits=2,
# axis=-3
)

# prior_variance[0] * bbox.xmin() * prior_width
# prior_variance[1] * bbox.ymin() * prior_height
builder.add_elementwise('bbox_xy_m',
input_names=['split_variances_xy', 'split_box_min_out', 'prior_wh_out'],
output_name='bbox_xy_m_out', # (2,num_prior_,1)
mode='MULTIPLY'
)
# prior_variance[0] * bbox.xmin() * prior_width + prior_center_x
# prior_variance[1] * bbox.ymin() * prior_height + prior_center_y
builder.add_elementwise('bbox_xy_a',
input_names=['bbox_xy_m_out', 'prior_xy_out'],
output_name='bbox_xy_a_out',
mode='ADD'
)

# prior_variance[2] * bbox.xmax()
# prior_variance[3] * bbox.ymax()
builder.add_elementwise('bbox_wh_m',
input_names=['split_variances_wh', 'split_box_max_out'],
output_name='bbox_wh_m_out',
mode='MULTIPLY'
)

# exp(prior_variance[2] * bbox.xmax())
# exp(prior_variance[3] * bbox.ymax())
builder.add_unary('bbox_wh_e',
input_name='bbox_wh_m_out',
output_name='bbox_wh_e_out',
mode='exp'
)

# exp(prior_variance[2] * bbox.xmax()) * prior_width
# exp(prior_variance[3] * bbox.ymax()) * prior_height
builder.add_elementwise('bbox_wh_m_m',
input_names=['bbox_wh_e_out', 'prior_wh_out'],
output_name='bbox_wh_m_m_out',
mode='MULTIPLY'
)

builder.add_elementwise('decoded_bbox',
input_names=['bbox_xy_a_out', 'bbox_wh_m_m_out'],
output_name='decoded_bbox_out',
mode='CONCAT'
)

builder.add_nms('nms',
input_names=['decoded_bbox_out', 'scores'],
output_names=['coordinates', 'confidence', 'nms_indices', 'nms_number'],
iou_threshold=0.3,
score_threshold=0.05,
max_boxes=200
)
builder.inspect_layers(verbose=True)

nms_model = ct.models.MLModel(builder.spec)
nms_model.save('nn.mlmodel')

print("success")
```

Contributor guide

Open the contributing guide

Research direction

Start with the builder.add_split and builder.add_elementwise calls in the supplied code, then compare the printed prior_variances, prior_wh, and prior_xy shapes with the compiler error shapes. Run builder.inspect_layers(verbose=True) before constructing MLModel and verify the tensor shapes through the bbox decoder; done means MLModel builds without the invalid blob shape error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.