apple / apple/coremltools

Linear layer without a bias gets explicit zero vector bias

Open
#1,847 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
When I create a linear layer in torch which doesn't have a bias the final spec still contains a bias. Is this intended?
To me it seems that the `linear` op would support not receiving a bias: https://apple.github.io/coremltools/source/coremltools.converters.mil.mil.ops.defs.html#module-coremltools.converters.mil.mil.ops.defs.iOS15.linear

Here's a minimal example:
```
import coremltools as ct
import torch
just_linear_model_traced = torch.jit.trace(just_linear_model, torch.randn(1, 768))
just_linear_model_coreml = ct.convert(
just_linear_model_traced,
convert_to="mlprogram",
inputs=[ct.TensorType(shape=(1, 768), name="input")],
outputs=[ct.TensorType(name="out")]
)
print(just_linear_model_coreml.get_spec())
```

I see the following spec:
```
....
operations {
type: "const"
outputs {
name: "var_4_bias_0_to_fp16"
type {
tensorType {
dataType: FLOAT16
rank: 1
dimensions {
constant {
size: 768
}
}
}
}
}
attributes {
key: "name"
value {
type {
tensorType {
dataType: STRING
}
}
immediateValue {
tensor {
strings {
values: "op_4_bias_0_to_fp16"
}
}
}
}
}
attributes {
key: "val"
value {
type {
tensorType {
dataType: FLOAT16
rank: 1
dimensions {
constant {
size: 768
}
}
}
}
blobFileValue {
fileName: "@model_path[/weights/weight.bin](https://file+.vscode-resource.vscode-cdn.net/weights/weight.bin)"
offset: 1179776
}
}
}
}
....
```

When loading the `blobFileValue` I see it's a zero vector. So this is a valid operation, however it is not clear to me why this explicit zero bias would be added. If this is intended feel free to close.

Contributor guide

Open the contributing guide

Research direction

No source file or test is named. Start with the minimal PyTorch trace and ct.convert(...), then inspect the generated spec through get_spec() and compare the emitted linear operation with the documented MIL iOS15 linear signature. Done means establishing whether a biasless layer should omit the bias or documenting why the explicit zero vector is intended.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.