Linear layer without a bias gets explicit zero vector bias
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🐞Describing the bug
When I create a linear layer in torch which doesn't have a bias the final spec still contains a bias. Is this intended?
To me it seems that the `linear` op would support not receiving a bias: https://apple.github.io/coremltools/source/coremltools.converters.mil.mil.ops.defs.html#module-coremltools.converters.mil.mil.ops.defs.iOS15.linear
Here's a minimal example:
```
import coremltools as ct
import torch
just_linear_model_traced = torch.jit.trace(just_linear_model, torch.randn(1, 768))
just_linear_model_coreml = ct.convert(
just_linear_model_traced,
convert_to="mlprogram",
inputs=[ct.TensorType(shape=(1, 768), name="input")],
outputs=[ct.TensorType(name="out")]
)
print(just_linear_model_coreml.get_spec())
```
I see the following spec:
```
....
operations {
type: "const"
outputs {
name: "var_4_bias_0_to_fp16"
type {
tensorType {
dataType: FLOAT16
rank: 1
dimensions {
constant {
size: 768
}
}
}
}
}
attributes {
key: "name"
value {
type {
tensorType {
dataType: STRING
}
}
immediateValue {
tensor {
strings {
values: "op_4_bias_0_to_fp16"
}
}
}
}
}
attributes {
key: "val"
value {
type {
tensorType {
dataType: FLOAT16
rank: 1
dimensions {
constant {
size: 768
}
}
}
}
blobFileValue {
fileName: "@model_path[/weights/weight.bin](https://file+.vscode-resource.vscode-cdn.net/weights/weight.bin)"
offset: 1179776
}
}
}
}
....
```
When loading the `blobFileValue` I see it's a zero vector. So this is a valid operation, however it is not clear to me why this explicit zero bias would be added. If this is intended feel free to close.
Contributor guide
Research direction
No source file or test is named. Start with the minimal PyTorch trace and ct.convert(...), then inspect the generated spec through get_spec() and compare the emitted linear operation with the documented MIL iOS15 linear signature. Done means establishing whether a biasless layer should omit the bias or documenting why the explicit zero vector is intended.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100