Expected type Float got Half in Loss function
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.5k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 3
Description
This is my loss model forward:
def forward(self, sentence_features: Iterable[Dict[str, Tensor]], labels: Tensor):
rep_a, rep_b = self.model(sentence_features)
output = cosine_similarity(rep_a.float(), rep_b.float()) # errors on this line unless I add the .float() cast
loss_fct = nn.MSELoss()
if labels is not None:
loss = loss_fct(output, labels.view(-1))
return loss
else:
return [rep_a, rep_b], output
Initialized with:
from apex import amp
model, optimizer = amp.initialize(loss_model, optimizer, opt_level='O1')
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided forward method, especially the cosine_similarity call, and the amp.initialize setup using opt_level='O1'. Reproduce the Float-versus-Half error and inspect how the loss inputs are typed under mixed precision. Done means the loss model handles the shown inputs without requiring the manual .float() cast.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100