clab / clab/dynet

pickneglogsoftmax returning [inf]

Open
#1,412 3 comments 0 reactions 0 assignees View on GitHub
need more info
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

> e_ = dy.pick(-dy.log(dy.softmax(e1)), k)

There is a chance for softmax to set 0 somewhere in the probability vector which will mess up the log. May be this should be handled internally? Otherwise, I can't think of any way of avoiding [inf] apart from if-condition, or a small additional noise value to avoid 0s. Is there any popular solution to this?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the expression using dy.pick, dy.log, and dy.softmax, then trace those entry points to the relevant implementation. Determine how zero probabilities produce [inf] and review the existing numerical behavior around softmax and log. Done means the issue has a documented, tested resolution for this expression, if the project agrees it should be handled internally.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.