pytorch / pytorch/vision

Add pos_weight in Focal Loss to trade off recall and precision

Open
#6,229 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🚀 The feature

I suggest adding the pos_weight (extra weights to positive examples) in Focal Loss implementation here.
F.binary_cross_entropy_with_logits already has a pos_weight argument, so to add this feature, it's enough to create a new parameter and pass it to

ce_loss = F.binary_cross_entropy_with_logits(inputs, targets, reduction="none", pos_weight=pos_weight)

With the addition of this feature it will be possible to trade off recall and precision by adding weights to positive examples.

Motivation, pitch

It’s possible to trade off recall and precision by adding weights to positive examples. For example, if a dataset contains $100$ positive and $300$ negative examples of a single class, then pos_weight for the class should be equal to
$\frac{300}{100}=3$. The loss would act as if the dataset contains $3*100 = 300$ positive examples.

This idea already used in BCEWithLogitsLoss (description) PyTorch implementation.

For Binary Focal Loss this improvement will mean the following: $$FL_c = ( l_{1,c}, ... , l_{N,c} )^T$$

$$
l_{n, c} = -[p_c\ y_{n, c}\ \alpha\ (1-\sigma(x_{n,c}))^{\gamma}\ log(\sigma(x_{n,c})) + (1 - y)\ (1-\alpha)\ \sigma(x_{n,c})^{\gamma}\ log(1-\sigma(x_{n,c})]
$$

where $c$ is the class number ($c > 1$ for multi-label binary classification, $c=1$ for single-label binary classification),
$n$ is the number of the sample in the batch and $p_c$ is the weight of the positive answer (pos_weight) for the class $c$.

$p_c > 1$ increases the recall, $p_c < 1$ increases the precision.

Alternatives

No response

Additional context

Ready to code it :)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in torchvision/ops/focal_loss.py, at the Focal Loss implementation linked in the issue, and compare its binary cross-entropy call with the documented pos_weight argument. Done means the public loss accepts pos_weight and forwards it so positive-example weighting can adjust recall and precision; verify the existing focal-loss behavior remains covered by the relevant tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.