facebookresearch / facebookresearch/clara

No update to posterior of psi for A > 1

Open
#2 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11
Forks
2
PR merge metrics
No merged PRs in 30d

Description

In the example repo's [example notebook](https://github.com/facebookresearch/clara/blob/master/examples.ipynb), it is clear that when A > 1, the posterior of psi is only updated for labeler of index = 0, the rest of which does not get updated.

This seems to be because when `self._init(ratings, labelers, true_ratings, scores)` is called in` ClaraGibbs.fit()`, initializing assignment only works for the label of index = 0, the `data` field was not updated for other labeler indices.

I managed to replicate the same problems using my own dataset.

Contributor guide

Open the contributing guide

Research direction

Start with examples.ipynb and trace ClaraGibbs.fit(), especially the call to self._init(ratings, labelers, true_ratings, scores). Compare how the data field is initialized for labeler index 0 versus other labeler indices when A > 1, then verify that the posterior of psi updates for every labeler using the example or the reporter’s replicated dataset.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.