facebookresearch / facebookresearch/deepconf
Majority Vote and Weighted majority vote does not group by equivalent answers?
- Dominant language
- Python
- Stars
- 412
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
Hi!
Thanks for open sourcing the implementation of DeepThink. I was looking into the implementation of `simple_majority_vote` and `weighted_majority_vote` [here](https://github.com/anirudhb11/deepconf/blob/main/deepconf/utils.py#L67-L90), and one thing I noticed is that the implementation does not group equivalent answers together. So for a given question, if the model answers `0.5` or `\frac{1}{2}`, these will be counted as two separate answers, when in reality they are the same answer.
Is this intended?
Thanks!
Contributor guide
Research direction
Start in deepconf/utils.py at lines 67-90 and inspect simple_majority_vote and weighted_majority_vote. Reproduce the reported case with answers such as 0.5 and \frac{1}{2}; done means equivalent answer representations are grouped consistently by both voting functions, with the behavior covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100