Possible unwanted float to double promotion
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 5.5k
- Forks
- 833
- Avg merge
- 13h 8m
- Merged PRs (30d)
- 2
Description
Hi,
i wanted to ask if the division by 4096 is supposed to be double or a float division.
The numerator is a float variable while the denominator defaults to a double constant. Therefore the division is carried out as a double operation.
Furthermore the first comparison against 0.0f is a double to float comparison.
Thanks in advance!
https://github.com/Netflix/vmaf/blob/540185eb927e85f27cb5b946e9e6046446844008/libvmaf/src/feature/integer_adm.c#L718-L720
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read libvmaf/src/feature/integer_adm.c at lines 718-720 and inspect the division and comparison types in their surrounding integer ADM calculations. Confirm whether the float-to-double promotions are intentional; done means the precision concern at that location is resolved or documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- computer-vision
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100