software-mansion / software-mansion/TypeGPU
bug: Numeric literals are forced into target precision unnecessarily
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 3.2k
- Forks
- 122
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 34
Description
Wrapping the literal 0.3 in a d.f32 constructor causes the number to be forced into 3d bits, adding more digits to the representation.
function foo() {
'use gpu';
return d.f32(0.3);
}
expect(tgpu.resolve([foo])).toMatchInlineSnapshot(`
"fn foo() -> f32 {
return 0.30000001192092896f;
}"
`);
This however is unnecessary, as 0.3f and 0.30000001192092896f are identical on the byte level. The following WGSL function has been confirmed to return true:
fn check() -> bool {
const coerced = 0.30000001192092896f;
const natural = 0.3;
return coerced == natural;
}
The cause
The coersion is happening at compile-time, in JS. That's to keep comptime operations at proper precision.
Potential fixes
We could instead perform the operations in the target precision, instead of coercing the numbers. Alternatively, when printing a numeric literal into shader code, we could check what smallest representation of a literal results in the same value.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the TypeScript compile-time numeric handling used by the d.f32 constructor and the tgpu.resolve path shown in the example. Compare the generated WGSL for 0.3, 0.3f, and 0.30000001192092896f, then verify that the chosen fix preserves comptime precision while emitting the smallest equivalent target-precision literal.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100