software-mansion / software-mansion/TypeGPU

bug: Numeric literals are forced into target precision unnecessarily

Open
#2,772 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
3.2k
Forks
122
Avg merge
3d 5h
Merged PRs (30d)
34

Description

Wrapping the literal 0.3 in a d.f32 constructor causes the number to be forced into 3d bits, adding more digits to the representation.

function foo() {
  'use gpu';
  return d.f32(0.3);
}

expect(tgpu.resolve([foo])).toMatchInlineSnapshot(`
  "fn foo() -> f32 {
    return 0.30000001192092896f;
  }"
`);

This however is unnecessary, as 0.3f and 0.30000001192092896f are identical on the byte level. The following WGSL function has been confirmed to return true:

fn check() -> bool {
  const coerced = 0.30000001192092896f;
  const natural = 0.3;
  return coerced == natural;
}

The cause

The coersion is happening at compile-time, in JS. That's to keep comptime operations at proper precision.

Potential fixes

We could instead perform the operations in the target precision, instead of coercing the numbers. Alternatively, when printing a numeric literal into shader code, we could check what smallest representation of a literal results in the same value.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the TypeScript compile-time numeric handling used by the d.f32 constructor and the tgpu.resolve path shown in the example. Compare the generated WGSL for 0.3, 0.3f, and 0.30000001192092896f, then verify that the chosen fix preserves comptime precision while emitting the smallest equivalent target-precision literal.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.