hashicorp / hashicorp/terraform-plugin-testing

Consider improving error messaging for `knownvalue.NumberExact` float comparisons

Open
#311 4 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
Go
Stars
68
Forks
22
Avg merge
2d 12h
Merged PRs (30d)
1

Description

### terraform-plugin-testing version

```
v1.7.0
```

### Use cases

When using `knownvalue.NumberExact`, it's possible to get confusing error messages in the situation where the provided `*big.Float` does not match the value parsed from a `json.Number` because of the mantissa using something like [`big.NewFloat`](https://pkg.go.dev/math/big#NewFloat).

#### Example
```go
func TestNumberFunction_recreate(t *testing.T) {
resource.UnitTest(t, resource.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"framework": providerserver.NewProtocol6WithError(New()),
},
Steps: []resource.TestStep{
{
Config: `output "test" {
value = 1.23
}`,
ConfigStateChecks: []statecheck.StateCheck{
statecheck.ExpectKnownOutputValue("test", knownvalue.NumberExact(big.NewFloat(1.23))),
},
},
},
})
}
```

#### Test error
```bash
--- FAIL: TestNumberFunction_recreate (0.43s)
/Users/austin.valle/code/terraform-provider-corner/internal/framework6provider/number_function_test.go:80: Step 1/1 error: Post-apply refresh state check(s) failed:
error checking value for output at path: test, err: expected value 1.23 for NumberExact check, got: 1.23
FAIL
FAIL github.com/hashicorp/terraform-provider-corner/internal/framework6provider 0.896s
FAIL
```
----------------

What can make this more confusing is when it accidentally is correct and the test passes 😅 :
```go
func TestNumberFunction_successful(t *testing.T) {
resource.UnitTest(t, resource.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"framework": providerserver.NewProtocol6WithError(New()),
},
Steps: []resource.TestStep{
{
Config: `output "test" {
value = 1234.5
}`,
ConfigStateChecks: []statecheck.StateCheck{
// Success!
statecheck.ExpectKnownOutputValue("test", knownvalue.NumberExact(big.NewFloat(1234.5))),
},
},
},
})
}
```

### Workaround
You can workaround this by creating the `*big.Float` exactly how the internal state check logic/Terraform creates it. Fixing the initial example in this issue:
```go
func TestNumberFunction_recreate(t *testing.T) {
matchingFloat, _, _ := big.ParseFloat("1.23", 10, 512, big.ToNearestEven)

resource.UnitTest(t, resource.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"framework": providerserver.NewProtocol6WithError(New()),
},
Steps: []resource.TestStep{
{
Config: `output "test" {
value = 1.23
}`,
ConfigStateChecks: []statecheck.StateCheck{
// Success!
statecheck.ExpectKnownOutputValue("test", knownvalue.NumberExact(matchingFloat)),
},
},
},
})
}
```

### Proposal
Improve the error message to also include the precision/rounding/mantissa information to make it more apparent why two floating point numbers don't match. Perhaps we could also make a suggestion about how best to create the `*big.Float` value if a comparison doesn't match because of the mantissa.

### References
- Found while writing dynamic tests in: https://github.com/hashicorp/terraform-provider-corner/pull/228

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.