hashicorp / hashicorp/terraform-plugin-testing
Consider improving error messaging for `knownvalue.NumberExact` float comparisons
- Dominant language
- Go
- Stars
- 68
- Forks
- 22
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 1
Description
### terraform-plugin-testing version
```
v1.7.0
```
### Use cases
When using `knownvalue.NumberExact`, it's possible to get confusing error messages in the situation where the provided `*big.Float` does not match the value parsed from a `json.Number` because of the mantissa using something like [`big.NewFloat`](https://pkg.go.dev/math/big#NewFloat).
#### Example
```go
func TestNumberFunction_recreate(t *testing.T) {
resource.UnitTest(t, resource.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"framework": providerserver.NewProtocol6WithError(New()),
},
Steps: []resource.TestStep{
{
Config: `output "test" {
value = 1.23
}`,
ConfigStateChecks: []statecheck.StateCheck{
statecheck.ExpectKnownOutputValue("test", knownvalue.NumberExact(big.NewFloat(1.23))),
},
},
},
})
}
```
#### Test error
```bash
--- FAIL: TestNumberFunction_recreate (0.43s)
/Users/austin.valle/code/terraform-provider-corner/internal/framework6provider/number_function_test.go:80: Step 1/1 error: Post-apply refresh state check(s) failed:
error checking value for output at path: test, err: expected value 1.23 for NumberExact check, got: 1.23
FAIL
FAIL github.com/hashicorp/terraform-provider-corner/internal/framework6provider 0.896s
FAIL
```
----------------
What can make this more confusing is when it accidentally is correct and the test passes 😅 :
```go
func TestNumberFunction_successful(t *testing.T) {
resource.UnitTest(t, resource.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"framework": providerserver.NewProtocol6WithError(New()),
},
Steps: []resource.TestStep{
{
Config: `output "test" {
value = 1234.5
}`,
ConfigStateChecks: []statecheck.StateCheck{
// Success!
statecheck.ExpectKnownOutputValue("test", knownvalue.NumberExact(big.NewFloat(1234.5))),
},
},
},
})
}
```
### Workaround
You can workaround this by creating the `*big.Float` exactly how the internal state check logic/Terraform creates it. Fixing the initial example in this issue:
```go
func TestNumberFunction_recreate(t *testing.T) {
matchingFloat, _, _ := big.ParseFloat("1.23", 10, 512, big.ToNearestEven)
resource.UnitTest(t, resource.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"framework": providerserver.NewProtocol6WithError(New()),
},
Steps: []resource.TestStep{
{
Config: `output "test" {
value = 1.23
}`,
ConfigStateChecks: []statecheck.StateCheck{
// Success!
statecheck.ExpectKnownOutputValue("test", knownvalue.NumberExact(matchingFloat)),
},
},
},
})
}
```
### Proposal
Improve the error message to also include the precision/rounding/mantissa information to make it more apparent why two floating point numbers don't match. Perhaps we could also make a suggestion about how best to create the `*big.Float` value if a comparison doesn't match because of the mantissa.
### References
- Found while writing dynamic tests in: https://github.com/hashicorp/terraform-provider-corner/pull/228
Contributor guide
Assessment
This issue has not been assessed yet.