aws-cloudformation / aws-cloudformation/cloudformation-cli

Contract tests test_input_equals_output does not work with arrays of primitive values

Open
#753 1 comment 1 reaction 0 assignees View on GitHub
bug contract tests
Dominant language
Python
Stars
336
Forks
172
Avg merge
3d 5m
Merged PRs (30d)
3

Description

Hey, I believe there is a bug in the `test_input_equals_output` portion of several contract tests.

### Issue
When comparing a list of strings, the tests will [call compare()](https://github.com/aws-cloudformation/cloudformation-cli/blob/master/src/rpdk/core/contract/resource_client.py#L352), which tries to iterate through the input. In the case of primitives including strings, it should probably just assert their equality rather than trying to iterate through the primitive type.

Our `inputs_1_create.json`:
```
{
"Id": "cfn-contract-test-endpoint",
...
"HlsPackage": {
...
"AdTriggers": [
"SPLICE_INSERT",
"PROVIDER_ADVERTISEMENT",
"DISTRIBUTOR_ADVERTISEMENT",
"PROVIDER_PLACEMENT_OPPORTUNITY",
"DISTRIBUTOR_PLACEMENT_OPPORTUNITY"
],
...
}
}
```

Stack trace:
```
________________________________________________________________________________________________________________________ contract_create_delete _________________________________________________________________________________________________________________________
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/suite/handler_create.py:47: in contract_create_delete
test_input_equals_output(resource_client, input_model, delete_model)
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/suite/handler_commons.py:168: in test_input_equals_output
pruned_output_model,
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:364: in compare
raise e
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:354: in compare
self.compare(inputs[key], outputs[key])
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:364: in compare
raise e
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:357: in compare
self.compare_list(inputs[key], outputs[key])
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:368: in compare_list
self.compare(inputs[index], outputs[index])
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:364: in compare
raise e
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:353: in compare
if isinstance(inputs[key], dict):
E TypeError: string indices must be integers
```

I am using cfn cli version `0.2.8`. `0.2.7` worked with the same inputs payload.

### Workaround
In our `inputs/input_1_create.json` (and _update and _invalid) files, we can temporarily just specify an empty array, which will prevent the tests from trying to iterate through characters of a string or iterate through a number/bool. Changing the `AdTriggers` in mentioned above to `[]` in each input file made the contract tests pass.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.