hashicorp / hashicorp/terraform-plugin-sdk
Upgrading OpenAPI Terraform provider to Terraform SDK 2.0: TypeMap with Elem*Resource not supported
- Dominant language
- Go
- Stars
- 485
- Forks
- 244
- Avg merge
- 19h 57m
- Merged PRs (30d)
- 4
Description
### Context
I am in the process of upgrading the [OpenAPI Terraform provider](https://github.com/dikhan/terraform-provider-openapi) to use the new Terraform SDK 2.0, and some int tests are now throwing the following error: `TypeMap with Elem*Resource not supported`
Previous versions of the SDK allowed TypeMap to have an associated Elem *schema.Resource; however the behaviour was unpredictable causing diff issues etc. The main goal was to have an object type definition that contained different properties types (eg: string, int, etc) and configurations (eg: mix of props where some are expected as input from the user and others are are computed aka readOnly). As per other issues raised in the past [21217](https://github.com/hashicorp/terraform/issues/21217#issuecomment-489699737) and [22511](https://github.com/hashicorp/terraform/issues/22511#issuecomment-522655851), this seemed not to be supported and the recommendation from Terraform maintainers back then was to configure the property instead as TypeList with Elem *Resource and limit size of 1. However, as per [21217 comment](https://github.com/hashicorp/terraform/issues/21217#issuecomment-489699737) it was suggested that future versions of the SDK would support complex objects natively without having to use the workaround of TypeList with 1 Elem and I'd really appreciate if you could shed some light on it.
After updating to v2.0.3, looks like Terraform no longer supports TypeMap with Elem *schema.Resource (as per debug output section) so I was wondering if the expectation/recommendation is still to configure complex objects using TypeList with Elem *Resource and limit size of 1?
### SDK version
```
github.com/hashicorp/terraform-plugin-sdk/v2 v2.0.3
```
### Relevant provider source code
The [OpenAPI Terraform provider](https://github.com/dikhan/terraform-provider-openapi) automatically converts the following `ContentDeliveryNetworkV1` definition into its corresponding Terraform schema.Schema as shown below (pasting the yaml too for the ref). Option 2 works as expected; Option 1 now fails due to SDK 2.0 no longer supporting TypeMap with Elem *Resource.
- Object definition
```yaml
ContentDeliveryNetworkV1:
type: "object"
properties:
...
object_property_argument: # Option 1 of handling complex objects (eg: object containing props with different types (eg: string, int, etc) or configurations (eg: mix of props where some are expected as input from the user and others are are computed aka readOnly))
type: "object"
properties:
account:
type: string
object_read_only_property:
type: string
readOnly: true
object_property_block: # Option 2 of handling complex objects, the flag `x-terraform-complex-object-legacy-config` tells the provider to set up the property using the block format, as suggested by Hashi maintainers in the past using TypeList with Elem *Resource and limit size of 1
type: "object"
x-terraform-complex-object-legacy-config: true
properties:
account:
type: string
object_read_only_property:
type: string
readOnly: true
````
- Corresponding Terraform resource schema (automatically generated based on the above definition)
```go
(map[string]*schema.Schema) (len=4) {
(string) (len=24) "object_property_argument": (*schema.Schema)(0xc0002152c0)({
Type: (schema.ValueType) TypeMap,
ConfigMode: (schema.SchemaConfigMode) 0,
Optional: (bool) true,
Required: (bool) false,
DiffSuppressFunc: (schema.SchemaDiffSuppressFunc) ,
Default: (interface {}) ,
DefaultFunc: (schema.SchemaDefaultFunc) ,
Description: (string) "",
InputDefault: (string) "",
Computed: (bool) false,
ForceNew: (bool) false,
StateFunc: (schema.SchemaStateFunc) ,
Elem: (*schema.Resource)(0xc00065c160)({
Schema: (map[string]*schema.Schema) (len=2) {
(string) (len=7) "account": (*schema.Schema)(0xc000215400)({
Type: (schema.ValueType) TypeString,
ConfigMode: (schema.SchemaConfigMode) 0,
Optional: (bool) true,
Required: (bool) false,
DiffSuppressFunc: (schema.SchemaDiffSuppressFunc) ,
Default: (interface {}) ,
DefaultFunc: (schema.SchemaDefaultFunc) ,
Description: (string) "",
InputDefault: (string) "",
Computed: (bool) false,
ForceNew: (bool) false,
StateFunc: (schema.SchemaStateFunc) ,
Elem: (interface {}) ,
MaxItems: (int) 0,
MinItems: (int) 0,
Set: (schema.SchemaSetFunc) ,
ComputedWhen: ([]string) ,
ConflictsWith: ([]string) ,
ExactlyOneOf: ([]string) ,
AtLeastOneOf: ([]string) ,
RequiredWith: ([]string) ,
Deprecated: (string) "",
ValidateFunc: (schema.SchemaValidateFunc) 0x1cf87a0,
ValidateDiagFunc: (schema.SchemaValidateDiagFunc) ,
Sensitive: (bool) false
}),
(string) (len=25) "object_read_only_property": (*schema.Schema)(0xc000215540)({
Type: (schema.ValueType) TypeString,
ConfigMode: (schema.SchemaConfigMode) 0,
Optional: (bool) true,
Required: (bool) false,
DiffSuppressFunc: (schema.SchemaDiffSuppressFunc) ,
Default: (interface {}) ,
DefaultFunc: (schema.SchemaDefaultFunc) ,
Description: (string) "",
InputDefault: (string) "",
Computed: (bool) true,
ForceNew: (bool) false,
StateFunc: (schema.SchemaStateFunc) ,
Elem: (interface {}) ,
MaxItems: (int) 0,
MinItems: (int) 0,
Set: (schema.SchemaSetFunc) ,
ComputedWhen: ([]string) ,
ConflictsWith: ([]string) ,
ExactlyOneOf: ([]string) ,
AtLeastOneOf: ([]string) ,
RequiredWith: ([]string) ,
Deprecated: (string) "",
ValidateFunc: (schema.SchemaValidateFunc) 0x1cf87a0,
ValidateDiagFunc: (schema.SchemaValidateDiagFunc) ,
Sensitive: (bool) false
})
},
SchemaVersion: (int) 0,
MigrateState: (schema.StateMigrateFunc) ,
StateUpgraders: ([]schema.StateUpgrader) ,
Create: (schema.CreateFunc) ,
Read: (schema.ReadFunc) ,
Update: (schema.UpdateFunc) ,
Delete: (schema.DeleteFunc) ,
Exists: (schema.ExistsFunc) ,
CreateContext: (schema.CreateContextFunc) ,
ReadContext: (schema.ReadContextFunc) ,
UpdateContext: (schema.UpdateContextFunc) ,
DeleteContext: (schema.DeleteContextFunc) ,
CustomizeDiff: (schema.CustomizeDiffFunc) ,
Importer: (*schema.ResourceImporter)(),
DeprecationMessage: (string) "",
Timeouts: (*schema.ResourceTimeout)(),
Description: (string) ""
}),
MaxItems: (int) 0,
MinItems: (int) 0,
Set: (schema.SchemaSetFunc) ,
ComputedWhen: ([]string) ,
ConflictsWith: ([]string) ,
ExactlyOneOf: ([]string) ,
AtLeastOneOf: ([]string) ,
RequiredWith: ([]string) ,
Deprecated: (string) "",
ValidateFunc: (schema.SchemaValidateFunc) ,
ValidateDiagFunc: (schema.SchemaValidateDiagFunc) ,
Sensitive: (bool) false
}),
(string) (len=21) "object_property_block": (*schema.Schema)(0xc0002157c0)({
Type: (schema.ValueType) TypeList,
ConfigMode: (schema.SchemaConfigMode) 0,
Optional: (bool) true,
Required: (bool) false,
DiffSuppressFunc: (schema.SchemaDiffSuppressFunc) ,
Default: (interface {}) ,
DefaultFunc: (schema.SchemaDefaultFunc) ,
Description: (string) "",
InputDefault: (string) "",
Computed: (bool) false,
ForceNew: (bool) false,
StateFunc: (schema.SchemaStateFunc) ,
Elem: (*schema.Resource)(0xc00065c210)({
Schema: (map[string]*schema.Schema) (len=2) {
(string) (len=25) "object_read_only_property": (*schema.Schema)(0xc0000fc000)({
Type: (schema.ValueType) TypeString,
ConfigMode: (schema.SchemaConfigMode) 0,
Optional: (bool) true,
Required: (bool) false,
DiffSuppressFunc: (schema.SchemaDiffSuppressFunc) ,
Default: (interface {}) ,
DefaultFunc: (schema.SchemaDefaultFunc) ,
Description: (string) "",
InputDefault: (string) "",
Computed: (bool) true,
ForceNew: (bool) false,
StateFunc: (schema.SchemaStateFunc) ,
Elem: (interface {}) ,
MaxItems: (int) 0,
MinItems: (int) 0,
Set: (schema.SchemaSetFunc) ,
ComputedWhen: ([]string) ,
ConflictsWith: ([]string) ,
ExactlyOneOf: ([]string) ,
AtLeastOneOf: ([]string) ,
RequiredWith: ([]string) ,
Deprecated: (string) "",
ValidateFunc: (schema.SchemaValidateFunc) 0x1cf87a0,
ValidateDiagFunc: (schema.SchemaValidateDiagFunc) ,
Sensitive: (bool) false
}),
(string) (len=7) "account": (*schema.Schema)(0xc0000fc3c0)({
Type: (schema.ValueType) TypeString,
ConfigMode: (schema.SchemaConfigMode) 0,
Optional: (bool) true,
Required: (bool) false,
DiffSuppressFunc: (schema.SchemaDiffSuppressFunc) ,
Default: (interface {}) ,
DefaultFunc: (schema.SchemaDefaultFunc) ,
Description: (string) "",
InputDefault: (string) "",
Computed: (bool) false,
ForceNew: (bool) false,
StateFunc: (schema.SchemaStateFunc) ,
Elem: (interface {}) ,
MaxItems: (int) 0,
MinItems: (int) 0,
Set: (schema.SchemaSetFunc) ,
ComputedWhen: ([]string) ,
ConflictsWith: ([]string) ,
ExactlyOneOf: ([]string) ,
AtLeastOneOf: ([]string) ,
RequiredWith: ([]string) ,
Deprecated: (string) "",
ValidateFunc: (schema.SchemaValidateFunc) 0x1cf87a0,
ValidateDiagFunc: (schema.SchemaValidateDiagFunc) ,
Sensitive: (bool) false
})
},
SchemaVersion: (int) 0,
MigrateState: (schema.StateMigrateFunc) ,
StateUpgraders: ([]schema.StateUpgrader) ,
Create: (schema.CreateFunc) ,
Read: (schema.ReadFunc) ,
Update: (schema.UpdateFunc) ,
Delete: (schema.DeleteFunc) ,
Exists: (schema.ExistsFunc) ,
CreateContext: (schema.CreateContextFunc) ,
ReadContext: (schema.ReadContextFunc) ,
UpdateContext: (schema.UpdateContextFunc) ,
DeleteContext: (schema.DeleteContextFunc) ,
CustomizeDiff: (schema.CustomizeDiffFunc) ,
Importer: (*schema.ResourceImporter)(),
DeprecationMessage: (string) "",
Timeouts: (*schema.ResourceTimeout)(),
Description: (string) ""
}),
MaxItems: (int) 1,
MinItems: (int) 0,
Set: (schema.SchemaSetFunc) ,
ComputedWhen: ([]string) ,
ConflictsWith: ([]string) ,
ExactlyOneOf: ([]string) ,
AtLeastOneOf: ([]string) ,
RequiredWith: ([]string) ,
Deprecated: (string) "",
ValidateFunc: (schema.SchemaValidateFunc) ,
ValidateDiagFunc: (schema.SchemaValidateDiagFunc) ,
Sensitive: (bool) false
})
}
```
### Terraform Configuration Files
```hcl
resource "openapi_cdn_v1" "my_cdn" {
...
object_property_block {
account = "my_account"
}
object_property_argument = {
account = "my_account"
object_read_only_property = "some computed value for object read only" // This is a known issue due to TypeMap with Elem*Resource unpredictability and a workaround users will have to do in order to fix the diff issues with objects that contain readOnly properties
}
...
}
```
### Debug Output
````
=== RUN TestAccCDN_Create_and_UpdateSubResource
2020/10/17 17:22:01 [INFO] data source 'openapi_cdn_v1' successfully registered in the provider (time:72.546µs)
gray_box_cdns_test.go:577: Step 1/2 error: Error running pre-apply refresh: 2020/10/17 17:22:01 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0
Error: InternalValidate
Internal validation of the provider failed! This is always a bug
with the provider itself, and not a user issue. Please report
this bug:
3 errors occurred:
* resource openapi_cdn_v1: object_property_argument: TypeMap with Elem
*Resource not supported,use TypeList/TypeSet with Elem *Resource or TypeMap
with Elem *Schema
* data source openapi_cdn_v1_instance: object_property_argument: TypeMap with
Elem *Resource not supported,use TypeList/TypeSet with Elem *Resource or
TypeMap with Elem *Schema
* data source openapi_cdn_v1: object_property_argument: TypeMap with Elem
*Resource not supported,use TypeList/TypeSet with Elem *Resource or TypeMap
with Elem *Schema
--- FAIL: TestAccCDN_Create_and_UpdateSubResource (0.33s)
FAIL
````
### Expected Behavior
TypeMap with Elem *Resource is supported so objects that objects containing props with different types (eg: string, int, etc) or configurations (eg: mix of props where some are expected as input from the user and others are are computed aka readOnly) can be defined in the terraform schema without having to use the workaround of configuring the property as TypeList with Elem *Resource and limit size of 1
*P.S: I was hoping this could be supported as part of the major upgrade to SDK 2.0.0 considering it would potentially involve introducing breaking changes and seemed like a good opportunity to do it.*
### Actual Behavior
TypeMap with Elem *Resource not supported
### Steps to Reproduce
Please list the full steps required to reproduce the issue, for example:
0. The provider exposes a resource with a schema containing one property set up as TypeMap with Elem *Resource (example provided above)
1. `terraform init`
2. `terraform apply`
3. Terraform will complain with the following output:
```
Error: InternalValidate
Internal validation of the provider failed! This is always a bug
with the provider itself, and not a user issue. Please report
this bug:
3 errors occurred:
* resource openapi_cdn_v1: object_property_argument: TypeMap with Elem
*Resource not supported,use TypeList/TypeSet with Elem *Resource or TypeMap
with Elem *Schema
```
### References
- [21217](https://github.com/hashicorp/terraform/issues/21217#issuecomment-489699737)
- [22511](https://github.com/hashicorp/terraform/issues/22511#issuecomment-522655851)
Contributor guide
Research direction
Start by reviewing the Terraform SDK v2 schema validation for TypeMap with Elem *schema.Resource and the linked historical discussions. Reproduce the failure from gray_box_cdns_test.go:577 using the OpenAPI provider examples, then determine whether native complex-object support or the TypeList workaround is the intended behavior. Done requires a documented resolution or an agreed implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, terraform
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100