Generate "expand" and "flatten" functions for associated external types

### Background

Given that Go uses static typing, when encountering “objects” with Go code it is very common that the API SDK (or other external code) implementation will use custom Go structure types for those objects. For example, a Terraform schema may need to describe an API with a list nested attribute (`types.List` of `types.Object`), whereas the API SDK may implement an array/slice of a custom Go type. The provider developer is responsible for converting to and from the Terraform SDK types to the external types to properly generate API requests (`Create`, `Update`, `Delete`) or handle API responses (`Read`). This data handling logic can be cumbersome and it is almost exclusively repetitive depending on the API. Previously with `terraform-plugin-sdk`, this type of provider logic was conventionally put into what developers called “expand” (Terraform to API) and “flatten” (API to Terraform) functions.

### Proposal

To ease developer burden, it is proposed that the [associated_external_type](https://github.com/hashicorp/terraform-plugin-codegen-spec/blob/main/spec/schema.json#L321), which can be optionally specified, be used in the generation of "expand" and "flatten" functions.

The following intermediate representation illustrates the usage of `associated_external_type` for a single nested block:

```json
{
"resources": [
{
"name": "aws_imagebuilder_image",
"schema": {
"blocks": [
{
"name": "image_tests_configuration",
"single_nested": {
"associated_external_type": {
"imports": [
{
"path": "github.com/aws/aws-sdk-go/service/imagebuilder"
}
],
"type": "*imagebuilder.ImageTestsConfiguration",
"mapping": {
"timeout_minutes": {
"name": "MinutesTimeout",
"type": "string"
}
}
},
"attributes": [
{
"name": "image_tests_enabled",
"bool": {
"computed_optional_required": "optional"
}
},
{
"name": "timeout_minutes",
"int64": {
"computed_optional_required": "computed_optional"
}
}
]
}
}
]
}
}
],
}
```

In addition to the schema, models and model helper functions that are generated from the IR, the code generation would also create the following "expand" and "flatten" functions:

```go
func (m ImageTestsConfigurationModel) ToImageTestsConfiguration(ctx context.Context, tfObject types.Object) (*imagebuilder.ImageTestsConfiguration, diag.Diagnostics) {
var diags diag.Diagnostics

if tfObject.IsNull() || tfObject.IsUnknown() {
return nil, diags
}

var tfModel ImageTestsConfigurationModel

diags.Append(tfObject.As(ctx, &tfModel, basetypes.ObjectAsOptions{})...)

if diags.HasError() {
return nil, diags
}

apiObject := &imagebuilder.ImageTestsConfiguration{
ImageTestsEnabled: tfModel.ImageTestsEnabled.ValueBoolPointer(),
TimeoutMinutes: tfModel.TimeoutMinutes.ValueInt64Pointer(),
}

return apiObject, diags
}

func (m ImageTestsConfigurationModel) FromImageTestsConfiguration(ctx context.Context, apiObject *imagebuilder.ImageTestsConfiguration) (types.Object, diag.Diagnostics) {
var diags diag.Diagnostics
var tfModel ImageTestsConfigurationModel

if apiObject == nil {
return m.objectNull(ctx), diags
}

tfModel.ImageTestsEnabled = types.BoolPointerValue(apiObject.ImageTestsEnabled)
tfModel.TimeoutMinutes = types.Int64PointerValue(apiObject.TimeoutMinutes)

return m.objectValueFrom(ctx)
}
```

### "Implicit" Mapping
The only fields that are currently defined within the schema for [associated_external_type](https://github.com/hashicorp/terraform-plugin-codegen-spec/blob/main/spec/schema.json#L321) are `type` and `import`. Consequently, assumptions have to be made about how fields within an intermediate representation (IR) that have an _associated_external_type_ defined should be handled when "expand" and "flatten" functions are generated.

#### Primitives

Primitives are defined as the following types:

| Schema Attribute Type | Model Type |
| ------------- | ------------- |
| schema.BoolAttribute | types.Bool, basetypes.BoolValue |
| schema.Float64Attribute | types.Float64, basetypes.Float64Value |
| schema.Int64Attribute | types.Int64, basetypes.Int64Value |
| schema.NumberAttribute | types.Number, basetypes.NumberValue |
| schema.StringAttribute | types.String, basetypes.StringValue |

If the IR contains an _associated_external_type_ for a primitive, the "expand" and "flatten" functions illustrated below will be generated.

Note that this assumes that the `associated_external_type.type` (e.g., `*apisdk.BoolType`) is something like the following:

```go
type BoolType *bool
```

**expand**

```go
func To(ctx context.Context, tfType types.) (*, diag.Diagnostics) {
var diags diag.Diagnostics

if tfType.IsNull() || tfType.IsUnknown() {
return nil, diags
}

var m

m = tfType.Value()

return &m, diags
}
```

**flatten**

```go
func From(ctx context.Context, apiObject *) (types., diag.Diagnostics) {
var diags diag.Diagnostics

if apiObject == nil {
return types.Null(), diags
}

m := types.(*apiObject)

return m, diags
}
```

The following assumptions have been made when implicitly mapping primitives:

- The _associated_external_type.type_ is a type which can be assigned a pointer of the field value type (i.e., types.Bool | basetypes.BoolValue <=> *bool etc).
- The type specified for the _associated_external_type.type_ in the IR will be a pointer.
- The usage of a pointer or reference in the "To<...>" and "From<...>" functions is used to emphasise where each is being used.

#### Collections

Collections are defined as the following types:

| Schema Attribute Type | Model Type |
| ------------- | ------------- |
| schema.ListAttribute | types.List, basetypes.ListValue |
| schema.MapAttribute | types.Map, basetypes.MapValue |
| schema.SetAttribute | types.Set, basetypes.SetValue |

If the IR contains an _associated_external_type_ for a collection, the "expand" and "flatten" functions illustrated below will be generated.

Note that this assumes that the `associated_external_type.type` (e.g., `*apisdk.BoolSliceType`) is something like the following:

```go
type BoolSliceType []*bool
```

**expand**

```go
func To(ctx context.Context, tfType types.) (*, diag.Diagnostics) {
var diags diag.Diagnostics

if tfType.IsNull() || tfType.IsUnknown() {
return nil, diags
}

var m

// ElementsAs() converts correctly from types.ListType{ElemType: types.BoolType}
// to BoolSliceType (i.e., []*bool), for instance.
diags.Append(tfType.ElementsAs(ctx, &m, false)...)

if diags.HasError() {
return nil, diags
}

return &m, diags
}
```

**flatten**

```go
func From(ctx context.Context, apiObject *) (types., diag.Diagnostics) {
var diags diag.Diagnostics

if apiObject == nil {
// The attr.Type required to call Null() can be obtained from
// schema.Attribute ElementType.
return types.Null(types.), diags
}

m, d := types.ValueFrom(ctx, types., apiObject)

diags.Append(d...)

if diags.HasError() {
return types.Null(types.), diags
}

return m, diags
}
```

The following assumptions have been made when implicitly mapping collections:

- The _associated_external_type.type_ is a type which can be passed to `ElementsAs(...)`. For example, if the schema defines the list attribute as `schema.ListAttribute{ElementType: types.BoolType}` then _associated_external_type.type_ would need to be a type that can be assigned `[]*bool`.

#### Object

Object is defined as the following type:

| Schema Attribute Type | Model Type |
| ------------- | ------------- |
| schema.ObjectAttribute | types.Object, basetypes.ObjectValue |

If the IR contains an _associated_external_type_ for an object, the "expand" and "flatten" functions listed below will be generated.

The "expand" and "flatten" functions are generated on the basis of `schema.ObjectAttribute` and `ApiObject` having the following form:

```go
schema.ObjectAttribute{
AttributeTypes: map[string]attr.Type{
"bool_field": types.BoolType,
"float64_field": types.Float64Type,
"list_field": types.ListType{
ElemType: types.StringType,
},
},
/* ... */
},
```

```go
type ApiObject struct {
BoolField *bool
Int64Field *int64
ListField []*string
}
```

**expand**

```go
func To(ctx context.Context, tfObject types.Object) (*, diag.Diagnostics) {
var diags diag.Diagnostics

if tfObject.IsNull() || tfObject.IsUnknown() {
return nil, diags
}

// Iterate over map[string]attr.Type from schema and generate
// struct as there won't be a corresponding model as they're
// only generated for list, map, set, single nested attributes/blocks.
type objStruct struct {
BoolField types.Bool `tfsdk:"bool_field"`
Float64Field types.Float64 `tfsdk:"float64_field"`
ListField types.List `tfsdk:"list_field"`
}

var objS objStruct

diags.Append(tfObject.As(ctx, &objS, basetypes.ObjectAsOptions{})...)

if diags.HasError() {
return nil, diags
}

// Generate on basis of corresponding field in AttributeTypes from schema.
// Need to detect if handling any non-primitive (i.e., list, map, object, or set).
var listField []*string

d := objS.ListField.ElementsAs(ctx, &listField, false)

diags.Append(d...)

if diags.HasError() {
return nil, diags
}

apiObject := &ApiObject{
BoolField: objS.BoolField.ValueBoolPointer(),
Float64Field: objS.Float64Field.ValueFloat64Pointer(),
ListField: listField,
}

return apiObject, diags
}
```

**flatten**

```go
func From(ctx context.Context, apiObject *ApiObject) (types.Object, diag.Diagnostics) {
var diags diag.Diagnostics

attrTypes := map[string]attr.Type{
"BoolField": types.BoolType,
"Float64Field": types.Float64Type,
"ListField": types.ListType{
ElemType: types.StringType,
},
}

if apiObject == nil {
return types.ObjectNull(attrTypes), diags
}

// Generate on basis of corresponding field in AttributeTypes from schema.
// Need to detect if handling any non-primitive (i.e., list, map, object, or set).
l, d := types.ListValueFrom(ctx, types.StringType, apiObject.ListField)

diags.Append(d...)

if diags.HasError() {
return types.ObjectNull(attrTypes), diags
}

o, d := types.ObjectValue(
attrTypes,
map[string]attr.Value{
"BoolField": types.BoolPointerValue(apiObject.BoolField),
"Float64Field": types.Float64PointerValue(apiObject.Float64Field),
"ListField": l,
},
)

diags.Append(d...)

return o, d
}
```

The following assumptions have been made when implicitly mapping collections:

- The _associated_external_type.type_ contains a one-to-one mapping of field names and types (e.g., if the `schema.ObjectAttribute` contains within `AttributeTypes` a `"BoolAttribute": types.BoolType` the expectation is that the `ApiObject` will contain a field called `BoolAttribute` which holds `*bool`).

#### Nested Attributes and Blocks

Nested attributes and blocks are defined as the following types:

| Schema Attribute Type | Model Type |
| ------------- | ------------- |
| schema.ListNestedAttribute | types.List, basetypes.ListValue |
| schema.ListNestedBlock | types.List, basetypes.ListValue |
| schema.MapNestedAttribute | types.Map, basetypes.MapValue |
| schema.SetNestedAttribute | types.Set, basetypes.SetValue |
| schema.SetNestedBlock | types.Set, basetypes.SetValue |
| schema.SingleNestedAttribute | types.Object, basetypes.ObjectValue |
| schema.SingleNestedBlock | types.Object, basetypes.ObjectValue |

##### Single Nested Attribute without nested _associated_external_type_

If the IR contains an _associated_external_type_ defined for a single nested attribute with no _associated_external_type_(s) defined on any of the attributes within the single nested attribute, then the "expand" and "flatten" functions listed below will be generated.

The "expand" and "flatten" functions are generated on the basis of `schema.SingleNestedAttribute`, and `ApiSingleNestedAttribute` having the following form:

```go
schema.SingleNestedAttribute{
Attributes: map[string]schema.Attribute{
"bool_attribute": schema.BoolAttribute{
Optional: true,
},
"int64_attribute": schema.Int64Attribute{
Optional: true,
},
},
/* ... */
},
```

```go
type ApiSingleNestedAttribute struct {
BoolAttribute *bool
Int64Attribute *int64
}
```

**expand**

```go
func ToSingleNestedAttribute(ctx context.Context, tfObject types.Object) (*ApiSingleNestedAttribute, diag.Diagnostics) {
var diags diag.Diagnostics

if tfObject.IsNull() || tfObject.IsUnknown() {
return nil, diags
}

var sna SingleNestedAttributeModel

diags.Append(tfObject.As(ctx, &sna, basetypes.ObjectAsOptions{})...)

if diags.HasError() {
return nil, diags
}

apiObject := &ApiSingleNestedAttribute{
BoolAttribute: sna.BoolAttribute.ValueBoolPointer(),
Int64Attribute: sna.Int64Attribute.ValueInt64Pointer(),
}

return apiObject, diags
}
```

**flatten**

```go
func FromSingleNestedAttribute(ctx context.Context, apiObject *ApiSingleNestedAttribute) (types.Object, diag.Diagnostics) {
var diags diag.Diagnostics
var sna SingleNestedAttributeModel

if apiObject == nil {
// ObjectNull() is pre-generated for nested attributes.
return sna.ObjectNull(ctx), diags
}

sna.BoolAttribute = types.BoolPointerValue(apiObject.BoolAttribute)
sna.Int64Attribute = types.Int64PointerValue(apiObject.Int64Attribute)

// ObjectValueFrom() is pre-generated for nested attributes.
return sna.ObjectValueFrom(ctx, sna)
}
```

##### Single Nested Attribute with nested _associated_external_type_(s)

If the IR contains an _associated_external_type_ defined for a single nested attribute with _associated_external_type_(s) defined on the attributes within the single nested attribute, then the "expand" and "flatten" functions listed below will be generated.

The "expand" and "flatten" functions are generated on the basis of `schema.SingleNestedAttribute`, and `ApiSingleNestedAttribute` having the following form:

```go
schema.SingleNestedAttribute{
Attributes: map[string]schema.Attribute{
"bool_attribute": schema.BoolAttribute{
Optional: true,
},
"int64_attribute": schema.Int64Attribute{
Optional: true,
},
},
/* ... */
},
```

```go
type ApiSingleNestedAttribute struct {
BoolAttribute *ApiBoolAttribute
Int64Attribute *int64
}

type ApiBoolAttribute *bool
```

**expand**

```go
func ToSingleNestedAttribute(ctx context.Context, tfObject types.Object) (*ApiSingleNestedAttribute, diag.Diagnostics) {
var diags diag.Diagnostics

if tfObject.IsNull() || tfObject.IsUnknown() {
return nil, diags
}

var sna SingleNestedAttributeModel

diags.Append(tfObject.As(ctx, &sna, basetypes.ObjectAsOptions{})...)

if diags.HasError() {
return nil, diags
}

toBoolAttribute, d := ToBoolAttribute(ctx, sna.BoolAttribute)

diags.Append(d...)

if diags.HasError() {
return nil, diags
}

apiObject := &ApiSingleNestedAttribute{
BoolAttribute: toBoolAttribute,
Int64Attribute: sna.Int64Attribute.ValueInt64Pointer(),
}

return apiObject, diags
}

func ToBoolAttribute(ctx context.Context, tfType types.Bool) (*ApiBoolAttribute, diag.Diagnostics) {
var diags diag.Diagnostics

if tfType.IsNull() || tfType.IsUnknown() {
return nil, diags
}

var m ApiBoolAttribute

m = tfType.ValueBoolPointer()

return &m, diags
}
```

**flatten**

```go
func FromSingleNestedAttribute(ctx context.Context, apiObject *ApiSingleNestedAttribute) (types.Object, diag.Diagnostics) {
var diags diag.Diagnostics
var sna SingleNestedAttributeModel

if apiObject == nil {
return sna.ObjectNull(ctx), diags
}

fromBoolAttribute, d := FromBoolAttribute(ctx, apiObject.BoolAttribute)

diags.Append(d...)

if diags.HasError() {
return sna.ObjectNull(ctx), diags
}

sna.BoolAttribute = fromBoolAttribute
sna.Int64Attribute = types.Int64PointerValue(apiObject.Int64Attribute)

return sna.ObjectValueFrom(ctx, sna)
}

func FromBoolAttribute(ctx context.Context, apiObject *ApiBoolAttribute) (types.Bool, diag.Diagnostics) {
var diags diag.Diagnostics

if apiObject == nil {
return types.BoolNull(), diags
}

b := types.BoolPointerValue(*apiObject)

return b, diags
}
```

### Further Considerations

- ~~Currently the [IR Schema](https://github.com/hashicorp/terraform-plugin-codegen-spec/blob/main/spec/schema.json) only permits the usage of `associated_external_type` on data source, provider and resource single nested attributes. This will need to be extended to all attribute and block types.~~
- [Add associated_external_type to all attributes and blocks](https://github.com/hashicorp/terraform-plugin-codegen-spec/pull/37#top)
- The [associated_external_type](https://github.com/hashicorp/terraform-plugin-codegen-spec/blob/main/spec/schema.json#L321) as it is currently defined allows specifying `type` and `import`. Consequently some assumptions will need to be made about how the fields within the model struct are mapped to/from the fields in the "object" that is used for interaction with the API. An initial assumption would be that there is a **direct one-to-one mapping** unless/until a "mapping" field is defined for `associated_external_type` which allows for specifying how this translation from model to API object should happen.
- [Consider adding "mapping" to associated_external_type](https://github.com/hashicorp/terraform-plugin-codegen-spec/issues/38#top)