hashicorp / hashicorp/hcl

Output still quoted after removing quote tokens?

Open
#430 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.8k
Forks
657
Avg merge
20h 36m
Merged PRs (30d)
6

Description

Hi, I'm trying to generate HCL Data resources from arbitrary JSON while handling the transform myself .This is mostly going well, but when I attempt to use Terraform 0.12 variable syntax it doesn't work quite as anticipated.

Here's the input data: ```"provider": "${aws.development-west2}"``` As you can see, we're using the tf 0.11 interpolation syntax in the JSON to indicate that a variable is present.

Here's the transform function

```
func (p Passport) RemoveQuotesAndInterpolation(attributeName string, block *hclwrite.Block) *hclwrite.Block{
attr := block.Body().GetAttribute(attributeName)
toks := attr.BuildTokens(nil)
var tokNew hclwrite.Tokens
for _, t := range toks {
b := string(t.Bytes)
switch {
case strings.Contains(b, "$$"):
// TODO clean this up with regex
t.Bytes = []byte(strings.ReplaceAll(strings.ReplaceAll(strings.ReplaceAll(b, "$$", ""), "{", ""), "}", ""))
tokNew = append(tokNew, t)

case b == "\"":
continue
default:
tokNew = append(tokNew, t)
}
}
spew.Dump(tokNew)
attr.BuildTokens(tokNew)
return block
}
```

Here's the object used to compose the hclwrite.Block:

```
type AWSVPC struct {
Provider *string `mapstructure:"provider,omitempty" required:"false" cty:"provider" hcl:"provider"`
ID *string `mapstructure:"id" required:"false" cty:"id" hcl:"id"`
}
```

Here's the tokens that result from this function:

```
(hclwrite.Tokens) (len=4 cap=4) {
(*hclwrite.Token)(0xc00029f230)({
Type: (hclsyntax.TokenType) TokenIdent,
Bytes: ([]uint8) (len=8 cap=8) {
00000000 70 72 6f 76 69 64 65 72 |provider|
},
SpacesBefore: (int) 0
}),
(*hclwrite.Token)(0xc00029f2c0)({
Type: (hclsyntax.TokenType) TokenEqual,
Bytes: ([]uint8) (len=1 cap=1) {
00000000 3d |=|
},
SpacesBefore: (int) 0
}),
(*hclwrite.Token)(0xc00029f140)({
Type: (hclsyntax.TokenType) TokenQuotedLit,
Bytes: ([]uint8) (len=21 cap=32) {
00000000 61 77 73 2e 64 65 76 65 6c 6f 70 6d 65 6e 74 2d |aws.development-|
00000010 77 65 73 74 32 |west2|
},
SpacesBefore: (int) 0
}),
(*hclwrite.Token)(0xc00029f380)({
Type: (hclsyntax.TokenType) TokenNewline,
Bytes: ([]uint8) (len=1 cap=1) {
00000000 0a |.|
},
SpacesBefore: (int) 0
})
}
```

Note lack of quote tokens. Yet when I review final output I receive this HCL:

```
data "aws_vpc" "west2" { provider = "aws.development-west2" id="blah" }
```

Rather than the expected output of

```
data "aws_vpc" "west2" { provider = aws.development-west2 id="blah" }
```

Here's the function used to assemble the slice of Data blocks for testing:

```
// AssembleHCL builds a slice of blocks into a string for use in testing
func AssembleHCLSlice(blocks []*hclwrite.Block) string {
dataFile := hclwrite.NewFile()
for _, b := range blocks {
dataFile.Body().AppendBlock(b)
}
return string(dataFile.Bytes())
}
```

And here's the token output when using an append for "\\"" instead of a continue:

```
(hclwrite.Tokens) (len=6 cap=8) {
(*hclwrite.Token)(0xc000281230)({
Type: (hclsyntax.TokenType) TokenIdent,
Bytes: ([]uint8) (len=8 cap=8) {
00000000 70 72 6f 76 69 64 65 72 |provider|
},
SpacesBefore: (int) 0
}),
(*hclwrite.Token)(0xc0002812c0)({
Type: (hclsyntax.TokenType) TokenEqual,
Bytes: ([]uint8) (len=1 cap=1) {
00000000 3d |=|
},
SpacesBefore: (int) 0
}),
(*hclwrite.Token)(0xc000281110)({
Type: (hclsyntax.TokenType) TokenOQuote,
Bytes: ([]uint8) (len=1 cap=1) {
00000000 22 |"|
},
SpacesBefore: (int) 0
}),
(*hclwrite.Token)(0xc000281140)({
Type: (hclsyntax.TokenType) TokenQuotedLit,
Bytes: ([]uint8) (len=21 cap=32) {
00000000 61 77 73 2e 64 65 76 65 6c 6f 70 6d 65 6e 74 2d |aws.development-|
00000010 77 65 73 74 32 |west2|
},
SpacesBefore: (int) 0
}),
(*hclwrite.Token)(0xc000281170)({
Type: (hclsyntax.TokenType) TokenCQuote,
Bytes: ([]uint8) (len=1 cap=1) {
00000000 22 |"|
},
SpacesBefore: (int) 0
}),
(*hclwrite.Token)(0xc000281380)({
Type: (hclsyntax.TokenType) TokenNewline,
Bytes: ([]uint8) (len=1 cap=1) {
00000000 0a |.|
},
SpacesBefore: (int) 0
})
}
```

What am I doing wrong that is resulting in quote tokens still being wrapped around my output rather than me receiving the unquoted output I expect after this token manipulation?

Thanks very much for taking a look!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with RemoveQuotesAndInterpolation and AssembleHCLSlice, then inspect hclwrite token handling around attr.BuildTokens and AppendBlock. Reproduce the quoted output using the supplied AWSVPC example and determine whether the token mutation is preserved; done means the behavior and supported path for the intended output are established.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.