Packer intermittent crashes: panic: ConfigSpec failed: connection is shut down [recovered]
- Dominant language
- Go
- Stars
- 15.8k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
#### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
#### Overview of the Issue
Intermittent crashes with packer build. See subject/title. Packer requested I report this crash:
!!!!!!!!!!!!!!!!!!!!!!!!!!! PACKER CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!Packer crashed! This is always indicative of a bug within Packer.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Packer[1] so that we can fix this.[1]: https://github.com/hashicorp/packer/issues
!!!!!!!!!!!!!!!!!!!!!!!!!!! PACKER CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
#### Reproduction Steps
packer build .
### Packer version
1.8.5
### Simplified Packer Template
Need private way to provide this. Also the crash.log.
### Operating system and Environment details
Ubuntu 22.04, x86_64
### Log Fragments and crash.log files
see above.
Set the env var `PACKER_LOG=1` for maximum log detail.
tail of crash log:
2023/02/27 17:57:27 ConfigSpec failed: connection is shut down
panic: ConfigSpec failed: connection is shut down [recovered]
panic: ConfigSpec failed: connection is shut downgoroutine 772 [running]:
log.Panic({0xc0014ccf88?, 0x1?, 0xc000bfc460?})
/opt/hostedtoolcache/go/1.18.9/x64/src/log/log.go:385 +0x65
github.com/hashicorp/packer/packer.(*cmdProvisioner).checkExit(0xc0004ab730?, {0x4017300, 0xc00088ed30}, 0x0)
/home/runner/work/packer/packer/packer/cmd_provisioner.go:47 +0x7f
github.com/hashicorp/packer/packer.(*cmdProvisioner).ConfigSpec.func1()
/home/runner/work/packer/packer/packer/cmd_provisioner.go:19 +0x39
panic({0x4017300, 0xc00088ed30})
/opt/hostedtoolcache/go/1.18.9/x64/src/runtime/panic.go:838 +0x207
github.com/hashicorp/packer-plugin-sdk/rpc.(*commonClient).ConfigSpec(0xc001095620)
/home/runner/go/pkg/mod/github.com/hashicorp/packer-plugin-sdk@v0.3.2/rpc/common.go:44 +0x297
github.com/hashicorp/packer/packer.(*cmdProvisioner).ConfigSpec(0x48fe4c0?)
/home/runner/work/packer/packer/packer/cmd_provisioner.go:22 +0x65
github.com/hashicorp/packer/hcl2template.decodeHCL2Spec({0x5927740, 0xc000c60600}, 0x50e5c69?, {0x7f89868582d8?, 0xc00109bce0?})
/home/runner/work/packer/packer/hcl2template/decode.go:17 +0x39
github.com/hashicorp/packer/hcl2template.(*HCL2Provisioner).HCL2Prepare(0xc0010cc450, 0xc0010a0ea0?)
/home/runner/work/packer/packer/hcl2template/types.hcl_provisioner.go:51 +0x447
github.com/hashicorp/packer/hcl2template.(*HCL2Provisioner).Provision(0xc0010cc450, {0x5926da0, 0xc000cc0600}, {0x592bb88, 0xc0010a0e70}, {0x5929f10, 0xc000cceee0}, 0xc000cceee0?)
/home/runner/work/packer/packer/hcl2template/types.hcl_provisioner.go:71 +0x4e
github.com/hashicorp/packer/packer.(*ProvisionHook).Run(0xc000665160?, {0x5926da0, 0xc000cc0600}, {0x7f89ad6fd108?, 0x20?}, {0x592bb88, 0xc0010a0e70}, {0x5929f10?, 0xc000cceee0}, {0x4335480, ...})
/home/runner/work/packer/packer/packer/provisioner.go:122 +0x1cb
github.com/hashicorp/packer-plugin-sdk/packer.(*DispatchHook).Run(0x0?, {0x5926da0, 0xc000cc0600}, {0xc000665160, 0x10}, {0x592bb88, 0xc0010a0e70}, {0x5929f10, 0xc000cceee0}, {0x4335480, ...})
/home/runner/go/pkg/mod/github.com/hashicorp/packer-plugin-sdk@v0.3.2/packer/hook.go:47 +0x172
github.com/hashicorp/packer-plugin-sdk/rpc.(*HookServer).Run(0xc000c3ddc0, 0xc0010a0c60, 0xc00088ea20)
/home/runner/go/pkg/mod/github.com/hashicorp/packer-plugin-sdk@v0.3.2/rpc/hook.go:75 +0x263
reflect.Value.call({0xc000bfe2a0?, 0xc001553d28?, 0x13?}, {0x50e42f9, 0x4}, {0xc0007c6ef8, 0x3, 0x3?})
/opt/hostedtoolcache/go/1.18.9/x64/src/reflect/value.go:556 +0x845
reflect.Value.Call({0xc000bfe2a0?, 0xc001553d28?, 0x0?}, {0xc0010f56f8, 0x3, 0x3})
/opt/hostedtoolcache/go/1.18.9/x64/src/reflect/value.go:339 +0xbf
net/rpc.(*service).call(0xc000c3de00, 0x0?, 0xc000a58fc0?, 0xc000664f80, 0xc001530480, 0xc0010f57d0?, {0x3e6bbe0?, 0xc0010a0c60?, 0xc0006ba540?}, {0x3edfbe0, ...}, ...)
/opt/hostedtoolcache/go/1.18.9/x64/src/net/rpc/server.go:381 +0x226
created by net/rpc.(*Server).ServeCodec
/opt/hostedtoolcache/go/1.18.9/x64/src/net/rpc/server.go:478 +0x3fe
Contributor guide
Research direction
Start with packer/cmd_provisioner.go and trace the ConfigSpec call through hcl2template/decode.go and hcl2template/types.hcl_provisioner.go, using the reported stack and packer build . as the entry point. Compare this path with packer-plugin-sdk/rpc/common.go and reproduce with a shareable template and crash log. Done means the intermittent ConfigSpec failure no longer causes the reported panic, with regression coverage for the failure path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100