Unexpected tokens in JSON jobspec when attributes are misquoted
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Nomad version
Tested on 1.4.6 and 1.6.1
### Issue
If you fail to put quotes around a value you assign to a namespace and parse that jobspec as HCL2, the return value will contain the unquoted value inside of `${` ... `}` delimiters. This seems to hold true for any attribute.
For example, `namespace = ns1` is returned in the JSON data as `"Namespace": "${ns1}",`
### Reproduction steps
Parse a job with right-hand quoting issues. I've reproduced this using `nomad job run -output example.nomad.hcl` to parse without executing the job and using the `v1/jobs/parse` endpoint (`jq -Rsc '{ JobHCL: ., Canonicalize: true }' example.nomad.hcl | nomad operator api /v1/jobs/parse`) on a running Nomad node.
The following is the example job I used to exercise the issue.
```
job "example" {
namespace = ns
group "cache" {
task "task" {
driver = raw_exec
config {
command = bash
args = ["-c", "while true; do sleep 1; done"]
}
}
}
}
```
#### Expected Result
Some sort of error
#### Actual Result
Click here to show Rendered JSON jobspec
```json
{
"Job": {
"Region": null,
"Namespace": "${ns}",
"ID": "example",
"Name": "example",
"Type": null,
"Priority": null,
"AllAtOnce": null,
"Datacenters": null,
"Constraints": null,
"Affinities": null,
"TaskGroups": [
{
"Name": "cache",
"Count": null,
"Constraints": null,
"Affinities": null,
"Tasks": [
{
"Name": "task",
"Driver": "${raw_exec}",
"User": "",
"Lifecycle": null,
"Config": {
"args": [
"-c",
"while true; do sleep 1; done"
],
"command": "${bash}"
},
"Constraints": null,
"Affinities": null,
"Env": null,
"Services": null,
"Resources": null,
"RestartPolicy": null,
"Meta": null,
"KillTimeout": null,
"LogConfig": null,
"Artifacts": null,
"Vault": null,
"Templates": null,
"DispatchPayload": null,
"VolumeMounts": null,
"Leader": false,
"ShutdownDelay": 0,
"KillSignal": "",
"Kind": "",
"ScalingPolicies": null
}
],
"Spreads": null,
"Volumes": null,
"RestartPolicy": null,
"ReschedulePolicy": null,
"EphemeralDisk": null,
"Update": null,
"Migrate": null,
"Networks": null,
"Meta": null,
"Services": null,
"ShutdownDelay": null,
"StopAfterClientDisconnect": null,
"MaxClientDisconnect": null,
"Scaling": null,
"Consul": null
}
],
"Update": null,
"Multiregion": null,
"Spreads": null,
"Periodic": null,
"ParameterizedJob": null,
"Reschedule": null,
"Migrate": null,
"Meta": null,
"ConsulToken": null,
"VaultToken": null,
"Stop": null,
"ParentID": null,
"Dispatched": false,
"DispatchIdempotencyToken": null,
"Payload": null,
"ConsulNamespace": null,
"VaultNamespace": null,
"NomadTokenID": null,
"Status": null,
"StatusDescription": null,
"Stable": null,
"Version": null,
"SubmitTime": null,
"CreateIndex": null,
"ModifyIndex": null,
"JobModifyIndex": null
}
}
```
## Hypothesis
Nomad uses a "tweaked" version of HCL2 because Nomad's jobspec string interpolation syntax looks like HCL2 interpolations, which created issues with HCL2 treating those interpolation strings as UnknownValues and erroring out. The tweaks allow for certain unknown values to be reinserted into the jobspec so that Nomad can process them in due course. This issue seems like a case where the parser encounters the unquoted RHS value, HCL2 treats it like a Traversal, evaluates it to an Unknown value, and treats it the same as a legitimate interpolated Nomad value (like `${NOMAD_TASK_DIR}`)
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the issue with the supplied example jobspec using `nomad job run -output example.nomad.hcl` and the `/v1/jobs/parse` endpoint. Trace how the HCL2 jobspec is parsed and how unquoted RHS traversals become `${...}` values; done means malformed unquoted attributes produce an error instead of a rendered JSON jobspec.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100