cloudflare / cloudflare/skills
turnstile-spin: validate.sh always fails — jq '//' operator swallows success:false
- Dominant language
- Shell
- Stars
- 2.8k
- Forks
- 272
- Avg merge
- 8h 48m
- Merged PRs (30d)
- 60
Description
## Summary
`skills/turnstile-spin/scripts/validate.sh` reports failure for every correctly-configured integration. The dummy-token check can never observe `success: false`, which is the exact response it is written to accept.
## Cause
Line:
```sh
success=$(echo "$dummy" | (jq -r '.success // "missing"' 2>/dev/null || echo "missing"))
```
jq's `//` is the *alternative* operator: it returns the right-hand side when the left is `null` **or `false`**. Since siteverify returns `"success": false` for a dummy token, `.success // "missing"` evaluates to `"missing"`, never `"false"`.
```console
$ echo '{"success":false}' | jq -r '.success // "missing"'
missing
$ echo '{"success":false}' | jq -r '.success'
false
```
Control flow then hits:
```sh
if [ "$success" != "false" ]; then
echo "{\"status\":\"error\",\"check\":\"dummy_siteverify\",...}"
exit 1
fi
```
so the `invalid-input-response` branch below it is unreachable.
## Reproduction
With a valid secret and any dummy token:
```console
$ TURNSTILE_SECRET= ./validate.sh --sitekey
validate: unexpected shape for dummy token: {"error-codes":["invalid-input-response"],"success":false,"messages":[]}
{"status":"error","check":"dummy_siteverify","detail":"expected success:false on a dummy token"}
```
That response is the documented success signal — SKILL.md's own edge-case table says `invalid-input-response` "means the secret IS valid. validate.sh treats this as success." It does not.
This is reproducible with the published test secret `1x0000000000000000000000000000000AA`, so it needs no real credentials.
## Impact
Step 10 (Validation) fails for every user with a working setup. SKILL.md instructs the agent to "surface the error and stop" on validation failure, so a correct integration is reported as broken at the final step.
## Suggested fix
```sh
success=$(echo "$dummy" | (jq -r 'if has("success") then .success else "missing" end' 2>/dev/null || echo "missing"))
```
Verified against live siteverify with both the always-pass and always-fail test secrets.
## Note on the Python fallback
The `||` fallbacks throughout these scripts mask this class of bug: when `jq` is absent the fallback yields `"missing"`, producing the same misleading error for a different reason. Worth distinguishing "parser unavailable" from "field absent".
Contributor guide
Research direction
Start with skills/turnstile-spin/scripts/validate.sh and inspect the jq expression that reads the dummy siteverify response, then run it with the published test secret and a dummy token. Done means a response containing success:false reaches the expected invalid-input-response success path instead of reporting validation failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 92/100