cloudflare / cloudflare/skills

turnstile-spin: validate.sh always fails — jq '//' operator swallows success:false

Open Beginner friendly
#86 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
2.8k
Forks
272
Avg merge
8h 48m
Merged PRs (30d)
60

Description

## Summary

`skills/turnstile-spin/scripts/validate.sh` reports failure for every correctly-configured integration. The dummy-token check can never observe `success: false`, which is the exact response it is written to accept.

## Cause

Line:

```sh
success=$(echo "$dummy" | (jq -r '.success // "missing"' 2>/dev/null || echo "missing"))
```

jq's `//` is the *alternative* operator: it returns the right-hand side when the left is `null` **or `false`**. Since siteverify returns `"success": false` for a dummy token, `.success // "missing"` evaluates to `"missing"`, never `"false"`.

```console
$ echo '{"success":false}' | jq -r '.success // "missing"'
missing
$ echo '{"success":false}' | jq -r '.success'
false
```

Control flow then hits:

```sh
if [ "$success" != "false" ]; then
echo "{\"status\":\"error\",\"check\":\"dummy_siteverify\",...}"
exit 1
fi
```

so the `invalid-input-response` branch below it is unreachable.

## Reproduction

With a valid secret and any dummy token:

```console
$ TURNSTILE_SECRET= ./validate.sh --sitekey
validate: unexpected shape for dummy token: {"error-codes":["invalid-input-response"],"success":false,"messages":[]}
{"status":"error","check":"dummy_siteverify","detail":"expected success:false on a dummy token"}
```

That response is the documented success signal — SKILL.md's own edge-case table says `invalid-input-response` "means the secret IS valid. validate.sh treats this as success." It does not.

This is reproducible with the published test secret `1x0000000000000000000000000000000AA`, so it needs no real credentials.

## Impact

Step 10 (Validation) fails for every user with a working setup. SKILL.md instructs the agent to "surface the error and stop" on validation failure, so a correct integration is reported as broken at the final step.

## Suggested fix

```sh
success=$(echo "$dummy" | (jq -r 'if has("success") then .success else "missing" end' 2>/dev/null || echo "missing"))
```

Verified against live siteverify with both the always-pass and always-fail test secrets.

## Note on the Python fallback

The `||` fallbacks throughout these scripts mask this class of bug: when `jq` is absent the fallback yields `"missing"`, producing the same misleading error for a different reason. Worth distinguishing "parser unavailable" from "field absent".

Contributor guide

Open the contributing guide

Research direction

Start with skills/turnstile-spin/scripts/validate.sh and inspect the jq expression that reads the dummy siteverify response, then run it with the published test secret and a dummy token. Done means a response containing success:false reaches the expected invalid-input-response success path instead of reporting validation failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
cli
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
92/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.