garrytan / garrytan/gstack

`design` still hardcodes `gpt-4o` in 11 places — and the obvious fix (bump to gpt-5.x) breaks 5 of them on `max_tokens`

Open
#2,807 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133k
Forks
19.9k
Avg merge
18h 46m
Merged PRs (30d)
26

Description

---

### Summary

Three coupled problems in `design/`, verified against `origin/main` @ `0d1bd561` (v1.79.0.0):

1. **`gpt-4o` is hardcoded at 11 call sites**, with no env override.
2. **5 of those 11 also pass `max_tokens`.** The gpt-5 family rejects that parameter outright, so bumping the model — the obvious fix for (1) — turns 5 working calls into hard 400s. The two changes have to ship together.
3. **`./setup` never rebuilds `design/dist/design`** when only `design/src` changes, so even a correct source fix does not reach users.

Reporting them as one issue because fixing any one alone leaves the user no better off.

---

### 1. Hardcoded `gpt-4o`, no override

```
$ git grep -c 'model: "gpt-4o"' origin/main -- design/src design/prototype.ts
prototype.ts:1 check.ts:1 design-to-code.ts:1 diff.ts:1
evolve.ts:2 generate.ts:1 iterate.ts:1 memory.ts:1 variants.ts:1 # 11 total
```

`design/src/auth.ts` reads `OPENAI_API_KEY` from the environment, but there is no
corresponding `*_MODEL` env var anywhere in `design/` — the model is a literal in
each file. Users who are required (by org policy, or by a model deprecation) to move
off `gpt-4o` have to patch and recompile, and lose the patch on the next upgrade.

Related but not the same ask: #990 (non-OpenAI image providers).

### 2. The trap: bumping the model breaks `max_tokens`

Five of the eleven sites POST to `/v1/chat/completions` with `max_tokens`:

| file | orchestrator | token param |
|---|---|---|
| `design/src/check.ts` | `:33` | `:60 max_tokens: 200` |
| `design/src/diff.ts` | `:39` | `:70 max_tokens: 600` |
| `design/src/memory.ts` | `:45` | `:69 max_tokens: 800` |
| `design/src/design-to-code.ts` | `:48` | `:72 max_tokens: 1000` |
| `design/src/evolve.ts` | `:124` | `:138 max_tokens: 400` |

The gpt-5 family rejects `max_tokens`:

```
$ curl -s -o /dev/null -w '%{http_code}\n' https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" -H 'Content-Type: application/json' \
-d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}],"max_tokens":16}'
200

$ ... same request with "model":"gpt-5.5"
400
{"error":{"message":"Unsupported parameter: 'max_tokens' is not supported with this model.
Use 'max_completion_tokens' instead.", ...}}
```

So a well-intentioned "replace gpt-4o with gpt-5.x" sweep silently converts five working
subcommands (`/design check`, `diff`, `memory`, `design-to-code`, `evolve`) into hard 400s.
The fix is `max_tokens` → `max_completion_tokens` on those five lines, in the same change.

`evolve.ts` is worth calling out: `:66` is a Responses/`image_generation` call and `:124`
is a chat call, both on `gpt-4o`. Bumping the file's model fixes one and breaks the other
unless `:138` moves too.

### 3. `./setup` never rebuilds the design binary

`setup:620-630` decides `NEEDS_BUILD` from four checks, all keyed on `$BROWSE_BIN`:

```sh
NEEDS_BUILD=0
if [ ! -x "$BROWSE_BIN" ]; then NEEDS_BUILD=1
elif [ -n "$(find "$SOURCE_GSTACK_DIR/browse/src" -type f -newer "$BROWSE_BIN" -print -quit 2>/dev/null)" ]; then NEEDS_BUILD=1
elif [ "$SOURCE_GSTACK_DIR/package.json" -nt "$BROWSE_BIN" ]; then NEEDS_BUILD=1
elif [ -f "$SOURCE_GSTACK_DIR/bun.lock" ] && [ "$SOURCE_GSTACK_DIR/bun.lock" -nt "$BROWSE_BIN" ]; then NEEDS_BUILD=1
fi
```

But `bun run build` (`scripts/build.sh:27-29`) compiles **four** binaries, including
`design/dist/design` — and `setup:662` already lists `design/dist/design` in its codesign
loop, so setup knows the artifact exists. `design/src` is simply absent from the trigger set.

Consequence: a change confined to `design/src` (no `browse/src`, no `package.json`, no
`bun.lock` churn) never triggers a rebuild, and `./setup` leaves the stale binary in place.
Two upstream commits already fit that shape (`2b08cfe7`, `f91ad61a`, both 2026-04-05).

Observed locally after editing `design/src` and running `./setup`:

```
$ ls -l design/dist/design
-rwxr-xr-x 60828000 3 Sep 20:39 design/dist/design # older than the source edit
$ strings -a design/dist/design | grep -oE 'gpt-4o' | wc -l
10 # source said otherwise
```

The failure is silent — the skill runs happily on the old binary — which is what makes it
worth fixing independently of (1) and (2).

**Suggested fix:** extend the staleness check to the sources of every artifact the build
produces, e.g. add `design/src` alongside `browse/src` and compare against the oldest of
the produced binaries rather than `$BROWSE_BIN` alone.

---

### Note on #1771

#1771 is still open, but the defect it describes appears to be gone:
`git grep -c 'gpt-image-2' origin/main -- design/src` returns 0, so the
`image_generation` tool no longer pins a model and falls back to `gpt-image-1`.
The `gpt-4o` orchestrator itself is what remains — hence this issue rather than a
comment there. Might be worth closing #1771 if that matches your read.

### Environment

gstack v1.79.0.0 (`0d1bd561`), macOS 15 arm64, bun 1.3.11.

Contributor guide

Open the contributing guide

Research direction

Start with the 11 model call sites in design/src and design/prototype.ts, then inspect setup:620-630 and scripts/build.sh:27-29. Run the existing design build and setup flow before changing anything. Done means model selection is no longer hardcoded, the five chat calls use the compatible token parameter, and setup rebuilds design/dist/design when design/src changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, shell, typescript
Domain
ai, build-system, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.