beyondcode / beyondcode/polyscope-community

[Bug]: Autopilot retries session-limit failures every ~1.5s and each retry consumes a max_iterations slot

Open
#196 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
20
Forks
0
PR merge metrics
No merged PRs in 30d

Description

### Platform

macOS

### Operating system version

macOS Tahoe 26.5.2 (25F84)

### System architecture

ARM64 (M1, M2, etc)

### PolyScope Version

0.24.1

### Bug description

When the Claude session limit is reached mid-goal, Autopilot enters a tight retry
loop instead of waiting for the reset time, and every failed attempt consumes one
of the goal's max_iterations slots. This leaves the goal permanently over its cap,
so from then on it re-pauses at every story boundary and needs a manual Resume for
each remaining story.

Measured on a single quota event, with two goals running in parallel (both hit at
the same moment). All timestamps below are UTC, as stored in the database.

- 22 new sessions were created in 30 seconds. The intervals between them are 1-2s
(mean 1.43s), i.e. ~42 retries/min, with no backoff of any kind.
- Each of those sessions recorded the same assistant message:
"You've hit your session limit · resets 1pm (Europe/Madrid)"
- autopilot_meta.iterations_completed ended at 28 against max_iterations 25 on both
goals.
- Subtracting the failed attempts from the counter reconstructs the real work
exactly: goal A 28 - 23 = 5 (4 completed stories + 1 in progress), goal B
28 - 22 = 6 (6 completed stories). So each retry costs exactly one iteration.

The retry loop seems to stop only when the counter crosses max_iterations: in both
goals it stopped immediately after passing 25. If that is correct, raising
max_iterations does not mitigate the problem, it just makes the storm longer and
still leaves the goal at its cap. This part is inferred from two samples and I
could not confirm it directly.

Consequences:

1. A single quota event costs the goal its entire remaining iteration budget,
multiplied by the number of goals running in parallel.
2. After the reset time passed, Autopilot did not resume on its own. The goal
stayed paused for about 1.5 hours until I clicked Resume manually.
3. Because the goal is now over its cap, each Resume runs exactly one story and
pauses again. A goal with 7 remaining stories needs 7 manual clicks.

Expected behaviour:

- An attempt that failed because of the provider's usage limit should not count
towards max_iterations.
- Autopilot should back off instead of retrying every 1.5s, and ideally wait until
the reset timestamp that Claude already provides in the error message, then
continue on its own.

### Steps to reproduce

1. Start an Autopilot goal with several stories, using Claude as the agent and the
default max_iterations (25).
2. Let it run until the Claude session limit is reached mid-goal. This is easiest to
trigger with two or more goals running in parallel.
3. Look at the Activity feed: the same "You've hit your session limit" message
repeats many times within a few seconds.
4. Check the counter: iterations_completed is now higher than max_iterations.
5. Wait until after the announced reset time. The goal stays paused.
6. Click Resume. Exactly one story runs and the goal pauses again. Repeat for every
remaining story.

### Relevant log output

```shell
# Retry storm: 22 sessions in 30 seconds, intervals 1-2s (UTC)
sqlite3 -readonly ~/.polyscope/polyscope.db \
"select created_at from sessions where worktree_id='' order by created_at;"

10:33:44 10:33:45 10:33:46 10:33:48 10:33:50 10:33:52 10:33:53 10:33:55
10:33:56 10:33:57 10:33:59 10:34:01 10:34:02 10:34:04 10:34:05 10:34:06
10:34:07 10:34:08 10:34:10 10:34:11 10:34:13 10:34:14

# Counter left above the cap on both goals
sqlite3 -readonly ~/.polyscope/polyscope.db \
"select status, iterations_completed, max_iterations from autopilot_meta;"

goal-a | running | 28 | 25
goal-b | paused | 28 | 25

# One iteration consumed per failed attempt
sqlite3 -readonly ~/.polyscope/polyscope.db \
"select am.worktree_id,
am.iterations_completed,
(select count(*) from messages m
join sessions s on s.id = m.session_id
where s.worktree_id = am.worktree_id
and m.role = 'assistant'
and m.content like '%session limit%') as failed_attempts
from autopilot_meta am;"

goal-a | 28 | 23 -> 28 - 23 = 5 real iterations (4 completed + 1 in progress)
goal-b | 28 | 22 -> 28 - 22 = 6 real iterations (6 completed)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.