Metaculus / Metaculus/forecasting-tools
Bots timeout occasionally, and seem to take much longer than they should
Open
@elisescu is already working on this.
Since Feb 18, 2026.
Low Priority
- Dominant language
- Python
- Stars
- 79
- Forks
- 38
- Avg merge
- 20m
- Merged PRs (30d)
- 12
Description
Some bots occasionally take a really long time to run. I'm not sure why.
Examplest errors
Most of this has been solved with longer timeouts per response. But a bot shouldn't need 5min to make a forecast I don't think. This makes me think I'm doing something wrong.
Hypotheses
- It actually takes bots this long
- The Metaculus Client has a sync timeout when waiting between requests in order to avoid rate limiting. If this triggers when one forecast finishes, then the other forecasts wait a while to finish (though the long timeouts happened even before the rate limiting change).
- Something to do with the concurrency limit 1 for research
- We are using 2 structure_outputs per forecast
- We messed up batching somewhere
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.