NetLogo / NetLogo/Netlogo-LLM-Extension
feat: batched tick-level inference (llm:chat-all)
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1
- Forks
- 0
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 4
Description
Roadmap priority: B4 — described in the progress report as "the single biggest simulation-speed win for population-scale models."
Summary
Add a batched, tick-level inference primitive — an llm:chat-all-style call that fires every agent's request concurrently with one barrier per tick, instead of N sequential per-agent calls.
Why it matters
The dominant cost in a population-scale LLM model is wall-clock, not tokens. ask turtles [ let r llm:chat ... ] serializes: 100 turtles at 2s per call is over 3 minutes per tick. Firing them together and joining once per tick collapses that to roughly the slowest single call.
llm:chat-async already exists but retrieval is a blocking runresult, so the modeler has to hand-roll the two-pass pattern (fire all, then collect all) and gets no concurrency control.
What needs to be done
- Decide the API shape. Options: a command that takes an agentset and a prompt reporter, a two-phase fire/collect pair, or a batched reporter returning a list aligned to the agentset.
- Respect NetLogo's execution model — results must land back on the right agents, and the barrier must complete before the primitive returns.
- Per-agent history must stay correct under concurrent commit (the
historyLock/commitExchangepath from14f7c35already handles this; verify it holds at batch scale). - Per-agent failure isolation — one failed call must not abort the batch.
- Must compose with throttling (#48): "fire all" needs to mean "queue all, run at most N concurrently."
Open questions
- What is the natural NetLogo idiom here? A primitive taking an agentset and a prompt-producing reporter is powerful but unusual for this codebase.
- Should the result be assigned to a turtle variable automatically, or returned as a list for the modeler to distribute?
- Interaction with
llm:chat-async— does this replace it, or sit alongside?
Related
- #48 throttling (hard dependency — unbounded batch fan-out is exactly the failure mode documented there)
- #49 token tracking (needed to report batch cost)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing llm:chat-async path, especially the blocking runresult behavior, and inspect the historyLock/commitExchange path referenced in the issue. Resolve the API and result-assignment design with #48 throttling and #49 token tracking before implementation; done means a barriered batch preserves per-agent history, isolates failures, and limits concurrency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- ai, backend-api-design, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100