NetLogo / NetLogo/Netlogo-LLM-Extension

feat: batched tick-level inference (llm:chat-all)

Open
#53 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Scala
Stars
1
Forks
0
Avg merge
3d 1h
Merged PRs (30d)
4

Description

Roadmap priority: B4 — described in the progress report as "the single biggest simulation-speed win for population-scale models."

Summary

Add a batched, tick-level inference primitive — an llm:chat-all-style call that fires every agent's request concurrently with one barrier per tick, instead of N sequential per-agent calls.

Why it matters

The dominant cost in a population-scale LLM model is wall-clock, not tokens. ask turtles [ let r llm:chat ... ] serializes: 100 turtles at 2s per call is over 3 minutes per tick. Firing them together and joining once per tick collapses that to roughly the slowest single call.

llm:chat-async already exists but retrieval is a blocking runresult, so the modeler has to hand-roll the two-pass pattern (fire all, then collect all) and gets no concurrency control.

What needs to be done

  1. Decide the API shape. Options: a command that takes an agentset and a prompt reporter, a two-phase fire/collect pair, or a batched reporter returning a list aligned to the agentset.
  2. Respect NetLogo's execution model — results must land back on the right agents, and the barrier must complete before the primitive returns.
  3. Per-agent history must stay correct under concurrent commit (the historyLock / commitExchange path from 14f7c35 already handles this; verify it holds at batch scale).
  4. Per-agent failure isolation — one failed call must not abort the batch.
  5. Must compose with throttling (#48): "fire all" needs to mean "queue all, run at most N concurrently."

Open questions

  • What is the natural NetLogo idiom here? A primitive taking an agentset and a prompt-producing reporter is powerful but unusual for this codebase.
  • Should the result be assigned to a turtle variable automatically, or returned as a list for the modeler to distribute?
  • Interaction with llm:chat-async — does this replace it, or sit alongside?

Related

  • #48 throttling (hard dependency — unbounded batch fan-out is exactly the failure mode documented there)
  • #49 token tracking (needed to report batch cost)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing llm:chat-async path, especially the blocking runresult behavior, and inspect the historyLock/commitExchange path referenced in the issue. Resolve the API and result-assignment design with #48 throttling and #49 token tracking before implementation; done means a barriered batch preserves per-agent history, isolates failures, and limits concurrency.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
ai, backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.