NetLogo / NetLogo/Netlogo-LLM-Extension
feat: BehaviorSpace-aware experiment logging to CSV
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1
- Forks
- 0
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 4
Description
Roadmap priority: B7
Summary
Automatically log every prompt, response, model, and token count per BehaviorSpace run to CSV, so LLM behaviour can be analysed alongside model outputs.
Why it matters
Every demo that needed this built it by hand — demos/provider-sensitivity/ has its own CSV writing and a Python analysis script, and crisis-triage does the same. That per-demo scaffolding should be a feature.
Without it, a sweep tells you what happened in the model but not what the LLM said, so there is no way to correlate outcomes with prompts or drift.
What needs to be done
- Detect BehaviorSpace context and the current run number from the workspace.
- Auto-write a per-run CSV: run number, tick, agent id, prompt, response, model, provider, token counts, latency, error (if any).
- Config key to enable and to set the output path; off by default.
- Handle concurrent BehaviorSpace runs writing to the same target without interleaving corruption.
- Escape newlines and quotes properly — prompts and responses are multi-line free text, which naive CSV writing will break.
Open questions
- CSV or JSONL? CSV is what the existing demo analysis scripts expect; JSONL handles multi-line text far better. Possibly both.
- Should this reuse the recording format from B6 (#55) rather than being a separate writer?
Related
- #49 token tracking (dependency — token counts are one of the requested columns)
- B6 record/replay (#55) — overlapping data capture
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing CSV writing and analysis code in demos/provider-sensitivity/ and the corresponding crisis-triage scaffolding. Review the dependency on #49 and the overlap with #55 before resolving the CSV versus JSONL and recording-format questions. Done means an opt-in BehaviorSpace-aware logger records the requested fields safely for concurrent runs, with correct escaping and configurable output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100