NetLogo / NetLogo/Netlogo-LLM-Extension

feat: BehaviorSpace-aware experiment logging to CSV

Open
#56 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Scala
Stars
1
Forks
0
Avg merge
3d 1h
Merged PRs (30d)
4

Description

Roadmap priority: B7

Summary

Automatically log every prompt, response, model, and token count per BehaviorSpace run to CSV, so LLM behaviour can be analysed alongside model outputs.

Why it matters

Every demo that needed this built it by hand — demos/provider-sensitivity/ has its own CSV writing and a Python analysis script, and crisis-triage does the same. That per-demo scaffolding should be a feature.

Without it, a sweep tells you what happened in the model but not what the LLM said, so there is no way to correlate outcomes with prompts or drift.

What needs to be done

  1. Detect BehaviorSpace context and the current run number from the workspace.
  2. Auto-write a per-run CSV: run number, tick, agent id, prompt, response, model, provider, token counts, latency, error (if any).
  3. Config key to enable and to set the output path; off by default.
  4. Handle concurrent BehaviorSpace runs writing to the same target without interleaving corruption.
  5. Escape newlines and quotes properly — prompts and responses are multi-line free text, which naive CSV writing will break.

Open questions

  • CSV or JSONL? CSV is what the existing demo analysis scripts expect; JSONL handles multi-line text far better. Possibly both.
  • Should this reuse the recording format from B6 (#55) rather than being a separate writer?

Related

  • #49 token tracking (dependency — token counts are one of the requested columns)
  • B6 record/replay (#55) — overlapping data capture

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing CSV writing and analysis code in demos/provider-sensitivity/ and the corresponding crisis-triage scaffolding. Review the dependency on #49 and the overlap with #55 before resolving the CSV versus JSONL and recording-format questions. Done means an opt-in BehaviorSpace-aware logger records the requested fields safely for concurrent runs, with correct escaping and configurable output.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.