feature: basic results analysis for BehaviorSpace
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 20/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Domain
- analytics, data-visualization
Research direction
The issue names no files, tests, or entry points; begin by locating BehaviorSpace's existing results and CSV handling. Narrow the broad notes to one agreed first capability, then verify it against a simple one-parameter experiment and the requested plot or export behavior.
Written by the indexing model from the issue text.
Description
some old notes from a 2009 devel meeting on this:
General agreement, I think that we need to make sure that simple,
broad, basic low-threshold functionality is there before we do anything
too extensive or advanced in any particular area (within this general
area of results collection and analysis). currently it's quite a pain
to do even the simplest stuff, e.g. familiar examples like varying
the density in Fire and getting a plot of the resulting burn percentagesGeneral agreement, I think, that the right next step is to do what
Forrest proposed, namely build something like Ben Shargel's original
tool, but instead of focusing on "heat maps" (which show two parameters
varying), focus on regular plots (which show one parameter varying).
such a UI can support one parameter experiments directly, and we should
support Paulo's proposed simplified output format. The same UI can
support multiple-parameter experiments as long as we require that you
pick one parameter at a time and fix the others temporarily in order to
get a plot.Further detail/comments on Forrest's suggestion:
- We should support multiple runs per combination of settings -- this is
basic. We could just take the mean of the runs and show that, probably
with an error bar or a mustache box or whatever it's called, or maybe
just plot a dot for each run and let the user eyeball the dots.- Once you have a plot there should always be an export button to export
just the data you are seeing Paulo-style.- It would be awesome if you could start examining the results as they
come in, rather than having to wait until the entire experiment is over.
We should probably plan for this from the start.- You will need be able to open a previously saved csv file and browse
it using this UI.- It would be awesome, and maybe not that hard, if you could open
multiple previously saved csv files and browse the combined results from
all of them. That would save cluster people the trouble of merging their
results files themselves.Other notes:
- Josh U recommended we look at TinkerPlots ("it kind of blew my mind
for ease of use")- Michelle says working in this space would be easier if she/we didn't
have to parse csv files and suck data out of plots. some kind of objects
holding the results or events notifying of results as they happen (in
NetLogoListener or whatever)- Seth said maybe "metrics" should be something that we
explicitly represent so they can be shared by plots, monitors,
behaviorspace, etc? currently the solution is to make a global variable
and/or reporter procedure. but then you have to put that in the
procedures tab. is that good or bad? this overlaps with the Two Joshes
plotting work. maybe it's fine the way it is. or maybe some things
could be made more convenient without making metrics separate and
explicit, for example, when making an experiment setup you could choose
from a list of reporters taken from monitors and plots.
- Dominant language
- Scala
- Stars
- 1.2k
- Forks
- 272
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NetLogo/NetLogo
-
doc
Difficulty 1/5 1-3 hours Newbie friendliness 72/100
-
compiler gui low priority
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
-
widgets
Difficulty 4/5 3-5 days Newbie friendliness 52/100
-
design gui
Difficulty 5/5 Over a week Newbie friendliness 45/100
-
infrastructure
Difficulty 5/5 Over a week Newbie friendliness 25/100
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
apache/datafusion-comet#6060 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
lichess-org/lila#21756 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
salesforce/evalon#10 ·