PAIR-code / PAIR-code/deliberate-lab

Reformat directory format for data downloads

Open
#755 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
TypeScript
Stars
96
Forks
40
Avg merge
2d 11h
Merged PRs (30d)
20

Description

[Fixed in #766 ] 1. Fix data population bug

To reproduce: go to an experiment with multiple survey fields, download data, and inspect. [experiment_name].json has the correct fields, but [experiment_name].participantData.csv is empty, and only contains an empty column for one survey question. Make sure that the correct data is populated in participantData.csv.

2. Reformat directory for ease-of-use

Current directory and file format:

[experiment_name]_Alerts.csv

[experiment_name]_ChatHistory_[CohortID]_[ChatDiscussionName].csv // For each cohort.

[experiment_name]_Logs.json

[experiment_name]_participantData.csv

[experiment_name].json

Proposed directory and file rename / reshuffling:

alerts.csv // Only if there were alerts (don't generate an empty CSV). This is how it works for payouts currently; if there are no payouts enabled, no file is generated.

For each unique chat stage:
[ChatDiscussionName]/ 
    [CohortID].csv
    ...

llmLogs.json 

participantData.csv 

experimentConfig.json 

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The data-population portion is marked fixed in #766. Start by locating the data-download generation entry point and comparing its output with the existing payout-file behavior; done means downloads use alerts.csv only when alerts exist, group chat CSVs under chat-stage directories, and use the requested llmLogs.json, participantData.csv, and experimentConfig.json names.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.