Devographics / Devographics/Monorepo

LLM-based freeform data coding

Open
#545 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
178
Forks
55
Avg merge
1m
Merged PRs (30d)
2

Description

We need two functions

### generateCodebook

```
generateCodebook(question: QuestionMetadata, potentialEntities: Entity[], rawAnswers: RawDataAnswer[]) // returns {codes: Code[]}

// with

type Code = {
id: string // id of the corresponding entity
existsInRepo: boolean // whether the code already exists in the entities repo
entity: Entity // when adding a new entity, should contain entity metadata
}
```

- we need the question so that we can submit the question prompt to the LLM.
- `entities` here are the existing potentially matching entities, as defined by the question's `matchTags` function. Ideally less than 1000 (or whatever the context cutoff is).
- It can potentially be empty if this is a brand new question covering a totally new topic.
- Any entities mentioned in previous years but not yet part of that set of entities should be manually included before the existing codebook is passed to the LLM.
- Any entity with `existsInRepo: true` should then manually be added to the Entities repo in the appropriate file.
- To save on time/data, it's ok if the LLM only returns the `id` for existing entities and not the entire `entity` object.

### generateMatches

```
generateMatches(question: QuestionMetadata, matchingEntities: entities[], rawAnswers: RawDataAnswer[]) // return result: matches: AnswerMatch[]

// where AnswerMatch is defined as:

type AnswerMatch = {
index: number;
answer: string;
answerId: string;
tokenIds: string[];
};
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the TypeScript definitions for QuestionMetadata, Entity, RawDataAnswer, Code, and AnswerMatch, then identify where the two requested functions belong. Done means generateCodebook handles existing and new entities as specified, and generateMatches returns the required match fields for the raw answers.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.