twitter / twitter/communitynotes

Feature Request --- Add LLM reasoning and suggestion disposition data to collaborative note records

Open
#431 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
350
Avg merge
6m
Merged PRs (30d)
2

Description

Is your feature request related to a problem? Please describe.
The current collaborative note data captures four elements: (1) which notes are collaborative, (2) who contributed each suggestion, (3) the suggestion content, and (4) note content changes. However, two important pieces of information are missing:

  • LLM reasoning traces — the "show my work" explanations displayed in the Community Notes interface that reveal how the model arrived at its output.
  • Suggestion disposition — whether each contributor's suggestion was ultimately accepted or rejected, and why.

Without these, researchers and developers cannot fully reconstruct the decision-making process behind a collaborative note's final form.

Describe the solution you'd like
Expose both of the above in the data:

  • LLM reasoning: Include the model's chain-of-thought or rationale associated with each collaborative note, as shown in the "show my work" UI.
  • Accept/reject status: For each suggestion, record whether it was accepted or rejected, and ideally, why.

This would enable deeper analysis of how collaborative notes evolve and how human and LLM judgments interact.

Describe alternatives you've considered
No alternatives identified — this information already exists in the system and would primarily need to be surfaced in the data.

Additional context
N/A

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are identified in the issue. Start by locating the data export or collaborative-note record models and the source of the “show my work” and suggestion disposition fields; done means defining and documenting a safe, complete representation for both kinds of data.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.