twitter / twitter/communitynotes
Request for Scored Output Files from Algorithm Execution
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 350
- Avg merge
- 6m
- Merged PRs (30d)
- 2
Description
Is your feature request related to a problem? Please describe.
I need aggregated results so I can analyze helpfulness on Notes level. I think the only way so far is to run the algorithm from scratch so I’m reproducing results using the downloaded data (notes, ratings, notes history, and user enrollment) on a 64-core Intel(R) Xeon(R) Gold 6448H CPU with 500GB memory (correct me if I am wrong). However, after 20 hours, the pre-scoring phase still hasn’t completed. It looks like it won't finish within one day which stops me from working on further analysis.
Since the algorithm runs every hour or so on the server, may I know:
- would it be possible to share the output files (scored_notes.tsv, helpfulness_scores.tsv, note_status_history.tsv, and aux_note_info.tsv)?
- and the hardware requirement and expected running time if I want to generate aggregated scores for notes myself?
This would greatly help for research analysis, as running the algorithm locally to aggregate helpfulness scores has been quite challenging.
Describe the solution you'd like
Would it be possible to share the output files (scored_notes.tsv, helpfulness_scores.tsv, note_status_history.tsv, and aux_note_info.tsv)? They don’t need to be the latest versions—files aligned with the current download page would be fine.
Describe alternatives you've considered
It would be nice to share the hardware requirement and expected running time if I want to generate aggregated scores for notes from scratch, or any intermediate process.
Additional context
Thank you so much for your contribution on this amazing project! I am a PhD student working on fact-checking in Natural Language Processing and I am very happy to explore and contribute more. I am actively working on this and any help in above questions would be much appreciated!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the current download page and the linked ranking-notes documentation to understand how algorithm outputs are currently exposed. Compare the requested scored_notes.tsv, helpfulness_scores.tsv, note_status_history.tsv, and aux_note_info.tsv files with the available data, and document whether publishing them or hardware and runtime guidance is feasible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100