twitter / twitter/communitynotes

CONCAT ERROR(S) when running CN algorithm

Open
#345 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
350
Avg merge
6m
Merged PRs (30d)
2

Description

I have the right packages installed and the correct Python version, so why does this error come up when running the CN algorithm?

MERGE ERROR(S) AT: /burg/home/sgg2140/communitynotes/sourcecode/scoring/pflip_plus_model.py, in _compute_scoring_cutoff, at line 348: scoringCutoff = scoringCutoff.merge(cutoffByRatings[[c.noteIdKey, "ratingMin"]])
PandasTypeError: Type expectation mismatch on noteId: found=Int64 expected=int64
PandasTypeError: Input mismatch on noteId: left=int64 vs right=Int64 (UNALLOWED)
PandasTypeError: Merge key mismatch on noteId: left=int64 vs right=Int64 (UNALLOWED)
PandasTypeError: Output mismatch on noteId: result=int64 expected=None (UNALLOWED)

INFO:birdwatch.constants:Fitting pflip model elapsed time: 11.41 secs (0.19 mins)
Traceback (most recent call last):
File "/burg/home/sgg2140/communitynotes/sourcecode/main.py", line 33, in
main()
File "/burg/home/sgg2140/communitynotes/sourcecode/scoring/runner.py", line 269, in main
return _run_scorer(args=args, dataLoader=dataLoader, extraScoringArgs=extraScoringArgs)
File "/burg/home/sgg2140/communitynotes/sourcecode/scoring/pandas_utils.py", line 682, in _inner
retVal = main(*args, **kwargs)
File "/burg/home/sgg2140/communitynotes/sourcecode/scoring/runner.py", line 222, in _run_scorer
scoredNotes, helpfulnessScores, newStatus, auxNoteInfo = run_scoring(
File "/burg/home/sgg2140/communitynotes/sourcecode/scoring/run_scoring.py", line 1987, in run_scoring
) = run_prescoring(
File "/burg/home/sgg2140/communitynotes/sourcecode/scoring/run_scoring.py", line 1268, in run_prescoring
pflipPlusModel.fit(notes, ratings, noteStatusHistory, prescoringRaterModelOutput)
File "/burg/home/sgg2140/communitynotes/sourcecode/scoring/pflip_plus_model.py", line 1566, in fit
self._prepare_note_info(
File "/burg/home/sgg2140/communitynotes/sourcecode/scoring/pflip_plus_model.py", line 1054, in _prepare_note_info
scoringCutoff = self._compute_scoring_cutoff(
File "/burg/home/sgg2140/communitynotes/sourcecode/scoring/pflip_plus_model.py", line 349, in _compute_scoring_cutoff
assert len(scoringCutoff) == beforeMerge
AssertionError
srun: error: g276: task 0: Exited with exit code 1
ERROR: main.py failed

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in sourcecode/scoring/pflip_plus_model.py at _compute_scoring_cutoff, especially the merge on line 348, and reproduce the CN algorithm run shown in the traceback. Trace the types of the noteId merge keys and check the assertion at line 349; the issue is done when the merge completes without PandasTypeError and the assertion passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.