twitter / twitter/communitynotes

Abnormal tweetId in Note request data

Open
#376 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
350
Avg merge
6m
Merged PRs (30d)
2

Description

The newly released note request data is said to have four columns "userId tweetId createdAtMillis sourceLink". I notice that a non-trivial amount of tweetId seem to be abnormal.

Example:

cat batSignals-00000.tsv| grep $'\t91540'
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	9154089611629224624	1724916211485
BE177514EB62FA0F486A310A3DD07698C591ACE5077E70E5B5E9AFC105DD2DCE	9154092414728527376	1726533818655

Usually, tweetId starts with "1" but the above two tweetIds start with "9". I also searched for all requests proposed by the first userId.

cat batSignals-00000.tsv| grep C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	977040036244765513	1724916298043
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	1828757772290400291	1724935602779
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	2684673839852374649	1724319335350
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	2951003568237083342	1725515064463
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	3324343820942848469	1724258023892
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	3561937426136355544	1724743384355
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	4137635619809309092	1724258370090
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	4952653969869310800	1724936298653
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	5651508576841648976	1725515613590
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	5724641268837992313	1724326146100
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	6905583205459998959	1724916234226
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	6985250573715648597	1724257857799
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	7195974708253445253	1724126266603
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	7538073314496696227	1724937322549
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	7820944466784416933	1725108506437
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	8558507768692594852	1725108335051
C23DBB56A37D9566AE659462EA431791A55D9403F8E0B54E5E5899E236B9526A	9154089611629224624	1724916211485

It looks like all of this person's requests are on some sort of abnormal tweetIds. All of these tweets are not available on X. Is there a bug in exporting tweetIds for the note request data?

Updated: The requests with abnormal tweetIds are very small fraction---about 0.09%.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the documented four-column format and the records in batSignals-00000.tsv, focusing on the abnormal tweetId examples and the affected user's rows. Trace how note request data is exported, then determine whether the IDs are changed during export and verify the reported roughly 0.09% frequency. Done means identifying the cause and confirming whether the published data is incorrect.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.