JustAnotherArchivist / JustAnotherArchivist/snscrape

Content of tweet includes non written mentions

Open
#992 4 comments 0 reactions 0 assignees View on GitHub
bug module:twitter
Dominant language
Python
Stars
5.4k
Forks
783
PR merge metrics
No merged PRs in 30d

Description

### Describe the bug

Then scrapping the following [tweet](https://twitter.com/10xTeamDev/status/1674020720458776576), the content returned starts like `"@GitHubCopilot @tabnine @Replit @vercel Have you tried them ?"` instead of just `"Have you tried them ?"` as expected.

### How to reproduce

Use the `TwitterTweetScraper` and pass the tweet id `1674020720458776576`.

### Expected behaviour

There should be no non-written mentions at the beginning of the content.

### Screenshots and recordings

_No response_

### Operating system

macOS 13.4.1

### Python version: output of `python3 --version`

3.9

### snscrape version: output of `snscrape --version`

0.7.0.20230622

### Scraper

TwitterTweetScraper

### How are you using snscrape?

Module (`import snscrape.modules.something` in Python code)

### Backtrace

_No response_

### Log output

_No response_

### Dump of locals

_No response_

### Additional context

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start from the TwitterTweetScraper entry point and reproduce the issue with tweet ID 1674020720458776576. Compare the returned content with the tweet's written text and trace how leading mentions are included. Done means the content begins with "Have you tried them ?" without the non-written mentions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.