dogsheep / dogsheep/twitter-to-sqlite

Import archive error 'withheld_in_countries'

Open
#63 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
416
Forks
24
PR merge metrics
No merged PRs in 30d

Description

Importing the twitter archive I received this error:
```bash
$ twitter-to-sqlite import archive.db twitter-2021-12-31-.zip
birdwatch-note-rating: not yet implemented
birdwatch-note: not yet implemented
branch-links: not yet implemented
community-tweet: not yet implemented
contact: not yet implemented
device-token: not yet implemented
direct-message-mute: not yet implemented
mute: not yet implemented
periscope-account-information: not yet implemented
periscope-ban-information: not yet implemented
periscope-broadcast-metadata: not yet implemented
periscope-comments-made-by-user: not yet implemented
periscope-expired-broadcasts: not yet implemented
periscope-followers: not yet implemented
periscope-profile-description: not yet implemented
professional-data: not yet implemented
protected-history: not yet implemented
reply-prompt: not yet implemented
screen-name-change: not yet implemented
smartblock: not yet implemented
spaces-metadata: not yet implemented
sso: not yet implemented
Traceback (most recent call last):
File "/home/paulox/.virtualenvs/dogsheep/bin/twitter-to-sqlite", line 8, in
sys.exit(cli())
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/cli.py", line 759, in import_
archive.import_from_file(db, filename, content)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/archive.py", line 246, in import_from_file
db[table_name].insert_all(rows, pk=pk, replace=True)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 2625, in insert_all
self.insert_chunk(
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 2406, in insert_chunk
result = self.db.execute(query, params)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 422, in execute
return self.conn.execute(sql, parameters)
sqlite3.OperationalError: table archive_tweet has no column named withheld_in_countries
```

I found only a single tweet with the key `withheld_in_countries` in `tweet.js` that seems the problems:
```JSON
[
{
"tweet" : {
"retweeted" : false,
"source" : "Twitter for Android",
"entities" : {
"hashtags" : [
{
"text" : "NowOnAndroid",
"indices" : [
"64",
"77"
]
}
],
"symbols" : [ ],
"user_mentions" : [
{
"name" : "Periscope",
"screen_name" : "PeriscopeCo",
"indices" : [
"3",
"15"
],
"id_str" : "1111111111",
"id" : "222222222"
}
],
"urls" : [
{
"url" : "https://t.co/xxxxxxxxx",
"expanded_url" : "https://vine.co/v/xxxxxxxxx",
"display_url" : "vine.co/v/xxxxxxxxxx",
"indices" : [
"78",
"101"
]
}
]
},
"display_text_range" : [
"0",
"101"
],
"favorite_count" : "0",
"id_str" : "1111111111111111111111",
"truncated" : false,
"retweet_count" : "0",
"withheld_in_countries" : [
"TR"
],
"id" : "000000000000000000",
"possibly_sensitive" : false,
"created_at" : "Fri Aug 14 06:04:03 +0000 2015",
"favorited" : false,
"full_text" : "RT @periscopeco: Travel the world. LIVE. The Global Map is here #NowOnAndroid https://t.co/NZXdsPWROk",
"lang" : "en"
}
}
]
```

I solved the error removing the key from the `tweet.js` but I'm reporting this error to improve the project.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.