틱톡 크롤링 - DB 연계 코드
- Dominant language
- Jupyter Notebook
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
```
import requests
from bs4 import BeautifulSoup
import time
import urllib.request
from selenium.webdriver import Chrome
from selenium import webdriver
import re
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
import datetime as dt
import pymysql
import pandas as pd
from sqlalchemy import create_engine
def tiktok_keyword():
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument("--single-process")
chrome_options.add_argument("--disable-dev-shm-usage")
path='./chromedriver'
driver = webdriver.Chrome(path, chrome_options=chrome_options)
driver.implicitly_wait(1)
driver.set_window_size(2048, 1200)
driver.get('https://www.tiktok.com/ko-KR/')
time.sleep(5)
keyword_list = driver.find_elements_by_xpath('/html/body/div/div/div[2]/div[1]/div/div[1]/div/div[1]/div[4]/div[2]/a')
if len(keyword_list) < 1:
print("No search, Try again")
tiktok_keyword()
else:
pass
conn = pymysql.connect(
user='admin',
passwd='rladlscjf00!',
host='kic.cj7mov3fe2u4.ap-northeast-2.rds.amazonaws.com',
db='toyou',
charset='utf8'
)
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
sql = """
DROP TABLE keywordTiktok
"""
cursor.execute(sql)
except:
pass
sql = """
CREATE TABLE keywordTiktok(
keywordTiktokNo INT primary key auto_increment,
keywordTiktokName VARCHAR(50),
keywordTiktokView VARCHAR(50),
keywordTiktokLink VARCHAR(300)
)
"""
cursor.execute(sql)
key_list= []
link_list=[]
view_list=[]
for keyword in keyword_list:
key = keyword.find_element_by_xpath('div/span').text
key_list.append(key)
link = keyword.get_attribute('href')
link_list.append(keyword.get_attribute('href'))
for link in link_list:
driver.get(link)
time.sleep(1.0)
try:
view = driver.find_element_by_xpath('/html/body/div/div/div[2]/div[2]/div/header/div[1]/div/h2/strong').text
except:
view = driver.find_element_by_xpath('/html/body/div/div/div[2]/div[2]/div/div[1]/h2[1]').text
view_list.append(view)
for i in range(len(link_list)):
sql = """insert into keywordTiktok(keywordTiktokName,keywordTiktokView,keywordTiktokLink)
values (%s, %s, %s)"""
cursor.execute(sql, (key_list[i], view_list[i], link_list[i]))
conn.commit()
cursor.close()
print("Upload Complete : Tiktok Trend")
```
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue only provides a tiktok_keyword() entry point and inline crawler/database code; start by confirming the intended database integration and expected behavior with the repository maintainers. Define completion criteria for the TikTok data and keywordTiktok table before locating the relevant project file and validating the agreed workflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, python
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100