AI4Finance-Foundation / AI4Finance-Foundation/FinNLP

How to use download news

Aperta
#31 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Jupyter Notebook
Stelle
1.5k
Fork
276
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

May be, proxing url broken?

start_date = "2024-09-01"
end_date = "2024-09-31"
config = {
"use_proxy": "us_free", # use proxies to prvent ip blocking
"max_retry": 5,
"proxy_pages": 5,
"token": "XXXXXX" # Available at https://finnhub.io/dashboard <-- inputed finnhub token api
}

news_downloader = Finnhub_Date_Range(config) # init
news_downloader.download_date_range_stock(start_date,end_date) # Download headers
news_downloader.gather_content() # Download contents
df = news_downloader.dataframe
selected_columns = ["headline", "content"]
df[selected_columns].head(10)

------
---> 10 news_downloader = Finnhub_Date_Range(config) # init
11 news_downloader.download_date_range_stock(start_date,end_date) # Download headers
12 news_downloader.gather_content() # Download contents

File ~/gitlab/FinNLP/finnlp/data_sources/news/finnhub_date_range.py:16, in Finnhub_Date_Range.__init__(self, args)
15 def __init__(self, args = {}):
---> 16 super().__init__(args)
17 assert "token" in args.keys(), "Please input your finnhub token. Avaliable at https://finnhub.io/dashboard"
18 self.finnhub_client = finnhub.Client(api_key=args["token"])

File ~/gitlab/FinNLP/finnlp/data_sources/news/_base.py:6, in News_Downloader.__init__(self, args)
5 def __init__(self, args = {}):
----> 6 super().__init__(args)
7 pass

File ~/gitlab/FinNLP/finnlp/data_sources/_base.py:22, in FinNLP_Downloader.__init__(self, args)
20 else:
21 self.proxy_id = 0
---> 22 self.proxy_list = self._update_proxy()
23 else:
24 self.proxy_list = []

File ~/gitlab/FinNLP/finnlp/data_sources/_base.py:44, in FinNLP_Downloader._update_proxy(self)
42 return get_china_free_proxy(self.proxy_pages)
43 else:
---> 44 return get_us_free_proxy(self.proxy_pages)

File ~/gitlab/FinNLP/finnlp/utils/get_proxy.py:86, in get_us_free_proxy(pages)
84 content = scripts[3].xpath(".//text()")
85 pattern = re.compile('LIST",data:(.+),added:')
---> 86 result_list = pattern.findall(content[0])
87 result_list = result_list[0].strip("[{").strip("}]").split("},{")
89 for result in result_list:

IndexError: list index out of range

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.