binary-husky / binary-husky/gpt_academic

[Feature]: 从arxiv上下载文件,配置国内访问加速,并增加进度条显示

Open
#1,917 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
71.4k
Forks
8.3k
PR merge metrics
No merged PRs in 30d

Description

### Class | 类型

程序主体

### Feature Request | 功能请求

直接使用原始arxiv_download代码,经常出现从网站下载文件失败的状况,建议配置国内访问加速。比较简易的更改方式是在requests.get请求参数中增加headers,配置方法为:
```python
url = file_url
headers = {
'User-Agent': 'Lynx'
}

response = requests.get(url, headers=headers, stream=True)
```

另外,增加进度条可直观显示下载进度,可通过以下代码实现:

```python
total_size = int(response.headers.get('content-length', 0)) # 获取文件总大小
block_size = 1024 # 每次下载的块大小

# 使用 tqdm 创建进度条
with open(filename, 'wb') as file, tqdm(
desc=filename,
total=total_size,
unit='iB',
unit_scale=True,
unit_divisor=1024,
) as bar:
for data in response.iter_content(block_size):
file.write(data) # 写入文件
bar.update(len(data)) # 更新进度条
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the arxiv_download implementation and locate the requests.get call that retrieves the file. Check how dependencies are declared before adding the suggested request headers and tqdm-based progress reporting. Done means downloads show progress when a content length is available and still save the complete file successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.