[Feature] Bash script to concurrently download from a list
- Ngôn ngữ chính
- Go
- Star
- 373
- Fork
- 43
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
## Feature Proposal
Hi, I've written a simple bash script that allows you to download multiple files from a list, using concurrency.
Essentially, the script requires a list of `n` URLs and will generate `n` processes that call `vimeo-dl`.
I just tried downloading ~50 files and it's worked flawlessly. Eventually, it could be integrated directly in the Go code.
## Bash script
```bash
#!/bin/bash
# Read the list of URLs from the file
urls=$(cat $1)
# Count the number of URLs
url_count=$(echo "$urls" | wc -l)
# User agent. You can change it as you wish
user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36"
# Function to download using vimeo-dl. Add flags if needed.
download_video() {
local url=$1
vimeo-dl \
--combine \
-i "$url" \
--user-agent "$user_agent"
}
# Export the function so it can be used by parallel
export -f download_video
# Run the download_video function concurrently for each URL
echo "$urls" | xargs -n 1 -P "$url_count" -I {} bash -c 'download_video "$@"' _ {}
```
## Usage
1. Generate a `list.txt` file containing master.json urls.
2. Execute the script and pass the list via args:
`./script.sh list.txt`
3. Your videos will be exported to your pwd.
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.