akiomik / akiomik/vimeo-dl

[Feature] Bash script to concurrently download from a list

オープン
#59 コメント 0 件 リアクション 2 件 担当者 0 名 GitHub で見る
主要言語
Go
スター
373
フォーク
43
PR マージ指標
30日以内にマージされた PR はありません

説明

## Feature Proposal
Hi, I've written a simple bash script that allows you to download multiple files from a list, using concurrency.

Essentially, the script requires a list of `n` URLs and will generate `n` processes that call `vimeo-dl`.
I just tried downloading ~50 files and it's worked flawlessly. Eventually, it could be integrated directly in the Go code.

## Bash script

```bash
#!/bin/bash

# Read the list of URLs from the file
urls=$(cat $1)

# Count the number of URLs
url_count=$(echo "$urls" | wc -l)

# User agent. You can change it as you wish
user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36"

# Function to download using vimeo-dl. Add flags if needed.
download_video() {
local url=$1
vimeo-dl \
--combine \
-i "$url" \
--user-agent "$user_agent"
}

# Export the function so it can be used by parallel
export -f download_video

# Run the download_video function concurrently for each URL
echo "$urls" | xargs -n 1 -P "$url_count" -I {} bash -c 'download_video "$@"' _ {}
```

## Usage

1. Generate a `list.txt` file containing master.json urls.
2. Execute the script and pass the list via args:
`./script.sh list.txt`
3. Your videos will be exported to your pwd.

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。