akiomik / akiomik/vimeo-dl

[Feature] Bash script to concurrently download from a list

Abierto
#59 0 comentarios 2 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Go
Estrellas
373
Forks
43
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Feature Proposal
Hi, I've written a simple bash script that allows you to download multiple files from a list, using concurrency.

Essentially, the script requires a list of `n` URLs and will generate `n` processes that call `vimeo-dl`.
I just tried downloading ~50 files and it's worked flawlessly. Eventually, it could be integrated directly in the Go code.

## Bash script

```bash
#!/bin/bash

# Read the list of URLs from the file
urls=$(cat $1)

# Count the number of URLs
url_count=$(echo "$urls" | wc -l)

# User agent. You can change it as you wish
user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36"

# Function to download using vimeo-dl. Add flags if needed.
download_video() {
local url=$1
vimeo-dl \
--combine \
-i "$url" \
--user-agent "$user_agent"
}

# Export the function so it can be used by parallel
export -f download_video

# Run the download_video function concurrently for each URL
echo "$urls" | xargs -n 1 -P "$url_count" -I {} bash -c 'download_video "$@"' _ {}
```

## Usage

1. Generate a `list.txt` file containing master.json urls.
2. Execute the script and pass the list via args:
`./script.sh list.txt`
3. Your videos will be exported to your pwd.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.