creativecommons / creativecommons/quantifying

Add WikiCommons Data Source

Aperta
#180 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub
✨ goal: improvement 🏁 status: ready for work 💻 aspect: code help wanted 🟩 priority: low
Lingua principale
Python
Stelle
48
Fork
74
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Problem
Hello, right now, the project collects data from Google Custom Search and GitHub, and work on adding Wikipedia is already in progress via PRs #176 and #167. Also, @TimidRobot commented about “more meaningful data” (for Wikipedia) suggesting they expect more than just basic counts — but WikiCommons they hasn’t been addressed yet.

However, WikiCommons is also an important source for Creative Commons–licensed media, and it’s not yet part of the automated system.
There’s an older version of it under `pre-automation/wikicommons/`, but it hasn’t been updated to the new structure.

## Description
Work can be done on adding WikiCommons as a new data source using the MediaWiki API.
This would collect counts of CC-licensed media files (like images, videos, and audio) by license type.

The plan is to:
- Review the old `pre-automation/wikicommons_scratcher.py` script.
- Rewrite it to match the new 3-step workflow (1-fetch, 2-process, 3-report).
- Make sure the new script uses the current shared helpers and output format.

This will help the project measure CC-licensed media content more accurately.

## Alternatives
It could be combined with the Wikipedia data, but keeping it separate makes it easier to track media content specifically.

## Additional context
- **Old script**: `pre-automation/wikicommons/`
- **API**:
- [MediaWiki Action API](https://commons.wikimedia.org/w/api.php)
- [API:Categorymembers - MediaWiki](https://www.mediawiki.org/wiki/API:Categorymembers/en)
- Builds on similar work done for **Wikipedia** (#159, #167, #176)

## Implementation
- [x] I would be interested in implementing this feature.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by reading pre-automation/wikicommons_scratcher.py and the related pre-automation/wikicommons/ files, then compare the Wikipedia work in issues #159, #167, and #176. Adapt the source to the 1-fetch, 2-process, 3-report workflow using the current shared helpers and output format; done means CC-licensed media counts are collected by license type through the MediaWiki API.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
data
Tipo di issue
Funzionalità
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
50/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.