hackforla / hackforla/data-science
Epic: MediaWiki Data Science Projects
- Dominant language
- Jupyter Notebook
- Stars
- 33
- Forks
- 22
- PR merge metrics
- No merged PRs in 30d
Description
### Overview
I had GPT distill some interesting data science questions that can be answered using the MediaWiki API, particularly in culturally or economically sensitive areas. That response is in the collapsed section below. I have turned each one into sub-issues for assignment to DS CoP members.
-chinaexpert1
### Action Items
- [x] Review MediaWiki API site [here](https://www.mediawiki.org/wiki/API:Main_page)
- [x] Make MediaWiki Project Template
- [x] Make all the sub-issues
- [ ] Assign to new and existing members after onboarding
### Resources/Instructions
15 MediaWiki questions proposed by GPT:
Great target API. The MediaWiki Action API (plus Wikibase on Wikidata) exposes edits, logs, links, categories, language links, coordinates, and more—perfect for sensitive, policy-relevant analyses. Here are concrete, non-toy questions you can answer with specific modules and an analysis sketch:
1. Representation gaps in biographies
• Question: How does the share of biographies by gender/region/occupation evolve over time?
• How: Seed pages via `list=categorymembers`, get revision timestamps via `prop=revisions`, enrich with Wikidata entity attributes via `action=wbgetentities` (gender, country, occupation). Compare time trends and category coverage. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ACategorymembers?utm_source=chatgpt.com)][1])
2. Language equity on sensitive topics
• Question: Do key public-health, migration, or conflict pages exist and get updated across low-resource languages as quickly as in English?
• How: For a topic set, pull interlanguage links with `prop=langlinks`, then per language fetch latest `prop=revisions` (timestamps/size). Compute “time-to-translation” and update lag. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ALanglinks?utm_source=chatgpt.com)][2])
3. Controversy & protection dynamics
• Question: Which policy-sensitive pages (e.g., policing, elections) see page protection spikes?
• How: Use `prop=info&inprop=protection` to snapshot protection; use `list=logevents` (letype=protect/unprotect) for change history; correlate with edit/revert bursts from `prop=revisions`. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3AInfo?utm_source=chatgpt.com)][3])
4. Deletion patterns and notability bias
• Question: Are articles about marginalized communities more likely to be nominated or deleted?
• How: Mine `list=logevents` (letype=delete, move, restore) and—where permitted—`prop=deletedrevisions` to mark outcomes; stratify by topic/region from categories/Wikidata. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ALogevents?utm_source=chatgpt.com)][4])
5. External link economy (payday lenders, crypto exchanges, gig platforms)
• Question: Which sensitive industries get linked most, and where?
• How: Use `list=exturlusage` for target domains; map to pages and track link addition/removal via page revisions. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3AExturlusage?utm_source=chatgpt.com)][5])
6. Edit-war & revert intensity on culturally charged pages
• Question: Which pages have the highest revert ratios and by whom (anon vs registered)?
• How: Parse `prop=revisions` for users and SHA1; compute revert chains; optionally add ORES “damaging/goodfaith” scores for edit quality. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ARevisions?utm_source=chatgpt.com)][6])
7. Gendered or respectful language drift over time
• Question: Did terms like “illegal immigrant” → “undocumented immigrant” decline across articles?
• How: Pull a page’s revision text via `prop=revisions` (rvslots=main\&rvprop=content) and run n-gram time-series; compare pre/post guidance changes. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ARevisions?utm_source=chatgpt.com)][6])
8. Geospatial knowledge coverage vs. need
• Question: Are locations in low-income regions under-documented?
• How: Use GeoData (`prop=coordinates` / `list=geosearch`) to map article density per area; normalize by population/economic indicators. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3AGeosearch/Sample_code_1?utm_source=chatgpt.com)][7])
9. Topic-network bias and agenda setting
• Question: Do sensitive pages link mostly to a narrow set of sources or topics?
• How: Build link graphs with `list=backlinks` and `prop=links`; compute centrality and modularity; flag echo chambers. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ABacklinks?utm_source=chatgpt.com)][8])
10. Watcher/attention asymmetry
• Question: Do sensitive topics have unusually high/low watcher counts or strict protection?
• How: `prop=info` (watchers, when allowed) + `inprop=protection`; compare with edit volumes and revert rates. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3AInfo?utm_source=chatgpt.com)][3])
11. Newcomer burden and gatekeeping
• Question: Are newcomer edits on sensitive pages more likely to be reverted?
• How: Join `list=usercontribs` for newcomer cohorts with page-level `prop=revisions`; add ORES for damaging/goodfaith probabilities to quantify triage behavior. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3AUsercontribs?utm_source=chatgpt.com)][9])
12. Category-level under/over-coverage
• Question: Which sensitive categories (e.g., Indigenous issues) are thin relative to peers?
• How: `list=categorymembers` counts over time; compare article length, quality assessments (PageAssessments extension) and interlanguage presence. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ACategorymembers?utm_source=chatgpt.com)][1])
13. Corporate pages around layoffs/scandals
• Question: Do protection levels and revert rates spike around major corporate events?
• How: For a company list, pull `prop=revisions` time-series and `list=logevents` (protect/unprotect); test for structural breaks. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ARevisions?utm_source=chatgpt.com)][6])
14. Cross-language topic presence for crisis terms
• Question: Are emergency-response articles present/updated during disasters in smaller Wikipedias?
• How: Use `prop=langlinks` to enumerate languages per key page; monitor `prop=revisions` timestamps per language. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ALanglinks?utm_source=chatgpt.com)][2])
15. Source-type auditing on sensitive pages
• Question: Are certain outlet types (state media, tabloids) over-represented?
• How: Extract references from wikitext in `prop=revisions` and join with `list=exturlusage` to classify URLs. ([[MediaWiki](https://www.mediawiki.org/wiki/API%3ARevisions?utm_source=chatgpt.com)][6])
Notes & guardrails
• Many of these benefit from adding Wikidata attributes via the Wikibase API (`action=wbgetentities`). ([[MediaWiki](https://www.mediawiki.org/wiki/Wikibase/API?utm_source=chatgpt.com)][10])
• Use protection and deletion data responsibly; some items are redacted/suppressed. Start with aggregated reporting and avoid profiling individual editors (follow Wikimedia norms). ([[MediaWiki](https://www.mediawiki.org/wiki/Manual%3AProtection?utm_source=chatgpt.com)][11])
GPT can turn any of the above into a scoped GitHub Epic with tasks (data pull, ETL, metrics, dashboards) and *ready-to-run notebooks* - this might be particularly useful.
[1]: https://www.mediawiki.org/wiki/API%3ACategorymembers?utm_source=chatgpt.com "API:Categorymembers"
[2]: https://www.mediawiki.org/wiki/API%3ALanglinks?utm_source=chatgpt.com "API:Langlinks"
[3]: https://www.mediawiki.org/wiki/API%3AInfo?utm_source=chatgpt.com "API:Info"
[4]: https://www.mediawiki.org/wiki/API%3ALogevents?utm_source=chatgpt.com "API:Logevents"
[5]: https://www.mediawiki.org/wiki/API%3AExturlusage?utm_source=chatgpt.com "API:Exturlusage"
[6]: https://www.mediawiki.org/wiki/API%3ARevisions?utm_source=chatgpt.com "API:Revisions"
[7]: https://www.mediawiki.org/wiki/API%3AGeosearch/Sample_code_1?utm_source=chatgpt.com "API:Geosearch/Sample code 1"
[8]: https://www.mediawiki.org/wiki/API%3ABacklinks?utm_source=chatgpt.com "API:Backlinks"
[9]: https://www.mediawiki.org/wiki/API%3AUsercontribs?utm_source=chatgpt.com "API:Usercontribs"
[10]: https://www.mediawiki.org/wiki/Wikibase/API?utm_source=chatgpt.com "Wikibase/API"
[11]: https://www.mediawiki.org/wiki/Manual%3AProtection?utm_source=chatgpt.com "Manual:Protection"
Additional Topics Proposed (if needed):
1. **POV/Dispute Template Lifecycles**
Track when pages gain/lose templates like `{{POV}}`, `{{Disputed}}`, `{{Advert}}`, `{{COI}}`; measure dwell time and reoccurrence by topic.
Modules: `list=embeddedin`, `prop=revisions` (content), `prop=info`.
2. **Redirect & Naming Neutrality**
Map redirect graphs for controversial terms (e.g., deprecated → neutral phrasing) and quantify which names “win” over time.
Modules: `prop=redirects`, `list=backlinks` (redirects only), `prop=revisions`.
3. **Image/Media License Equity (Commons)**
For sensitive pages, analyze how many images exist, where they come from, and license mix (CC BY/SA, ND/NC, PD).
Modules: page `prop=images` (enwiki) → Commons `imageinfo&prop=extmetadata`.
4. **Citation Freshness & Archive Coverage**
Measure % dead or archived links, age of citations, and adoption of `|archive-url|archive-date` in references.
Modules: `prop=revisions` (content), optional `list=exturlusage` for present-day links.
5. **Protection Scope vs Topic Size**
Normalize protection prevalence by topic size: what share of pages in each sensitive topic are currently semi/full protected?
Modules: `list=categorymembers`, `prop=info&inprop=protection`.
6. **Infobox Completeness Gaps**
Within topic-specific infoboxes (e.g., health orgs, municipalities), quantify missing key fields across pages and regions.
Modules: `prop=revisions` (content), template parsing.
Contributor guide
Assessment
This issue has not been assessed yet.