freeCodeCamp / freeCodeCamp/devdocs

[low priority refactoring] generalize executing JS during scraping

Aperta
#789 1 commento 1 reazione 0 assegnatari Vedi su GitHub
Lingua principale
Ruby
Stelle
39.5k
Fork
2.6k
Merge medio
4g 32m
PR unite (30g)
13

Descrizione

If doc sites use client-side JavaScript to produce or fetch some of their content, we need to launch a headless browser and execute it. Currently scraper subclasses do that individually using capybara, e.g.
https://github.com/freeCodeCamp/devdocs/blob/master/lib/docs/scrapers/angularjs.rb#L42-L45
but it might be better to extract a helper method or config option, a little more like this:
https://github.com/freeCodeCamp/devdocs/blob/master/lib/docs/scrapers/webpack.rb#L43-L46

cc @dangeranger

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by comparing lib/docs/scrapers/angularjs.rb#L42-L45 with lib/docs/scrapers/webpack.rb#L43-L46 to understand the existing Capybara-based JavaScript execution patterns. Trace how scraper configuration and helpers are organized, then define a shared approach that preserves the relevant scraper behavior and verify affected scrapers still produce their expected content.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
ruby
Ambito
documentation, tooling
Tipo di issue
Refactoring
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.