gosom / gosom/google-maps-scraper
add place id to results table in distributed scraping
Open
- Dominant language
- Go
- Stars
- 5.9k
- Forks
- 924
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 3
Description
hi,
i think the `results` table in the seed job db should include the google maps `place_id` to avoid deduplication.
it could be taken even a step further, combining the `place_id` with a `timestamp` in a primary key.
this way a place can be rescraped after some arbitrary amount of time, ensuring updated data and minimizing
scraping twice the same place.
let me know what u think about it.
thanks
mattia papaccioli
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.