algolia / algolia/docsearch-scraper
The prompt record is 0 after running the container
- Dominant language
- Python
- Stars
- 322
- Forks
- 115
- PR merge metrics
- No merged PRs in 30d
Description
config.json
```
{
"index_name": "pptfz",
"start_urls": [
"https://blog.pptfz.cn/"
],
"sitemap_urls": [
"https://blog.pptfz.cn/sitemap.xml"
],
"sitemap_alternate_links": true,
"stop_urls": [],
"selectors": {
"lvl0": {
"selector": "(//ul[contains(@class,'menu__list')]//a[contains(@class, 'menu__link menu__link--sublist menu__link--active')]/text() | //nav[contains(@class, 'navbar')]//a[contains(@class, 'navbar__link--active')]/text())[last()]",
"type": "xpath",
"global": true,
"default_value": "Documentation"
},
"lvl1": "header h1",
"lvl2": "article h2",
"lvl3": "article h3",
"lvl4": "article h4",
"lvl5": "article h5, article td:first-child",
"lvl6": "article h6",
"text": "article p, article li, article td:last-child"
},
"strip_chars": " .,;:#",
"custom_settings": {
"separatorsToIndex": "_",
"attributesForFaceting": [
"language",
"version",
"type",
"docusaurus_tag"
],
"attributesToRetrieve": [
"hierarchy",
"content",
"anchor",
"url",
"url_without_anchor",
"type"
]
},
"conversation_id": [
"833762294"
],
"nb_hits": 9510
}
```
After running, the following message is displayed
```
docker run -it --env-file=/docusaurus/algolia/.env -e "CONFIG=$(cat /docusaurus/algolia/config.json | jq -r tostring)" algolia/docsearch-scraper
> DocSearch: https://blog.pptfz.cn/ 0 records)
Crawling issue: nbHits 0 for pptfz
```
How should this problem be solved
Contributor guide
Research direction
Start by reproducing the documented docker run command with config.json and inspect the scraper output for why the crawl reports zero records. Done means the container successfully crawls the configured site and no longer reports nbHits 0 for the index.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- backend, search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100