algolia / algolia/docsearch-scraper

The prompt record is 0 after running the container

Open
#577 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
322
Forks
115
PR merge metrics
No merged PRs in 30d

Description

config.json
```
{
"index_name": "pptfz",
"start_urls": [
"https://blog.pptfz.cn/"
],
"sitemap_urls": [
"https://blog.pptfz.cn/sitemap.xml"
],
"sitemap_alternate_links": true,
"stop_urls": [],
"selectors": {
"lvl0": {
"selector": "(//ul[contains(@class,'menu__list')]//a[contains(@class, 'menu__link menu__link--sublist menu__link--active')]/text() | //nav[contains(@class, 'navbar')]//a[contains(@class, 'navbar__link--active')]/text())[last()]",
"type": "xpath",
"global": true,
"default_value": "Documentation"
},
"lvl1": "header h1",
"lvl2": "article h2",
"lvl3": "article h3",
"lvl4": "article h4",
"lvl5": "article h5, article td:first-child",
"lvl6": "article h6",
"text": "article p, article li, article td:last-child"
},
"strip_chars": " .,;:#",
"custom_settings": {
"separatorsToIndex": "_",
"attributesForFaceting": [
"language",
"version",
"type",
"docusaurus_tag"
],
"attributesToRetrieve": [
"hierarchy",
"content",
"anchor",
"url",
"url_without_anchor",
"type"
]
},
"conversation_id": [
"833762294"
],
"nb_hits": 9510
}
```

After running, the following message is displayed
```
docker run -it --env-file=/docusaurus/algolia/.env -e "CONFIG=$(cat /docusaurus/algolia/config.json | jq -r tostring)" algolia/docsearch-scraper
> DocSearch: https://blog.pptfz.cn/ 0 records)

Crawling issue: nbHits 0 for pptfz
```

How should this problem be solved

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the documented docker run command with config.json and inspect the scraper output for why the crawl reports zero records. Done means the container successfully crawls the configured site and no longer reports nbHits 0 for the index.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
backend, search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.