bisohns / bisohns/search-engine-parser

google cannot parse "Tallest mountain in the world"

Open
#149 7 comments 0 reactions 0 assignees View on GitHub
bug help wanted
Dominant language
Python
Stars
491
Forks
87
PR merge metrics
No merged PRs in 30d

Description

**Description**
The google engine cannot parse the return results of "Tallest mountain in the world"

**To Reproduce**
Steps to reproduce the behavior:
```python
from search_engine_parser.core.engines.google import Search
searcher = Search()
results = searcher.search("Tallest mountain in the world")
```

**Expected behavior**
Correctly parsed results

**Screenshots**

```
Traceback (most recent call last):
File "XXXXX/.conda/envs/info/lib/python3.9/site-packages/search_engine_parser/core/base.py", line 240, in get_results
search_results = self.parse_result(results, **kwargs)
File "XXXXX/.conda/envs/info/lib/python3.9/site-packages/search_engine_parser/core/base.py", line 151, in parse_result
rdict = self.parse_single_result(each, **kwargs)
File "XXXXX/.conda/envs/info/lib/python3.9/site-packages/search_engine_parser/core/engines/google.py", line 74, in parse_single_result
title = r_elem.find('div', class_='BNeawe').text
AttributeError: 'NoneType' object has no attribute 'text'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "XXXXX/temp.py", line 4, in
results = searcher.search("Tallest mountain in the world")
File "XXXXX/.conda/envs/info/lib/python3.9/site-packages/search_engine_parser/core/base.py", line 270, in search
return self.get_results(soup, **kwargs)
File "XXXXX/.conda/envs/info/lib/python3.9/site-packages/search_engine_parser/core/base.py", line 243, in get_results
raise NoResultsOrTrafficError(
search_engine_parser.core.exceptions.NoResultsOrTrafficError: The returned results could not be parsed. This might be due to site updates or server errors. Drop an issue at https://github.com/bisoncorps/search-engine-parser if this persists
```

**Desktop (please complete the following information):**
- OS: [Linux]
- Python Version [3.9.5]
- Search-engine-parser version [0.6.2] (latest)

**Additional context**
The result that cannot be parsed:
```

Rank
Mountain
Country
1.
Everest
Nepal/Tibet
2.
K2 (Mount Godwin Austen)
Pakistan/China
3.
Kangchenjunga
India/Nepal
4.
Lhotse
Nepal/Tibet

```
The corresponding result of https://github.com/bisoncorps/search-engine-parser/blob/0418867b3529980d5a4eb71899dec37092fe7df1/search_engine_parser/core/engines/google.py#L66
```
[
,
]
```
The first `div` does not contain the title.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.