jina-ai / jina-ai/reader

Reader returning duplicated content/content not set to visible

Open
#1,195 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
12k
Forks
882
PR merge metrics
No merged PRs in 30d

Description

Using the reader endpoint results in duplicate information as it is returning text from elements set not to be visible.

```
curl "https://r.jina.ai/https://github.com/trust-center/privacy" \
-H "Authorization: Bearer MYTOKEN"
```

Image

The html for the duplicate content has `Display: none`.

Is there a way for us to only return content that would be visible on the page?

Image

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reader endpoint request for the linked GitHub URL and inspect how its HTML is converted to reader content. Check how elements with `display: none` are handled; done means the response no longer includes hidden content or duplicates the visible text. No source file or test is identified in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.