Notion page extraction drops mention and equation rich-text segments
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 610
Description
## Description
`NotionExtractor` preserves all Notion rich-text segments in database properties and table cells, but the normal page/block extraction paths still keep only segments that contain a `text` object.
Notion rich text can also contain `mention` and `equation` objects. Those segments expose their rendered value through `plain_text`, so they are currently skipped when a Notion page is imported into a knowledge base.
This affects both:
- `_get_notion_block_data()` for top-level page blocks;
- `_read_block()` for recursively-read child blocks.
## Reproduction
On current `main`, a paragraph payload containing mixed rich text such as:
```python
[
{"type": "text", "plain_text": "See ", "text": {"content": "See "}},
{"type": "mention", "plain_text": "Project Alpha", "mention": {"type": "page", "page": {"id": "page-id"}}},
{"type": "equation", "plain_text": "E = mc^2", "equation": {"expression": "E = mc^2"}},
]
```
is processed by the current loop as only:
```text
See
```
The mention and equation text are omitted because the implementation checks `if "text" in rich_text` and then reads only `rich_text["text"]["content"]`.
## Expected behavior
All rendered rich-text segments should be preserved in extracted page content, using each segment's `plain_text` value (with the existing `text.content` shape retained only as a fallback if needed).
For the payload above, the extracted text should contain:
```text
See Project AlphaE = mc^2
```
## Upstream precedent
This is consistent with the recently merged table-cell fix in #41993. During review, `plain_text` was explicitly preferred because Notion rich text can contain `text`, `mention`, and `equation` segments, and the merged `_get_cell_text()` helper now follows that rule.
The recently merged database-property fix in #42050 also joins all `plain_text` segments instead of taking only a subset.
The two ordinary page/block paths appear to be the remaining sibling omission in the same extractor.
## Proposed scope
Keep the fix narrow:
1. preserve `plain_text` for every rich-text segment in `_get_notion_block_data()`;
2. do the same in `_read_block()`;
3. add focused regression coverage for top-level and nested blocks containing text + mention + equation segments;
4. leave database-property and table-cell behavior unchanged.
I have the scoped change ready and will link the PR here.
Contributor guide
Research direction
Start with the rich-text handling in _get_notion_block_data() and _read_block(), comparing it with the recently merged _get_cell_text() behavior from #41993. Add focused regression coverage for top-level and nested blocks containing text, mention, and equation segments; done means every segment's plain_text is preserved while the existing text.content fallback remains supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100