Regex-based Markdown parsing fails on valid Markdown structures
- Dominant language
- Python
- Stars
- 451
- Forks
- 707
- Avg merge
- 22h 59m
- Merged PRs (30d)
- 91
Description
### Describe the bug
The current implementation in `RepositoryBasedEntityModel` uses complex, regex-based parsing to extract leaders, email addresses, and URLs from raw Markdown content. This approach fails to correctly handle valid Markdown constructs such as nested lists, inline formatting (bold/italic), and alternate link formats, leading to incomplete or incorrect data extraction.
---
### To Reproduce
Steps to reproduce the behavior:
1. Add or update a repository README with valid Markdown that includes:
- Nested lists for leaders
- Inline formatting (e.g. `**bold**`, `_italic_`)
- Links or emails using different Markdown-supported formats
2. Trigger the parsing logic in `RepositoryBasedEntityModel`.
3. Inspect the extracted leaders, emails, or URLs.
4. Observe that some values are missing or incorrectly parsed.
---
### Expected behavior
All valid Markdown content should be parsed reliably, and leaders, emails, and URLs should be correctly extracted regardless of formatting variations or nesting.
---
### Are you going to work on fixing this?
- [x] Yes
- [ ] No
---
### Screenshots
Not applicable.
---
### Desktop
- OS: Not environment-specific
- Browser: Not applicable
- Version: Not applicable
---
### Smartphone
- Device: Not applicable
- OS: Not applicable
- Browser: Not applicable
- Version: Not applicable
---
### Additional context
This can be fixed by replacing regex-based Markdown parsing with a structured approach:
- Convert Markdown to HTML using the existing `markdown` dependency.
- Traverse the HTML using `lxml` or Python’s built-in `html.parser`.
- Extract leaders from `
Contributor guide
Assessment
This issue has not been assessed yet.