OWASP / OWASP/Nest

Regex-based Markdown parsing fails on valid Markdown structures

Open
#3,579 1 comment 0 reactions 0 assignees View on GitHub
bug enhancement
Dominant language
Python
Stars
451
Forks
707
Avg merge
22h 59m
Merged PRs (30d)
91

Description

### Describe the bug
The current implementation in `RepositoryBasedEntityModel` uses complex, regex-based parsing to extract leaders, email addresses, and URLs from raw Markdown content. This approach fails to correctly handle valid Markdown constructs such as nested lists, inline formatting (bold/italic), and alternate link formats, leading to incomplete or incorrect data extraction.

---

### To Reproduce
Steps to reproduce the behavior:

1. Add or update a repository README with valid Markdown that includes:
- Nested lists for leaders
- Inline formatting (e.g. `**bold**`, `_italic_`)
- Links or emails using different Markdown-supported formats
2. Trigger the parsing logic in `RepositoryBasedEntityModel`.
3. Inspect the extracted leaders, emails, or URLs.
4. Observe that some values are missing or incorrectly parsed.

---

### Expected behavior
All valid Markdown content should be parsed reliably, and leaders, emails, and URLs should be correctly extracted regardless of formatting variations or nesting.

---

### Are you going to work on fixing this?
- [x] Yes
- [ ] No

---

### Screenshots
Not applicable.

---

### Desktop
- OS: Not environment-specific
- Browser: Not applicable
- Version: Not applicable

---

### Smartphone
- Device: Not applicable
- OS: Not applicable
- Browser: Not applicable
- Version: Not applicable

---

### Additional context
This can be fixed by replacing regex-based Markdown parsing with a structured approach:
- Convert Markdown to HTML using the existing `markdown` dependency.
- Traverse the HTML using `lxml` or Python’s built-in `html.parser`.
- Extract leaders from `

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.