BOHICA-LABS / BOHICA-LABS/writescore
Strip YAML frontmatter before analysis
- Dominant language
- Python
- Stars
- 2
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
When analyzing markdown files with YAML frontmatter (common in static site generators like Jekyll, Hugo, etc.), the tool includes the frontmatter in its semantic coherence analysis. This causes false positives:
- Frontmatter (lines between `---` delimiters) is detected as "disconnected paragraphs"
- The Material Purpose or similar metadata sections are flagged as having "weak transitions" to the article content
- This artificially lowers semantic coherence scores
## Example
A file structured like:
```markdown
---
title: "Article Title"
status: draft
---
## Material Purpose (for Blue Quill Submission)
This article addresses...
---
# Article Title
Your organization has more...
```
Gets flagged with:
- `Para 1->2 (too disconnected, sim=0.24)` - frontmatter to heading
- `Para 2->3 (too disconnected, sim=0.08)` - heading to Material Purpose
- `Para 4->5 (sim=0.01)` - Material Purpose to second `---` delimiter
## Proposed Solution
Before analysis, strip:
1. YAML frontmatter (content between opening `---` and closing `---` at the start of the file)
2. Optionally: common metadata sections like "Material Purpose" that are clearly not article content
This would give more accurate scores for the actual article content that will be published.
## Impact
Currently seeing ~3.8 pts potential improvement blocked by frontmatter issues in semantic coherence scoring.
Contributor guide
Assessment
This issue has not been assessed yet.