Automattic / Automattic/harper
Harper incorrectly tags code/commands as issues — detect code blocks and avoid false positives (2026)
- Dominant language
- Rust
- Stars
- 15.4k
- Forks
- 627
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 106
Description
**Describe the bug**
Harper is tagging code snippets, CLI commands, logs, and other code-like text as “issues” when they are simply examples or output. This creates many false-positive issue flags in posts and notes that contain examples, tutorials, or code samples.
**Where**
Web: https://writewithharper.com/
Repo: https://github.com/Automattic/harper
Observed behavior in Harper editor/analysis (2026)
**To reproduce**
Open the Harper editor on writewithharper.com or create a new note/post in Harper.
Paste any of the examples below into the editor content.
Save or let Harper scan/analyze the content.
Observe that pieces of the code/commands/logs are flagged as “issues” (false positives) rather than being recognized as code examples.
# Example A: fenced shell code block (should be recognized as code)
```bash
$ npm install my-package
node index.js
```
```text name=example-inline-and-prompt.md
# Example B: inline code + prompt lines (should NOT be flagged as issues)
This is how you run the tool: `mytool --serve` or from the shell run `$ mytool start`.
The REPL shows:
> Ready on http://localhost:3000
```
**Expected behavior**
Harper should recognize and exclude:
Markdown fenced code blocks (```), indented blocks, and inline code spans (inline).
Shell/CLI prompt lines starting with $, >, or common prompts (PS C:>).
Plain log/output lines and stack traces.
Common file fragments (package.json, Dockerfile, shell scripts) and not tag them as issues by default.
Only flag text as an “issue” when the content is clearly describing a problem (e.g., “I get an error: …”, “it fails with X”) or when the user explicitly marks it.
**Suggested heuristics / mitigations**
Skip issue detection inside:
Markdown fenced code blocks (```), YAML/JSON codefences, and inline backtick spans.
Indented code blocks (4+ spaces).
Treat lines that look like prompts/logs as non-issues:
Leading tokens: $, >, PS C:>, [INFO], [WARN], timestamps (YYYY- or ISO-like), stack trace patterns (at Module.func).
Add an editor toggle/setting: “Ignore code blocks & logs when auto-detecting issues”.
Provide a simple inline UI affordance: “Mark this selection as example/code” to opt-out from issue detection for that block.
Optionally, a conservative NLP filter: run issue-detection only on paragraph text (not code blocks) and require presence of human-language problem indicators (“error”, “fail”, “unexpected”, “bug”, “exception”) before tagging.
**Why this matters**
Authors frequently include commands, example outputs, and small code fragments in tutorials and documentation. False-positive issue tagging disrupts their workflow and makes Harper less useful for technical writing.
**What got flagged?**
From https://github.com/ublue-os/bazzite (README.md):
"Verification
These images are signed with sigstore's [cosign](https://docs.sigstore.dev/cosign/key_management/overview/). You can verify the signature by downloading the cosign.pub key from this repo and running the following command:
cosign verify --key cosign.pub ghcr.io/ublue-os/bazzite"
**Why is this incorrect?**
Got one Capitalization problem: This sentence does not start with a capital letter.
pub
**Example of correct usage:**
No problem in this case.
Contributor guide
Research direction
Start by reproducing the false positives in the Harper editor/analysis described in the issue, then locate the issue-detection entry point in the repository. Trace how fenced and inline Markdown, prompts, logs, and stack traces are analyzed, and add coverage for the listed examples. Done means code-like content is excluded while clearly described problems can still be flagged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, shell
- Domain
- devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100