Automattic / Automattic/harper

Harper incorrectly tags code/commands as issues — detect code blocks and avoid false positives (2026)

Open
#4,012 0 comments 0 reactions 0 assignees View on GitHub
bug false-positive harper-core linting
Dominant language
Rust
Stars
15.4k
Forks
627
Avg merge
1d 15h
Merged PRs (30d)
106

Description

**Describe the bug**
Harper is tagging code snippets, CLI commands, logs, and other code-like text as “issues” when they are simply examples or output. This creates many false-positive issue flags in posts and notes that contain examples, tutorials, or code samples.

**Where**
Web: https://writewithharper.com/
Repo: https://github.com/Automattic/harper
Observed behavior in Harper editor/analysis (2026)

**To reproduce**
Open the Harper editor on writewithharper.com or create a new note/post in Harper.
Paste any of the examples below into the editor content.
Save or let Harper scan/analyze the content.

Observe that pieces of the code/commands/logs are flagged as “issues” (false positives) rather than being recognized as code examples.

# Example A: fenced shell code block (should be recognized as code)
```bash
$ npm install my-package
node index.js
```

```text name=example-inline-and-prompt.md
# Example B: inline code + prompt lines (should NOT be flagged as issues)
This is how you run the tool: `mytool --serve` or from the shell run `$ mytool start`.
The REPL shows:
> Ready on http://localhost:3000
```

**Expected behavior**
Harper should recognize and exclude:
Markdown fenced code blocks (```), indented blocks, and inline code spans (inline).
Shell/CLI prompt lines starting with $, >, or common prompts (PS C:>).
Plain log/output lines and stack traces.

Common file fragments (package.json, Dockerfile, shell scripts) and not tag them as issues by default.
Only flag text as an “issue” when the content is clearly describing a problem (e.g., “I get an error: …”, “it fails with X”) or when the user explicitly marks it.

**Suggested heuristics / mitigations**
Skip issue detection inside:
Markdown fenced code blocks (```), YAML/JSON codefences, and inline backtick spans.
Indented code blocks (4+ spaces).

Treat lines that look like prompts/logs as non-issues:
Leading tokens: $, >, PS C:>, [INFO], [WARN], timestamps (YYYY- or ISO-like), stack trace patterns (at Module.func).
Add an editor toggle/setting: “Ignore code blocks & logs when auto-detecting issues”.

Provide a simple inline UI affordance: “Mark this selection as example/code” to opt-out from issue detection for that block.
Optionally, a conservative NLP filter: run issue-detection only on paragraph text (not code blocks) and require presence of human-language problem indicators (“error”, “fail”, “unexpected”, “bug”, “exception”) before tagging.

**Why this matters**
Authors frequently include commands, example outputs, and small code fragments in tutorials and documentation. False-positive issue tagging disrupts their workflow and makes Harper less useful for technical writing.

**What got flagged?**
From https://github.com/ublue-os/bazzite (README.md):

"Verification
These images are signed with sigstore's [cosign](https://docs.sigstore.dev/cosign/key_management/overview/). You can verify the signature by downloading the cosign.pub key from this repo and running the following command:
cosign verify --key cosign.pub ghcr.io/ublue-os/bazzite"

**Why is this incorrect?**
Got one Capitalization problem: This sentence does not start with a capital letter.
pub
**Example of correct usage:**
No problem in this case.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the false positives in the Harper editor/analysis described in the issue, then locate the issue-detection entry point in the repository. Trace how fenced and inline Markdown, prompts, logs, and stack traces are analyzed, and add coverage for the listed examples. Done means code-like content is excluded while clearly described problems can still be flagged.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, shell
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.