Automattic / Automattic/harper

Feature request: flag slang / informal words (and phrases?)

Open
#3,018 0 comments 0 reactions 0 assignees View on GitHub
enhancement harper-core linting
Dominant language
Rust
Stars
15.4k
Forks
627
Avg merge
1d 15h
Merged PRs (30d)
106

Description

A change I made to the dictionary has just been merged which allows any word to flagged as slang/informal. We're not distinguishing between the two for now at least.

There are two annotation flags: `/†` for words that only have slang or informal senses/uses and `/‡` for words that have slang/informal senses or uses as well as regular uses.

This adds an internal flag to words. Only a few of the most obvious words have so far been marked this way, such as `ain't`, `bro`, `dude`. The internal flag is not yet propagated up through the system where linters can see it.

To propagate up a new flag, the plumbing needs to be added to `DictWordMetadata`, `TokenKind`, and `SequenceExpr`. It's usually pretty easy to add the plumbing by following the rust errors in your IDE when you start trying to use a new flag in one place of the code. I think it might most closely resemble the swear word flag and the dialect flags.

The thinking behind having a "only informal" and "has informal senses" was to avoid needing to have a "not slang or informal" flag that would need to be added to every word in the dictionary. This was how I approached the mass noun vs countable noun logic and how I updated the single vs plural logic. Nouns don't have to be marked for singular or countable since those are the default. But because of this there needs to be code added between the `DictWordMetadata` and `TokenKind` levels to handle the default. This is the more fiddly part to get right. Use lots of unit tests.

After the plumbing infrastructure is done it will be possible to add a linter on top. I would again look to the `AvoidCurses` and `Regionalisms` linters. Ideally it should:

- Suggest one or more standard words for each word flagged as slang/informal
- Handle singular and plural for nouns (possibly possessive too?) and inflections of verbs
- Ideally, should take some context into account to only offer suggestions that make sense for the sentence. This is something the swear flagger doesn't do that well yet.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing swear-word and dialect flag plumbing through DictWordMetadata, TokenKind, and SequenceExpr, then review the AvoidCurses and Regionalisms linters. Use unit tests to verify the default handling for words without slang flags and propagation of both annotation flags. Done includes working plumbing that can support a slang/informal linter with appropriate suggestions and inflection handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.