Automattic / Automattic/harper

Split out `harper-dict` from `harper-core` so we can build tools using Harper's POS logic without pulling in the entire massive core

Open
#3,818 0 comments 1 reaction 0 assignees View on GitHub
dictionary enhancement harper-core
Dominant language
Rust
Stars
15.4k
Forks
627
Avg merge
1d 15h
Merged PRs (30d)
102

Description

**What problem does this solve?**

I often want to make tools for Harper, but I have an unexpandable MacBook Air with the biggest SSD I could afford at the time and it's full several times a week. `harper-core` is very big and I'm already using share build artefact directories to mitigate this. Build times are also slow.

**Proposed Solution**

Create a new `harper-dict` that has just the functionality for using the `FstDictionary` and `DictWordMetadata`.
It doesn't need the functionality for building the curated dictionary, or the functionality of the User dictionaries.
It also doesn't need tokenization of documents or `LintKind`.

**Examples**

My new [**ctx**](https://github.com/hippietrail/ctx) tool I'm working on to analyse phrases in context to help making new Linters would benefit from being able to look up the POSes of the context words.

**Component**
- [x] Core engine
- [ ] Plugin/Extension
- [ ] Other: _____

**Additional Context**

I had a quick attempt at this without an agent but asking the CoPilot on GitHub's Harper repo page.

This hits more than a few areas of Rust I haven't learned yet. It seems that there are things that are used in `harper-core` that would also be used in `harper-dict` but we of course wouldn't want to make core a dep of dict. So that might mean also separating out a `harper-common` or `harper-tools` and moving things like `CharString` there ... or maybe duping/vendoring? such things into `harper-dict`

It was at this point I decided it's more productive to stop working on it myself and post a feature request about it instead to get some feedback.

Contributor guide

Open the contributing guide

Research direction

Start by tracing FstDictionary and DictWordMetadata through harper-core and reviewing which shared types they require. Define the package boundary and dependency structure, then verify that harper-dict exposes dictionary lookup and metadata without curated-dictionary building, user dictionaries, tokenization, or LintKind.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.