Automattic / Automattic/harper
Split out `harper-dict` from `harper-core` so we can build tools using Harper's POS logic without pulling in the entire massive core
- Dominant language
- Rust
- Stars
- 15.4k
- Forks
- 627
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 102
Description
**What problem does this solve?**
I often want to make tools for Harper, but I have an unexpandable MacBook Air with the biggest SSD I could afford at the time and it's full several times a week. `harper-core` is very big and I'm already using share build artefact directories to mitigate this. Build times are also slow.
**Proposed Solution**
Create a new `harper-dict` that has just the functionality for using the `FstDictionary` and `DictWordMetadata`.
It doesn't need the functionality for building the curated dictionary, or the functionality of the User dictionaries.
It also doesn't need tokenization of documents or `LintKind`.
**Examples**
My new [**ctx**](https://github.com/hippietrail/ctx) tool I'm working on to analyse phrases in context to help making new Linters would benefit from being able to look up the POSes of the context words.
**Component**
- [x] Core engine
- [ ] Plugin/Extension
- [ ] Other: _____
**Additional Context**
I had a quick attempt at this without an agent but asking the CoPilot on GitHub's Harper repo page.
This hits more than a few areas of Rust I haven't learned yet. It seems that there are things that are used in `harper-core` that would also be used in `harper-dict` but we of course wouldn't want to make core a dep of dict. So that might mean also separating out a `harper-common` or `harper-tools` and moving things like `CharString` there ... or maybe duping/vendoring? such things into `harper-dict`
It was at this point I decided it's more productive to stop working on it myself and post a feature request about it instead to get some feedback.
Contributor guide
Research direction
Start by tracing FstDictionary and DictWordMetadata through harper-core and reviewing which shared types they require. Define the package boundary and dependency structure, then verify that harper-dict exposes dictionary lookup and metadata without curated-dictionary building, user dictionaries, tokenization, or LintKind.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100