google / google/mozc

DictionaryTool consumes unbounded memory (~100 GB) with a ~42k-entry user dictionary on macOS Apple Silicon (Google Japanese Input v3.33)

Open
#1,545 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
3k
Forks
525
PR merge metrics
No merged PRs in 30d

Description

### Environment

- **Product:** Google Japanese Input **v3.33.6130** (proprietary build; the `DictionaryTool` here is the same code shared with Mozc)
- **OS:** macOS 26.5.2 (build 25F84)
- **Hardware:** MacBook Air (`Mac17,3`), Apple **M5**, **16 GB** RAM (Apple Silicon native build)
- **User dictionary:** [DMiME 1.1](https://osdn.net/projects/dmime/) medical-term dictionary — **42,467 entries**, plain UTF-8, tab-separated (`readingwordpos`), imported as a single user dictionary. (Almost all entries are `名詞`/NOUN.)

### Summary

When the user dictionary contains ~42,000 entries, **`DictionaryTool` allocates memory without bound during "テーブルの更新" (table update/rebuild)** — climbing to roughly **100 GB** on this 16 GB machine, so the only way out is a force quit.

This happens in two situations:
1. **During import** of the full 42k-entry file.
2. **Every time `DictionaryTool` is opened afterwards**, because it rebuilds the table view of the already-stored user dictionary. So once the data is present, the tool becomes unusable and re-balloons on each launch. Uninstalling/reinstalling the app does **not** help, because `user_dictionary.db` survives.

**Splitting does not help.** Importing as 9 separate ~5,000-entry dictionaries still balloons — memory had already reached ~**5.4 GB at 20,000 entries** and kept growing. The cost scales with the **total** number of entries across all dictionaries, not per-dictionary.

### Important: only the GUI is affected

The conversion engine (`GoogleJapaneseInputConverter`) handles the full 42,467-entry dictionary **without any problem** (tens of MB RSS, normal). The dictionary data is written **correctly and completely** to `~/Library/Application Support/Google/JapaneseInput/user_dictionary.db` (verified by parsing the protobuf: all 42,467 entries present with correct POS). **Only the `DictionaryTool` GUI balloons.**

### Steps to reproduce

1. Fresh install of Google Japanese Input v3.33 on an Apple Silicon Mac.
2. Get `DMiME-1.1.txt` (42,467 entries) or any user dictionary of comparable size.
3. `DictionaryTool` → 管理 → 新規辞書にインポート → select the file.
4. Watch `DictionaryTool` RSS in Activity Monitor climb without bound (~100 GB) → force quit required.
5. Re-open `DictionaryTool`: the same balloon occurs on "テーブルの更新" before you can do anything.

### Expected

Importing / displaying a ~42k-entry user dictionary should complete with **bounded** memory. This size is far below the documented user-dictionary limits, and the same DMiME dictionary has reportedly been imported without issue on Windows and earlier (Intel) Mac builds for years — which suggests this may be a **regression specific to the Apple Silicon–native macOS build (v3.33)**.

### Workaround (for other affected users)

Bypass `DictionaryTool` entirely. The data is already saved in `user_dictionary.db`, so placing/restoring a valid `user_dictionary.db` and restarting the Converter (`killall GoogleJapaneseInputConverter`) gives full medical-term conversion — as long as you never open `DictionaryTool`.

### Notes

- This is distinct from #692 (that was a *crash on open* due to a missing `Frameworks` dir); here the tool launches and then consumes unbounded memory.
- I'm aware this is the proprietary Google Japanese Input build and that such reports are sometimes redirected to the product forum. Filing here because `DictionaryTool` is Mozc-shared code and this may well reproduce in OSS Mozc with a large user dictionary. Happy to move it if preferred.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting Mozc's DictionaryTool implementation for the table-update/rebuild path and reproduce it with the 42,467-entry DMiME dictionary. Compare memory behavior during import and reopening against the converter path. Done means the GUI handles a dictionary of this size with bounded memory while retaining all entries.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.