diegomura / diegomura/react-pdf
Arabic loses base letterforms: slice() start boundary uses last-of-group glyph index
- Dominant language
- TypeScript
- Stars
- 16.8k
- Forks
- 1.3k
- Avg merge
- 5h 6m
- Merged PRs (30d)
- 52
Description
## Describe the bug
Word-initial forms of most of the Arabic alphabet lose their **base letterform**, leaving only the dot mark. The output is not blank and not tofu — it is fluent, well-formed Arabic that is the *wrong word*, so nothing signals the corruption:
| Input | Rendered |
| --- | --- |
| `هلیا نجاتی` (a person's name) | `هلیا جاتی` |
| `نون` | `ون` |
| `بابا` | `انا` |
| `مرحبا` (no word-initial dotted letter) | `مرحبا` ✅ |
Affects every letter whose glyph decomposes into a dotless base plus a zero-advance dot mark — ب ت ث ن ي ف ق ز ذ ض ظ غ خ ج — i.e. most of the alphabet.
Confirmed on the **rasterised page** (`pdftoppm`), not only via text extraction: the dot is visibly painted and the letterform is absent.
This surfaced on customer-facing KYC/KYB compliance documents, where an entity's or officeholder's legal name is rendered incorrectly with no visual indication.
## Root cause
Arabic letters are shaped as a dotless base plus a separate zero-advance mark. In Noto Sans Arabic, `ب` becomes `uni066E.init` (base, `codePoints: [0x628]`) + `dotbelowar` (mark, `codePoints: []`).
`glyph-indices/resolve.ts` assigns a zero-codepoint glyph the **previous** glyph's starting character index:
```ts
const length = glyph?.codePoints?.length || 0;
const value = length === 0 ? result[i - 1] : currentIndex;
```
So a base+mark pair appears in `glyphIndices` as two consecutive **equal** values — one character, two glyphs.
`run/glyphIndexAt.ts` deliberately returns the **last** glyph of such a group (its own docstring says so: *"When multiple consecutive glyphs point to the same string index (e.g. decomposed characters), returns the last glyph in that group"*).
That is correct for a slice's **end** boundary, where the whole group must be included. It is wrong for the **start** boundary — `run/slice.ts` uses it for both:
```ts
const startIndex = glyphIndexAt(start, run); // ← last-of-group; drops leading glyphs
const endIndex = glyphIndexAt(end - 1, run); // ← last-of-group; correct here
...
const glyphs = (run.glyphs || []).slice(sliceStart, sliceEnd);
```
Worked example for `با` (`fontFamily: 'NotoSansArabic'`):
```
fontkit layout (textkit forces 'ltr'):
0: uni066E.init codePoints [0x628] ← the ب letterform
1: dotbelowar codePoints [] ← its dot
2: uni0627.fina codePoints [0x627]
glyphIndices = [0, 0, 1]
glyphIndexAt(0) → 1 (last of the {0,1} group — the dot)
sliceStart = 1
glyphs.slice(1, 3) → [dotbelowar, uni0627.fina] ← ب letterform discarded
```
Instrumenting `renderRun` in `@react-pdf/render` confirms it receives **2** glyphs where fontkit shaped **3**, with `stringIndices: [0]` instead of `[0, 2]`.
## Proposed fix
For the start boundary only, walk back to the first glyph sharing that character index. In `packages/textkit/src/run/slice.ts`:
```ts
/**
* Return the FIRST glyph index of the group mapping to the given string index.
*
* `glyphIndexAt` returns the last, which is right for an end boundary but at a
* start boundary discards the leading glyphs of a decomposed character — for
* Arabic, the base letterform.
*/
const glyphGroupStartAt = (index: number, run: Run) => {
const glyphIndices = run?.glyphIndices;
const last = glyphIndexAt(index, run);
if (!glyphIndices || last <= 0 || last >= glyphIndices.length) return last;
let i = last;
while (i > 0 && glyphIndices[i - 1] === glyphIndices[i]) i -= 1;
return i;
};
// ...
const startIndex = glyphGroupStartAt(start, run);
const endIndex = glyphIndexAt(end - 1, run);
```
**Ligatures are unaffected.** A ligature is a single glyph carrying several codepoints, so its group has one member and this returns exactly what `glyphIndexAt` did. Behaviour changes only where a character decomposes into multiple glyphs: Arabic, Hebrew with niqqud, Devanagari, combining accents.
Verified against this fix applied to `@react-pdf/textkit@7.0.1`:
- all the cases above render correctly, including on a `['', 'NotoSansArabic']` fallback stack;
- seven scripts (Japanese, Simplified/Traditional Chinese, Korean, Arabic, Thai, Hebrew, Latin) round-trip through `pdftotext` in a real multi-page document;
- a 201-test suite covering Latin and CJK reports shows no regressions.
## Reproduction
Self-contained — no external binaries, exits non-zero while the bug is present:
```js
// repro.mjs — node repro.mjs
import { Document, Font, Page, Text, renderToBuffer } from '@react-pdf/renderer'
import zlib from 'node:zlib'
import React from 'react'
// Noto Sans Arabic Regular, e.g. from
// notofonts/notofonts.github.io fonts/NotoSansArabic/unhinted/ttf/
const FONT = './NotoSansArabic-Regular.ttf'
Font.register({ family: 'Arabic', src: FONT })
const countEmittedGlyphs = (buffer) => {
const latin = buffer.toString('latin1')
let total = 0
const re = /stream\r?\n/g
let m
while ((m = re.exec(latin)) !== null) {
const start = m.index + m[0].length
const end = latin.indexOf('endstream', start)
if (end === -1) continue
let content
try {
content = zlib.inflateSync(buffer.subarray(start, end)).toString('latin1')
} catch { continue }
for (const op of content.matchAll(/(\[[^\]]*\]|<[0-9a-fA-F]*>)\s*T[Jj]/g))
for (const hex of op[1].matchAll(/<([0-9a-fA-F]*)>/g))
total += hex[1].length / 4
}
return total
}
const descriptor = { fontFamily: 'Arabic', fontWeight: 400 }
await Font.load(descriptor)
const font = Font.getFont(descriptor).data
let failed = 0
for (const text of ['با', 'نون', 'بابا', 'نجاتی', 'مرحبا']) {
// textkit forces 'ltr' so fontkit does not reverse the run itself
const shaped = font.layout(text, undefined, undefined, undefined, 'ltr').glyphs.length
const buffer = await renderToBuffer(
React.createElement(Document, null,
React.createElement(Page, { size: 'A4', style: { fontFamily: 'Arabic', fontSize: 16 } },
React.createElement(Text, null, text))))
const emitted = countEmittedGlyphs(buffer)
if (emitted !== shaped) failed += 1
console.log(`${text} shaped=${shaped} emitted=${emitted} ${emitted === shaped ? 'OK' : 'DROPPED'}`)
}
process.exitCode = failed ? 1 : 0
```
Output on 7.0.1:
```
با shaped=3 emitted=2 DROPPED
نون shaped=5 emitted=4 DROPPED
بابا shaped=6 emitted=5 DROPPED
نجاتی shaped=8 emitted=7 DROPPED
مرحبا shaped=6 emitted=6 OK
```
## Relationship to existing issues
- **#3404 / #3405** (glyph with empty `codePoints`) — different bug. I backported #3405 and measured it: it is a **no-op** here. Our mark glyph sits between two correctly-mapped glyphs, so `assignPendingCodePoints` has no codepoints to distribute.
- **#3406 / #3407** (dedupe by `glyph.id` in `reorderLine`) — targets the 6.x implementation. 7.0.1 rewrote bidi reordering around `reverseRun`/`reorderRuns` and no longer dedupes, so that path is gone.
Also ruled out: the font build (`unhinted`, `hinted`, `full` and Noto **Naskh** Arabic all decompose these letters identically and all dropped the base), and `direction: 'rtl'` (no effect).
Note that `@react-pdf/renderer` 4.5.1 → 4.9.0 *did* fix a related hard crash — `TypeError: Cannot read properties of undefined (reading 'id')` in `reorderLine` on mixed-font RTL lines — but not this glyph drop.
## Separate issue, for the record
A dotless skeleton glyph is shared between letters (`uni066E.init` serves ب ت ث ن ي), and `encodeGlyphs` in `@react-pdf/render` writes `font.unicode[gid]` only once:
```ts
if (font.unicode[gid] == null) { font.unicode[gid] = glyph.codePoints; }
```
Since a PDF subset carries one `ToUnicode` entry per glyph id, a document containing both letters can extract one as the other (`بابا` → `نانا`) even when rendering is correct. Properly resolving that needs `/ActualText` marked content. Happy to open a separate issue if useful.
Related but not the cause: `fontkit`'s `Font#getGlyph(glyph, characters)` caches by glyph id and ignores `characters` on a cache hit, so a shared skeleton keeps the codePoints of whichever character first produced it (`f.layout('با'); f.layout('نون')` — the second reports `[0x628]` instead of `[0x646]`). Real, but patching it changes nothing here because of the `font.unicode[gid]` write-once above.
## Environment
- `@react-pdf/renderer` 4.9.0
- `@react-pdf/textkit` 7.0.1
- `@react-pdf/layout` 5.2.0
- `@react-pdf/render` 4.7.0
- `fontkit` 2.0.4
- Node 24.13.1, macOS (also reproduced in a node:24-alpine container)
Contributor guide
Research direction
Start in packages/textkit/src/run/slice.ts and read glyphIndexAt.ts to understand how repeated glyph indices map character boundaries. Run the self-contained repro.mjs with NotoSansArabic and compare shaped versus emitted glyph counts. Done means Arabic base letterforms are retained at slice starts while the listed scripts and existing regression coverage remain intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- react, typescript
- Domain
- internationalization, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100