python-pillow / python-pillow/Pillow

Warn that pre-applying arabic_reshaper + python-bidi corrupts text on a Raqm build

Open Beginner friendly
#9,907 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Documentation Font Rendering Third Party Framework
Dominant language
Python
Stars
13.8k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
89

Description

Summary

The docs say libraqm "provides bidirectional text support (using FriBiDi), shaping (using HarfBuzz)". That is accurate, but nothing says what happens if the caller also does those two steps — which is what most Arabic-with-Pillow advice online recommends:

text = get_display(arabic_reshaper.reshape(text))   # the widely-copied recipe
draw.text((x, y), text, font=font)

On a Raqm build this is applied twice and the text comes out wrong. I would like to propose a short docs note, because the failure is silent to anyone who does not read Arabic: it renders cleanly, it just says something else.

Environment

Reproduced on Pillow 12.3.0 (and 12.2.0), Python 3.14, macOS.
raqm 0.10.5, harfbuzz 14.2.1, fribidi 1.0.16.

Reproducer
from PIL import Image, ImageDraw, ImageFont, features
import arabic_reshaper
from bidi.algorithm import get_display

print(features.check("raqm"), features.version("raqm"))   # True 0.10.5

FONT = "/System/Library/Fonts/SFArabic.ttf"   # any Arabic-capable font
text   = "الإمارات"                            # "the Emirates"
recipe = get_display(arabic_reshaper.reshape(text))

for name, s in (("plain", text), ("recipe", recipe)):
    img = Image.new("L", (700, 110), 255)
    ImageDraw.Draw(img).text((20, 15), s, font=ImageFont.truetype(FONT, 64), fill=0)
    img.save(f"out_{name}.png")

out_plain.png is correct. out_recipe.png is the same word rendered reversed — Raqm applies the bidirectional algorithm to a string that was already reordered into visual order.

Two things that are checkable without reading Arabic

1. The string length changes. The recipe merges the lam-alef pair into a single presentation codepoint:

input : 8 codepoints  0x627 0x644 0x625 0x645 0x627 0x631 0x627 0x62a
recipe: 7 codepoints  0xfe95 0xfe8d 0xfead 0xfe8e 0xfee3 0xfef9 0xfe8d

All 7 are in Arabic Presentation Forms-B (U+FE70–U+FEFF). Anything downstream doing an exact-match search, a length check, or an index into that string is now working on a different string than the user typed.

2. Diacritics are deleted. arabic_reshaper.reshape() drops tashkeel silently:

input    : 'مَرْحَبًا'   4 diacritics
reshaped : 'ﻣﺮﺣﺒﺎ'      0 diacritics

Pillow renders all four correctly on its own; after the recipe they are gone. For vocalised text — teaching material, Qur'anic text, TTS input — that is data loss rather than a display quirk.

Why this is worth a doc line

The recipe is not simply wrong. On a build without Raqm it is still necessary, because nothing else shapes or reorders the text. So the correct advice is conditional, and the condition is a runtime property of the environment doing the rendering — not of the Pillow version:

if not features.check("raqm"):
    text = get_display(arabic_reshaper.reshape(text))

Nothing in the docs connects those two facts, so people copy an unconditional recipe and it is a coin-flip whether it helps or corrupts. #3081 is still receiving comments eight years on, and the recipe is recommended in that thread alongside instructions for enabling Raqm — after which it becomes harmful.

Proposed change

A short note in the text-rendering docs, near the direction / features / language parameters that already carry "Requires libraqm":

Complex scripts. When libraqm is available, Pillow performs shaping and applies the bidirectional algorithm itself. Do not pre-process text with arabic_reshaper and python-bidi in that case — the reordering is applied twice and the result is incorrect. Check PIL.features.check("raqm") at runtime, in the environment doing the rendering, to decide whether that pre-processing is needed.

Happy to open the PR if that wording is welcome, or to adjust it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the text-rendering documentation near the direction, features, and language parameters that mention libraqm, and review the existing guidance around complex scripts. Add a conditional warning based on PIL.features.check("raqm") explaining when pre-processing is unnecessary, then verify the note accurately distinguishes Raqm and non-Raqm builds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.