docling-project / docling-project/docling
GB/T standard PDF: force_full_page_ocr + tesseract chi_sim still yields garbled CID text on cover page (docling-serve v1.22.0)
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
## Summary
When processing a Chinese national standard PDF (**GB/T 37136-2018**) through **docling-serve v1.22.0** (`POST /v1/chunk/hybrid/file`), the embedded text layer produces garbled Han characters and PDF glyph references on the cover page. Enabling full-page OCR with **Tesseract CLI + `chi_sim` + `eng`** does **not** recover readable cover text. Output still starts from page 2 (table of contents); page 1 title/metadata remain missing or garbled.
This appears related to PDFs with **CID-keyed fonts and missing/incorrect ToUnicode mappings** (similar class of issues as #2963, #3081), but we are filing a **standalone report** because `force_full_page_ocr` via the serve API does not improve page 1 in our setup.
## Environment
- **Image**: `quay.io/docling-project/docling-serve:v1.22.0`
- **API endpoint**: `POST /v1/chunk/hybrid/file`
- **OCR engine**: `tesseract_cli` (`convert_ocr_preset=tesseract`)
- **Languages**: `chi_sim`, `eng` (verified in container: `tesseract --list-langs` → `chi_sim`, `eng`, `osd`)
- **Custom image layer**: copied `chi_sim.traineddata` + Noto CJK fonts into the official image (Tesseract Chinese pack confirmed installed)
## Sample document
- **File**: GB/T 37136-2018 (Chinese national standard PDF, publicly available standard document)
- Can provide the PDF privately to maintainers if needed
## Steps to reproduce
1. Run docling-serve v1.22.0
2. POST the PDF to `/v1/chunk/hybrid/file` with OCR enabled (parameters below)
3. Inspect `documents[].content.md_content` / `text_content` or hybrid chunks for page 1
### Default parse (no OCR)
Garbled cover examples from extracted text:
- `犌犅/犜 37136-2018`
- English title garbled as spaced Han + glyph refs: `... /G27 /G28 /G2B /G2C /G2D /G2E /`
- First usable content starts at page 2 (TOC: `目次`)
### With force OCR (same result for page 1)
We tried all of the following; **chunk count and cover content did not improve**:
| Attempt | `convert_page_range` |
|---------|----------------------|
| 1 | `[1, 8]` |
| 2 | `[1, 1]` |
| 3 | `[1, 3]` |
| 4 | omitted (full document) |
## Request parameters used (OCR recovery profile)
**Multipart form fields:**
convert_do_ocr=true convert_force_ocr=true convert_do_table_structure=false convert_images_scale=3 convert_ocr_preset=tesseract convert_ocr_lang=chi_sim convert_ocr_lang=eng convert_page_range=[1,1] convert_ocr_custom_config={"kind":"tesseract_cli","lang":["chi_sim","eng"],"force_full_page_ocr":true}
**`options` JSON (sent in parallel):**
```json
{
"do_ocr": true,
"force_ocr": true,
"do_table_structure": false,
"images_scale": 3,
"ocr_preset": "tesseract",
"ocr_lang": ["chi_sim", "eng"],
"page_range": [1, 1],
"ocr_custom_config": {
"kind": "tesseract_cli",
"lang": ["chi_sim", "eng"],
"force_full_page_ocr": true
}
}
Contributor guide
Assessment
This issue has not been assessed yet.