docling-project / docling-project/docling

GB/T standard PDF: force_full_page_ocr + tesseract chi_sim still yields garbled CID text on cover page (docling-serve v1.22.0)

Open
#3,582 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

## Summary

When processing a Chinese national standard PDF (**GB/T 37136-2018**) through **docling-serve v1.22.0** (`POST /v1/chunk/hybrid/file`), the embedded text layer produces garbled Han characters and PDF glyph references on the cover page. Enabling full-page OCR with **Tesseract CLI + `chi_sim` + `eng`** does **not** recover readable cover text. Output still starts from page 2 (table of contents); page 1 title/metadata remain missing or garbled.

This appears related to PDFs with **CID-keyed fonts and missing/incorrect ToUnicode mappings** (similar class of issues as #2963, #3081), but we are filing a **standalone report** because `force_full_page_ocr` via the serve API does not improve page 1 in our setup.

## Environment

- **Image**: `quay.io/docling-project/docling-serve:v1.22.0`
- **API endpoint**: `POST /v1/chunk/hybrid/file`
- **OCR engine**: `tesseract_cli` (`convert_ocr_preset=tesseract`)
- **Languages**: `chi_sim`, `eng` (verified in container: `tesseract --list-langs` → `chi_sim`, `eng`, `osd`)
- **Custom image layer**: copied `chi_sim.traineddata` + Noto CJK fonts into the official image (Tesseract Chinese pack confirmed installed)

## Sample document

- **File**: GB/T 37136-2018 (Chinese national standard PDF, publicly available standard document)
- Can provide the PDF privately to maintainers if needed

## Steps to reproduce

1. Run docling-serve v1.22.0
2. POST the PDF to `/v1/chunk/hybrid/file` with OCR enabled (parameters below)
3. Inspect `documents[].content.md_content` / `text_content` or hybrid chunks for page 1

### Default parse (no OCR)

Garbled cover examples from extracted text:

- `犌犅/犜 37136-2018`
- English title garbled as spaced Han + glyph refs: `... /G27 /G28 /G2B /G2C /G2D /G2E /`
- First usable content starts at page 2 (TOC: `目次`)

### With force OCR (same result for page 1)

We tried all of the following; **chunk count and cover content did not improve**:

| Attempt | `convert_page_range` |
|---------|----------------------|
| 1 | `[1, 8]` |
| 2 | `[1, 1]` |
| 3 | `[1, 3]` |
| 4 | omitted (full document) |

## Request parameters used (OCR recovery profile)

**Multipart form fields:**
convert_do_ocr=true convert_force_ocr=true convert_do_table_structure=false convert_images_scale=3 convert_ocr_preset=tesseract convert_ocr_lang=chi_sim convert_ocr_lang=eng convert_page_range=[1,1] convert_ocr_custom_config={"kind":"tesseract_cli","lang":["chi_sim","eng"],"force_full_page_ocr":true}

**`options` JSON (sent in parallel):**

```json
{
"do_ocr": true,
"force_ocr": true,
"do_table_structure": false,
"images_scale": 3,
"ocr_preset": "tesseract",
"ocr_lang": ["chi_sim", "eng"],
"page_range": [1, 1],
"ocr_custom_config": {
"kind": "tesseract_cli",
"lang": ["chi_sim", "eng"],
"force_full_page_ocr": true
}
}

Image

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.