docling-project / docling-project/docling
How to include page numbers in markdown / Conditional OCR when text layer is missing
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
Hi, thank you for the great project! I've been reading the README and documentation carefully, but I couldn't find answers to the following two questions. I would appreciate any clarification you can provide:
### 1. Including Page Numbers in Markdown Output
Is there a way to include page numbers in the output when using `export_to_markdown()` or `save_as_markdown()`?
I tried using `page_break_placeholder=""`, but it seems that the `{page_no}` variable is not recognized and is inserted as a literal string. Is there a built-in way to insert the actual page number into the markdown output?
### 2. Conditional OCR Based on Text Layer
I'm trying to build a pipeline that:
- uses the text layer **when it exists**, and
- falls back to OCR **only when the text layer is missing**.
However, when I set `do_ocr = True`, it doesn't seem to run OCR properly unless I also set `force_full_page_ocr = True`. But with `force_full_page_ocr = True`, even PDFs that already have a text layer get OCRed, which is not ideal.
I confirmed using `fitz` (PyMuPDF) that the document *does* have an existing text layer. Still, `docling` either doesn't pick it up or forces OCR regardless.
Is there a recommended or correct way in Docling to conditionally perform OCR **only if the PDF has no usable text layer**?
This is my code, and I hope I can solve this problem.
```
def conver_pdf_to_assets(file_path, output_folder):
logging.basicConfig(level=logging.INFO)
render_options = PdfPipelineOptions()
render_options.images_scale = IMAGE_ZOOM_FACTOR
render_options.generate_picture_images = True
render_options.generate_page_images = True
render_options.do_ocr = True
render_options.do_table_structure = True
render_options.table_structure_options.do_cell_matching = True
ocr_options = TesseractCliOcrOptions(force_full_page_ocr=True)
render_options.ocr_options = ocr_options
pdf_processor = DocumentConverter(
format_options= {
InputFormat.PDF: PdfFormatOption(
pipeline_cls=StandardPdfPipeline,
backend=PyPdfiumDocumentBackend,
accelerator_options=AcceleratorOptions(device=AcceleratorDevice.CUDA, num_threads=8),
pipeline_options= render_options,
),
})
conversion_result = pdf_processor.convert(file_path)
table_index = 0
figure_index=0
for content_element, _ in conversion_result.document.iterate_items():
if isinstance(content_element, TableItem):
table_index += 1
img = content_element.get_image(conversion_result.document)
if img :
table_img_file = output_folder/f"table-{table_index}.png"
with table_img_file.open('wb') as tbl_out:
content_element.get_image(conversion_result.document).save(tbl_out, format='PNG')
else:
logging.warning(f"Table {table_index} does not have an image associated with it.")
if isinstance(content_element, PictureItem):
figure_index += 1
figure_img_file = output_folder/f"figure-{figure_index}.png"
with figure_img_file.open('wb') as fig_out:
content_element.get_image(conversion_result.document).save(fig_out, format='PNG')
embedded_md_path = output_folder/f"AJ-with-images.md"
conversion_result.document.save_as_markdown(
filename=embedded_md_path,
image_mode=ImageRefMode.REFERENCED,
page_break_placeholder=f""
)
```
Thank you in advance for your help — and again, thank you for building and sharing this powerful tool!
Contributor guide
Assessment
This issue has not been assessed yet.