docling-project / docling-project/docling

Question: Optimal env settings on ppc64le

Open
#3,334 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
3d 4h
Merged PRs (30d)
98

Description

### Question

Problem: We are experiencing difficulty finding optimal threading configuration for Docling document processing in our RAG (Retrieval-Augmented Generation) pipeline on ppc64le. Despite extensive testing with various combinations of threading environment variables and pipeline options, we cannot achieve optimal performance for both `digitize_docs` (document conversion) and downstream processing (`process_text` and `process_tables`).

Platform: IBM Power
CPU: 16 physical cores with SMT2 (Simultaneous Multithreading level 2) = 32 vCPU
Deployment: Podman & OpenShift

Application context (`python3.12`):
-We use 2 separate sizes of ProcessPool:
- Larger pool size of 4 workers for light docs (<500 pages)
- Smaller pool size of 2 workers for heavy docs (>500 pages)

-Each worker in the `ProcessPoolExecutor` creates its own `DocumentConverter` instance
-A separate ThreadPoolExecutor of size 4 to process tables and text

Pipeline Options have following config
```
pipeline_options.do_table_structure = True
pipeline_options.table_structure_options.do_cell_matching = True
pipeline_options.do_ocr = False
```

We're unclear about the interaction between:
- `OMP_NUM_THREADS` (OpenMP)
- `MKL_NUM_THREADS` (Intel MKL)
- `OPENBLAS_NUM_THREADS` (OpenBLAS)
- `DOCLING_NUM_THREADS` (Docling-specific)
- `layout_batch_size` & `table_batch_size` (ThreadedPdfPipelineOptions)

Our observations:
- With keeping the ProcessPool and ThreadPool size same
1. OMP_NUM_THREADS = 8 -> Boosted our document conversion time, but degraded our process_tables speed and ability to detect tables
2. layout_batch_size = 12 and table_batch_size = 12 -> No improvement in process_tables or process_text speeds
3. MKL_NUM_THREADS = 8, OPENBLAS_NUM_THREADS = 2, OMP_NUM_THREADS = 8 -> Degraded all (document conversion, process_tables and process_text speeds)
4. Increasing ProcessPool size also caused a degradation in process_tables with no impact on document conversion.

Questions:
- Should these be set to physical cores (16), logical cores (32), or something else?
- How do these interact with Python's `ProcessPoolExecutor` and `ThreadPoolExecutor`?
- Is there a recommended ratio between these values and our worker pool sizes?
- Are there known performance characteristics on ppc64le vs. x86_64?
- Which env var has correlation with PID count?
- Does Docling have specific optimizations or recommendations for SMT architectures?

We could not find a clear correlation between threading parameter changes and performance improvements.
Please do ask away in case more details are needed. Thanks in advance!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported benchmarks on ppc64le with the stated ProcessPoolExecutor and ThreadPoolExecutor sizes, pipeline options, and threading environment variables. Compare document conversion, table processing, and text processing separately; done means documenting the observed relationships and a supported configuration or clear guidance on which additional measurements are needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.