docling-project / docling-project/docling
Question: Optimal env settings on ppc64le
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 98
Description
### Question
Problem: We are experiencing difficulty finding optimal threading configuration for Docling document processing in our RAG (Retrieval-Augmented Generation) pipeline on ppc64le. Despite extensive testing with various combinations of threading environment variables and pipeline options, we cannot achieve optimal performance for both `digitize_docs` (document conversion) and downstream processing (`process_text` and `process_tables`).
Platform: IBM Power
CPU: 16 physical cores with SMT2 (Simultaneous Multithreading level 2) = 32 vCPU
Deployment: Podman & OpenShift
Application context (`python3.12`):
-We use 2 separate sizes of ProcessPool:
- Larger pool size of 4 workers for light docs (<500 pages)
- Smaller pool size of 2 workers for heavy docs (>500 pages)
-Each worker in the `ProcessPoolExecutor` creates its own `DocumentConverter` instance
-A separate ThreadPoolExecutor of size 4 to process tables and text
Pipeline Options have following config
```
pipeline_options.do_table_structure = True
pipeline_options.table_structure_options.do_cell_matching = True
pipeline_options.do_ocr = False
```
We're unclear about the interaction between:
- `OMP_NUM_THREADS` (OpenMP)
- `MKL_NUM_THREADS` (Intel MKL)
- `OPENBLAS_NUM_THREADS` (OpenBLAS)
- `DOCLING_NUM_THREADS` (Docling-specific)
- `layout_batch_size` & `table_batch_size` (ThreadedPdfPipelineOptions)
Our observations:
- With keeping the ProcessPool and ThreadPool size same
1. OMP_NUM_THREADS = 8 -> Boosted our document conversion time, but degraded our process_tables speed and ability to detect tables
2. layout_batch_size = 12 and table_batch_size = 12 -> No improvement in process_tables or process_text speeds
3. MKL_NUM_THREADS = 8, OPENBLAS_NUM_THREADS = 2, OMP_NUM_THREADS = 8 -> Degraded all (document conversion, process_tables and process_text speeds)
4. Increasing ProcessPool size also caused a degradation in process_tables with no impact on document conversion.
Questions:
- Should these be set to physical cores (16), logical cores (32), or something else?
- How do these interact with Python's `ProcessPoolExecutor` and `ThreadPoolExecutor`?
- Is there a recommended ratio between these values and our worker pool sizes?
- Are there known performance characteristics on ppc64le vs. x86_64?
- Which env var has correlation with PID count?
- Does Docling have specific optimizations or recommendations for SMT architectures?
We could not find a clear correlation between threading parameter changes and performance improvements.
Please do ask away in case more details are needed. Thanks in advance!
Contributor guide
Research direction
Start by reproducing the reported benchmarks on ppc64le with the stated ProcessPoolExecutor and ThreadPoolExecutor sizes, pipeline options, and threading environment variables. Compare document conversion, table processing, and text processing separately; done means documenting the observed relationships and a supported configuration or clear guidance on which additional measurements are needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100