docling-project / docling-project/docling-jobkit
implement file/ZIP export for chunking pipeline (support to_formats + ZipTarget)
- Dominant language
- Python
- Stars
- 36
- Forks
- 47
- Avg merge
- 7d 3h
- Merged PRs (30d)
- 17
Description
## Summary
The chunking pipeline correctly returns `ChunkedDocumentResult` for `InBodyTarget`,
but fails when exporting chunks/documents to a file/zip target. Sending a chunking
request with a file target (`target_type=ZipTarget`/non-InBody) and `to_formats`
results in `"Saving chunks to a file is not yet supported."` The chunk endpoints in `docling-serve`
also exclude `to_formats` for chunk requests.
## How to reproduce
1. POST to `/v1/chunk/hybrid/file/async` with `target_type` set to ZIP and `convert_to_formats=json`
2. Observe worker logs: chunking succeeds but the final worker fails with:
`RuntimeError("Saving chunks to a file is not yet supported.")`
3. Result is 404 for `/v1/result/`
## Root cause
`process_chunk_results()` consumes the lazy `conv_results` iterator during chunking,
Th
`process_chunk_results()` explicitly return when traget_type is not inbody
`NotImplementedError("Saving chunks to a file is not yet supported.")`
`process_export_results()` (in `results.py`) already handles this correctly by
materializing `conv_results` into a list first — chunking should mirror that pattern.
Additionally, `app.py` excludes `to_formats` from chunk endpoints via:
`FormDepends(..., excluded_fields=["to_formats"])`
## Expected behavior
- Chunk file endpoints accept `to_formats` (same semantics as `/v1/convert/file/async`)
- Chunking pipeline supports file/zip export — writes converted documents into `output/`
and produces a ZIP (same pattern as `process_export_results()`)
- If `to_formats` includes JSON/MD/HTML/TXT, those exports are included in the zip
- Chunk data (e.g., `chunks.json`) is also written inside the zip
## Files to review
- `docling_jobkit/convert/chunking.py`
- `docling_jobkit/convert/results.py` — reference implementation
- `app.py:918-980` — chunk endpoint excluding `to_formats`
## Additional context
- Needs integration test: POST to `/v1/chunk/.../file/async` with
`convert_to_formats=json` + `target_type=ZIP`, validate zip contains
both converted docs and chunk files
- Needs docs example showing chunk-to-zip usage
Contributor guide
Research direction
Start with process_chunk_results() in docling_jobkit/convert/chunking.py, compare its file-target path with process_export_results() in docling_jobkit/convert/results.py, and inspect app.py lines 918-980 for the excluded to_formats field. Add an integration test for a ZIP chunk request with JSON output, verify the ZIP contains converted documents and chunk data, and update the documentation with a chunk-to-ZIP example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100