docling-project / docling-project/docling-mcp

local_document_cache is never evicted: RSS grows ~0.65 MB/page for the life of the server

Open
#136 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
742
Forks
138
Avg merge
11h 30m
Merged PRs (30d)
5

Description

`local_document_cache` and `local_stack_cache` are never evicted from, so a long-running server grows monotonically.

Verified against `v3.2.0` (installed from PyPI, remote conversion mode).

## Where it breaks

`docling_mcp/shared.py:20-21` declares both caches as plain module-level dicts:

```python
local_document_cache: dict[str, DoclingDocument] = {}
local_stack_cache: dict[str, list[NodeItem]] = {}
```

Every conversion inserts into `local_document_cache` (`tools/conversion.py`, `tools/converters/{local,remote}.py`), and the generation/manipulation tools insert into both. Nothing anywhere in the package removes an entry: there is no `.pop()`, no `.clear()`, no `del`, no size or TTL bound, and no tool that lets a client drop a document it is finished with.

The only memory-facing helper is `cleanup_memory()` (`tools/conversion.py:32`), called after each conversion at lines 89 and 141:

```python
def cleanup_memory() -> None:
"""Force garbage collection to free up memory."""
logger.info("Performed memory cleanup")
gc.collect()
```

`gc.collect()` cannot help here — the documents are still strongly referenced by the dict, so they are not garbage. The log line reads as though memory was reclaimed when nothing was.

## Impact

Measured with `DOCLING_MCP_KEEP_IMAGES=false`, remote mode (no models in-process), one container, RSS via `docker stats`:

| state | RSS |
|---|---|
| idle after startup | ~133 MiB |
| + a 16-page PDF | +5 MiB |
| + a 75-page PDF | +49 MiB |

≈ **0.65 MB per converted page, never released**. `KEEP_IMAGES=true` would be considerably worse.

For a short-lived client session this is fine. For a server left running as a shared long-lived process — which is the deployment the streamable-http transport invites — RSS only ever goes up, and the operator's only remedy is restarting the process, which also throws away every still-wanted document key. With no container memory limit set, a busy day can crowd whatever else shares the host.

## Repro

1. Start the server: `docling-mcp-server --transport streamable-http --host 0.0.0.0 --port 8053 conversion`
2. Note RSS.
3. Call `convert_document_into_docling_document` on a few dozen-page PDFs, using a distinct source each time so the content-keyed cache does not short-circuit.
4. RSS climbs by roughly 0.65 MB/page and never comes back down — including after `cleanup_memory()` runs, and with no client sessions open.

## Possible fixes

Roughly in order of effort:

1. Bound the caches — an LRU (`functools.lru_cache`-style or `OrderedDict` + `move_to_end`) with a configurable `DOCLING_MCP_CACHE_MAX_DOCUMENTS` / max-bytes, evicting oldest first.
2. Add an explicit eviction tool (e.g. `drop_document_from_local_cache(document_key)`), so a client that knows it is done can say so. Cheap, and useful regardless of 1.
3. Make `cleanup_memory()` honest — either have it evict, or drop the claim that it frees memory.

Happy to open a PR for (1)+(2) if that shape sounds right to the maintainers.

Contributor guide

Open the contributing guide

Research direction

Start with the cache declarations in docling_mcp/shared.py:20-21, then trace writes from tools/conversion.py and tools/converters/{local,remote}.py and inspect cleanup_memory() in tools/conversion.py. Reproduce the RSS growth using the streamable-http command and distinct PDFs; done means an agreed cache-eviction or explicit-drop behavior is implemented and the misleading memory-cleanup claim is addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.