Getting frequent 504s with medium sized PDFs and ocr... | Unstructured-IO/unstructured-api

Nobody has claimed this yet.

Describe the bug
Sending a single PDF (this one: https://arxiv.org/abs/2310.12931, embedded text, 39 pages) to the self hosted API, with either ocr_only or hi_res strategy.
The server has 32GB RAM, 8 CPU cores and a CUDA enabled GPU, ressources are below 20% CPU and 5% RAM when processing the PDF.
The Unstructured API version is v0.0.61.

The server responds with 504 after some 20 to 30s and the client caller via partition_via_api will try again for some time.
In the server logs I can see, that each time a new request is made it is properly worked on, printing out '[...] unstructured INFO Processing entire page OCR [...]' and the server does not crash.
However the client detaches and discards the request, so when the server is done processing, it is just discarded.

On the client side I get

INFO: Response status code: 504 Retry attempt #1. Sleeping 1.4 seconds before retry.
INFO: <html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx</center>
</body>
</html>

leaving the process running on the server and triggering a new one which gets detached in a similar fashion.

I tested the same call on the officially hosted API and the results was there after some 5m:30s.

Question: Is there some setting I can make on the server side to avoid that? It is obviously running on the officially hosted service so the answer should be yes. Could you hint me the right way to go with this?

To Reproduce
Calling the API like this:

from unstructured.partition.api import partition_via_api
API_BASE_URL = "....."

result = partition_via_api(
    "files/2310.12931.pdf",
    strategy="ocr_only",
    api_url=f"{API_BASE_URL}/general/v0/general/",
)

Filetype: PDF (see above for exact file)
Any additional API parameters: The server runs with UNSTRUCTURED_MEMORY_FREE_MINIMUM_MB="4096"

Environment:

Using the hosted API or self hosting?
- using the self-hosted API
How are you calling the API? (Langchain, SDKs, cUrl, etc.)
- Calling via Unstructured API SDK partition_via_api

Additional context
Update: The same happens when calling the API with cURL like so:

curl -X "POST" "$API_BASE_URL/general/v0/general" \
-H 'accept: application/json' \
-H 'Content-Type: multipart/form-data' \
-F files=@files/2310.12931.pdf \
-F 'strategy=ocr_only'

minus the retry part, because cURL simply stops after a single attempt.
On the server side the behaviour stays the same, the process keeps running until properly finished and is then discarded because the client detached.

Start with the self-hosted API request path exercised by partition_via_api and the equivalent curl request, then inspect how the nginx gateway timeout relates to the server's long-running PDF OCR processing. Reproduce with the linked 39-page PDF and ocr_only or hi_res, and consider the issue done when the request completes without a 504 or discarded server-side work.

Written by the indexing model from the issue text.

Getting frequent 504s with medium sized PDFs and ocr_only and hi_res strategies

Description

Contributor guide

First steps

Research direction

Assessment