pulp / pulp/pulpcore

[PULP-957] Error during long running synchronous upload API

Open
#7,061 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Issue
Dominant language
Python
Stars
598
Forks
168
Avg merge
1d 4h
Merged PRs (30d)
86

Description

Describe the bug
Trying to make a synchronous upload API with a large file (so that the request takes a lot of time) is failing with:

(pulp [c13d1db3342142918bf40e0de7757e9c]: 10.129.30.13 - - [30/Oct/2025:16:41:03 +0000] "POST /api/pulp/hyagi-s3/api/v3/content/rpm/packages/upload/ HTTP/1.0" 201 8071 "-" "curl/8.6.0" 587817)
pulp [None]: pulpcore.app.entrypoint:ERROR: Api App '1228@pulp-api-6f5cfb9878-22zn7' failed to write a heartbeat to the database.
[2025-10-30 16:41:03 +0000] [1228] [INFO] Worker exiting (pid: 1228)
[2025-10-30 16:41:04 +0000] [1] [ERROR] Worker (pid:1228) exited with code 3
[2025-10-30 16:41:04 +0000] [1264] [INFO] Worker exiting (pid: 1264)
[2025-10-30 16:41:04 +0000] [1244] [INFO] Worker exiting (pid: 1244)
[2025-10-30 16:41:04 +0000] [1240] [INFO] Worker exiting (pid: 1240)
[2025-10-30 16:41:04 +0000] [1238] [INFO] Worker exiting (pid: 1238)
[2025-10-30 16:41:08 +0000] [1] [ERROR] Shutting down: Master
[2025-10-30 16:41:08 +0000] [1] [ERROR] Reason: Worker failed to boot.

To Reproduce
Steps to reproduce the behavior:

# download a sample file
wget https://download.copr.fedorainfracloud.org/results/@dotnet-sig/dotnet-preview/fedora-41-x86_64/09584046-dotnet10.0/dotnet10.0-10.0.100~rc.1.25451.107-0.7.fc41~bootstrap.src.rpm 

# upload it using synchronous upload API
curl  https://${BASE_ADDR}/api/pulp/${DOMAIN}/api/v3/content/rpm/packages/upload/ -F file=@dotnet10.0-10.0.100~rc.1.25451.107-0.7.fc41~bootstrap.src.rpm

# alternative to make the request take longer
curl --limit-rate 5M  https://${BASE_ADDR}/api/pulp/${DOMAIN}/api/v3/content/rpm/packages/upload/ -F file=@dotnet10.0-10.0.100~rc.1.25451.107-0.7.fc41~bootstrap.src.rpm

Additional context
As a workaround, we increased the API_APP_TTL

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the synchronous upload API endpoint shown in the reproduction and the API_APP_TTL setting mentioned in the workaround. Review the heartbeat and worker-exit errors in the provided logs to determine how long-running uploads are handled. Done means the reproduced large or rate-limited upload completes without the API app losing its database heartbeat or shutting down.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.