500 error when listing a lot of RPM packages
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 598
- Forks
- 168
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 86
Description
Version
3.25.3
Describe the bug
[2024-05-24 17:33:02 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:14)
[2024-05-24 17:33:03 +0000] [14] [ERROR] Error handling request /api/pulp/default/api/v3/content/rpm/packages/?offset=0&limit=1
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/sync.py", line 178, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/lib/python3.9/site-packages/opentelemetry/instrumentation/wsgi/__init__.py", line 589, in __call__
iterable = self.wsgi(environ, start_response)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/utils.py", line 1711, in runner
return sentry_patched_function(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/__init__.py", line 174, in sentry_patched_wsgi_handler
return SentryWsgiMiddleware(bound_old_app, use_x_forwarded_for)(
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/wsgi.py", line 109, in __call__
reraise(*_capture_exception())
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/utils.py", line 1640, in reraise
raise value
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/wsgi.py", line 102, in __call__
response = self.app(
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/wsgi.py", line 124, in __call__
response = self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/__init__.py", line 462, in sentry_patched_get_response
rv = old_get_response(self, request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 140, in get_response
response = self._middleware_chain(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django_guid/middleware.py", line 79, in middleware
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 134, in __call__
response = response or self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/whitenoise/middleware.py", line 124, in __call__
return self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 134, in __call__
response = response or self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 134, in __call__
response = response or self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 134, in __call__
response = response or self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 134, in __call__
response = response or self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 134, in __call__
response = response or self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/utils/deprecation.py", line 134, in __call__
response = response or self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/middleware.py", line 169, in __call__
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/pulpcore/middleware.py", line 24, in __call__
response = self.get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 84, in sentry_wrapped_callback
return callback(request, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
return view_func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/rest_framework/viewsets.py", line 124, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/rest_framework/mixins.py", line 40, in list
page = self.paginate_queryset(queryset)
File "/usr/local/lib/python3.9/site-packages/rest_framework/generics.py", line 175, in paginate_queryset
return self.paginator.paginate_queryset(queryset, self.request, view=self)
File "/usr/local/lib/python3.9/site-packages/rest_framework/pagination.py", line 395, in paginate_queryset
self.count = self.get_count(queryset)
File "/usr/local/lib/python3.9/site-packages/rest_framework/pagination.py", line 530, in get_count
return queryset.count()
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 608, in count
return self.query.get_count(using=self.db)
File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/query.py", line 568, in get_count
return obj.get_aggregation(using, {"__count": Count("*")})["__count"]
File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/query.py", line 554, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/utils.py", line 1711, in runner
return sentry_patched_function(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/__init__.py", line 641, in execute
result = real_execute(self, sql, params)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/psycopg/cursor.py", line 728, in execute
self._conn.wait(
File "/usr/local/lib/python3.9/site-packages/psycopg/connection.py", line 969, in wait
return waiting.wait(gen, self.pgconn.socket, timeout=timeout)
File "psycopg_binary/_psycopg/waiting.pyx", line 193, in psycopg_binary._psycopg.wait_c
File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/base.py", line 203, in handle_abort
sys.exit(1)
SystemExit: 1
('pulp [-]: ::ffff:10.131.8.12 - - [24/May/2024:17:33:03 +0000] "GET /api/pulp/default/api/v3/content/rpm/packages/?offset=0&limit=1 HTTP/1.1" 500 0 "-" "-"',)
To Reproduce
Load up pulp with many thousands of packages. Then run:
$ pulp rpm content list --limit 1
Error: <html>
<head>
<title>Internal Server Error</title>
</head>
<body>
<h1><p>Internal Server Error</p></h1>
</body>
</html>
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here. Please provide links to any previous discussions via Discourse or Bugzilla.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with pulp rpm content list --limit 1 against an installation containing many thousands of packages, then inspect the /api/pulp/default/api/v3/content/rpm/packages/ listing path and its pagination count step. Use the traceback to focus on the database count reached through Django REST Framework pagination. Done means the endpoint returns a valid package-list response instead of timing out with HTTP 500.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- api, backend, databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100