UsageTracking failure - Postgrex.Protocol (#PID<0.4680.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.53147.0> (Lightning.UsageTracking.ReportWorker) timed out because it queued and checked out the connection for longer than 70000ms
Open
Nobody has claimed this yet.
bug
- Dominant language
- Elixir
- Stars
- 296
- Forks
- 86
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 50
Description
Sentry Issue: LIGHTNING-16F
Postgrex.Protocol (#PID<0.4680.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.53147.0> (Lightning.UsageTracking.ReportWorker) timed out because it queued and checked out the connection for longer than 70000ms
#PID<0.53147.0> (Lightning.UsageTracking.ReportWorker) was at location:
(lightning 2.15.6) lib/lightning/usage_tracking/workflow_metrics_service.ex:69: anonymous fn/3 in Lightning.UsageTracking.WorkflowMetricsService.metric_updater/1
(elixir 1.18.3) lib/enum.ex:2546: Enum."-reduce/3-lists^foldl/2-0-"/3
(lightning 2.15.6) lib/lightning/repo.ex:2: anonymous fn/1 in Lightning.Repo.transaction/2
(ecto 3.13.2) lib/ecto/repo/transaction.ex:7: anonymous fn/2 in Ecto.Repo.Transaction.transact/4
(ecto_sql 3.13.2) lib/ecto/adapters/sql.ex:1458: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection 2.8.1) lib/db_connection.ex:1753: DBConnection.run_transaction/4
(lightning 2.15.6) lib/lightning/usage_tracking/workflow_metrics_service.ex:29: Lightning.UsageTracking.WorkflowMetricsService.metrics_for_runs/2
(lightning 2.15.6) lib/lightning/usage_tracking/workflow_metrics_service.ex:19: Lightning.UsageTracking.WorkflowMetricsService.generate_metrics/3
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in lib/lightning/usage_tracking/workflow_metrics_service.ex at lines 19, 29, and 69, following generate_metrics/3 through metrics_for_runs/2 and metric_updater/1. Use the Sentry report and Postgrex timeout details to identify the long connection checkout or transaction path; done means the UsageTracking.ReportWorker no longer disconnects after exceeding 70000ms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir, postgresql
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100