MeltanoLabs / MeltanoLabs/target-postgres
Long lived Sinks seem to time out, pool doesn't seem to fix this automatically
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13
- Forks
- 23
- Avg merge
- 4h 45m
- Merged PRs (30d)
- 4
Description
> @BuzzCutNorman tried this PR here https://meltano.slack.com/archives/C01RKUVUG4S/p1688075229854589?thread_ts=1688060563.197079&cid=C01RKUVUG4S , I thought this would address an issue like
cur.execute(b''.join(parts)) cmd_type=elb consumer=True name=target-postgres producer=False stdio=stderr string_id=target-postgres 2023-06-29T20:23:10.757683Z [info ] sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly cmd_type=elb consumer=True name=target-postgres producer=False stdio=stderr string_id=target-postgresBy using Pooling properly but we're still hitting it. Going to take a look a bit deeper but wanted to capture this somewhere
@BuzzCutNorman can you try this again?
Originally posted by @visch in https://github.com/MeltanoLabs/target-postgres/issues/153#issuecomment-1622556921
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported failure with a long-lived sink and review the target-postgres pooling behavior around the psycopg2 connection. Confirm whether the pool handles a server-closed connection automatically, and document or test the behavior that should prevent the sink from timing out.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100