canonical / canonical/livepatch-k8s-operator
database-relation-changed hook fails with SQLSTATE 42P05 (prepared statement "lrupsc_1_0" already exists) when pgbouncer-k8s is redeployed
- Dominant language
- Python
- Stars
- 4
- Forks
- 12
- Avg merge
- 13h 21m
- Merged PRs (30d)
- 4
Description
**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated log analysis and triage.**
Filed by @canonical/solutions-qa
---
## Summary
The `database-relation-changed` hook in `canonical-livepatch-server-k8s` rev 76 raises an uncaught exception when `pgbouncer-k8s` (the database proxy) is redeployed or scaled. The `livepatch-schema-tool check` command fails with PostgreSQL error `SQLSTATE 42P05: prepared statement "lrupsc_1_0" already exists`, causing the unit to enter error state.
## Test Observer
- **Execution**: https://test-observer.canonical.com/#/charms/405104?testExecutionId=413592&testResultId=9533915
- **Test plan**: `integration/postgresql-k8s:database/postgresql_client/pgbouncer-k8s:backend-database`
- **CI run**: https://github.com/canonical/charm-integration-testing/actions/runs/22491507153
- **Charm**: `canonical-livepatch-server-k8s` rev 76 (`latest/stable`), pgbouncer-k8s rev 520 (`1/edge`)
## Error (from juju debug-log)
Observed at `15:34:19` during idempotent redeploy of pgbouncer-k8s, and again at `15:20:17` during scale-in/scale-out:
```
ERROR unit.canonical-livepatch-server-k8s/0.juju-log database:11:
Failed to determine if schema upgrade required: non-zero exit code 1 executing
['/usr/local/bin/livepatch-schema-tool', 'check', '/etc/livepatch/schema-upgrades',
'--db', 'postgresql://[REDACTED]@target-service.../livepatch-server'],
stderr='Error: ERROR: prepared statement "lrupsc_1_0" already exists (SQLSTATE 42P05)
ERROR: prepared statement "lrupsc_1_0" already exists (SQLSTATE 42P05)'
ERROR unit.canonical-livepatch-server-k8s/0.juju-log database:11:
Uncaught exception while in charm code:
ERROR juju.worker.uniter.operation hook "database-relation-changed"
(via hook dispatching script: dispatch) failed: exit status 1
```
## Root Cause
When pgbouncer-k8s is redeployed or scaled, a `database-relation-changed` event fires. The charm calls `livepatch-schema-tool check` which internally prepares a PostgreSQL statement named `lrupsc_1_0`. In pgBouncer **session pooling** mode, the backend PostgreSQL connection may be reused across multiple invocations, meaning a previously prepared statement `lrupsc_1_0` still exists on the server-side connection. The subsequent `PREPARE` then fails with `42P05`.
The charm has no error handling for this exit code from `livepatch-schema-tool`, resulting in an uncaught exception and a hook failure.
## Recurrence
This failure is reproducible across multiple test executions for pgbouncer-k8s rev 520:
| Execution | Result |
|-----------|--------|
| 413592 | FAILED |
| 413565 | FAILED |
| 413563 | FAILED |
| 413589 | FAILED |
## Suggested Fix
One or more of the following:
1. **In `livepatch-schema-tool`**: Execute `DEALLOCATE lrupsc_1_0` (or `DEALLOCATE ALL`) before re-preparing the statement, so it is idempotent.
2. **In `livepatch-schema-tool`**: Treat `SQLSTATE 42P05` as non-fatal — if the statement already exists, it can safely be reused or re-prepared after deallocation.
3. **In the charm's `database-relation-changed` handler**: Catch the exit-code-1 error from `livepatch-schema-tool` and handle `42P05` gracefully instead of propagating as uncaught exception.
Contributor guide
Assessment
This issue has not been assessed yet.