canonical / canonical/postgresql-single-kernel-library
planned_units() reads crash config render with ModelError while a force-removed cross-model SAAS lingers in goal-state
- Dominant language
- Python
- Stars
- 4
- Forks
- 3
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 26
Description
## Issue
After a dead-DC teardown that force-removes a cross-model SAAS (`juju remove-saas --force`), the removed SAAS application keeps lingering in goal-state, and `Application.planned_units()` fails with `ModelError` for every hook invocation while it does. The library reads `planned_units()` unguarded in two places, so every event that re-renders the Patroni config crashes the hook — and `update-status` re-crashes it on every run for as long as the SAAS lingers (observed for the full 12-minute window in a live repro).
Observed on the PostgreSQL VM charm (`postgresql-operator` revN built on this library, substrate VM/LXD; library pinned at 16.3.6 and verified still present on `16/edge`):
```
ops.model.ModelError: ERROR saas application "db1" not found
_on_async_relation_broken (async_replication.py:559)
-> charm.update_config (charm.py:2774)
-> lib config.py:509 update_config
-> config.py:687 render_patroni_yml_file
-> state.py:505 synchronous_node_count [cached_property]
-> peer_relation.py:414 planned_units
-> ops planned_units -> goal_state() [fails]
```
## Where
`core/state.py` (16.3.6; unchanged on `16/edge`):
```python
@cached_property
def synchronous_node_count(self) -> int:
"""Number of expected sync standbys."""
planned_units = self.application.planned_units # <- raises ModelError
```
`core/peer_relation.py` (both `PostgreSQLPeer` and `PostgreSQLApplication`):
```python
@cached_property
def planned_units(self) -> int:
"""Get the number of planned units for the application."""
return self.app.planned_units() # <- unguarded ops goal-state read
```
`synchronous_node_count` is evaluated unconditionally during `render_patroni_yml_file` (the synchronous block is rendered even when synchronous mode is off), so *any* `update_config()` path is affected — not only replication-related handlers.
## Why a charm-side guard is not enough
The charm (`canonical/postgresql-operator`) added a `_planned_units` fallback for its own direct read (https://github.com/canonical/postgresql-operator/pull/1913), but the library's internal read chain cannot be intercepted from the charm: `charm.update_config()` is called from ~12 sites (update-status, leader-elected, databases-changed, peer-relation events, TLS push, restore completion, password rotation...), and only one of them catches `ModelError` — the others catch `RetryError`/`psycopg2.OperationalError` or nothing, so the lib read crashes those hooks during the lingering-goal-state window.
## Suggested fix
Make the library resilient to transient goal-state failures, e.g.:
- catch `ModelError` in `planned_units` (both `PostgreSQLPeer` and `PostgreSQLApplication`) and fall back to the count of currently known units (`len(self.relation.units)` style), or return `None` and have `synchronous_node_count` skip the synchronous block when the count is unavailable;
- and/or guard `synchronous_node_count`/`synchronous_configuration` so a failed `planned_units` read does not abort the whole config render.
A goal-state outage should degrade the synchronous block, not crash every hook that renders config.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in core/peer_relation.py with planned_units for PostgreSQLPeer and PostgreSQLApplication, then trace synchronous_node_count in core/state.py through render_patroni_yml_file. Compare the two unguarded goal-state reads and the available relation-unit fallback. Done means a lingering SAAS goal-state ModelError no longer aborts configuration rendering or repeated hook invocations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100