canonical / canonical/mongodb-operator

A broken relation causes an uncaught exception

Open
#368 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
HCL
Stars
14
Forks
15
Avg merge
9h 44m
Merged PRs (30d)
17

Description

## Steps to reproduce

Follow the guide to deploy 5G SD-Core.
https://canonical-charmed-5g.readthedocs-hosted.com/en/1.3-beta_a/how-to/deploy_sdcore_standalone/
Then run `juju remove-application amf`.

## Versions

Operating system: Ubuntu 23.04

Juju CLI: 3.4.0

Juju agent:

```
"/run/user/1000/doc": lstat /run/user/1000/doc: permission denied
Model Controller Cloud/Region Version SLA Timestamp
test1 microk8s-localhost microk8s/localhost 3.4.0 unsupported 18:30:41+03:00

App Version Status Scale Charm Channel Rev Address Exposed Message
amf waiting 1 sdcore-amf-k8s 2 10.152.183.42 no installing agent
ausf active 1 sdcore-ausf-k8s 1.3/edge 47 10.152.183.114 no
grafana-agent-k8s 0.35.2 waiting 1 grafana-agent-k8s latest/stable 58 10.152.183.27 no installing agent
mongodb-k8s waiting 1 mongodb-k8s 6/beta 38 10.152.183.216 no installing agent
nms active 1 sdcore-nms-k8s 1.3/edge 37 10.152.183.201 no
nrf active 1 sdcore-nrf-k8s 1.3/edge 53 10.152.183.25 no
nssf active 1 sdcore-nssf-k8s 1.3/edge 42 10.152.183.102 no
pcf active 1 sdcore-pcf-k8s 1.3/edge 43 10.152.183.242 no
self-signed-certificates active 1 self-signed-certificates beta 72 10.152.183.135 no
smf active 1 sdcore-smf-k8s 1.3/edge 49 10.152.183.48 no
traefik-k8s 2.10.4 active 1 traefik-k8s latest/stable 166 10.0.0.3 no
udm active 1 sdcore-udm-k8s 1.3/edge 40 10.152.183.127 no
udr active 1 sdcore-udr-k8s 1.3/edge 42 10.152.183.110 no
upf active 1 sdcore-upf-k8s 1.3/edge 52 10.152.183.152 no
webui active 1 sdcore-webui-k8s 1.3/edge 26 10.152.183.206 no

Unit Workload Agent Address Ports Message
amf/0* blocked idle 10.1.146.24 Waiting for fiveg_nrf relation
ausf/0* active idle 10.1.146.38
grafana-agent-k8s/0* blocked idle 10.1.146.25 logging-consumer: off, grafana-cloud-config: off
mongodb-k8s/0* error idle 10.1.146.58 hook failed: "database-relation-broken"
nms/0* active idle 10.1.146.9
nrf/0* active idle 10.1.146.33
nssf/0* active idle 10.1.146.30
pcf/0* active idle 10.1.146.63
self-signed-certificates/0* active idle 10.1.146.27
smf/0* active idle 10.1.146.22
traefik-k8s/0* active idle 10.1.146.60
udm/0* active idle 10.1.146.48
udr/0* active idle 10.1.146.18
upf/0* active idle 10.1.146.21
webui/0* active idle 10.1.146.5
```

Charm revision: 6/beta Rev: 38

LXD: 5.20

## Log output

Juju debug log:

```
controller-0: 18:19:44 INFO juju.worker.caasapplicationprovisioner.amf scaling application "amf" to desired scale 0
controller-0: 18:19:45 WARNING juju.worker.caasapplicationprovisioner.amf update units application "amf" not found
controller-0: 18:19:47 WARNING juju.worker.caasapplicationprovisioner.amf update units application "amf" not found
unit-mongodb-k8s-0: 18:19:47 INFO juju.worker.uniter awaiting error resolution for "relation-broken" hook
unit-mongodb-k8s-0: 18:19:48 WARNING unit.mongodb-k8s/0.juju-log database:4: 'app' expected but not received.
unit-mongodb-k8s-0: 18:19:48 WARNING unit.mongodb-k8s/0.juju-log database:4: 'app_name' expected in snapshot but not found.
controller-0: 18:19:48 WARNING juju.worker.caasapplicationprovisioner.amf update units application "amf" not found
unit-mongodb-k8s-0: 18:19:48 INFO unit.mongodb-k8s/0.juju-log database:4: Update relation user: relation-26 on free5gc
unit-mongodb-k8s-0: 18:19:48 INFO unit.mongodb-k8s/0.juju-log database:4: Updating relation data according to diff
unit-mongodb-k8s-0: 18:19:48 ERROR unit.mongodb-k8s/0.juju-log database:4: Uncaught exception while in charm code:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/model.py", line 2946, in _run
result = subprocess.run(args, **kwargs) # type: ignore
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('/var/lib/juju/tools/unit-mongodb-k8s-0/relation-get', '-r', '4', '-', 'mongodb-k8s', '--app', '--format=json')' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/./src/charm.py", line 1137, in
main(MongoDBCharm)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/main.py", line 436, in main
_emit_charm_event(charm, dispatcher.event_name)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/main.py", line 144, in _emit_charm_event
event_to_emit.emit(*args, **kwargs)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/framework.py", line 340, in emit
framework._emit(event)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/framework.py", line 842, in _emit
self._reemit(event_path)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/framework.py", line 931, in _reemit
custom_handler(event)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/lib/charms/mongodb/v0/mongodb_provider.py", line 116, in _on_relation_event
self.oversee_users(departed_relation_id, event)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/lib/charms/mongodb/v0/mongodb_provider.py", line 168, in oversee_users
self._diff(event)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/lib/charms/mongodb/v0/mongodb_provider.py", line 191, in _diff
old_data = json.loads(event.relation.data[self.charm.model.app].get("data", "{}"))
File "/usr/lib/python3.10/_collections_abc.py", line 824, in get
return self[key]
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/model.py", line 1686, in __getitem__
return super().__getitem__(key)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/model.py", line 790, in __getitem__
return self._data[key]
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/model.py", line 774, in _data
data = self._lazy_data = self._load()
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/model.py", line 1570, in _load
return self._backend.relation_get(self.relation.id, self._entity.name, self._is_app)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/model.py", line 3027, in relation_get
raw_data_content = self._run(*args, return_output=True, use_json=True)
File "/var/lib/juju/agents/unit-mongodb-k8s-0/charm/venv/ops/model.py", line 2948, in _run
raise ModelError(e.stderr) from e
ops.model.ModelError: ERROR permission denied (unauthorized access)

unit-mongodb-k8s-0: 18:19:49 ERROR juju.worker.uniter.operation hook "database-relation-broken" (via hook dispatching script: dispatch) failed: exit status 1
```

## Additional context

Contributor guide

Open the contributing guide

Research direction

Start with the relation event flow in lib/charms/mongodb/v0/mongodb_provider.py, especially _on_relation_event, oversee_users, and _diff, then trace how src/charm.py dispatches the hook. Reproduce the failure with `juju remove-application amf` after following the linked deployment guide. Done means the broken relation no longer raises an uncaught ModelError and the database-relation-broken hook completes successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.