canonical / canonical/charm-integration-testing

katib-db-manager: update-status hook fails — bundle includes deprecated mysql relation alongside relational-db, blocking workload start

Open
#473 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
6
Forks
1
Avg merge
1d 20h
Merged PRs (30d)
98

Description

**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated log analysis and triage.**
Filed by @canonical/solutions-qa

---

## Problem

The integration test for `katib-db-manager` (rev 253, channel `0.15/stable`) has been failing **continuously since at least April 2025** across all test plan variants. The test bundle deploys both the deprecated `mysql` interface **and** the new `relational-db` interface simultaneously, which causes the charm to refuse to start its workload container.

With the workload never starting, the Pebble health check `katib-db-manager-up` is never registered, and the `update-status` hook crashes with `ops.model.ModelError: check 'katib-db-manager-up' not found`.

**Test Observer artefact page:** https://test-observer.canonical.com/#/charms/139999?testExecutionId=468867&testResultId=10487173
**CI run:** https://github.com/canonical/charm-integration-testing/actions/runs/24401663364

## Environment

- Charm: `katib-db-manager` rev 253, channel `0.15/stable`
- Juju: 3.6.21 / juju:3/stable
- Kubernetes: ubuntu:20.04 kubernetes amd64
- Co-deployed: `mysql-k8s` rev 400 (8.0/stable), `mysql-router-k8s` rev 902 (8.0/stable)

## Root Cause

The test bundle for test plan `integration/mysql-router-k8s:database/mysql_client/katib-db-manager:relational-db` includes **both** relations:
- `mysql-k8s:mysql → target:mysql` (deprecated direct mysql interface)
- `neighbor:database → target:relational-db` (new interface, via mysql-router-k8s)

The charm explicitly fails when both are present. From the debug log:
```
ERROR unit.target/0.juju-log Relation mysql is deprecated. Remove deprecated mysql relation to unblock.
ERROR unit.target/0.juju-log Missing attribute 'root_password' in mysql relation data
```

Because the workload never starts, the Pebble check is never registered, causing `update-status` to crash:
```
WARNING File "./src/charm.py", line 360, in _get_check_status
WARNING return self.container.get_check("katib-db-manager-up").status
WARNING ops.model.ModelError: check 'katib-db-manager-up' not found
WARNING File "./src/charm.py", line 368, in _refresh_status
WARNING raise GenericCharmRuntimeError("Failed to run health check on workload container")
ERROR hook "update-status" (via hook dispatching script: dispatch) failed: exit status 1
```

## Proposed Fix

Remove the deprecated `mysql-k8s:mysql → target:mysql` relation from the test bundle for `katib-db-manager` when testing the `relational-db` interface path. The charm should be deployed with **only** the `relational-db` interface (via `mysql-router-k8s:database → katib-db-manager:relational-db`).

## Related

- [canonical/katib-operators#128](https://github.com/canonical/katib-operators/issues/128) — upstream open issue tracking the missing `grpc_health_probe` binary for the Pebble health check (separate but related charm bug)
- Failure is consistent across all test plan variants; no successful runs observed since April 2025

Contributor guide

Open the contributing guide

Research direction

Start by locating the bundle for test plan integration/mysql-router-k8s:database/mysql_client/katib-db-manager:relational-db and inspect its relation definitions. Remove the deprecated mysql-k8s:mysql to target:mysql relation, retain the relational-db path, and rerun the katib-db-manager integration test to confirm the workload starts and update-status completes without the missing health-check error.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
devops, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.