canonical / canonical/charm-integration-testing

`grafana-agent-k8s` and `vault-k8s` appear to use incompatible tls-certificates implementations

Open
#512 6 comments 0 reactions 0 assignees View on GitHub
bug charm-deployment
Dominant language
Python
Stars
6
Forks
1
Avg merge
1d 22h
Merged PRs (30d)
96

Description

**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated log analysis and triage.**
Filed by @canonical/solutions-qa

## Summary
In Test Observer execution 502676, test_deploy failed with a 15-minute wait_for_idle timeout. At timeout, target (grafana-agent-k8s) remained in waiting state with message Waiting for TLS certificate., while other applications were active.

## Observed environment
- Target charm: grafana-agent-k8s
- Target revision/channel: 168 on 1/candidate
- Neighbor charm: vault-k8s rev 502 on 1.16/stable
- Other deployed apps: prometheus-k8s rev 287, self-signed-certificates rev 586
- Model: model-25185463723-260430193620
- Controller/cloud: 25185463723-260430193620 on k8s-production
- Juju version: 3.6.21

## Failing test
- Test name: test_deploy
- Test result id: 10815555
- Test execution id: 502676
- Artefact id: 246744
- Failure message:
juju.backend.JujuWaitTimeoutError: Timed out while waiting for applications: [active], units: [active] (applications: ['target'], units: ['target/0'])

## Key observations from logs
- At timeout (20:00:47Z), Juju status shows:
- target app: waiting - Waiting for TLS certificate.
- target/0 unit: waiting - Waiting for TLS certificate.
- neighbor, prometheus-k8s, self-signed-certificates: active
- In debug-log.txt during certificates-relation-changed on target/0:
- Relation data did not pass JSON Schema validation
- A later status snapshot still showed target waiting for TLS certificate.

## Steps to reproduce
1. Run charm-integration-testing against grafana-agent-k8s revision 168 on channel 1/candidate.
2. Use a bundle with:
- neighbor: vault-k8s (1.16/stable, rev 502)
- self-signed-certificates (1/stable, rev 586)
- prometheus-k8s (2/stable, rev 287)
- target: grafana-agent-k8s (1/candidate, rev 168)
3. Include relations:
- neighbor:vault-pki -> target:certificates
- neighbor:metrics-endpoint -> target:metrics-endpoint
- prometheus-k8s:receive-remote-write -> target:send-remote-write
- neighbor:tls-certificates-pki -> self-signed-certificates:certificates
4. Execute test_deploy and wait for idle with a 15-minute timeout.
5. Observe timeout with target still waiting for TLS certificate.

## Reference
- Test Observer UI:
https://test-observer.canonical.com/#/charms/246744?testExecutionId=502676&testResultId=10815555

## Requested investigation
Please investigate whether this failure is caused by:
- relation data/schema incompatibility in the TLS path,
- delayed certificate issuance/propagation,
- or a test orchestration/readiness expectation mismatch.

Contributor guide

Open the contributing guide

Research direction

Start with the failing test_deploy execution and its debug-log.txt output, focusing on the certificates-relation-changed event and the JSON Schema validation failure. Compare the grafana-agent-k8s and vault-k8s TLS relation data, then verify whether issuance timing or readiness handling contributes. Done means identifying and documenting the confirmed cause with evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.