canonical / canonical/test_observer
Support artefact-scoped attachment rules (low priority; easy workaround exists)
- Dominant language
- Python
- Stars
- 13
- Forks
- 13
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 11
Description
**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated triage experience.**
Filed by @canonical/solutions-qa
## Priority
**Low.** There is a straightforward workaround (see below), so this is a usability/ergonomics improvement rather than a blocker.
## Summary
Allow issue-to-test-result attachment rules to be scoped to a specific artefact (charm/snap). Today the attachment-rule create schema (`IssueTestResultAttachmentRulePostRequest`) accepts `families`, `environment_names`, `test_case_names`, `template_ids`, `test_result_statuses`, and `execution_metadata` — but no artefact dimension.
## Problem
When a single charm fails with a **variable** failure signature, there is no way to write one clean rule:
- Keying on `execution_metadata` (a specific endpoint/message) matches only one variant — you'd need N rules for N signatures.
- Keying on `test_case_names` (e.g. `test_build_bundle`) matches that test case for **every** charm, which massively over-attaches.
- Net result: the issue must be attached by explicit result-ID lists, and future recurrences are not auto-attached.
## Workaround (why this is low priority)
The situation is easy to work around today:
- Attach the issue to the affected results directly via an explicit `test_result_ids` list (precise, no over-matching), or
- Create one narrow rule per stable `execution_metadata` signature.
The only real cost is manual re-attachment when new failures of the same variable-signature bug appear.
## Evidence
`self-signed-certificates` bundle-build fails across ~30 different neighbor-endpoint signatures (charm-integration-testing#747). With no artefact-scoped rule available, the issue was manually attached three times (revisions v650 ×2, v652), ~236 explicit result IDs total, with no auto-attach going forward. By contrast, `kubeapi-load-balancer` and `nova-cloud-controller` had a single stable signature, so a normal `execution_metadata` rule worked cleanly.
## Proposed change
1. **API**: add optional artefact filters to `IssueTestResultAttachmentRulePostRequest` and the rule-matching logic — at minimum `artefacts` (names); ideally also `artefact_tracks`, `artefact_versions`, `artefact_stages` to mirror the dimensions already returned by `GET .../attachment-rule-filters` under `artefact_filters`.
2. **MCP server (`canonical/test-observer-mcp`)**: expose the new filter(s) in the `create_attachment_rule` tool (add `artefacts` to `createAttachmentRuleInput`) and forward them.
## Example enabled after the change
```
create_attachment_rule(
issue_id=,
artefacts=["self-signed-certificates"],
test_case_names=["test_build_bundle"],
test_result_statuses=["FAILED"],
)
```
Matches every SSC bundle-build failure regardless of which neighbor endpoint is unfulfilled, without touching any other charm.
## Acceptance criteria
- A rule with `artefacts=[X]` matches only results whose artefact name is X.
- Combinable with existing filters (test_case, families, status, execution_metadata).
- `matching_test_results_count` reflects the artefact scope.
- MCP `create_attachment_rule` accepts and forwards `artefacts` (+ optional track/version/stage).
## Caveat
Artefact-scoped rules should still generally be paired with a failure-signature dimension (test_case / template / execution_metadata) so a rule doesn't attach an issue to *all* of a charm's failures across unrelated bugs.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the IssueTestResultAttachmentRulePostRequest schema and the attachment-rule matching logic, then inspect the artefact_filters returned by the attachment-rule-filters endpoint. Update the MCP createAttachmentRuleInput and create_attachment_rule forwarding path as needed. Done means artefact filters combine with existing filters, matching_test_results_count respects the scope, and only results for the selected artefact are matched.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100