azurenoops / azurenoops/spin_agent

[HIGH-5] Fabricated confidence scores

Open
#645 2 comments 0 reactions 1 assignee Claimed by @jrspinella View on GitHub
area:scoring bug priority:high
Dominant language
C#
Stars
3
Forks
1
Avg merge
18h 9m
Merged PRs (30d)
43

Description

## Problem

Confidence scores displayed to ISSOs (and potentially included in ATO artifacts) are hardcoded to `0.8` and `0.6`. They are presented as if they were reliability-derived metrics, but they are constants with no connection to model output probability, retrieval confidence, or any runtime signal. This is misleading to compliance officers who may use these figures when making authorization decisions.

## Evidence / Location

- Confidence score assignment in compliance response builder / narrative generator
- Values `0.8` (primary) and `0.6` (secondary/fallback) are literals, not computed

## Fix

1. Either compute confidence from real signals (model logprobs, retrieval score, citation count) or remove the field entirely.
2. If a placeholder is required short-term, label it explicitly as `"confidence": null` or `"confidence": "not-computed"` — never return a fake numeric value.
3. Document the intended confidence computation contract in the API schema.

## Source Reviewers

Vision · Banner

---
> ⚠️ **Caveat:** All findings are static read-reviews; build/tests were not run. Reproduce before treating fix as confirmed.
> 📋 **Note:** War Machine's ATO/RMF findings are still pending and will be folded in as a second batch.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.