CycloneDX / CycloneDX/specification

# Proposal: AI Behavioral Risk Assertions for CycloneDX 2.0

Open
#956 3 comments 0 reactions 1 assignee Claimed by @stevespringett View on GitHub
Dominant language
XSLT
Stars
547
Forks
93
Avg merge
7h 11m
Merged PRs (30d)
37

Description

This proposal explores a minimal extension to CycloneDX 2.0 that enables machine-readable representation of **AI-specific behavioral risks and exploitability assertions**.

The intent is **not** to introduce a separate exchange format. Rather, the proposal extends existing CycloneDX `risk`, `vulnerability`, and VEX `analysis` constructs to represent AI risks that are not adequately captured by traditional CVEs.

---

## Motivation

CycloneDX already provides:

* Components and services
* Vulnerabilities and CWEs
* VEX analysis
* Risk assessments
* Impact and consequence categories

However, AI systems introduce risks that frequently exist **without a corresponding CVE**.

Examples include:

* Prompt injection
* Indirect prompt injection through retrieved content
* Unsafe tool invocation
* Agent privilege escalation
* Retrieval poisoning
* Unsafe model behavior
* Model behavior drift

Today, these are difficult to represent in a machine-readable and interoperable manner.

---

## Gap Analysis

Current vulnerability models answer:

* Is a CVE applicable?
* Is the component affected?
* Is remediation available?

AI systems often require answering:

* Is the model susceptible to prompt injection?
* Can retrieved content alter model behavior?
* Can the agent invoke tools outside intended boundaries?
* What are the safety consequences if exploitation occurs?

These questions are behavioral and exploitability assertions that may exist independently of a CVE.

---

## Proposed Extension

Introduce AI-specific categories under the existing `risk` object.

Example:

```json
{
"component": {
"type": "ml-model",
"name": "medical-triage-llm"
},

"risk": {

"category": "prompt-injection",

"analysis": {

"state": "affected",

"vector": "indirect",

"confidence": "high"
},

"impact": [

"health",

"safety"

]
}
}
```

Possible categories include:

* prompt-injection
* retrieval-poisoning
* unsafe-tool-use
* agent-privilege-escalation
* model-drift
* unsafe-output
* hallucination-risk

---

## Relationship to Existing CycloneDX Constructs

This proposal does not introduce a separate top-level AI security object.

Instead:

* `component`
represents the model, agent, or AI service

* `risk`
represents AI behavioral risks

* VEX `analysis`
represents exploitability and applicability

* `impact`
represents safety, health, societal, or operational consequences

This approach preserves compatibility with existing CycloneDX 2.0 design principles.

---

## Safety-Critical Example

```json
{
"component": {
"type": "ml-model",
"name": "medical-triage-llm"
},

"risk": {

"category": "prompt-injection",

"analysis": {

"state": "affected",

"vector": "indirect"

},

"impact": [

"health",

"patient-safety"

],

"rating": "high"

}
}
```

A consumer or policy engine could then evaluate:

```text
if risk.category == "prompt-injection"
and risk.rating >= HIGH
and analysis.state == affected

then

priority = immediate
```

---

## Discussion

Would CycloneDX benefit from:

1. Standardized AI behavioral risk categories under `risk`?

2. AI-specific exploitability semantics within existing VEX analysis?

3. A common vocabulary for AI behavioral and safety risks that remains compatible with the CycloneDX 2.0 architecture?

I believe this direction preserves existing CycloneDX design principles while enabling machine-readable communication of AI exploitability and behavioral risk assertions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.