ansible / ansible/ansible-chatbot-stack
Safety shields
- Dominant language
- Python
- Stars
- 2
- Forks
- 17
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 49
Description
## Summary
The `inline::llama-guard` safety provider is loaded at startup but **never invoked on any request**. The `registered_resources.shields` list is empty, which causes `client.shields.list()` to return nothing, making `run_shield_moderation()` a no-op on every query.
## Evidence
**1. No shields registered in the configmap**
```yaml
registered_resources:
shields: [] # <-- empty
```
The provider and default shield ID are declared, but without a shield entry here, nothing references the provider:
```yaml
providers:
safety:
- provider_id: llama-guard
provider_type: inline::llama-guard
config:
excluded_categories: []
registered_resources:
safety:
- default_shield_id: llama-guard
```
**2. Application-layer moderation silently passes**
`streaming_query.py` calls `run_shield_moderation()` on every request. This function discovers shields via `client.shields.list()`. With an empty list, the moderation loop never executes and returns `ShieldModerationPassed()`:
```
streaming_query.py -> retrieve_response_generator() -> run_shield_moderation()
-> client.shields.list() # returns []
-> shields_to_run = list(all_shields) # empty
-> for shield in shields_to_run: ... # never executes
-> return ShieldModerationPassed()
```
**3. No shield activity in pod logs**
Logs show only readiness probes and `POST /v1/streaming_query` requests. Zero entries for shield execution, safety violations, or guard invocations.
**4. Observed on:** prod2-west cluster, namespace `wisdom-ls-stack`, deployment `lm-stack-server` (image `ansible-chatbot-stack:0.9.202603241302`)
## How to Enable
The `inline::llama-guard` provider is a prompt-based safety classifier. It wraps user input in a structured prompt with safety categories (S1-S14), sends it to any inference model via `inference_api.chat_completion()`, and parses the response for `safe` or `unsafe\nS1,S2,...`. It does **not** require an actual Llama Guard model — any LLM that can follow the prompt format works.
### Option 1: Use the existing main inference model (all providers)
The simplest option — point the shield at whichever inference model is already registered. This works for any provider (Vertex AI, vLLM, OpenAI) since the shield just calls `inference_api.chat_completion()` with the given model ID.
```yaml
registered_resources:
shields:
- shield_id: llama-guard
provider_id: llama-guard
provider_resource_id:
```
For example, if the main model is registered as `vertexai/gemini-2.0-flash` or `my_rhoai_dev/granite-3.1`, just use that same ID. No additional model registration needed.
**Tradeoff:** Every request makes two LLM calls to the same model (one for safety, one for the actual response), which adds latency.
### Option 2: Register a dedicated smaller/faster guardrail model (recommended)
Use a lighter model for the safety check to minimize latency overhead. The guardrail model only needs to classify content, not generate full responses.
#### Vertex AI
```yaml
registered_resources:
models:
# ADD: dedicated guardrail model
- metadata: {}
model_id: vertexai/gemini-2.0-flash
provider_id: vertexai
provider_model_id: gemini-2.0-flash
shields:
- shield_id: llama-guard
provider_id: llama-guard
provider_resource_id: vertexai/gemini-2.0-flash
```
#### OpenAI
```yaml
registered_resources:
models:
- metadata: {}
model_id: openai/gpt-4o-mini
provider_id: openai
provider_model_id: gpt-4o-mini
shields:
- shield_id: llama-guard
provider_id: llama-guard
provider_resource_id: openai/gpt-4o-mini
```
### Option 3: Deploy a dedicated Llama Guard model (vLLM)
Serve `meta-llama/Llama-Guard-4-12B` (or `3-8B` for lower resource usage) on a separate vLLM endpoint, register it as an additional inference provider, and point the shield to it. This gives the best classification accuracy since the model is purpose-built for this prompt format.
### Verification
After updating the configmap and restarting:
1. Check startup logs for `"Available shields: ['llama-guard']"` instead of `"No available shields. Disabling safety"`
2. Hit the `GET /shields` endpoint to confirm the shield is listed
3. Send a test query with harmful content and verify it gets blocked
## Notes
- The guardrail model receives a structured Llama Guard prompt and must respond with `safe` or `unsafe\nS1,S2,...`. Models not fine-tuned for this format may need testing for consistency.
- `excluded_categories: []` means all 14 categories are active (S1 Violent Crimes through S14 Code Interpreter Abuse).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the configmap's registered_resources.shields entry and trace the streaming_query.py path through retrieve_response_generator() to run_shield_moderation(). Register the llama-guard shield with an available model, then restart and verify the startup log reports it, GET /shields lists it, and harmful test input is blocked.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, security
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100