Message Alerts and Admin Workflow for Sensitive Content

@paullizer is already working on this.

Since Aug 13, 2025.

Add an alert mechanism that monitors user and AI messages for configured keywords, regex, and/or concepts. When a rule matches, alert admins and optionally auto-trigger a workflow (archive/delete/redact and/or inform the user). This feature will integrate with a new notification system (in-app and external) we plan to build.

Problem
We lack proactive monitoring of sensitive content across conversations. Admins can’t be alerted or take consistent actions (e.g., archive/delete) when content policy rules are violated.

Goals
- Detect rule-based matches in user and AI messages (keywords, regex, categories).
- Create an Alert record with context for admin review.
- Notify admins via a pluggable channel (in-app first, external later).
- Allow manual and optional auto-actions: archive, delete, redact, and/or notify user.
- Provide an admin UI to review alerts, filter, take action, and audit outcomes.

Non-Goals
- Full DLP or enterprise eDiscovery scope.
- Guaranteed zero-false-positives semantic detection in v1.
- Complex role-based approval workflows (v1 is single-step action).

Scope
- Message monitoring for content in messages produced by users and the AI assistant.
- Keyword/regex rule engine; optional Azure AI Content Safety categories when enabled.
- Alert persistence, admin review UI, actions, and auditing.
- In-app notifications for admins (MVP), with extension points for email/Teams later.

Proposed Design

Detection
- Rule types:
- Keyword and regex lists (case-sensitive/insensitive, word-boundary options).
- Category-based using Azure AI Content Safety if available (`CLIENTS["content_safety_client"]`).
- Rule storage:
- Global rules in `settings` (Cosmos `settings` container), e.g., `settings.alert_rules` list.
- Example rule (MVP):
```json
{
"id": "rule-pii-1",
"enabled": true,
"match": {
"type": "regex",
"pattern": "\\b(SSN|Social Security|\\d{3}-\\d{2}-\\d{4})\\b",
"flags": ["i"]
},
"scope": ["user_messages", "ai_messages"],
"severity": "high",
"autoActions": ["archive_conversation"],
"notifyUser": true
}
```
- Trigger point:
- On message creation (user or AI), enqueue a background scan via `flask_executor` to avoid request latency.
- Ensure idempotency with a per-message “scanned” marker.

System Flow
1. Message created -> enqueue `scan_message(message_id, conversation_id, author_type)`.
2. Scanner loads rules and optionally calls Content Safety.
3. On match -> write an Alert record (Cosmos).
4. Notify admins (in-app notifications MVP; plug-in adapters for Teams/email later).
5. If rule has autoActions -> execute (archive/delete/redact) and optionally inform the user.
6. Record an audit entry for all actions (who, what, when, why).

Data Model (Cosmos)
- Container: alerts (new) or reuse existing `safety` container.
- Alert document:
```json
{
"id": "",
"type": "alert",
"conversation_id": "",
"message_id": "",
"actor": {"id": "", "kind": "user|ai"},
"matched": {
"rule_id": "rule-pii-1",
"severity": "high",
"keywords": ["SSN"],
"categories": ["Hate", "Violence"],
"evidence": [{"start": 10, "end": 20, "snippet": "..." }]
},
"status": "open|actioned|closed",
"created_at": "",
"updated_at": "",
"actions": [
{
"kind": "archive_conversation|delete_message|redact_message|notify_user",
"actor": "",
"at": "",
"notes": "optional"
}
],
"notes": ""
}
```
- Container: notifications (new; in-app MVP)
- Notification document:
```json
{
"id": "",
"type": "notification",
"audience": {"role": "admin"},
"title": "Alert: PII detected",
"body": "Conversation matched rule-pii-1",
"resource": {"kind": "alert", "id": ""},
"status": "unread|read",
"created_at": ""
}
```
- Rules config stored in `settings` container:
- `settings.id = "global"`, `settings.alert_rules = [...]`, `settings.notification_channels = {...}`.

Admin UI (MVP)
- Alerts list: filter by severity, status, created_at, rule_id.
- Alert details: matched snippets, rule, conversation/message links, actions.
- Actions:
- Archive conversation: move to `archived_conversations`/`archived_messages` (containers already exist).
- Delete message: remove from `messages`; log action in alert.
- Redact message: replace matched spans; preserve original in audit trail.
- Notify user: add a system message and/or email (future).
- Settings page: manage rules (create/edit/enable/disable), test-rule utility, toggle Content Safety usage.

Notifications
- MVP: In-app notifications for admins (polling or websocket-lite later).
- Extensibility:
- Email via Azure Communication Services (future).
- Teams via Incoming Webhook or Graph API (future).
- Respect existing environment toggles; no hard dependency on external channels for MVP.

Permissions, Auditing, and Observability
- Admin-only access to alerts and actions (align with existing group/role model).
- Every action is appended to `alert.actions` with actor and timestamp.
- Metrics: counts by rule/severity/status, time-to-action, false-positive flag.
- Logs with correlation IDs (conversation_id/message_id/alert_id).

Performance & Reliability
- All scanning async via `Executor`; retry with backoff; mark scan result to prevent rework.
- Rate-limit Content Safety calls and handle timeouts gracefully.
- Idempotent actions (especially archive/delete).

Security & Privacy
- Avoid storing full sensitive payloads in alerts; include minimal snippets only when necessary.
- Encrypt PII at rest if needed; respect data residency per Azure environment config.
- Admin access only; audit reads of alert details if required.

Rollout
- Phase 1: Detection + Alert persistence + In-app admin notifications + Manual actions.
- Phase 2: Auto-actions per rule + user notifications.
- Phase 3: External channels (email/Teams), semantic concepts, advanced analytics.

Risks & Mitigations
- False positives: provide test-rule tool and soft actions (redact before delete).
- Latency: run async; cap cost on external API calls.
- Admin overload: severity-based routing and daily digest option (future).

Open Questions
- Do we reuse the existing `safety` container or create a dedicated `alerts` container?
- Preferred external notification channel priority (Teams vs email)?
- Should user notifications be a system message, email, or both?
- Redaction policy defaults (replace with [REDACTED], or mask pattern?).

Acceptance Criteria
- Rules can be created, enabled/disabled, and tested in settings.
- New messages are scanned asynchronously; matches create Alert records.
- Admins see in-app notifications for new Alerts.
- Admins can archive conversation, delete or redact message, and notify user.
- All admin actions are audited in the Alert record.
- Feature is gated via settings; no regressions when disabled.

Implementation Checklist
- [ ] Data models: create `alerts` container (or confirm reuse of `safety`).
- [ ] Settings: extend global settings with `alert_rules`, `notification_channels`, feature flag.
- [ ] Scanner: background job to evaluate rules (+ optional Content Safety).
- [ ] Actions service: archive/delete/redact/notify with idempotency + audits.
- [ ] Notifications: in-app MVP container + UI badge + list/detail view.
- [ ] Admin UI: Alerts list, detail, actions; Settings for rules.
- [ ] Telemetry: structured logs and basic metrics.
- [ ] Tests: unit tests for rule matching, actions, and idempotency; E2E happy path + FP case.
- [ ] Docs: admin guide, rule examples, ops notes.