ai-cfia / ai-cfia/louis

Enable potentially abusive user detection insights from calls to Azure endpoint

Offen
#15 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
1
Forks
0
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

# Context

To comply with the Government of Canada's Digital Standards, we are implementing potentially abusive user detection insights in our applications. This initiative focuses on addressing security and privacy risks, designing ethical services, and ensuring responsible data stewardship. The detection system utilizes Azure OpenAI service to analyze user requests and identify potentially harmful content or behaviors. When flagged, these instances are summarized in a report available in the Azure OpenAI Studio, enabling proactive management and a safer user environment.

# to discuss

As of now, the requests made from our different applications don't have user information. We would need to implement changes so that calls include the user doing the prompt. We need to evaluate if this is worth considering what the metric actually adds. This is what we would get from adding user information :
![image](https://github.com/user-attachments/assets/b1d30ed2-eaa0-42e9-b0c8-8df6d068e52d)

Take note that it doesn't give the prompt to have an idea if the blocking request is a false postive.

# Actions to take

- [ ] Deploy a Azure Data Explorer Cluster to collect metrics on potentially abusive usage of models
- [ ] Connect models metrics to Data Explorer Cluster
- [ ] Update Louis deployment to potentially add user metadata to litellm calls to our models to enable user tracking of abusive usage
- [ ] Update Librechat deployment to potentially add user metadata to litellm calls to our models to enable user tracking of abusive usage
- [ ] Update Fertiscan deployment to potentially add user metadata to litellm calls to our models to enable user tracking of abusive usage
- [ ] Update Finesse deployment to potentially add user metadata to litellm calls to our models to enable user tracking of abusive usage

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.