elastic / elastic/detection-rules

[New Rule] New LLM Jacking Rules

Open
#6,154 0 comments 0 reactions 2 assignees Assigned to @Mikaayenson View on GitHub
community Rule: New Team: TRADE
Dominant language
Python
Stars
2.7k
Forks
696
Avg merge
4d 17h
Merged PRs (30d)
87

Description

### Description

Hello, I'd like to suggest some rules to detect LLMJacking based on the research shared by https://permiso.io/blog/exploiting-hosted-models and https://www.sysdig.com/blog/llmjacking-stolen-cloud-credentials-used-in-new-ai-attack. These detections are based on CloudTrail logs.

I did create a pull request to sigma https://github.com/SigmaHQ/sigma/pull/5903 showing the logic of two possible rules:
- AWS Bedrock Model Recon Activity => this query detects attempts done by attackers to detect model availability
`event.action:"InvokeModel" and event.provider:"bedrock.amazonaws.com" and event.dataset:"aws.cloudtrail" and aws.cloudtrail.user_identity.access_key_id:AKIA* and aws.cloudtrail.error_code:"ValidationException"`

It could work also with a more generalized `event.outcome:"failure"` as seen in the research by Permiso
Image the error code is not always the same.

Image
- Uncommon AWS Bedrock API Calls From Long Lived Access Keys => this query detects attempts to call uncommon APIs from AKIA keys.

`(event.action:"CreateFoundationModelAgreement" or event.action:"GetFoundationModelAvailability" or event.action:"GetUseCaseForModelAccess" or event.action:"PutFoundationModelEntitlement" or event.action:"PutUseCaseForModelAccess") and event.provider:"bedrock.amazonaws.com" and event.dataset:"aws.cloudtrail" and aws.cloudtrail.user_identity.access_key_id:AKIA*`

I'd like to extend these rules with some more rules based on the research by Permiso:
```
AKIA* with Mozilla UA: A long lived access key with a user-agent that includes Mozilla rarely occurs legitimately.

AKIA* with no UA touching Bedrock: A long lived access key with no user-agent touching the Bedrock service rarely occurs legitimately.
```
I'd create a rule: Uncommon User Agent Interacting with Bedrock From Long Lived Access Keys
`event.provider:"bedrock.amazonaws.com" and event.dataset:"aws.cloudtrail" and aws.cloudtrail.user_identity.access_key_id:AKIA* and (user_agent.original:Mozilla* or user_agent.original:"")`

Then I'd like to propose a new Threat Hunting rule to monitor high interactions from AKIA keys with InvokeModel and InvokeModelWithResponseStream.
```
FROM logs-*
| WHERE event.action IN ("InvokeModel", "InvokeModelWithResponseStream")
| WHERE STARTS_WITH(aws.cloudtrail.user_identity.access_key_id, "AKIA")
| STATS count = COUNT(*) BY aws.cloudtrail.user_identity.access_key_id
| SORT count DESC
```
ECS Version:

Image

### Target Ruleset

aws

### Target Rule Type

None

### Tested ECS Version

8.11.0

### Query

Standard Rules:
1. `event.action:"InvokeModel" and event.provider:"bedrock.amazonaws.com" and event.dataset:"aws.cloudtrail" and aws.cloudtrail.user_identity.access_key_id:AKIA* and aws.cloudtrail.error_code:"ValidationException"`
2. `(event.action:"CreateFoundationModelAgreement" or event.action:"GetFoundationModelAvailability" or event.action:"GetUseCaseForModelAccess" or event.action:"PutFoundationModelEntitlement" or event.action:"PutUseCaseForModelAccess") and event.provider:"bedrock.amazonaws.com" and event.dataset:"aws.cloudtrail" and aws.cloudtrail.user_identity.access_key_id:AKIA*`
3. `event.provider:"bedrock.amazonaws.com" and event.dataset:"aws.cloudtrail" and aws.cloudtrail.user_identity.access_key_id:AKIA* and (user_agent.original:Mozilla* or user_agent.original:"")`

Threat Hunting Rule:
```
FROM logs-*
| WHERE event.action IN ("InvokeModel", "InvokeModelWithResponseStream")
| WHERE STARTS_WITH(aws.cloudtrail.user_identity.access_key_id, "AKIA")
| STATS count = COUNT(*) BY aws.cloudtrail.user_identity.access_key_id
| SORT count DESC
```
The description of this issue has the description for each rule.

### New fields required in ECS/data sources for this rule?

_No response_

### Related issues or PRs

_No response_

### References

https://permiso.io/blog/exploiting-hosted-models and https://www.sysdig.com/blog/llmjacking-stolen-cloud-credentials-used-in-new-ai-attack

### Redacted Example Data

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.