kestra-io / kestra-io/plugin-aws
Add GuardDuty tasks for AWS Plugin
- Dominant language
- Java
- Stars
- 22
- Forks
- 30
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 10
Description
## Summary
Amazon GuardDuty is AWS's managed threat-detection service: it continuously analyses CloudTrail, VPC Flow Logs, DNS logs and EKS/S3/RDS telemetry and emits **findings** (compromised credentials, crypto-mining, unusual API calls, …). This sub-plugin lets a Kestra flow list, fetch, archive and rate GuardDuty findings, and react to new findings with a polling trigger — so triage, enrichment and remediation steps (isolate an EC2 instance, rotate a key, open a ticket, notify Slack) can be orchestrated in the same flow as the rest of the AWS estate.
## Motivation
- Today, teams wire GuardDuty to EventBridge → Lambda → custom glue code, or poll `aws guardduty list-findings` from a shell task and parse JSON by hand. Neither gives them retries, observability or a declarative audit trail.
- Security engineers and platform teams already running Kestra for AWS operations (S3, Lambda, CloudWatch, EventBridge tasks) get a native way to close the loop from *detection* to *response* without leaving the flow.
- It complements `io.kestra.plugin.aws.cloudwatch` (metric-driven alerting) and `io.kestra.plugin.aws.eventbridge` (event fan-out): GuardDuty findings become first-class inputs for remediation flows.
## Context
Part of the security integrations initiative alongside the existing CrowdStrike, Aikido and Snyk plugins.
Reference implementation inside this repository: the `io.kestra.plugin.aws.cloudwatch` sub-package — `AbstractCloudWatch` (abstract connection base building the SDK client through `ConnectionUtils.configureSyncClient(...)`), `Query` (task) and `Trigger` (polling trigger implementing `AbstractConnectionInterface`). `io.kestra.plugin.aws.sns.AbstractSns` shows the same pattern with a minimal surface.
## API Reference
- **Official docs**: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_Operations.html
- **Authentication**: standard AWS credentials — static access key / secret key / session token, STS AssumeRole, or the default credential provider chain — all already handled by `io.kestra.plugin.aws.AbstractConnection` / `ConnectionUtils`. Nothing new to implement.
- **Base URL pattern**: regional endpoint `https://guardduty..amazonaws.com` (resolved by the SDK; `endpointOverride` remains supported for LocalStack-style testing).
- **SDK / client library**: `software.amazon.awssdk:guardduty` (AWS SDK for Java v2) — `GuardDutyClient`, https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/services/guardduty/GuardDutyClient.html
Key operations:
| Operation | Purpose |
|---|---|
| `ListDetectors` | Resolve the detector ID of the account/region (one detector per region) |
| `ListFindings` | Page through finding IDs matching a `FindingCriteria` (severity, type, `service.archived`, `updatedAt`, …), sorted by `updatedAt` |
| `GetFindings` | Hydrate up to 50 finding IDs into full finding objects |
| `ArchiveFindings` / `UnarchiveFindings` | Move findings out of / back into the active queue |
| `UpdateFindingsFeedback` | Mark findings `USEFUL` / `NOT_USEFUL` with a comment |
| `CreateSampleFindings` | Generate sample findings — useful for tests and demos |
## Gradle Dependencies
Add to `build.gradle`, next to the other `software.amazon.awssdk` entries (the version is managed by Kestra's platform BOM `io.kestra:platform`, exactly like `cloudwatch`, `sns`, `sqs`, … — do **not** pin a version):
```groovy
// AWS GuardDuty
api 'software.amazon.awssdk:guardduty'
```
> Use the latest stable version available on Maven Central / jcenter.
>
> **Kestra framework inclusions (do not add as dependencies):**
> Kestra's internal HTTP client (`io.kestra.core.http.client`) and Jackson serializers
> are provided by the framework. Only list an explicit HTTP dependency when an official
> SDK (e.g. AWS SDK, GCP client libraries) is required — never list OkHttp, Apache
> HttpComponents, or `java.net.http` wrappers.
## Plugin Structure
- **Repository**: `plugin-aws` (existing)
- **Namespace**: `io.kestra.plugin.aws.guardduty`
- **Sub-plugins**: `guardduty` (single new sub-package, flat — tasks directly under it, no `tasks` package)
- **Categories**: `CLOUD, INFRASTRUCTURE`
*(Each sub-package may have its own `@PluginSubGroup(category = …)` if the categories differ across sub-plugins.)*
> **Task class naming**: task class names must **not** repeat the plugin or package name as
> a prefix or suffix. The fully-qualified type already carries the namespace context.
> Use concise action or resource names: `List`, `Create`, `Get`, `Delete`, `Trigger` — not
> `PhpIpamList`, `ListPhpIpam`, `SubnetsListPhpipam`, etc.
Proposed classes (findings are the primary resource of the sub-package, so the bare verbs refer to findings):
| Class | Kind | SDK call(s) |
|---|---|---|
| `AbstractGuardDuty` | abstract base extends `AbstractConnection` | exposes `GuardDutyClient client(RunContext)` via `ConnectionUtils.configureSyncClient(clientConfig, GuardDutyClient.builder())` |
| `ListDetectors` | task | `ListDetectors` |
| `List` | task | `ListFindings` (+ `GetFindings` in batches of 50 when `hydrate: true`), `fetchType` FETCH / FETCH_ONE / STORE |
| `Get` | task | `GetFindings` for an explicit list of IDs |
| `Archive` | task | `ArchiveFindings` / `UnarchiveFindings` (single `action` enum: `ARCHIVE` / `UNARCHIVE`) |
| `UpdateFeedback` | task | `UpdateFindingsFeedback` |
| `Trigger` | polling trigger, `AbstractTrigger implements PollingTriggerInterface, AbstractConnectionInterface` | `ListFindings` with `updatedAt > watermark`, hydrated through `GetFindings` |
Connection properties (`region`, `accessKeyId`, `secretKeyId`, `sessionToken`, `stsRoleArn`, `endpointOverride`, …) come from `AbstractConnection` and are redeclared on the trigger exactly as `cloudwatch.Trigger` does today.
`detectorId` should be **optional** everywhere: when absent, resolve it once with `ListDetectors` (there is a single detector per account/region) and log which one was picked.
## Suggested Tasks
1. Create `AbstractGuardDuty extends AbstractConnection` with a `client(RunContext)` factory mirroring `AbstractCloudWatch`
2. `ListDetectors` — return the detector IDs of the account/region
3. `List` — findings: `Property> criteria` (mapped to `FindingCriteria`), `Property minSeverity` (bounded 0–8, `@Min/@Max`), `Property includeArchived` (default `false`), `Property maxResults` (bounded, default 50), `Property hydrate` (default `true`), `Property fetchType` (default `STORE` — findings are large JSON documents); outputs `ids`, `findings`/`uri`, `count`
4. `Get` — findings by ID (`Property> findingIds`, max 50 per SDK call, chunk internally)
5. `Archive` — archive / unarchive a list of finding IDs
6. `UpdateFeedback` — `Property feedback` (`USEFUL` / `NOT_USEFUL`), `Property comment`
7. `Trigger` — polling trigger firing on new or updated findings: persist the last seen `updatedAt` (epoch millis) in the namespace KV store (`guardduty_watermark___`), advance it on **every** poll, filter with `minSeverity` / `findingTypes` / `includeArchived`; outputs `findings` (list) and `count`
8. Write unit + integration tests — use LocalStack via Testcontainers (GuardDuty is covered by LocalStack Pro only; fall back to WireMock stubs of the JSON protocol via `endpointOverride`, and gate any live-AWS test behind `.github/setup-unit.sh`)
9. Add `package-info.java` with `@PluginSubGroup(title = "GuardDuty", categories = { PluginSubGroup.PluginCategory.CLOUD, PluginSubGroup.PluginCategory.INFRASTRUCTURE })`
10. Add `src/main/resources/metadata/guardduty.yaml` (`group: io.kestra.plugin.aws.guardduty`, same schema as `cloudwatch.yaml`) and `src/main/resources/icons/io.kestra.plugin.aws.guardduty.svg` (reuse the official GuardDuty architecture icon — do not draw one)
11. Add YAML examples (`full = true`) on every task/trigger, update `AGENTS.md` and `src/main/resources/doc/io.kestra.plugin.aws.md`
## YAML Examples
### Example 1 — List high-severity active findings and store them
```yaml
id: guardduty_list_high_findings
namespace: company.security
inputs:
- id: region
type: STRING
defaults: eu-west-1
tasks:
- id: list_findings
type: io.kestra.plugin.aws.guardduty.List
region: "{{ inputs.region }}"
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
minSeverity: 7
includeArchived: false
fetchType: STORE
- id: log_count
type: io.kestra.plugin.core.log.Log
message: "{{ outputs.list_findings.count }} high-severity GuardDuty findings stored at {{ outputs.list_findings.uri }}"
```
### Example 2 — Rate a finding as useful, then archive it
```yaml
id: guardduty_triage_finding
namespace: company.security
inputs:
- id: findingId
type: STRING
tasks:
- id: get_finding
type: io.kestra.plugin.aws.guardduty.Get
region: eu-west-1
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
findingIds:
- "{{ inputs.findingId }}"
- id: feedback
type: io.kestra.plugin.aws.guardduty.UpdateFeedback
region: eu-west-1
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
findingIds:
- "{{ inputs.findingId }}"
feedback: USEFUL
comment: "Confirmed by SOC, remediation flow {{ execution.id }}"
- id: archive
type: io.kestra.plugin.aws.guardduty.Archive
region: eu-west-1
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
action: ARCHIVE
findingIds:
- "{{ inputs.findingId }}"
```
### Example 3 — React to every new critical finding
```yaml
id: guardduty_on_new_finding
namespace: company.security
triggers:
- id: on_finding
type: io.kestra.plugin.aws.guardduty.Trigger
interval: PT2M
region: eu-west-1
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
minSeverity: 8
findingTypes:
- "UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWS"
- "CryptoCurrency:EC2/BitcoinTool.B!DNS"
tasks:
- id: notify
type: io.kestra.plugin.core.log.Log
message: "{{ trigger.count }} new GuardDuty finding(s): {{ trigger.findings | jq('[.[].title]') }}"
```
## Acceptance Criteria
### Functional
- [ ] Authentication task / abstract base class implemented (`AbstractGuardDuty` reusing `AbstractConnection` + `ConnectionUtils`)
- [ ] Core CRUD tasks for each resource group (`ListDetectors`, `List`, `Get`, `Archive`, `UpdateFeedback`)
- [ ] At least one polling trigger (`Trigger`, watermark on `updatedAt` persisted in namespace KV, advanced on every poll)
- [ ] Unit + integration tests pass (`./gradlew test`)
- [ ] Build passes with `./gradlew build`
### Kestra Plugin Coding Standards
- [ ] HTTP calls use Kestra's internal HTTP client (`io.kestra.core.http.client`) — no OkHttp, Apache HttpClient, or similar (the AWS SDK is the only external client, as for every other `plugin-aws` sub-package)
- [ ] All new properties use `Property` — no legacy `@PluginProperty(dynamic = true)` on new code
- [ ] Secret/credential properties annotated with `@PluginProperty(secret = true)` (inherited from `AbstractConnection`; redeclared copies on the trigger keep `secret = true` + `@ToString.Exclude`)
- [ ] Every property and output has a `@Schema` annotation
- [ ] Task classes carry the five mandatory Lombok annotations (`@SuperBuilder`, `@ToString`, `@EqualsAndHashCode`, `@Getter`, `@NoArgsConstructor`)
- [ ] Logging via `runContext.logger()` only
- [ ] JSON serialization uses Jackson mappers from `io.kestra.core.serializers`
- [ ] All `Property` fields support Kestra expression language (template rendering)
### Documentation & Structure
- [ ] `@Plugin(examples = ...)` entries each set `full = true` with a complete runnable flow (id + namespace + tasks/triggers)
- [ ] Sensitive values in examples use `{{ secret('SECRET_NAME') }}`
- [ ] `package-info.java` per sub-package with `@PluginSubGroup(category = PluginSubGroup.PluginCategory.)` matching the categories above (CLOUD, INFRASTRUCTURE)
- [ ] `metadata/guardduty.yaml`, `index.yaml` updated, and sub-group icon SVG present
---
## Repository Setup Checklist
> The repository already exists — the Scaffold step is intentionally omitted.
### 2. Add to Sanity check page
Add the new `guardduty` sub-plugin to the [Sanity check Notion page](https://www.notion.so/kestra-io/32736907f7b580cbb00dc7c061e624b1?v=32736907f7b58002ac2b000ccc63d8a2).
### 3. Run scoped Terraform apply
Run the following from `infra/terraform/github`:
```bash
terraform apply \
-target='github_repository.repo["plugin-aws"]' \
-target='github_issue_labels.plugins["plugin-aws"]' \
-target='github_repository_ruleset.branch["plugin-aws"]'
```
## Notes for the developer
- `FindingCriteria` is a nested map of `{ "": { "eq": [...], "gte": , ... } }`. Model it as `Property>` and translate to the SDK `Condition` builder — do not expose the SDK types as properties.
- GuardDuty severity is a decimal 0–8.9; `minSeverity` maps to `severity.gte`. Document the mapping in `@Schema`.
- `GetFindings` accepts at most 50 IDs per call — chunk in `List`/`Get`/`Trigger`.
- LocalStack Community does not ship GuardDuty; if LocalStack Pro is not available in CI, stub the JSON protocol with WireMock and point `endpointOverride` at it.
---
*[View as Artifact](https://claude.ai/code/artifact/4026c5d2-5ec5-4741-b85c-711d2a48bcdb)*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the existing io.kestra.plugin.aws.cloudwatch sub-package, especially AbstractCloudWatch, Query, Trigger, and cloudwatch.yaml, then inspect AbstractSns and build.gradle for the shared connection and dependency patterns. Implement the proposed GuardDuty tasks, trigger, metadata, documentation, examples, and tests under the listed paths. Done means ./gradlew test and ./gradlew build pass and the functional and coding-standard acceptance criteria are met.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- cloud, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100