aws-samples / aws-samples/sample-aiml-security-assessment

Allow users to select which service assessments run (Bedrock / SageMaker / AgentCore)

Open
#47 0 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
HTML
Stars
43
Forks
20
Avg merge
2d 23h
Merged PRs (30d)
4

Description

## Summary

Add a way for users to **select which service assessments run** — Amazon Bedrock, Amazon SageMaker AI, and Amazon Bedrock AgentCore — instead of always running all three.

Today, every deployment scans Bedrock, SageMaker, and AgentCore unconditionally. The FinServ GenAI module is already opt-in via the `EnableFinServAssessment` CloudFormation parameter, but the three core service assessments have no equivalent toggle.

## Motivation

- **Not every account uses every service.** An account that only uses Bedrock still incurs the SageMaker and AgentCore Lambda invocations (and the associated IAM permission surface) on every run, across every target region.
- **Scoped assessments.** Users running a targeted review of a single service (e.g. "just audit our Bedrock footprint") currently get findings for services they don't care about, cluttering the consolidated report.
- **Cost / runtime.** Skipping unused branches reduces Step Functions execution time and Lambda invocations, which matters most in multi-account, multi-region runs where the per-region `Parallel` branch fans out across many accounts.
- **Consistency.** The pattern already exists for FinServ; extending it to the core services makes the selection model uniform.

## Current behavior

In `aiml-security-assessment/statemachine/assessments.asl.json`, the per-region `Map` runs a `Parallel` state with a branch for Bedrock, SageMaker, and AgentCore — all three are unconditional. Only the FinServ branch is gated by a `Choice` state:

```
"FinServ Enabled?": {
"Type": "Choice",
"Choices": [{ "And": [
{ "Variable": "$.OriginalInput.enableFinServ", "IsPresent": true },
{ "Variable": "$.OriginalInput.enableFinServ", "StringEquals": "true" },
{ "Variable": "$.RegionIndex", "NumericEquals": 0 }
], "Next": "FinServ Security Assessment" }],
"Default": "FinServ Assessment Skipped"
}
```

The `EnableFinServAssessment` CloudFormation parameter flows through the deployment templates into the Step Functions input as `enableFinServ`. There is no analogous parameter for Bedrock / SageMaker / AgentCore.

## Proposed behavior

Let users choose any subset of `{Bedrock, SageMaker, AgentCore}` to assess. Reasonable design, mirroring the existing FinServ pattern:

1. **Deployment parameters** — add per-service enable toggles (e.g. `EnableBedrockAssessment`, `EnableSageMakerAssessment`, `EnableAgentCoreAssessment`, all defaulting to `true` to preserve current behavior). Alternatively, a single `Services` list parameter (e.g. `"bedrock,sagemaker,agentcore"`). Decide on one shape and apply it consistently.
2. **Step Functions** — wrap each of the three service branches in a `Choice` gate keyed off the corresponding input flag (same structure as `FinServ Enabled?`), routing to a `... Assessment Skipped` `Pass` state when disabled.
3. **Report layer** — the consolidated report should reflect only the selected areas. Confirm that a service being *deselected* is presented distinctly from a service being *present but N/A* (no resources / unavailable), so users aren't misled into thinking a skipped service was assessed and found clean.

## Acceptance criteria

- [ ] Users can enable/disable each of Bedrock, SageMaker, and AgentCore independently at deploy time.
- [ ] Default (no options set) preserves today's behavior: all three run.
- [ ] Disabled service branches are skipped in the Step Functions execution (no Lambda invocation, no wasted runtime).
- [ ] The consolidated HTML report clearly distinguishes "not selected" from "assessed, no findings / N/A".
- [ ] Both single-account (`template.yaml` / `aiml-security-single-account.yaml`) and multi-account (`template-multi-account.yaml` / `2-aiml-security-codebuild.yaml`) deployment paths support the new parameters.
- [ ] Docs updated: `README.md`, `docs/DEVELOPER_GUIDE.md`, and any check-count references.

## Areas likely to change

- `aiml-security-assessment/statemachine/assessments.asl.json` — add `Choice` gates for the three service branches.
- `aiml-security-assessment/template.yaml` and `template-multi-account.yaml` — new parameters + pass-through into the Step Functions input.
- `deployment/aiml-security-single-account.yaml` and `deployment/2-aiml-security-codebuild.yaml` — expose the parameters (mirror how `EnableFinServAssessment` is threaded through).
- `aiml-security-assessment/buildspec.yml` — propagate the flags if the CodeBuild path passes them into the execution input.
- Report layer (`generate_consolidated_report/`) — handle deselected areas distinctly from N/A.

## Notes / open questions

- Parameter shape: individual boolean toggles vs. a single comma-separated `Services` list — needs a decision.
- Should the FinServ toggle be folded into the same selection mechanism for a uniform UX, or left as-is?
- The Agentic AI Security lens is *synthesized* from Bedrock and AgentCore findings (`AGENTIC_BEDROCK_CHECK_MAPPINGS` / `AGENTIC_AGENTCORE_CHECK_MAPPINGS`). Disabling Bedrock and/or AgentCore will reduce the AG- coverage — decide and document how the Agentic lens behaves when its source services are deselected.

Contributor guide

Open the contributing guide

Research direction

Trace EnableFinServAssessment through template.yaml, template-multi-account.yaml, deployment/aiml-security-single-account.yaml, and deployment/2-aiml-security-codebuild.yaml, then compare its flow with assessments.asl.json. Inspect generate_consolidated_report/ and buildspec.yml for report and execution-input handling. Done means independently selectable services, preserved defaults, skipped Step Functions branches, distinct report status, and updated README.md and docs/DEVELOPER_GUIDE.md.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, devops, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.