aws-samples / aws-samples/sample-aiml-security-assessment
Allow users to select which service assessments run (Bedrock / SageMaker / AgentCore)
- Dominant language
- HTML
- Stars
- 43
- Forks
- 20
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 4
Description
## Summary
Add a way for users to **select which service assessments run** — Amazon Bedrock, Amazon SageMaker AI, and Amazon Bedrock AgentCore — instead of always running all three.
Today, every deployment scans Bedrock, SageMaker, and AgentCore unconditionally. The FinServ GenAI module is already opt-in via the `EnableFinServAssessment` CloudFormation parameter, but the three core service assessments have no equivalent toggle.
## Motivation
- **Not every account uses every service.** An account that only uses Bedrock still incurs the SageMaker and AgentCore Lambda invocations (and the associated IAM permission surface) on every run, across every target region.
- **Scoped assessments.** Users running a targeted review of a single service (e.g. "just audit our Bedrock footprint") currently get findings for services they don't care about, cluttering the consolidated report.
- **Cost / runtime.** Skipping unused branches reduces Step Functions execution time and Lambda invocations, which matters most in multi-account, multi-region runs where the per-region `Parallel` branch fans out across many accounts.
- **Consistency.** The pattern already exists for FinServ; extending it to the core services makes the selection model uniform.
## Current behavior
In `aiml-security-assessment/statemachine/assessments.asl.json`, the per-region `Map` runs a `Parallel` state with a branch for Bedrock, SageMaker, and AgentCore — all three are unconditional. Only the FinServ branch is gated by a `Choice` state:
```
"FinServ Enabled?": {
"Type": "Choice",
"Choices": [{ "And": [
{ "Variable": "$.OriginalInput.enableFinServ", "IsPresent": true },
{ "Variable": "$.OriginalInput.enableFinServ", "StringEquals": "true" },
{ "Variable": "$.RegionIndex", "NumericEquals": 0 }
], "Next": "FinServ Security Assessment" }],
"Default": "FinServ Assessment Skipped"
}
```
The `EnableFinServAssessment` CloudFormation parameter flows through the deployment templates into the Step Functions input as `enableFinServ`. There is no analogous parameter for Bedrock / SageMaker / AgentCore.
## Proposed behavior
Let users choose any subset of `{Bedrock, SageMaker, AgentCore}` to assess. Reasonable design, mirroring the existing FinServ pattern:
1. **Deployment parameters** — add per-service enable toggles (e.g. `EnableBedrockAssessment`, `EnableSageMakerAssessment`, `EnableAgentCoreAssessment`, all defaulting to `true` to preserve current behavior). Alternatively, a single `Services` list parameter (e.g. `"bedrock,sagemaker,agentcore"`). Decide on one shape and apply it consistently.
2. **Step Functions** — wrap each of the three service branches in a `Choice` gate keyed off the corresponding input flag (same structure as `FinServ Enabled?`), routing to a `... Assessment Skipped` `Pass` state when disabled.
3. **Report layer** — the consolidated report should reflect only the selected areas. Confirm that a service being *deselected* is presented distinctly from a service being *present but N/A* (no resources / unavailable), so users aren't misled into thinking a skipped service was assessed and found clean.
## Acceptance criteria
- [ ] Users can enable/disable each of Bedrock, SageMaker, and AgentCore independently at deploy time.
- [ ] Default (no options set) preserves today's behavior: all three run.
- [ ] Disabled service branches are skipped in the Step Functions execution (no Lambda invocation, no wasted runtime).
- [ ] The consolidated HTML report clearly distinguishes "not selected" from "assessed, no findings / N/A".
- [ ] Both single-account (`template.yaml` / `aiml-security-single-account.yaml`) and multi-account (`template-multi-account.yaml` / `2-aiml-security-codebuild.yaml`) deployment paths support the new parameters.
- [ ] Docs updated: `README.md`, `docs/DEVELOPER_GUIDE.md`, and any check-count references.
## Areas likely to change
- `aiml-security-assessment/statemachine/assessments.asl.json` — add `Choice` gates for the three service branches.
- `aiml-security-assessment/template.yaml` and `template-multi-account.yaml` — new parameters + pass-through into the Step Functions input.
- `deployment/aiml-security-single-account.yaml` and `deployment/2-aiml-security-codebuild.yaml` — expose the parameters (mirror how `EnableFinServAssessment` is threaded through).
- `aiml-security-assessment/buildspec.yml` — propagate the flags if the CodeBuild path passes them into the execution input.
- Report layer (`generate_consolidated_report/`) — handle deselected areas distinctly from N/A.
## Notes / open questions
- Parameter shape: individual boolean toggles vs. a single comma-separated `Services` list — needs a decision.
- Should the FinServ toggle be folded into the same selection mechanism for a uniform UX, or left as-is?
- The Agentic AI Security lens is *synthesized* from Bedrock and AgentCore findings (`AGENTIC_BEDROCK_CHECK_MAPPINGS` / `AGENTIC_AGENTCORE_CHECK_MAPPINGS`). Disabling Bedrock and/or AgentCore will reduce the AG- coverage — decide and document how the Agentic lens behaves when its source services are deselected.
Contributor guide
Research direction
Trace EnableFinServAssessment through template.yaml, template-multi-account.yaml, deployment/aiml-security-single-account.yaml, and deployment/2-aiml-security-codebuild.yaml, then compare its flow with assessments.asl.json. Inspect generate_consolidated_report/ and buildspec.yml for report and execution-input handling. Done means independently selectable services, preserved defaults, skipped Step Functions branches, distinct report status, and updated README.md and docs/DEVELOPER_GUIDE.md.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, devops, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100