aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap

[Resource Type] - [BUG] - Bedrock Prompt Management: Only allowing 1 prompt variant per prompt

Open
#2,330 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
No language data
Stars
1.1k
Forks
62
PR merge metrics
No merged PRs in 30d

Description

### Name of the resource

Other

### Resource Name

AWS::Bedrock::Prompt

### Issue Description

Describe the bug
Trying to deploy a prompt with 2 variants and getting an error that it only expected 1 variant.

The original issue is reported here - https://github.com/awslabs/generative-ai-cdk-constructs/issues/988

### Expected Behavior

It should allow a maximum of 3 variants.

https://aws.amazon.com/bedrock/prompt-management/#:~:text=Create%20up%20to%20three%20prompt,and%20department%20for%20enterprise%20management.

### Observed Behavior

Getting the following error:

❌ PromptManagerStack failed: Error: The stack named PromptManagerStack failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Properties validation failed for resource flightParametersPromptE31C7D46 with message:
[#/Variants: expected maximum item count: 1, found: 2]

### Test Cases

`AWSTemplateFormatVersion: '2010-09-09'
Description: |
CloudFormation test case demonstrating the variant limitation issue in AWS::Bedrock::Prompt.
According to AWS documentation, prompts should support up to 3 variants, but CloudFormation
validation currently fails with "expected maximum item count: 1, found: 2" when using 2 variants.

This template is designed to FAIL deployment to demonstrate the bug for AWS Support ticket.

Parameters:
KMSKeyId:
Type: String
Description: KMS Key ID for prompt encryption (optional)
Default: ""

Conditions:
HasKMSKey: !Not [!Equals [!Ref KMSKeyId, ""]]

Resources:
# KMS Key for prompt encryption (if not provided)
PromptEncryptionKey:
Type: AWS::KMS::Key
Condition: !Not [!Condition HasKMSKey]
Properties:
Description: KMS key for Bedrock prompt encryption - test case
KeyPolicy:
Statement:
- Sid: Enable IAM User Permissions
Effect: Allow
Principal:
AWS: !Sub 'arn:aws:iam::${AWS::AccountId}:root'
Action: 'kms:*'
Resource: '*'
- Sid: Allow Bedrock Service
Effect: Allow
Principal:
Service: bedrock.amazonaws.com
Action:
- kms:Decrypt
- kms:GenerateDataKey
Resource: '*'

PromptEncryptionKeyAlias:
Type: AWS::KMS::Alias
Condition: !Not [!Condition HasKMSKey]
Properties:
AliasName: alias/bedrock-prompt-test-key
TargetKeyId: !Ref PromptEncryptionKey

# This prompt resource will FAIL deployment due to CloudFormation validation error
# Error: "#/Variants: expected maximum item count: 1, found: 2"
TestPromptWithTwoVariants:
Type: AWS::Bedrock::Prompt
Properties:
Name: cloudformation-variant-test-prompt
Description: Test prompt with 2 variants - should work according to AWS docs but fails in CloudFormation
CustomerEncryptionKeyArn: !If
- HasKMSKey
- !Sub 'arn:aws:kms:${AWS::Region}:${AWS::AccountId}:key/${KMSKeyId}'
- !GetAtt PromptEncryptionKey.Arn
DefaultVariant: variant-1
Tags:
Purpose: CloudFormationBugReport
Issue: VariantLimitationBug
ExpectedBehavior: ShouldSupport3Variants
ActualBehavior: OnlySupports1Variant
Variants:
# Variant 1 - This should work
- Name: variant-1
ModelId: anthropic.claude-3-5-sonnet-20241022-v2:0
TemplateType: TEXT
TemplateConfiguration:
Text:
Text: "This is the first variant. Please help with: {{request}}"
InputVariables:
- Name: request
InferenceConfiguration:
Text:
Temperature: 0.7
TopP: 0.9
MaxTokens: 1000
# Variant 2 - This causes the CloudFormation validation error
- Name: variant-2
ModelId: anthropic.claude-3-5-sonnet-20241022-v2:0
TemplateType: TEXT
TemplateConfiguration:
Text:
Text: "This is the second variant with different settings. Please help with: {{request}}"
InputVariables:
- Name: request
InferenceConfiguration:
Text:
Temperature: 0.3
TopP: 0.95
MaxTokens: 1500

# This prompt works fine with only 1 variant (for comparison)
TestPromptWithOneVariant:
Type: AWS::Bedrock::Prompt
Properties:
Name: cloudformation-single-variant-test-prompt
Description: Test prompt with 1 variant - this works fine
CustomerEncryptionKeyArn: !If
- HasKMSKey
- !Sub 'arn:aws:kms:${AWS::Region}:${AWS::AccountId}:key/${KMSKeyId}'
- !GetAtt PromptEncryptionKey.Arn
DefaultVariant: single-variant
Tags:
Purpose: CloudFormationBugReport
Status: WorksCorrectly
VariantCount: "1"
Variants:
- Name: single-variant
ModelId: anthropic.claude-3-5-sonnet-20241022-v2:0
TemplateType: TEXT
TemplateConfiguration:
Text:
Text: "This single variant prompt works correctly. Please help with: {{request}}"
InputVariables:
- Name: request
InferenceConfiguration:
Text:
Temperature: 0.5
TopP: 0.9
MaxTokens: 2000

Outputs:
ErrorDescription:
Description: Expected CloudFormation deployment error
Value: |
This template will fail with error:
"Properties validation failed for resource TestPromptWithTwoVariants with message:
[#/Variants: expected maximum item count: 1, found: 2]"

ExpectedBehavior:
Description: What should happen according to AWS documentation
Value: |
According to AWS Bedrock documentation, prompts should support up to 3 variants.
The CloudFormation AWS::Bedrock::Prompt resource should allow 1-3 variants in the Variants array.

ActualBehavior:
Description: Current CloudFormation behavior
Value: |
CloudFormation validation incorrectly limits Variants array to maximum 1 item,
preventing users from creating prompts with multiple variants as documented.

SuggestedFix:
Description: Recommended fix for CloudFormation team
Value: |
Update the AWS::Bedrock::Prompt CloudFormation resource schema to allow:
- Minimum 1 variant (current behavior is correct)
- Maximum 3 variants (currently incorrectly limited to 1)
- This aligns with the AWS Bedrock service limits and API documentation

TestInstructions:
Description: How to reproduce the issue
Value: |
1. Deploy this CloudFormation template
2. Observe the validation error for TestPromptWithTwoVariants resource
3. Note that TestPromptWithOneVariant deploys successfully
4. Compare with AWS Bedrock API documentation showing 3 variant limit

WorkingPromptArn:
Description: ARN of the successfully created single-variant prompt
Value: !GetAtt TestPromptWithOneVariant.Arn

FailingPromptName:
Description: Name of the prompt that fails due to CloudFormation bug
Value: !Ref TestPromptWithTwoVariants
Condition: false # This will never output since the resource fails to create
`

### Other Details

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by deploying the inline CloudFormation template and compare the one-variant and two-variant resources, using the reported validation error as the baseline. Confirm the AWS::Bedrock::Prompt resource schema and related CloudFormation coverage materials; done means the two-variant case deploys successfully and up to three variants are accepted without breaking the one-variant case.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.