aws / aws/aws-cdk

Bedrock: Cross Region Inference capability

Open
#31,570 2 comments 1 reaction 0 assignees View on GitHub
@aws-cdk/aws-lambda effort/medium feature-request p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the feature

The [new](https://eu-central-1.console.aws.amazon.com/bedrock/home?region=eu-central-1#/cross-region-inference) cross-region inference profiles allow us to use the same Bedrock model in other regions by switching out the foundation model ARN with this inference profile ARN:
image

However, in the Amazon Bedrock Construct Library only allows for Foundation Models or Provisioned Models.

Can we incorprate this new cross-region inference profiles feature ASAP?

### Use Case

I am invoking the Bedrock API via Step Functions tasks with this code currently:

```
claude_model_id = bedrock.FoundationModelIdentifier.ANTHROPIC_CLAUDE_V2_1
invoke_model = tasks.BedrockInvokeModel(
scope,
id + " (Invoke Model)",
model=bedrock.FoundationModel.from_foundation_model_id(
scope,
"Model",
claude_model_id,
),...
```
ideally I'd like to simply switch out the `claude_model_id` for the inference profile ARN.

### Proposed Solution

Allow for custom ARNs instead of enforcing the `bedrock.FoundationModelIdentifier` class for the model IDs.

### Other Information

_No response_

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

2.155.0

### Environment details (OS name and version, etc.)

MacOS Ventura, M2 Chip

Contributor guide

Open the contributing guide

Research direction

Start with the Amazon Bedrock Construct Library APIs used by tasks.BedrockInvokeModel, especially FoundationModel.from_foundation_model_id and FoundationModelIdentifier. Trace how the model ARN is validated, then verify that a cross-region inference profile ARN can be supplied while existing Foundation and Provisioned Model usage continues to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.