Bedrock: Cross Region Inference capability
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the feature
The [new](https://eu-central-1.console.aws.amazon.com/bedrock/home?region=eu-central-1#/cross-region-inference) cross-region inference profiles allow us to use the same Bedrock model in other regions by switching out the foundation model ARN with this inference profile ARN:
However, in the Amazon Bedrock Construct Library only allows for Foundation Models or Provisioned Models.
Can we incorprate this new cross-region inference profiles feature ASAP?
### Use Case
I am invoking the Bedrock API via Step Functions tasks with this code currently:
```
claude_model_id = bedrock.FoundationModelIdentifier.ANTHROPIC_CLAUDE_V2_1
invoke_model = tasks.BedrockInvokeModel(
scope,
id + " (Invoke Model)",
model=bedrock.FoundationModel.from_foundation_model_id(
scope,
"Model",
claude_model_id,
),...
```
ideally I'd like to simply switch out the `claude_model_id` for the inference profile ARN.
### Proposed Solution
Allow for custom ARNs instead of enforcing the `bedrock.FoundationModelIdentifier` class for the model IDs.
### Other Information
_No response_
### Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
### CDK version used
2.155.0
### Environment details (OS name and version, etc.)
MacOS Ventura, M2 Chip
Contributor guide
Research direction
Start with the Amazon Bedrock Construct Library APIs used by tasks.BedrockInvokeModel, especially FoundationModel.from_foundation_model_id and FoundationModelIdentifier. Trace how the model ARN is validated, then verify that a cross-region inference profile ARN can be supplied while existing Foundation and Provisioned Model usage continues to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100