cloudflare / cloudflare/ai

AI Gateway: Bedrock Application Inference Profile ARN fails via /compat/chat/completions

Open
#583 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.2k
Forks
345
Avg merge
13h 31m
Merged PRs (30d)
1

Description

A Bedrock Application Inference Profile ARN works via AI Gateway's Bedrock native endpoint, but fails via `/compat/chat/completions`.

Works

```bash
curl -X POST \
"https://gateway.ai.cloudflare.com/v1///aws-bedrock/bedrock-runtime/us-east-2/model/arn%3Aaws%3Abedrock%3Aus-east-2%3A%3Aapplication-inference-profile%2F/invoke" \
-H "cf-aig-authorization: Bearer " \
-H "Content-Type: application/json" \
-d '{
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 1024,
"messages": [{
"role": "user",
"content": [{ "type": "text", "text": "Reply only OK" }]
}]
}'
```

Response:

```json
{
"model": "claude-sonnet-...",
"type": "message",
"role": "assistant",
"content": [{ "type": "text", "text": "OK" }],
"stop_reason": "end_turn",
"usage": { "input_tokens": 13, "output_tokens": 4 }
}
```

Fails: raw ARN

```bash
curl -X POST \
"https://gateway.ai.cloudflare.com/v1///compat/chat/completions" \
-H "cf-aig-authorization: Bearer " \
-H "Content-Type: application/json" \
-d '{
"model": "aws-bedrock/arn:aws:bedrock:us-east-2::application-inference-profile/",
"messages": [{ "role": "user", "content": "Reply only OK" }],
"max_tokens": 16
}'
```

Fails: URL-encoded ARN

```json
{
"model": "aws-bedrock/arn%3Aaws%3Abedrock%3Aus-east-2%3A%3Aapplication-inference-profile%2F",
"messages": [{ "role": "user", "content": "Reply only OK" }],
"max_tokens": 16
}
```

Also tried:

```text
aws-bedrock/application-inference-profile/
aws-bedrock/
aws-bedrock/us-east-2/arn:aws:bedrock:us-east-2::application-inference-profile/
```

Question: is Bedrock Application Inference Profile ARN supported through /compat/chat/completions? If yes, what is the correct model format?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Bedrock native request and the failing /compat/chat/completions requests with raw and URL-encoded Application Inference Profile ARNs. Trace the compatibility endpoint's model parsing and compare it with the native Bedrock path; done means establishing whether these ARNs are supported and identifying the accepted model format or the missing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
ai, api, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.