aws-samples / aws-samples/bedrock-chat

[BUG] Incorrect Handling of Mistral Model Configuration

Open
#871 0 comments 0 reactions 0 assignees View on GitHub
needs-triage
Dominant language
TypeScript
Stars
1.3k
Forks
535
Avg merge
1d 12h
Merged PRs (30d)
10

Description

## 🚨 **Please Note** 🚨

To ensure efficient investigation of the issue, please fill out the fields below with as much detail as possible. **Reports that do not follow this template may be closed without notification.** We appreciate your cooperation.

## 🐞Describe the bug

The code incorrectly attempts to access reasoning_params configuration when Mistral model is enabled, despite Mistral not supporting this feature. This leads to KeyError exceptions in two specific scenarios:

Bot Object Creation

When ENABLE_MISTRAL=true and find_public_bot_by_id() is called
During BotModel instantiation, GenerationParamsModel attempts to access non-existent 'reasoning_params' from DEFAULT_MISTRAL_GENERATION_CONFIG
Results in KeyError as this configuration is Claude-specific

Model Inference Configuration

When ENABLE_MISTRAL=true and compose_args_for_converse_api() is called with enable_reasoning=true
Function attempts to access DEFAULT_GENERATION_CONFIG["reasoning_params"]
Fails as Mistral's configuration doesn't include reasoning parameters

## 🔄 To Reproduce

Steps to reproduce the behavior:

Set "enableMistral": true in cdk.json
Run npx cdk deploy
Try to use any feature that enables reasoning with Mistral models
Observe the KeyError in logs

## 📷 Screenshots

n/a

## 🔎 Logs for Chat Issues

`[ERROR] 2025-05-27T20:14:46.820Z f0f3ec65-be4b-411d-90bc-7dfaaeb1817e Failed to run stream handler: 'reasoning_params'
Traceback (most recent call last):
File "/var/task/app/usecases/chat.py", line 71, in prepare_conversation
conversation = find_conversation_by_id(user_id, chat_input.conversation_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/app/repositories/conversation.py", line 292, in find_conversation_by_id
raise RecordNotFoundError(f"No conversation found with id: {conversation_id}")
app.repositories.common.RecordNotFoundError: No conversation found with id: 01JW9P41T5K59HRED3S18QQT49

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/var/task/app/websocket.py", line 179, in process_chat_input
chat(
File "/var/task/app/usecases/chat.py", line 244, in chat
user_msg_id, conversation, bot = prepare_conversation(user_id, chat_input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/app/usecases/chat.py", line 112, in prepare_conversation
owned, bot = fetch_bot(user_id, chat_input.bot_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/app/usecases/bot.py", line 465, in fetch_bot
return False, find_public_bot_by_id(bot_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/app/repositories/custom_bot.py", line 581, in find_public_bot_by_id
"budget_tokens": DEFAULT_GENERATION_CONFIG["reasoning_params"]["budget_tokens"], # type: ignore
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
KeyError: 'reasoning_params'`

## 🔎 Logs for Bot Creation/Update Issues

n/a

## 📝 Additional context

This is a compatibility issue where Claude-specific features (reasoning/thinking parameters) are being applied to Mistral models without appropriate checks. The current code structure doesn't properly differentiate between model capabilities when Mistral is enabled.

Suggested solution: Add appropriate guards to prevent reasoning parameter access when Mistral is enabled by:

Adding model type checks before accessing reasoning parameters
Creating separate parameter handling paths for Claude vs Mistral models
Ensuring the DEFAULT_MISTRAL_GENERATION_CONFIG doesn't attempt to reference Claude-specific parameters

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with enableMistral enabled and reasoning requested. Inspect app/repositories/custom_bot.py, especially find_public_bot_by_id(), then trace compose_args_for_converse_api() and the generation configuration paths. Done means Mistral bot creation and inference no longer access reasoning_params, while Claude reasoning handling still works.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.