aws-samples / aws-samples/bedrock-chat
[BUG] Incorrect Handling of Mistral Model Configuration
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 535
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 10
Description
## 🚨 **Please Note** 🚨
To ensure efficient investigation of the issue, please fill out the fields below with as much detail as possible. **Reports that do not follow this template may be closed without notification.** We appreciate your cooperation.
## 🐞Describe the bug
The code incorrectly attempts to access reasoning_params configuration when Mistral model is enabled, despite Mistral not supporting this feature. This leads to KeyError exceptions in two specific scenarios:
Bot Object Creation
When ENABLE_MISTRAL=true and find_public_bot_by_id() is called
During BotModel instantiation, GenerationParamsModel attempts to access non-existent 'reasoning_params' from DEFAULT_MISTRAL_GENERATION_CONFIG
Results in KeyError as this configuration is Claude-specific
Model Inference Configuration
When ENABLE_MISTRAL=true and compose_args_for_converse_api() is called with enable_reasoning=true
Function attempts to access DEFAULT_GENERATION_CONFIG["reasoning_params"]
Fails as Mistral's configuration doesn't include reasoning parameters
## 🔄 To Reproduce
Steps to reproduce the behavior:
Set "enableMistral": true in cdk.json
Run npx cdk deploy
Try to use any feature that enables reasoning with Mistral models
Observe the KeyError in logs
## 📷 Screenshots
n/a
## 🔎 Logs for Chat Issues
`[ERROR] 2025-05-27T20:14:46.820Z f0f3ec65-be4b-411d-90bc-7dfaaeb1817e Failed to run stream handler: 'reasoning_params'
Traceback (most recent call last):
File "/var/task/app/usecases/chat.py", line 71, in prepare_conversation
conversation = find_conversation_by_id(user_id, chat_input.conversation_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/app/repositories/conversation.py", line 292, in find_conversation_by_id
raise RecordNotFoundError(f"No conversation found with id: {conversation_id}")
app.repositories.common.RecordNotFoundError: No conversation found with id: 01JW9P41T5K59HRED3S18QQT49
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/task/app/websocket.py", line 179, in process_chat_input
chat(
File "/var/task/app/usecases/chat.py", line 244, in chat
user_msg_id, conversation, bot = prepare_conversation(user_id, chat_input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/app/usecases/chat.py", line 112, in prepare_conversation
owned, bot = fetch_bot(user_id, chat_input.bot_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/app/usecases/bot.py", line 465, in fetch_bot
return False, find_public_bot_by_id(bot_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/app/repositories/custom_bot.py", line 581, in find_public_bot_by_id
"budget_tokens": DEFAULT_GENERATION_CONFIG["reasoning_params"]["budget_tokens"], # type: ignore
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
KeyError: 'reasoning_params'`
## 🔎 Logs for Bot Creation/Update Issues
n/a
## 📝 Additional context
This is a compatibility issue where Claude-specific features (reasoning/thinking parameters) are being applied to Mistral models without appropriate checks. The current code structure doesn't properly differentiate between model capabilities when Mistral is enabled.
Suggested solution: Add appropriate guards to prevent reasoning parameter access when Mistral is enabled by:
Adding model type checks before accessing reasoning parameters
Creating separate parameter handling paths for Claude vs Mistral models
Ensuring the DEFAULT_MISTRAL_GENERATION_CONFIG doesn't attempt to reference Claude-specific parameters
Contributor guide
Research direction
Start by reproducing the failure with enableMistral enabled and reasoning requested. Inspect app/repositories/custom_bot.py, especially find_public_bot_by_id(), then trace compose_args_for_converse_api() and the generation configuration paths. Done means Mistral bot creation and inference no longer access reasoning_params, while Claude reasoning handling still works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100