aws-samples / aws-samples/bedrock-chat
[BUG / Feature request] Not working as expected when using Aurora as vector store
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 535
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 10
Description
## 🐞Describe the bug
When using Aurora as vector store of a bedrock knowledge base it does not work well when aurora which is default is set for minimum of 0 ACUs.
Since the first prompt when it is stopped it needs about 10 seconds to be in available state, the first question tries to reach the knowledge base and since gets error below it does not use the KB at all.
Screenshot of the error:
After the error occurs for first time, all subsequent chats inside this conversation not even try to use the knowledge base at all.
## 🔄 To Reproduce
Steps to reproduce the behaviour:
1. Create a Knowledge base in Bedrock and select aurora as vector store.
2. Bedrock will create an Aurora serverless with min ACU of 0
3. Create a new bot and use the KB id as part of it, so it can use it.
5. Wait Aurora to be in paused state. By default it has a 3 hour of unusage to pause but you can change that.
6. Start chatting and you should get the error on first conversation.
## Why that is needed
Allowing this solution to work seamlessly with Aurora vector stores will make it very cost effective for new PoCs.
Since most PoCs have low usage, Aurora will be in paused state charging almost 0 which makes a good case for PoC.
In comparison to using OpenSearch, this will be way more cost effective since OS has a min cost of around 170 per month per KB, but I do not expect to replace the existing logic from bedrock chat as it uses OpenSearch when creating KBs from it but to at least work well when creating KBs on Bedrock directly and using Aurora as vector store.
## What is expected
I Expect that when Aurora Serverless gets into paused state (After inactivity) for an aurora vector store, that it retries after 15 seconds so that it can keep using it as expected.
Note: This only happens on first conversation, so If error occurs due to paused state on first conversation, starting a new one would work well if waiting about 15 seconds from starting the first one since Aurora would be in start state by this time.
Contributor guide
Research direction
No files or tests are named. Start by tracing the Bedrock knowledge-base retrieval path and its Aurora vector-store error handling, then reproduce the first request while Aurora is paused. Done means a failed first request retries after Aurora resumes and subsequent messages continue using the knowledge base.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- ai, cloud, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100