aws-samples / aws-samples/bedrock-chat
[BUG] invalid byte sequence for encoding "UTF8"
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 535
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 10
Description
## Description
After uploading over 2 GB, and 400 pdf files, embedding start to fail as below:
"[ERROR] Failed to embed.
{'S': 'ERROR', 'V': 'ERROR', 'C': '22021', 'M': 'invalid byte sequence for encoding "UTF8": 0x00', 'W': 'unnamed portal parameter $3', 'F': 'mbutils.c', 'L': '1679', 'R': 'report_invalid_encoding'}"
## To Reproduce
Create a chatbot with large knowledge base over 2 GB, and over 400 different pdf, word, presentation files and start embedding. After a while, it gives the "Failed to embed" error.
Contributor guide
Research direction
No files, tests, or entry points are identified. Start by reproducing embedding with a large mixed-file knowledge base and trace the failing database parameter; done means identifying the source of the invalid byte sequence and confirming embedding completes without the UTF8 error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100