aws-samples / aws-samples/bedrock-chat

[BUG] invalid byte sequence for encoding "UTF8"

Open
#308 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
1.3k
Forks
535
Avg merge
1d 12h
Merged PRs (30d)
10

Description

## Description

After uploading over 2 GB, and 400 pdf files, embedding start to fail as below:
"[ERROR] Failed to embed.
{'S': 'ERROR', 'V': 'ERROR', 'C': '22021', 'M': 'invalid byte sequence for encoding "UTF8": 0x00', 'W': 'unnamed portal parameter $3', 'F': 'mbutils.c', 'L': '1679', 'R': 'report_invalid_encoding'}"

## To Reproduce

Create a chatbot with large knowledge base over 2 GB, and over 400 different pdf, word, presentation files and start embedding. After a while, it gives the "Failed to embed" error.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are identified. Start by reproducing embedding with a large mixed-file knowledge base and trace the failing database parameter; done means identifying the source of the invalid byte sequence and confirming embedding completes without the UTF8 error.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.