microsoft / microsoft/rag-experiment-accelerator

02_qa_generation fails with exception

Open
#485 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
311
Forks
111
PR merge metrics
No merged PRs in 30d

Description

Running 02_qa_generation step fails with exception:

2024-04-15 20:42:21,690 - ERROR - rag_experiment_accelerator.ingest_data.acs_ingest - could not generate a valid json so moving over to next question! Error message: Expecting value: line 1 column 1 (char 0)
2024-04-15 20:42:21,720 - ERROR - rag_experiment_accelerator.ingest_data.acs_ingest - Traceback (most recent call last):
File "/Users/<>/repos/rag-experiment-accelerator/rag_experiment_accelerator/ingest_data/acs_ingest.py", line 128, in generate_qna
response_dict = json.loads(
^^^^^^^^^^^
File "/Users/<>/anaconda3/envs/rag-experiment/lib/python3.11/json/init.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/<>/anaconda3/envs/rag-experiment/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/<>/anaconda3/envs/rag-experiment/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

the generated response fails to load as json in generate_qna

'json[ { "question": "What are the top features and benefits of Surface Pro 8?", "answer": "Surface Pro 8s top features and benefits include being the most powerful pro ever, extending the ultimate desktop experience with Thunderbolt™ 4 ports, and having the most advanced display in a pro with improved video call capabilities." }, { "question": "How does Surface Pro 8s display enhance the user experience?", "answer": "Surface Pro 8s display is 11% larger, 10.8% higher resolution, 12.5% brighter, individually calibrated, virtually edge-to-edge, and now with up to 120Hz refresh rate for an even smoother pen experience and more responsive touch." }]'

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in rag_experiment_accelerator/ingest_data/acs_ingest.py at generate_qna around line 128 and reproduce the 02_qa_generation step. Inspect the generated response shown in the issue and the JSON-loading path; done means the generated Q&A response is handled successfully instead of raising JSONDecodeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.