ASSERT-KTH / ASSERT-KTH/Mokav

The output in feedback should correspond with the response in history

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10
Forks
0
PR merge metrics
No merged PRs in 30d

Description

The response saved the history is always the first one: https://github.com/ASSERT-KTH/DET-Gen/blob/main/src/test_generator.py#L88

But the output in the feedback may be computed for another response, if the first one is invalid: https://github.com/ASSERT-KTH/DET-Gen/blob/main/src/code_runner.py#L148

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the history-saving logic at src/test_generator.py#L88 with the feedback execution path at src/code_runner.py#L148. Trace how responses are selected when the first response is invalid, then verify that the response recorded in history is the one whose output appears in feedback. Done means these responses remain aligned in that case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.