Improving the review feature
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- jupyter, typescript
- Domain
- ai
Research direction
Start by reading PR #28 and tracing the review feature’s current comparison path. Reproduce cases where the tutor misses differences, then evaluate the deterministic diff approach alongside model temperature and system-prompt effects. Done means differences between initial and student-written code are detected consistently for the affected models.
Written by the indexing model from the issue text.
Description
One of the main issues with the review feature right now is its inconsistency as mentioned in PR #28 (at least for some models): sometimes the tutor can't spot the difference between the initial code and the student written code. This could be due to the model not being good enough, the temperature and system prompt could also have an effect.
One solution I tried implementing for this is a deterministic diff calculator so that we don't have to rely on the LLM to find the differences. Using our own comparison tool could be a source of error, but I think this feature still needs to be improved some way in this aspect.
- Dominant language
- TypeScript
- Stars
- 4
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from QuantStack/jupyter-ai-tutor
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
QuantStack/jupyter-ai-tutor#39 · 4 comments · 1 reaction ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 42/100
-
Difficulty 3/5 Half a day Newbie friendliness 64/100
All issues in QuantStack/jupyter-ai-tutor
Similar issues
-
clawsweeper:fix-shape-clear clawsweeper:queueable-fix clawsweeper:source-repro impact:ux-friction issue-rating: 🦞 diamond lobster no-stale P3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
community first-timers-only good first issue hacktoberfest help wanted low hanging fruit up-for-grabs
Difficulty 1/5 Under an hour Newbie friendliness 76/100
-
code-quality refactoring
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
github/gh-aw-firewall#8816 ·
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
vercel/react-tweet#225 ·