OpenHands / OpenHands/extensions

[Research] Set up automatic failure clustering and improvement suggestion

Open
#141 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
148
Forks
90
Avg merge
1d 17h
Merged PRs (30d)
36

Description

Summary

utils/analysis/laminar_signals/analyze.py already turns Laminar signal data into grounded recommendations. We also need it to analyze critic-failure datasets so skill improvements can be based on failed trajectories, not only on signal_events.

Target repo

  • OpenHands/extensions

Dependencies

  • Merge after a critic-failure dataset is available for historical conversations.

Scope

Add a second input mode for critic-failure data and keep the existing Laminar signal mode intact.

Files to update

  • utils/analysis/laminar_signals/analyze.py
  • utils/analysis/laminar_signals/README.md
  • utils/analysis/laminar_signals/templates/default.j2
  • utils/analysis/laminar_signals/templates/pr_review.j2

Acceptance criteria

  • The analyzer can read a critic-failure dataset as input.
  • Output stays structured and points to concrete skill or plugin files.
  • Existing Laminar signal analysis still works.
  • The README documents both input modes.

References

  • OpenHands/extensions/utils/analysis/laminar_signals/analyze.py
  • OpenHands/extensions/utils/analysis/laminar_signals/README.md

This issue was drafted by an AI assistant (OpenHands) on behalf of the user.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with utils/analysis/laminar_signals/analyze.py and the existing Laminar signal flow, then read README.md and both templates to understand the current structured output. The critic-failure dataset dependency must be available before this can be completed. Done means both input modes work, existing signal analysis remains intact, outputs identify concrete skill or plugin files, and the README documents both modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.