norrisaftcc / norrisaftcc/algorithm-liza
M4: Emit SHODANN-shaped CodeMetrics from eval runs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
Per task iteration, extract coverage, test_count, complexity, loc, functions, docstrings, lint_issues and syntax_errors from LIZA's generated code, using SHODANN's field names exactly.
Generated code frequently does not parse, so the extractor has to handle that as data rather than as an exception — syntax_errors is a field for exactly this reason, and a run producing broken code should score badly rather than crash the harness.
This is worth doing on its own merits even if the scoring in the next issue is later dropped. Rationale in ADR-0003.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read ADR-0003, then locate the evaluation-run loop that processes each task iteration and the existing code-analysis utilities. Trace how generated code is passed into metrics extraction; completion means emitting all eight SHODANN-named fields per iteration, recording syntax failures as data, and allowing broken code to produce a poor score without crashing the harness.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100