classifier: occasional UNSAFE on benign odd shapes (bun -e embedded text; verdict contradicting its own SAFE reasoning)
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Active
- Tech stack
- bun, nodejs, typescript
Research direction
Reproduce the two listed probes and inspect INLINE_CODE_INTERPRETERS plus the parser and overlay paths. Done requires choosing among interpreter scoping, verdict/reason reconciliation, or accepting the behavior as noise, then verifying the benign commands no longer prompt if a fix is selected.
Written by the indexing model from the issue text.
Description
This issue was generated by AI during post-fix verification.
Residual classifier over-flagging after the 2026-08-26/27 fix chain
All systematic over-prompting bugs from this week are fixed (fb2e5a9, a4614e7: VERDICT-prefix parsing, inline-code scoping, credential-material scoping, gh/tag carve-outs) and verified live. Two residual classes remain, both model-variance rather than systematic:
-
Reasoning/verdict mismatch. The model sometimes answers UNSAFE while its own reason resolves to SAFE. Log example (pid 79753, 2026-08-27T09:42:46):
verdict=UNSAFE reason="Reading logs is safe, but exfiltrates local data via grep/cut? No remote endpoint; resolve to SAFE." cmd="L=/Users/samuelreed/.omp/logs/...; grep -c ..." -
bun -ewith embedded command text flagged. Probes likebun -e ' const mod = await import("./index.ts"); ...'gotUNSAFE — executes local script via bun, importing and running code(2026-08-27T10:08, 10:16).bunis not inINLINE_CODE_INTERPRETERS, so the overlay cannot flag it consistently — the verdict is pure model judgment, and embedding command text (test probes that quote log lines) tips it.
Both prompt dialogs for legitimate local-only work. Volume is low (2 of ~20 commands in the verification session). Candidate directions if this keeps biting: (a) add bun/node/tsx to INLINE_CODE_INTERPRETERS with the same obfuscation-marker scoping, (b) teach the parser to reconcile a reason that resolves to the opposite verdict, or (c) accept as known noise. Filing so the shape is discoverable without re-deriving this session.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 1
- Avg merge
- 1h 20m
- Merged PRs (30d)
- 32
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from STRML/omp-classifier
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
STRML/omp-classifier#81 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
STRML/omp-classifier#80 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
STRML/omp-classifier#78 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
STRML/omp-classifier#82 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
STRML/omp-classifier#73 ·
All issues in STRML/omp-classifier
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
area:tools bug good first issue help wanted priority:P2
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
TaewoooPark/Motifcode#14 ·
-
bug 🐞
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
[Bounty proposal] fix(web): memory insights count an evening memory on the next day ($25 proposed) Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
BasedHardware/omi#15320 ·
-
Difficulty 2/5 Half a day Newbie friendliness 78/100
vercel/vercel-plugin#199 ·