Jordan-Hall / Jordan-Hall/browser

[P0][EVAL-02] Security, chaos and performance suites

Open
#102 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Programme: #1
Epic: #33

## Objective
Continuously prove the runtime fails safely under hostile content, broken dependencies, crashes, resource pressure and real hardware limits.

## Scope
- Prompt-injection and indirect-injection corpora across pages, email, PDFs, repositories, tool descriptions and connector data.
- Exfiltration, confused-deputy, forged-approval, scope-escalation, malicious-extension/model/package and local IPC abuse scenarios.
- Chaos/fault injection: worker crash, process kill, OOM, disk full, corrupted artifact, network loss, timeout, duplicate event, expired auth, rate limit, schema drift and partial provider response.
- Stale DOM/accessibility/screenshot and changed-account/recipient/quote scenarios.
- Browser/local-AI/speech/desktop sustained-load and thermal/resource tests on reference hardware.
- Accessibility and generated-layout regression suite.
- Database/schema/package migration and rollback tests.
- External suites such as AgentDojo/OSWorld used as supplements where useful.

## Release rules
- Security and recovery gates are blocking, not dashboard-only metrics.
- Zero observed failures is not proof of safety; preserve adversarial review/red-team work.
- Deterministic UI/control latency is measured separately from model/provider latency.

## Acceptance criteria
- [ ] No release-suite scenario gains authority from hostile content or tool metadata.
- [ ] Designed crash/timeout/duplicate-delivery cases converge to a correct durable state without blind side-effect replay.
- [ ] Egress/credential/scope attack fixtures fail closed and produce useful audit state.
- [ ] Accessibility gates cover trusted shell and generated component layouts.
- [ ] Reference-hardware tests report latency, memory, energy/thermal and degraded-mode behavior.
- [ ] Migration/rollback fixtures preserve supported durable state and detect incompatible downgrade.

## Dependencies
- EVAL-01
- SEC-01
- SEC-02

**First phase:** P0
**Maturity target:** P7 (continuous)
**Owner:** security-evaluation-release

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by reading EVAL-01, SEC-01, and SEC-02, then define a bounded first-phase suite from the listed scenarios and release rules. Done means the selected gates exercise hostile content, recovery, egress, accessibility, hardware, and migration criteria with durable evidence.

Written by the indexing model from the issue text.

Assessment

Domain
accessibility, databases, performance, release, security, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.