0x0pointer / 0x0pointer/agent-smith

Unconfirmed wordlist paths are registered as real endpoints, inflating the coverage matrix ~3x

Open
#180 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
129
Forks
10
Avg merge
8h 52m
Merged PRs (30d)
5

Description

What happens

coverage_matrix.json from a single web engagement:

endpoints   561    of which 360 (64%) are wordlist-shaped paths that do not exist
cells      5422    of which 3240 (59%) belong to those non-existent endpoints
pending    3182    ← Phase B can never drain this

The noise is pure extension-permutation fuzzing — one stem x 12 extensions:

/.bash_history  /.bash_history.txt  /.bash_history.php  /.bash_history.bak
/.bash_history.old  /.bash_history.conf  /.bash_history.xml  /.bash_history.json
/.bash_history.env  /.bash_history.log  ...

Top stems by permutation count: /.bash_history (12), /.bashrc (12), /.cache (12), /.config (12), /.cvs (12).

Sampling 40 of these paths against the live target returned 25x 403, 15x 302, zero 200s — none exist.

By contrast, only 20 endpoints came from channels that confirm existence (manual-crawl 16, react-props 4).

The provenance bug (this is the important part)

All 360 noise endpoints are labelled discovered_by: "spider":

>>> collections.Counter(e['discovered_by'] for e in noise)
{'spider': 360}
>>> collections.Counter(e['discovered_by'] for e in all_endpoints)
{'spider': 541, 'manual-crawl': 16, 'react-props': 4}

Nothing in the matrix is labelled ffuf. So either ffuf results are registered under the spider's provenance, or the spider performs extension permutation itself. Either way the noise cannot be filtered by source, because it claims to come from the most trustworthy discovery channel. Any fix keyed on discovered_by == "ffuf" will match nothing.

Tell-tale: the 360 noise endpoints carry 0 params between them, so they generate only generic cross-cutting cells — 3,240 cells that can never be legitimately closed because there is nothing behind them.

Why this matters
  • pending is useless as a progress signal; Phase B cannot reach 0 cells.
  • session(complete) gates on coverage, so the scan is blocked by unclosable cells.
  • The headline completeness figure overstates reality by ~3x. The genuinely tested surface here was ~20 endpoints / ~2,182 cells, not 5,422.
  • It also destroys the knowledge-graph view — see the companion issue.
Expected

A path discovered by wordlist fuzzing becomes an endpoint only once a response confirms it exists (non-404/403, or differing from the fuzz baseline).

Suggested fix
  1. Fix the provenance label first — wordlist-derived paths must not be recorded as spider.
  2. Gate endpoint registration on a liveness check for wordlist-derived paths.
  3. Report confirmed vs candidate endpoints/cells as separate counters so completeness reflects reality.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with coverage_matrix.json and trace endpoint registration through the spider/wordlist discovery path, checking how provenance is assigned before Phase B consumes the endpoints. Reproduce the inflated candidate set, then verify that wordlist paths require the stated liveness signal and that confirmed and candidate endpoints and cells are reported separately without blocking session(complete).

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.