code_search / sde_search: reliability score double-counts age, base="master" PR counts, unguarded parse crash, blocking metadata fetch

Summary

While packaging the Code Search Agent as a Claude plugin (NASA-IMPACT/akd-plugins#6) I ported the sde_search and repository_search tools and, in the process, reviewed the upstream implementations on develop. This issue collects several correctness / robustness / efficiency problems found in akd_ext/tools/code_search/ and akd_ext/tools/sde_search.py.

Grouped by confidence. "Confirmed" = fires on normal input today. "Latent" = a real gap that does not reproduce against the current SDE endpoint but has no guard (verified live — see notes).

All line numbers are against develop at time of filing.

---

Confirmed

1. Reliability score counts repository age twice

akd_ext/tools/code_search/utils.py:55 and :124-133

fetch_github_metadata unconditionally sets repository_metadata.first_commit_date = None on every successful fetch (and the field also defaults to ""). In calculate_reliability_score, the Development-History component only diverges from Age when first_commit_date is truthy:

effective_start_date = created_at
if repository_metadata.first_commit_date:   # always None/"" -> never taken
    ...
days_of_development = (now - effective_start_date).total_seconds() / (24 * 3600)
score_history = min(days_of_development / 1460 * 100, 100) * 0.15

So effective_start_date is always created_at, and History (0.15) computes the identical value to Age (0.20). Net effect: ~35% of every score is a single signal (days-since-creation) counted twice; there is effectively no independent "development history" signal.

Fix options: (a) populate first_commit_date with a real first-commit lookup (one extra GitHub /commits?per_page=1 + Link rel="last" call per repo), or (b) drop the History term and redistribute its 0.15 (e.g. into Age) so no signal is double-weighted, and update the docstring/weights accordingly.

2. PR counts hardcode base="master"

akd_ext/tools/code_search/utils.py:53-54

repository_metadata.pulls = repo.get_pulls(state="open", sort="created", base="master").totalCount
repository_metadata.closed_pulls = repo.get_pulls(state="closed", sort="created", base="master").totalCount

base="master" only counts PRs whose base branch is literally named master. Repositories whose default branch is main (the majority of repos created since ~2020) therefore report pulls and closed_pulls as 0. These fields are surfaced in repository_metadata output (they are not used in the score).

Fix: query the repo's actual default branch (repo.default_branch) instead of hardcoding, or drop the base filter entirely.

3. Metadata fetch is async def but does blocking I/O

akd_ext/tools/code_search/utils.py:36-60, used at repository_search.py:190-199

fetch_github_metadata is declared async but uses the synchronous PyGithub client (with Github(...) as g: repo = g.get_repo(...)), which performs blocking HTTP. RepositorySearchTool._arun fans these out with asyncio.gather(*tasks), but because each task blocks, there is no real concurrency and the event loop is blocked for the duration of every GitHub round-trip (a problem when the tool runs inside the MCP server / an async host).

Fix: either make the fetch truly async (httpx.AsyncClient against the GitHub REST API) or run the blocking client via asyncio.to_thread(...) so gather can overlap the calls.

4. Four GitHub API calls per repository

akd_ext/tools/code_search/utils.py:47-54

Each repo triggers get_repo plus get_issues(state="open").totalCount, get_pulls(state="open", ...).totalCount, and get_pulls(state="closed", ...).totalCount — 4 calls per repo. Only the get_repo fields (stars/forks/dates) feed the reliability score; the other three back fields that are output-only. Under the unauthenticated GitHub limit (60 req/hr) this exhausts the budget after ~15 repos. Separately, get_issues(state="open") counts pull requests as issues (GitHub API quirk), inflating open_issues.

Fix: drop the issues/pulls calls (or make them opt-in), and derive open_issues from repo.open_issues_count if that field is wanted.

---

Latent (real gap, not reproducing against the current endpoint)

5. Unguarded owner, repo parse can crash the whole search

akd_ext/tools/code_search/repository_search.py:206-208 (with :190-199)

parsed_url = urlparse(url)
path_parts = parsed_url.path.strip("/").split("/")
owner, repo = path_parts[0], path_parts[1]

There is no host check and no try/except. Any result URL with fewer than 2 path segments (a bare github.com/owner, an ascl.net/1612.017, a project homepage, etc.) raises IndexError. Because _arun uses asyncio.gather(*tasks) without return_exceptions=True, a single bad URL aborts the entire query rather than that one result.

I probed the live /api/code/search endpoint (40 docs across 4 queries) and every URL was a well-formed github.com/owner/repo, so this is not firing today — but the code depends entirely on that endpoint contract with no guard.

Fix: skip enrichment for non-github.com hosts / <2-segment paths (return the item unenriched), and/or pass return_exceptions=True to gather and handle failures per-item.

6. Strict-enum parse can fail an entire SDE query

akd_ext/tools/sde_search.py:213-214 (with :272)

division=NASASMDDivision(division) if division else None,
doc_type=SDEIndexedDocumentType(doc_type) if doc_type else None,

Both are StrEnums. A non-empty value outside the vocabulary raises ValueError, and parsing runs in an unguarded list comprehension (documents = [self._parse_document(doc, ...) for doc in ...]), so one out-of-vocabulary document fails the whole search. SDEIndexedDocumentType has no catch-all member (division at least has Other).

Probed 300 live docs across 6 queries — all division/document_type values are currently in-enum, so this is a hardening gap rather than a live bug.

Fix: map unknown values to None/Other (e.g. NASASMDDivision(x) if x in NASASMDDivision._value2member_map_ else None), or wrap _parse_document per-item so a bad document is skipped rather than fatal.

---

Minor (sde_search.py)

• _check_url_exists uses HTTP HEAD (:149-168): servers that reject HEAD (405/403) but serve GET are dropped as inaccessible. Only affects the non-default validate_urls=True path. Consider GET with a small range, or treating 405 as "accessible".
• New httpx.AsyncClient per URL in the validation gather (:278-279 via _check_url_exists): reuse a single client.
• filtered_count (:309) is computed after truncation to limit, so it reports the returned count, not the number that survived validation.
• score = doc.get("score") or doc.get("_score") or 0.0 (:182): a legitimate 0.0 in score falls through to _score.

---

Notes

• Discovered while building the Code Search Agent Claude plugin (NASA-IMPACT/akd-plugins#6). The plugin's ported sde_tools.py already guards #5 (host check before parsing) and documents #1, so the plugin is not affected by those two; the rest live in the shared logic that the deployed MCP RepositorySearchTool / SDESearchTool run.
• Happy to open PRs for any subset of these.

Start with akd_ext/tools/code_search/utils.py, repository_search.py, and sde_search.py at the referenced lines. Trace metadata fetching, URL parsing, enum conversion, validation, and result counting, then inspect any existing tests for these tools. Done means the confirmed inefficiencies and failure paths are addressed without one bad result or value aborting the whole search.