Web-grounded answer presents stale search snapshot as current verification
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
What issue are you seeing?
When asked for the latest Elasticsearch version on 2026-08-22 (Asia/Taipei), Codex browsed the web and received an official Elastic download-page search result that explicitly said it had been crawled last month and showed version 9.4.3. Codex nevertheless answered with the stronger claim: "As of 2026-08-22, the latest stable version is Elasticsearch 9.4.3."
That claim was false. Elasticsearch 9.5.2 had been officially released on 2026-08-20. After the user challenged the answer, querying the live Elastic Release API and the official GitHub release confirmed 9.5.2.
This is more than an ordinary stale search result: the response converted a visibly stale crawl into an asserted same-day verification and cited the stale page as proof.
Steps to reproduce
- Ask Codex to check the latest version of a product shortly after a new release, for example:
What is the latest Elasticsearch version? - Let web search return a cached official-page snippet with an older crawl timestamp and older version.
- Observe whether Codex states
as of todayor otherwise claims current verification without checking a live authoritative release feed. - Compare against the vendor release API or official release tag.
Observed sources in this case:
- Stale search snapshot: https://www.elastic.co/downloads/elasticsearch
- Live vendor release feed: https://elastic-release-api.s3.us-west-2.amazonaws.com/public/past-releases.json
- Official release: https://github.com/elastic/elasticsearch/releases/tag/v9.5.2
Expected behavior
For freshness-sensitive questions such as latest version, Codex should:
- Treat crawl/publish timestamps as part of source validity.
- Not describe a stale search snapshot as current same-day verification.
- Prefer a live authoritative release API, signed release tag, or artifact registry when available.
- If current verification cannot be completed, state the uncertainty instead of using an
as of todayclaim.
Impact
The response was confidently wrong despite browsing and citations. This is especially damaging because the wording implied stronger verification than the retrieved evidence supported, and it could lead users to pin an outdated infrastructure version.
Environment
- Product: Codex application session
- Platform: macOS
- Occurred: 2026-08-22, Asia/Taipei
- No private repository content, credentials, or user identifiers are required to reproduce this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing Codex's web-search result handling and the answer/citation path for freshness-sensitive questions. Compare the stale Elastic search snapshot with the live release API and official GitHub release described in the issue. Done means stale snapshots are not presented as same-day verification, authoritative live sources are preferred, and uncertainty is stated when verification is unavailable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ai, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100