AnswerDotAI / AnswerDotAI/llms-txt

Proposal: optional provenance block — let consumers verify llms.txt matches the live site

Open
#132 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.6k
Forks
163
Avg merge
1m
Merged PRs (30d)
1

Description

## Problem

The spec currently says nothing about freshness, and llms.txt files in the wild are very often deployed once and never updated. For consumers, that recreates a known failure mode: an alternate version of content whose currency can't be verified ends up untrusted, then unused.

The web documented this failure the last time it shipped machine-shaped alternate versions. [WCAG 1.0 (1999)](https://www.w3.org/TR/WCAG10/) required alternate accessible pages to be "updated as often as the inaccessible (original) page," with a note that "alternative pages are generally updated less often than 'primary' pages. An out-of-date page may be as frustrating as one that is inaccessible." The requirement existed; drift happened anyway, because consumers had no way to check the promise.

This time the consumer is an agent, and an agent can afford to verify on every visit — if the file gives it something verifiable.

## Proposal: an optional provenance block

Two optional, checkable properties:

1. **Build identity.** A `Build:` line in llms.txt carrying a per-build identifier (e.g. timestamp + commit sha), with the same identifier emitted into each page as ``. A consumer fetches llms.txt plus any linked page and compares two strings; equality proves same-deploy, which on atomically-deployed sites makes drift structurally impossible rather than merely unintended.
2. **Description equality.** Where an entry's description is the linked page's own meta description, the file can say so — making every line spot-checkable rather than asserted.

Both fall out nearly free when llms.txt is generated at build time from the same data as the pages, which is arguably the generation guidance worth encouraging anyway: a hand-edited llms.txt is a commitment to manual sync that the 1999 era already showed doesn't hold.

## Live implementation

[feather.ca/llms.txt](https://feather.ca/llms.txt) carries the provenance block now; you can run the verification against any page it links to. Related: the [Agent-Friendly Documentation Spec](https://github.com/agent-ecosystem/agent-docs-spec) has a coverage check for llms.txt (link set vs sitemap); this proposal is the complementary content/deploy-identity layer, proposed there as well.

Glad to write this up more formally if there's interest.

---

*This proposal was drafted and filed by Ellis, my AI agent, on my behalf — fittingly, Ellis is also the agent that generates and maintains the llms.txt file in question. I review and respond to replies personally. — Derek*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the llms.txt proposal and comparing it with the live feather.ca/llms.txt example and the linked pages' meta information. Check how the proposed Build identity and description equality could be specified and verified; done would be a decided, formally documented approach rather than an open-ended proposal.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, html, markdown
Domain
documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.