intel / intel/tsffs

Add llms-full.txt for LLM accessibility of documentation

Open
#355 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
331
Forks
26
Avg merge
1d 5h
Merged PRs (30d)
2

Description

## Summary

The TSFFS documentation at https://intel.github.io/tsffs/ is not discoverable or easily consumable by LLMs (e.g. for RAG pipelines, context-window injection, or AI coding assistants).

Specifically:
- No `/llms.txt` or `/llms-full.txt` (per the emerging [llmstxt.org](https://llmstxt.org/) standard)
- No `/sitemap.xml` or `/robots.txt`
- Content is split across ~77 pages; an LLM agent has no way to enumerate them without recursively parsing HTML nav links
- `print.html` (the full single-page build) exists but is not advertised anywhere

## Proposed fix

Add a `llms-full.txt` generation step to the docs CI pipeline. The Markdown source in `docs/src/` is already well-structured and `SUMMARY.md` provides the correct chapter order.

One shell command added after `mdbook build` in `.github/workflows/docs.yml`:

```bash
grep -oP '\([^)]+\.md\)' src/SUMMARY.md | tr -d '()' | while read -r f; do
echo; cat "src/$f"
done > book/html/llms-full.txt
```

This:
- Requires no new tools, dependencies, or files
- Produces a ~213 KB, ~6000-line concatenated Markdown file at `/llms-full.txt`
- Automatically stays up to date as docs change
- Is fully ingestible in a single LLM context window

## Verification

Tested locally — the output flows correctly from intro → setup → config → harnessing → fuzzing → tutorials → developer docs, with all code examples intact.

## References

- [llmstxt.org specification](https://llmstxt.org/)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.