AnswerDotAI / AnswerDotAI/llms-txt
Proposal: HTML data-llm Attributes for Enhanced AI Content Understanding
- Dominant language
- Jupyter Notebook
- Stars
- 2.6k
- Forks
- 163
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
## Summary
I propose extending the LLMs.txt standard to include **inline HTML data attributes** that provide AI-friendly structured data directly within web page elements. This would complement the existing `/llms.txt` file approach by solving context preservation issues, particularly for complex content like comparison tables, pricing information, and structured data.
## Problem Statement
Current LLMs.txt helps AI systems locate important content, but doesn't address the fundamental challenge of **semantic disambiguation** within that content. Specifically:
### Table and Structured Content Issues
- **Lost context**: When RAG systems scrape comparison tables, they often confuse "our pricing" with "competitor pricing"
- **Relationship fragmentation**: Table headers become disconnected from their data during embedding
- **Ambiguous ownership**: Content like "$50/month" loses meaning without knowing which company/product it refers to
### Real-World Example
On pages like comparison tables (e.g., "Formester vs Fillout"), current AI systems might incorrectly extract:
- ❌ "Formester costs $20/month" (actually Fillout's price)
- ❌ "Our basic plan includes 20MB uploads" (actually competitor's feature)
## Proposed Solution: `data-llm` Attributes
Add standardized `data-llm` attributes to HTML elements containing structured JSON that provides AI-friendly context and semantics.
### Basic Syntax
```html
```
### Example Implementations
#### Pricing Comparison Tables
```html
```
#### Product Information
```html
```
#### Contact Information
```html
```
## Benefits
### 1. **Solves Context Preservation**
- AI systems can definitively distinguish "our" vs "competitor" information
- Table relationships are explicitly maintained in structured form
- No more pricing confusion in RAG responses
### 2. **Backward Compatible**
- Doesn't interfere with existing HTML, CSS, or JavaScript
- Works alongside current LLMs.txt files
- Search engines ignore unknown data attributes
### 3. **Developer Friendly**
- Easy to implement during development
- Single source of truth - update once, both human and AI versions stay current
- No separate file management required
### 4. **Scalable**
- Works for any type of content, not just tables
- Extensible schema system for different content types
- Can be validated against JSON schemas
## Integration with LLMs.txt
This proposal **complements** rather than replaces LLMs.txt:
1. **LLMs.txt** - Guides AI to important pages and sections
2. **`data-llm` attributes** - Provides semantic understanding of content within those pages
### Updated LLMs.txt Example
```markdown
# Formester
> AI-powered form builder with advanced features
## Pricing Information
- [Pricing comparison](https://formester.com/pricing): Compare our plans with competitors
- Note: Contains `data-llm` attributes for accurate pricing extraction
- [Feature matrix](https://formester.com/features): Detailed feature breakdown
- Note: Uses structured attributes for feature categorization
```
## Implementation Strategy
### Phase 1: Schema Definition
- Define common content types (`pricing_comparison`, `our_product`, `company_contact`, etc.)
- Create JSON schema specifications for validation
- Document best practices and examples
### Phase 2: Tooling
- Build parsers for common RAG frameworks
- Create validation tools for developers
- Develop browser extensions for testing
### Phase 3: Community Adoption
- Share with RAG system builders
- Integrate
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the /llms.txt approach and the proposed HTML data-llm examples, then review the schema-definition, parser, validation, and browser-extension phases described here. This issue is done only after the schema, JSON validation rules, implementation scope, and documentation are agreed; no repository file or test is named yet.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, documentation, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100