Proposal: WebPageSemanticRepresentation type for AI agent consumption

Open
#4,786 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
json
Domain
data

Research direction

Read the linked full proposal in plasmate-labs/plasmate/docs/SCHEMA-ORG-PROPOSAL.md, then review Schema.org's existing type hierarchy and related properties. Done means determining whether the proposed WebPageSemanticRepresentation type and its listed properties fit Schema.org's scope and can be modeled consistently.

Written by the indexing model from the issue text.

Description

no-issue-activity

Summary

As AI agents increasingly consume web content, there is a growing need for a standardized way to describe machine-readable semantic representations of web pages.

We propose a new Schema.org type, WebPageSemanticRepresentation, that allows websites to advertise the availability of semantic page representations optimized for AI consumption.

Motivation

AI agents currently interact with web pages by fetching raw HTML and parsing the DOM (often via headless Chrome). This is expensive, slow, and wasteful - 90%+ of tokens sent to LLMs are presentational noise.

Several projects now produce structured semantic representations:

  • Plasmate SOM (Semantic Object Model)
  • Jina Reader API
  • Firecrawl
  • Accessibility trees

A Schema.org type would let websites signal that a semantic representation is available, its format, and where to fetch it.

Proposed Properties

Property Type Description
representationFormat Text Format (e.g., 'SOM/1.0', 'markdown')
representationUrl URL Where to fetch the representation
sourceUrl URL The original page URL
tokenCount Integer Approximate token count
compressionRatio Number HTML tokens / representation tokens
generatedBy SoftwareApplication Tool that generated it
contentScope Text What's included (full-page, main-content, etc.)
lastCrawled DateTime When the source was last fetched

Example

{
  "@context": "https://schema.org",
  "@type": "WebPageSemanticRepresentation",
  "sourceUrl": "https://example.com/article",
  "representationUrl": "https://cache.example.com/som?url=https://example.com/article",
  "representationFormat": "SOM/1.0",
  "tokenCount": 1200,
  "compressionRatio": 14.2
}

Context

This proposal comes from the W3C Web Content Browser for AI Agents Community Group, where we are incubating the Semantic Object Model (SOM) and Agent Web Protocol (AWP) specifications.

Full proposal: https://github.com/plasmate-labs/plasmate/blob/master/docs/SCHEMA-ORG-PROPOSAL.md

We would welcome feedback on whether this fits within Schema.org's scope and how to best model it within existing type hierarchies.

Dominant language
HTML
Stars
6.3k
Forks
962
Avg merge
18h 7m
Merged PRs (30d)
9

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from schemaorg/schemaorg

All issues in schemaorg/schemaorg

Similar issues

More Data Engineering issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.