Automattic / Automattic/blocks-engine
Replace CommonMark with a Blocks Engine-owned Markdown parser
- Dominant language
- PHP
- Stars
- 14
- Forks
- 2
- Avg merge
- 2h 10m
- Merged PRs (30d)
- 561
Description
## Goal
Make HTML the canonical content-format contract and remove the PHP Transformer's direct third-party Markdown runtime dependencies while preserving first-class Markdown conversion in the standard package. Specialized source-vendored builds may continue to omit the Markdown adapter through the existing optional runtime seam.
## Context
`FormatBridge\MarkdownAdapter` currently delegates Markdown-to-HTML conversion to `league/commonmark` and HTML-to-Markdown conversion to `league/html-to-markdown`. Composer consumers therefore inherit the full dependency graph even though WordPress.com already vendors a lean `src/` build that excludes `MarkdownAdapter.php` and League for its HTML-first site-builder pipeline.
Markdown remains a required product capability. Intelligence stores wiki content canonically as Markdown, Data Machine converts at render/editor boundaries, Markdown Database Integration uses the transformer for import/export, and Static Site Importer accepts nested `.md`/`.markdown` documents. Storage and product layers should consume conversion rather than own it.
The generic conversion primitive belongs upstream in WordPress PHP Toolkit. [WordPress/php-toolkit#313](https://github.com/WordPress/php-toolkit/issues/313) tracks extracting a direct Markdown `<->` HTML codec and replacing the toolkit Markdown component's embedded CommonMark graph. Blocks Engine should adapt that maintained primitive into its result and block contracts rather than duplicate a generic parser.
The canonical composition is:
```text
Markdown <-> HTML <-> WordPress blocks
```
Direct Markdown-to-HTML and HTML-to-Markdown requests should not make a lossy round trip through blocks.
## Delivery Plan
1. Formalize and test direct Markdown-to-HTML and HTML-to-Markdown routing while retaining League as the reference implementation ([#1523](https://github.com/Automattic/blocks-engine/pull/1523)).
2. Contribute the reusable self-contained Markdown `<->` HTML codec upstream through WordPress/php-toolkit#313.
3. Build differential corpus coverage from transformer/SSI fixtures and representative Markdown-first content.
4. Replace the League internals of `MarkdownAdapter` with the released PHP Toolkit codec.
5. Remove both direct League dependencies and their transitive dependency graph from PHP Transformer.
6. Update downstream pins and packaging manifests, then enable self-contained Markdown support in the WordPress.com vendored copy.
## Acceptance Criteria
- HTML is the explicit canonical interchange contract for Markdown and block composition.
- Direct Markdown-to-HTML and HTML-to-Markdown conversions do not route through blocks.
- The standard PHP Transformer package retains Markdown support without direct dependencies on `league/commonmark` or `league/html-to-markdown`.
- `MarkdownAdapter` consumes the generic PHP Toolkit codec rather than carrying a duplicate parser.
- The existing optional adapter contract remains valid for specialized builds that omit `MarkdownAdapter.php`.
- Markdown-to-blocks and blocks-to-Markdown compose through HTML and retain stable result envelopes.
- Tables, headings, lists, links, images, blockquotes, code fences, raw HTML islands, dynamic block comments, and normalization remain deterministic.
- Corpus, fuzz, bounded-input, and performance coverage validates the upstream codec before adoption.
- Intelligence/Data Machine render and editor-save flows preserve canonical Markdown storage.
- MDI import/export and Static Site Importer mixed-source imports retain product behavior.
- Static Site Importer no longer packages duplicate League transitive vendor namespaces solely for Blocks Engine Markdown conversion.
- WordPress.com can vendor Markdown support from maintained WordPress-owned source without the current Composer vendor trees.
- Storage layers continue to persist bytes without acquiring Markdown conversion logic.
## Migration
Keep the adapter and result contracts stable so downstream callers need no compatibility layer. Land direct HTML routing first, validate the PHP Toolkit codec differentially against the existing implementation, then change the adapter dependency in a normal PHP Transformer release and update downstream package pins.
---
AI assistance disclosure: OpenAI gpt-5.6-sol via OpenCode inspected Blocks Engine, WordPress PHP Toolkit, Intelligence, Data Machine, Markdown Database Integration, Static Site Importer, and WordPress.com integration boundaries and refined this tracker. Chris Huber directed the upstream ownership decision and will review the implementation.
Contributor guide
Research direction
Start with FormatBridge\MarkdownAdapter.php and the direct HTML routing work tracked in #1523. Read the upstream PHP Toolkit codec proposal in php-toolkit#313, then inspect transformer and Static Site Importer fixtures for differential coverage. Done means stable HTML-based composition, preserved adapter contracts, deterministic corpus results, and removal of the League dependency graph.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- markdown, php
- Domain
- backend, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100