vercel-labs / vercel-labs/json-render
Enable `catalog.prompt()`-generated system prompts support for smaller context window sizes
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 16.8k
- Forks
- 901
- Avg merge
- 3h 14m
- Merged PRs (30d)
- 4
Description
The system prompt generated by catalog.prompt() is approximately 4,300 tokens for our catalog of 18 components.
This creates problems for small language models and models running locally on browsers, mobile devices, and other edge environments. Many of these models support relatively small context windows (at most ~4k tokens). A 4,300-token system prompt, alone, cannot fit inside this context window.
For now, we manually reduced the system prompt to approximately 2,000 tokens, so json-render can work on a SLM.
Desired outcome
Provide a way in PromptOptions to enable the caller of catalog.prompt() to generate a much smaller system prompt.
Implementation Ideas:
sizeoption: a rough t-shirt size (sm,md,lg) for high level prompt length control, (wheresmtargets the prompt to ~1k tokens,md2k tokens, andlgthe current size)sectionsoptions +sizeoption: grandular control over the specificity at each sectiontemplateoption (with sections + size hooks): allow user to add a handlebars-like template with preset helpers for sections. This is the most robust and flexible but could potentially introduce a new dependency.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with PromptOptions in packages/core/src/schema.ts and locate the catalog.prompt() implementation that assembles the system prompt. Compare the proposed size, sections, and template approaches against the existing prompt structure. Done means callers can request a substantially smaller prompt suitable for approximately 2k-token context budgets without losing the required catalog behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, api
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100