Engineering Blueprint: Serving /llms.txt, /pricing.md & /ai/summary.json Without SPA Soft-404s
Updated 2026-11-14 · Empirical telemetry across 2459 B2B SaaS domains
Modern React, Vite, and Next.js marketing sites frequently return HTTP 200 text/html catch-all shells on unknown routes instead of deterministic plain text or JSON. When OAI-SearchBot, ClaudeBot, or PerplexityBot requests /llms.txt, /pricing.md, or /ai/summary.json, receiving a 239.7 KB HTML shell with 29.0 client scripts means automated HTTP checks receive client-hydrated HTML instead of plain-text product and pricing facts.
1. Required Deterministic Agent Endpoints (14 Routes)
| Route | Content-Type | Retrieval Purpose | Missing Across 2459 SaaS Domains |
|---|---|---|---|
/llms.txt & /llms-full.txt | text/plain; charset=utf-8 | Canonical entity summary & full technical corpus | 35.7% missing /llms.txt |
/pricing.md & /answers.md | text/markdown; charset=utf-8 | Exact USD fee schedule & 50-word factual answers | 87.6% missing |
/ai/summary.json & /ai/faq.json | application/json; charset=utf-8 | Typed entity attributes, SOC2/SLA facts & Q&A | 99.8% missing |
/.well-known/ai.txt, mcp.json & agent-card.json | text/plain / application/json | 27-crawler permissions, MCP, A2A v0.2.5 & OpenAPI 3.1 links | >98% missing |
2. Zero-Origin Edge Interception Pattern (<15ms)
Intercept agent discovery routes at your CDN edge (Cloudflare Worker, Vercel middleware.ts, Astro/SvelteKit src/middleware.ts, FastAPI, or Nginx) before requests hit your SPA router, and append RFC 8288 Link: </llms.txt>; rel="llms-txt" headers on HTML responses. Every pre-built teardown in our 2459-domain index includes copy-pasteable routers for all 5 stacks.