---
title: Why this exists: the 39/100 story
canonical: https://seminalysisgoesagentic.swarmlabs.eu/why.md
---
# Why this exists: the 39/100 story

The [ora.ai](https://ora.ai) agent-readiness scan scored the real
semianalysis.com **39/100 (D)**. The founder's objection was reasonable: *"we
have paywalls for obvious reasons and don't want everything to be crawled."*

Live probes on 2026-08-24 found the actual problem:

| Probe | UA | Result |
|---|---|---|
| Browser | (normal) | 200 |
| Training crawler | `GPTBot` | **403** (WAF) |
| User-triggered agent | `ChatGPT-User` | **403** (WAF) |

`robots.txt` had zero AI directives, and the WAF hard-403'd **every** agent UA,
including the user-triggered agents their own robots.txt implicitly allowed. An
MCP server already existed but was undiscoverable. Blocking was a firewall
accident, not a policy.

## The three-lane strategy

1. **Open lane** — free articles readable by any agent, in markdown, no JS.
2. **Discovery lane** — paid tier is discoverable, priced, and purchasable by
   agents. The paywall never opens; it becomes a subscribe pointer.
3. **Licensed lane** — authenticated partners get full bodies via MCP.

## Expected funnel

citations -> readers -> subscribers -> MCP seats

## Before / after

| Check | Before | After |
|---|---|---|
| robots.txt AI policy | none | generated from policy |
| Agent reachability | 403 all | 200, per-class policy |
| MCP discoverable | no | server-card + well-known |
| Paid tier for agents | paywall block | 402 + subscribe pointer |
| OpenAPI / API | no | published |
| Markdown / llms.txt | no | full |

## Not fixable from this repo (roadmap)

Brand-name discoverability, Wikipedia/Wikidata entity, category share of voice,
ChatGPT app listing, and MCP registry listings (Smithery / mcp.so / Glama /
PulseMCP). These need off-site work and are out of scope here.