Prototype Benchmarks: Five Generative UI Approaches Compared

Five prototypes built in parallel on July 6, 2026. All produce standalone HTML. Data from a real benchmark dataset (1280 documents, 8 domains, semantic boundaries).

The Five Prototypes

P1: Static Level (AG-UI Simulator)

User writes a prompt. The LLM decides which pre-registered components to show (chart, card, table). Components render with real data.

  • 3 registered components: BarChart (Chart.js), MetricCard, DataTable
  • 6 quick-prompts + free input
  • LLM receives prompt + component catalog, responds with JSON specifying which components and data
  • Fallback by keyword matching if API is unavailable

Tokens: ~200-500 per interaction (decision only, no code generation) Latency: <2s (decision + instant local rendering) Interactivity: Prompt -> response. Unidirectional.

P2: Full Generative (Level 3)

The LLM generates ALL HTML+CSS+JS. No pre-built components. Each page is unique.

Three pages generated with measured metrics:

Page Input Tokens Output Tokens Latency HTML chars
Architecture overview 221 2,400 16.1s 8,732
Learning curve analysis 228 3,564 25.0s 14,004
Error analysis 295 4,065 27.7s 14,430
Average 248 3,343 22.9s 12,389
Total 744 10,029 68.8s 37,166

Interactivity: None. Static generated pages.

P3: Declarative (A2TL-Web Level 2)

Compact A2TL-Web specs processed by an existing renderer into standalone HTML. No LLM in the rendering process.

Spec Spec chars Spec tokens HTML chars
Architecture overview 1,967 492 10,965
Learning curve 1,572 393 10,419
Error analysis 1,952 488 10,081
Total 5,491 1,373 31,465

Compression ratio: A2TL-Web is 5.7x more compact than the resulting HTML. Latency: Instantaneous (local rendering, ~100ms). Interactivity: Interactive charts (Chart.js hover/tooltips), but not bidirectional.

P4: Bidirectional Loop

Agent generates UI. User interacts. Interaction goes back to agent. Agent generates new UI. Continuous back-and-forth.

  • Cycle 0 (local): dashboard with domain buttons + analysis buttons + free input
  • User clicks or asks -> action sent to LLM with full context + history (last 6 cycles)
  • LLM generates new HTML with buttons/forms -> renders -> new buttons auto-connect -> loop continues
  • Sidebar log: cycle number, action, tokens, timestamp

Tokens: ~3,000-4,000 per cycle (similar to Level 3, but cumulative) Latency: ~20-30s per cycle Interactivity: Maximum. Each interaction produces new UI. The loop is infinite.

P5: Vault-to-Page (Deterministic Pipeline)

Reads a markdown file -> converts to UIDL (deterministic, no LLM) -> renderer generates HTML -> opens in browser.

Step Chars Time
MD source 6,112 —
UIDL generated 5,839 2ms
HTML final 14,653 102ms
Total — 310ms

LLM Tokens: 0 (entirely deterministic) MD-to-UIDL ratio: 0.96x (nearly 1:1) UIDL-to-HTML ratio: 2.51x Interactivity: None in the page, but the script accepts any markdown file as input.

Comparison Table

P1 Static P2 Generative P3 A2TL-Web P4 Bidirectional P5 Vault-to-Page
Level 1 3 2 3+ (collaborative) 2
Tokens/page ~300/interaction 3,343/page 0 (or ~458 if agent generates spec) ~3,500/cycle 0
Latency <2s 22.9s <0.1s 20-30s/cycle 0.3s
HTML output Dynamic 12,389 chars/page 10,488 chars/page Dynamic 14,653 chars
Interactivity Prompt->response None Chart hover Bidirectional continuous None
Visual quality Consistent (components) Variable (depends on LLM) Consistent (design system) Variable Consistent
Predictability High Low Very high Very low Very high
Cost per page Low High Zero Very high (cumulative) Zero
Requires LLM Yes (decision) Yes (generation) No* Yes (continuous generation) No

*The A2TL-Web spec can be generated by a human or an agent. The renderer is deterministic.

Key Findings

1. Level 2 (A2TL-Web) is 7.3x more token-efficient than Level 3

For the same information (architecture overview, learning curve, error analysis):

  • Level 3: 3,343 tokens average per page (LLM generates all HTML)
  • Level 2 A2TL-Web: 458 tokens average per spec (LLM only generates the description)
  • Ratio: 7.3x fewer tokens with A2TL-Web

2. Level 3 latency is prohibitive for interaction

22.9 seconds to generate a page. Level 2 is instantaneous (<100ms). For UIs shown once (reports, dashboards), Level 3 is acceptable. For interactive UIs, it is not.

3. The bidirectional loop works but is expensive

P4 demonstrates the concept is viable: agent generates UI, user interacts, agent re-generates. But each cycle costs ~3.5K tokens and ~25s. For a product, this requires: caching previous cycles, using Level 2 (A2TL-Web) instead of Level 3 to reduce tokens, and pre-generating probable options.

4. Vault-to-Page is the most efficient pipeline

0 LLM tokens. 310ms. Functional HTML. The MD -> UIDL -> HTML pipeline is deterministic, predictable, and free. For content that already exists in structured form, an LLM is not needed. A good parser suffices.

5. Level 3 visual quality is inconsistent

The three HTML files generated by the LLM are functional but each has different styles. Level 2 (A2TL-Web) produces consistent output because it uses the same design system (renderer.js). This matters for a product.

Use Case Recommended Tokens Latency
Existing content (docs, courses) P5: Vault-to-Page 0 <1s
Dashboards and reports P3: A2TL-Web generated by agent ~458 <1s
Personalized UI per user P1: Static/AG-UI pattern ~300 <2s
Free exploration / adaptive tutoring P4: Bidirectional loop ~3.5K/cycle 25s
Demos / one-shots P2: Full Level 3 ~3.3K 23s

The approach is not one level for everything. An orchestrating agent decides which level to use based on the situation: how much personalization is needed, how time-sensitive the interaction is, and whether the content already exists in structured form.