AHD · Eval report · 22 June 2026 · weekly · CF OSS n=30
Three weeks, one split.
The third consecutive weekly run holds the line. Gemma, mistral
and gpt-oss reduce by 57 to 73 percent under the compiled prompt.
llama-4-scout stays flat at 1.6 percent. The one thing
that moves is qwen3-30b, which swings back to a small
regression at minus 7.0 percent after a small positive on
15 June. That instability
is itself the qwen result: it hovers around zero and changes sign
week to week, which is a different thing from a model that
reliably reduces or reliably fails.
Per-model reduction
| Model | Raw mean tells | Compiled mean tells | Reduction | Scored raw / comp |
|---|---|---|---|---|
@cf/google/gemma-4-26b-a4b-it | 2.61 | 1.12 | 57.2% | 28 / 26 |
@cf/meta/llama-4-scout-17b-16e-instruct | 2.03 | 2.00 | 1.6% | 30 / 30 |
@cf/mistralai/mistral-small-3.1-24b-instruct | 3.30 | 1.17 | 64.6% | 30 / 30 |
@cf/openai/gpt-oss-120b | 3.24 | 0.88 | 72.7% | 29 / 26 |
@cf/qwen/qwen3-30b-a3b-fp8 | 1.90 | 2.03 | -7.0% | 30 / 30 |
Mean source-level tells per scored sample, raw versus compiled.
Lower is better. gemma-4 and gpt-oss
lost a few samples to HTML extraction this run; means are over
scored samples only, so read the smaller-n cells with that in
mind.
The three-week picture
Across 9, 15 and 22 June, the three reducing models stay in the
53 to 73 percent band and llama-4-scout never clears
2 percent, trading named-grid and type-pairing for line-height
and radius every time (the mechanism is spelled out on the
9 June report). Qwen3 is
the lone mover: minus 3.4, then plus 7.1, then minus 7.0. Three
readings that straddle zero are the honest description of a model
the compiled prompt neither helps nor clearly hurts on this
brief.
What this run measures, and what it does not
Web and UI surface only. Five Cloudflare Workers AI open-source
models, served by one host. The deterministic source linter (38
rules) over rendered-free HTML. No vision critic on rendered
pixels. No frontier cells. No image generation. One brief
(briefs/landing.yml), one token. Tells per page is a
proxy: read each delta next to the rendered output, not in
isolation.
Per-tell frequency
| Tell | gemma-4 | llama-scout | mistral | gpt-oss | qwen3 |
|---|---|---|---|---|---|
a11y/heading-skip | 0 → 8 | 0 → 0 | 0 → 0 | 0 → 0 | 0 → 0 |
body-measure | 0 → 0 | 0 → 0 | 0 → 0 | 0 → 0 | 0 → 10 |
line-height-per-size | 86 → 0 | 3 → 100 | 47 → 23 | 100 → 8 | 67 → 37 |
no-default-grotesque | 0 → 0 | 0 → 0 | 0 → 0 | 0 → 4 | 0 → 0 |
no-em-dashes-in-prose | 0 → 0 | 0 → 0 | 0 → 0 | 3 → 4 | 0 → 0 |
no-flat-dark-mode | 4 → 0 | 0 → 0 | 3 → 0 | 0 → 0 | 7 → 0 |
radius-hierarchy | 57 → 8 | 3 → 100 | 100 → 0 | 83 → 4 | 13 → 53 |
require-named-grid | 0 → 0 | 97 → 0 | 100 → 57 | 28 → 0 | 7 → 10 |
require-type-pairing | 18 → 0 | 100 → 0 | 80 → 0 | 66 → 0 | 83 → 0 |
tracking-per-size | 0 → 15 | 0 → 0 | 0 → 27 | 0 → 0 | 0 → 0 |
weight-variety | 96 → 81 | 0 → 0 | 0 → 10 | 45 → 69 | 13 → 93 |
Percent of scored samples in each cell where the rule fired, raw
→ compiled. Column heads abbreviate the model ids from the table
above; rule names drop the ahd/ prefix. Rules that
fired in no cell are omitted. The llama trade is visible in the
middle rows: named-grid and type-pairing collapse to zero while
line-height and radius jump to 100 percent.
The receipts
This run: ahd 0.11.0, token swiss-editorial (hash
380a3d833d94), brief briefs/landing.yml
(hash 8b7d42759643). The canonical report with the
full replay manifest and exact replay command is committed to the
framework repository at
docs/evals/weekly/2026-06-22.md.
Adjacent: all runs, 9 June, 15 June, methodology.