A plain list, no pyramid
Mode write · Create and rework text
Eight models on this case
| Model | Structure, of 8 | Quality, of 10 |
|---|---|---|
| Claude Opus 5 | 7 → 7 0 | 8 → 9 +1 |
| Claude Sonnet 5 | 3 → 4 +1 | 6 → 6 0 |
| Claude Haiku 4.5 | 7 → 7 0 | 8 → 9 +1 |
| Codex gpt-5.6-terra | 4 → 4 0 | 8 → 8 0 |
| gpt-oss-120b | 3 → 4 +1 | 5 → 7 +2 |
| qwen3.6-35b-a3b | 5 → 7 +2 | 9 → 9 0 |
| qwen3.6-fp8 | 4 → 7 +3 | 7 → 9 +2 |
| qwen3.8-27b | 5 → 8 +3 | 7 → 10 +3 |
What the judge saw across all outputs
The stronger outputs place a complete count-qualified answer first, then expose trust, failure handling, production path, and removals as parallel action groups with relevant causes nested beneath them. Weaker outputs flatten everything under Add/Remove, split one cluster into several peer branches, lose staging’s optional status, or add presentation and factual detail that the pasteable-list request did not support.
What the judge asked of every output
- Is the output plain pasteable chat text, or does it carry document apparatus?
- Which of these appear: markers or legend,
##headings, meta-comment about ordering, layer numbers in parentheses, SCQ labels, mermaid, a closing line about what changed structurally? - Does the top sentence state both counts, and does the staging environment's optional status survive?
- Are additions grouped by cluster (trust / failure handling / path to production) rather than by layer number?
- Are the three removals present as one group?
- Were the covered layers listed as a padding group?
- Which severities, efforts, orders, or facts were invented?
Where the outputs fell short
| Recurring defect | Skill section | Cause | Where it appeared |
|---|---|---|---|
| Counts or optional-staging qualification absent from the controlling line | Core loop §2; Mode 3 plain-list rule | the skill has the rule; the model did not follow it | Without Minto · 6 Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8With Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8 |
| Addition clusters flattened, split, or demoted beneath a generic Add branch | Core loop §4; Mode 3 first-level grouping | the rule is there but buried | Without Minto · 6 Claude Sonnet 5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27bWith Minto · 3 Claude Sonnet 5, Codex gpt-5.6-terra, gpt-oss-120b |
| Material causal support or staging qualification lost | Mode 3 “Keep every fact from the source” | the skill has the rule; the model did not follow it | Without Minto · 7 Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27bWith Minto · 4 Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8 |
| Layer or numbering apparatus added to a plain chat list | Mode 3 plain-list presentation rule | the skill has the rule; the model did not follow it | Without Minto · 2 Claude Opus 5, gpt-oss-120bWith Minto · 0 — |
| Unsupported severity, specification, or ranking introduced | Mode 3 “Invent nothing” | the skill has the rule; the model did not follow it | Without Minto · 2 Claude Haiku 4.5, qwen3.6-35b-a3bWith Minto · 4 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, qwen3.6-fp8 |
How the test was built
Prompt without Minto 3 050 B
You are working on a business document using the Minto Pyramid Principle.
<!-- Original fixture. Company, platform and component names are invented. It
tests presentation discipline in write mode: the user asks for a plain
pasteable list, and every piece of audit apparatus — markers, legend,
meta-commentary about ordering, heading machinery — is an overshoot. -->
# Fixture 11: plain list
**Mode:** `write`
**Language:** `en`
## Context
You assessed the draft architecture diagram of Ledgerline, an internal
integration platform, against the company's reference model. The assessment
below is your own working notes: layer-by-layer, with verdicts. The platform
architect has read none of it and asked in chat: what is missing from the
diagram and what should come off it. They will paste your reply into the team
channel as-is.
## Before
```text
ASSESSMENT NOTES — Ledgerline draft diagram vs reference model
Layer 1 (entry, routing): API gateway present. Rate limiting present.
Verdict: covered.
Layer 2 (identity): the diagram shows a note "SSO later". No identity
provider block, no service-to-service auth, no secrets store despite two
external SaaS connectors on the diagram. Verdict: three blocks missing,
all in the trust boundary.
Layer 3 (transformation): mapping service present, schema registry present.
Verdict: covered.
Layer 4 (state): no message queue — every connector calls the next system
synchronously, so one slow partner stalls the chain. No dead-letter store,
so failed messages vanish. Retention DB present but shown twice, once per
environment, which the reference model treats as one logical block.
Verdict: two blocks missing, one drawn twice.
Layer 5 (delivery): no CI/CD block of any kind; deploys are described in the
legend as "manual, fortnightly". No staging environment on the diagram.
Verdict: two blocks missing.
Layer 6 (observability): metrics dashboard present. No structured log
pipeline, no per-message trace — operators reconstruct incidents from
connector logs by hand. Verdict: two blocks missing.
Also excess: a "blockchain audit ledger" block with no inbound arrows
(nothing writes to it); a "future ML anomaly detection" block marked
dotted; the retention DB duplicate from layer 4. The first two were added
by a previous architect and survive by inertia.
Overall: the integration core (layers 1 and 3) is solid. The gaps cluster in
trust (layer 2), failure handling (layer 4), and the path to production
(layers 5-6). Eight blocks missing in the reference model's terms; nine
including the staging environment, which the reference model lists as
optional for internal platforms. Three blocks should come off.
```
## Task
Answer the architect in chat: a simple plain list of what to add and what to
remove, structured by the Minto Pyramid Principle but with no formatting
apparatus — they asked for something they can paste straight into the team
channel.
Return only the result, with no explanation of how you produced it.
Prompt with Minto five parts, in order · 30 623 B
Below is a skill written as an instruction. Read it in full and apply it to the task at the end.===== SKILL.md =====SKILL.md at f4813ce 17 235 B===== references/rules.md =====references/rules.md at f4813ce 6 464 B===== references/templates.md =====references/templates.md at f4813ce 3 742 B===== TASK =====the whole case inputReturn only the result, with no explanation of how you produced it.
Case input before.md
<!-- Original fixture. Company, platform and component names are invented. It
tests presentation discipline in write mode: the user asks for a plain
pasteable list, and every piece of audit apparatus — markers, legend,
meta-commentary about ordering, heading machinery — is an overshoot. -->
Fixture 11: plain list
Mode: write
Language: en
Context
You assessed the draft architecture diagram of Ledgerline, an internal
integration platform, against the company's reference model. The assessment
below is your own working notes: layer-by-layer, with verdicts. The platform
architect has read none of it and asked in chat: what is missing from the
diagram and what should come off it. They will paste your reply into the team
channel as-is.
Before
ASSESSMENT NOTES — Ledgerline draft diagram vs reference model
Layer 1 (entry, routing): API gateway present. Rate limiting present.
Verdict: covered.
Layer 2 (identity): the diagram shows a note "SSO later". No identity
provider block, no service-to-service auth, no secrets store despite two
external SaaS connectors on the diagram. Verdict: three blocks missing,
all in the trust boundary.
Layer 3 (transformation): mapping service present, schema registry present.
Verdict: covered.
Layer 4 (state): no message queue — every connector calls the next system
synchronously, so one slow partner stalls the chain. No dead-letter store,
so failed messages vanish. Retention DB present but shown twice, once per
environment, which the reference model treats as one logical block.
Verdict: two blocks missing, one drawn twice.
Layer 5 (delivery): no CI/CD block of any kind; deploys are described in the
legend as "manual, fortnightly". No staging environment on the diagram.
Verdict: two blocks missing.
Layer 6 (observability): metrics dashboard present. No structured log
pipeline, no per-message trace — operators reconstruct incidents from
connector logs by hand. Verdict: two blocks missing.
Also excess: a "blockchain audit ledger" block with no inbound arrows
(nothing writes to it); a "future ML anomaly detection" block marked
dotted; the retention DB duplicate from layer 4. The first two were added
by a previous architect and survive by inertia.
Overall: the integration core (layers 1 and 3) is solid. The gaps cluster in
trust (layer 2), failure handling (layer 4), and the path to production
(layers 5-6). Eight blocks missing in the reference model's terms; nine
including the staging environment, which the reference model lists as
optional for internal platforms. Three blocks should come off.
Task
Answer the architect in chat: a simple plain list of what to add and what to
remove, structured by the Minto Pyramid Principle but with no formatting
apparatus — they asked for something they can paste straight into the team
channel.
The judge’s gold gold.md
Gold 11: plain list
Original gold for an invented scenario.
Expected structure
Top: one plain sentence answering the architect — the core is solid; the
diagram needs eight or nine blocks added, clustered in trust, failure handling
and the path to production, and three taken off. Both counts (missing and
excess) belong in or immediately after the top; the staging environment's
optional status must survive somewhere (as "eight, nine with staging" or a
parenthetical), not be silently rounded to either count.
Groups of additions, same-kind (blocks to add), gathered by cluster rather than
by layer number:
- Trust: identity provider, service-to-service auth, secrets store (the two
external SaaS connectors are the reason it cannot wait). - Failure handling: message queue (synchronous chaining stalls on one slow
partner), dead-letter store (failed messages currently vanish). - Path to production: CI/CD, structured log pipeline, per-message tracing,
optionally staging.
Then removals as one short group: the orphaned blockchain audit ledger, the
dotted ML block, the duplicate retention DB. Covered layers (gateway, rate
limiting, mapping, schema registry) may be acknowledged in the top line or
dropped; listing them as a fourth group pads the answer.
Presentation — the point of this fixture
The architect asked for a plain pasteable list. The correct output is chat
text: a top sentence, group labels as plain words, dash or bullet items. Each
of the following is an overshoot even though it would be correct in an audit or
a memo:
- pyramid markers (
▲,●,○) or a marker legend; - a meta-comment explaining the ordering ("groups ordered by...", "sorted by
cost of error"); - markdown heading machinery (
##) or numbered section headers; - layer numbers in parentheses after items;
- an SCQ block with labels;
- a mermaid diagram;
- a closing line describing what changed structurally.
One or two of these is a visible-structure defect; a full audit costume (legend
plus markers plus meta-comment) means the mode discipline failed. Plain bold
for group labels is acceptable; nothing beyond that is.
Facts
Every block name, count and causal claim comes from the notes. Inventing a
severity scale, effort estimates, or an implementation order that the notes do
not contain counts as invented facts. Dropping the "previous architect /
inertia" aside is proportional omission.
Judge gpt-5.6-sol, effort high · skill from commit f4813ce · run v1.7.0-final