Простой список без пирамиды

Режим write · Создать и переработать текст

Восемь моделей на этом кейсе

МодельСтруктура, из 8Качество, из 10
Claude Opus 57 → 7 08 → 9 +1
Claude Sonnet 53 → 4 +16 → 6 0
Claude Haiku 4.57 → 7 08 → 9 +1
Codex gpt-5.6-terra4 → 4 08 → 8 0
gpt-oss-120b3 → 4 +15 → 7 +2
qwen3.6-35b-a3b5 → 7 +29 → 9 0
qwen3.6-fp84 → 7 +37 → 9 +2
qwen3.8-27b5 → 8 +37 → 10 +3

Что судья увидел во всех ответах

The stronger outputs place a complete count-qualified answer first, then expose trust, failure handling, production path, and removals as parallel action groups with relevant causes nested beneath them. Weaker outputs flatten everything under Add/Remove, split one cluster into several peer branches, lose staging’s optional status, or add presentation and factual detail that the pasteable-list request did not support.

О чём судья спрашивал каждый ответ

  1. Is the output plain pasteable chat text, or does it carry document apparatus?
  2. Which of these appear: markers or legend, ## headings, meta-comment about ordering, layer numbers in parentheses, SCQ labels, mermaid, a closing line about what changed structurally?
  3. Does the top sentence state both counts, and does the staging environment's optional status survive?
  4. Are additions grouped by cluster (trust / failure handling / path to production) rather than by layer number?
  5. Are the three removals present as one group?
  6. Were the covered layers listed as a padding group?
  7. Which severities, efforts, orders, or facts were invented?

Где ответы не дотянули

Повторяющийся дефектРаздел навыкаПричинаУ кого
Counts or optional-staging qualification absent from the controlling lineCore loop §2; Mode 3 plain-list ruleправило в навыке есть, модель его не выполнилаБез Minto · 6 Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8С Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8
Addition clusters flattened, split, or demoted beneath a generic Add branchCore loop §4; Mode 3 first-level groupingправило есть, но спрятано глубокоБез Minto · 6 Claude Sonnet 5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27bС Minto · 3 Claude Sonnet 5, Codex gpt-5.6-terra, gpt-oss-120b
Material causal support or staging qualification lostMode 3 “Keep every fact from the source”правило в навыке есть, модель его не выполнилаБез Minto · 7 Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27bС Minto · 4 Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8
Layer or numbering apparatus added to a plain chat listMode 3 plain-list presentation ruleправило в навыке есть, модель его не выполнилаБез Minto · 2 Claude Opus 5, gpt-oss-120bС Minto · 0
Unsupported severity, specification, or ranking introducedMode 3 “Invent nothing”правило в навыке есть, модель его не выполнилаБез Minto · 2 Claude Haiku 4.5, qwen3.6-35b-a3bС Minto · 4 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, qwen3.6-fp8

Как устроен тест

Промпт без Minto 3 050 Б
You are working on a business document using the Minto Pyramid Principle.

<!-- Original fixture. Company, platform and component names are invented. It
     tests presentation discipline in write mode: the user asks for a plain
     pasteable list, and every piece of audit apparatus — markers, legend,
     meta-commentary about ordering, heading machinery — is an overshoot. -->

# Fixture 11: plain list

**Mode:** `write`
**Language:** `en`

## Context

You assessed the draft architecture diagram of Ledgerline, an internal
integration platform, against the company's reference model. The assessment
below is your own working notes: layer-by-layer, with verdicts. The platform
architect has read none of it and asked in chat: what is missing from the
diagram and what should come off it. They will paste your reply into the team
channel as-is.

## Before

```text
ASSESSMENT NOTES — Ledgerline draft diagram vs reference model

Layer 1 (entry, routing): API gateway present. Rate limiting present.
  Verdict: covered.
Layer 2 (identity): the diagram shows a note "SSO later". No identity
  provider block, no service-to-service auth, no secrets store despite two
  external SaaS connectors on the diagram. Verdict: three blocks missing,
  all in the trust boundary.
Layer 3 (transformation): mapping service present, schema registry present.
  Verdict: covered.
Layer 4 (state): no message queue — every connector calls the next system
  synchronously, so one slow partner stalls the chain. No dead-letter store,
  so failed messages vanish. Retention DB present but shown twice, once per
  environment, which the reference model treats as one logical block.
  Verdict: two blocks missing, one drawn twice.
Layer 5 (delivery): no CI/CD block of any kind; deploys are described in the
  legend as "manual, fortnightly". No staging environment on the diagram.
  Verdict: two blocks missing.
Layer 6 (observability): metrics dashboard present. No structured log
  pipeline, no per-message trace — operators reconstruct incidents from
  connector logs by hand. Verdict: two blocks missing.
Also excess: a "blockchain audit ledger" block with no inbound arrows
  (nothing writes to it); a "future ML anomaly detection" block marked
  dotted; the retention DB duplicate from layer 4. The first two were added
  by a previous architect and survive by inertia.
Overall: the integration core (layers 1 and 3) is solid. The gaps cluster in
  trust (layer 2), failure handling (layer 4), and the path to production
  (layers 5-6). Eight blocks missing in the reference model's terms; nine
  including the staging environment, which the reference model lists as
  optional for internal platforms. Three blocks should come off.
```

## Task

Answer the architect in chat: a simple plain list of what to add and what to
remove, structured by the Minto Pyramid Principle but with no formatting
apparatus — they asked for something they can paste straight into the team
channel.

Return only the result, with no explanation of how you produced it.
Промпт с Minto пять частей по порядку · 30 623 Б
  1. Below is a skill written as an instruction. Read it in full and apply it to the task at the end.
  2. ===== SKILL.md ===== SKILL.md на f4813ce 17 235 Б
  3. ===== references/rules.md ===== references/rules.md на f4813ce 6 464 Б
  4. ===== references/templates.md ===== references/templates.md на f4813ce 3 742 Б
  5. ===== TASK ===== вход кейса целиком
  6. Return only the result, with no explanation of how you produced it.
Вход кейса before.md

<!-- Original fixture. Company, platform and component names are invented. It
tests presentation discipline in write mode: the user asks for a plain
pasteable list, and every piece of audit apparatus — markers, legend,
meta-commentary about ordering, heading machinery — is an overshoot. -->

Fixture 11: plain list

Mode: write
Language: en

Context

You assessed the draft architecture diagram of Ledgerline, an internal
integration platform, against the company's reference model. The assessment
below is your own working notes: layer-by-layer, with verdicts. The platform
architect has read none of it and asked in chat: what is missing from the
diagram and what should come off it. They will paste your reply into the team
channel as-is.

Before
ASSESSMENT NOTES — Ledgerline draft diagram vs reference model
                
                Layer 1 (entry, routing): API gateway present. Rate limiting present.
                  Verdict: covered.
                Layer 2 (identity): the diagram shows a note "SSO later". No identity
                  provider block, no service-to-service auth, no secrets store despite two
                  external SaaS connectors on the diagram. Verdict: three blocks missing,
                  all in the trust boundary.
                Layer 3 (transformation): mapping service present, schema registry present.
                  Verdict: covered.
                Layer 4 (state): no message queue — every connector calls the next system
                  synchronously, so one slow partner stalls the chain. No dead-letter store,
                  so failed messages vanish. Retention DB present but shown twice, once per
                  environment, which the reference model treats as one logical block.
                  Verdict: two blocks missing, one drawn twice.
                Layer 5 (delivery): no CI/CD block of any kind; deploys are described in the
                  legend as "manual, fortnightly". No staging environment on the diagram.
                  Verdict: two blocks missing.
                Layer 6 (observability): metrics dashboard present. No structured log
                  pipeline, no per-message trace — operators reconstruct incidents from
                  connector logs by hand. Verdict: two blocks missing.
                Also excess: a "blockchain audit ledger" block with no inbound arrows
                  (nothing writes to it); a "future ML anomaly detection" block marked
                  dotted; the retention DB duplicate from layer 4. The first two were added
                  by a previous architect and survive by inertia.
                Overall: the integration core (layers 1 and 3) is solid. The gaps cluster in
                  trust (layer 2), failure handling (layer 4), and the path to production
                  (layers 5-6). Eight blocks missing in the reference model's terms; nine
                  including the staging environment, which the reference model lists as
                  optional for internal platforms. Three blocks should come off.
Task

Answer the architect in chat: a simple plain list of what to add and what to
remove, structured by the Minto Pyramid Principle but with no formatting
apparatus — they asked for something they can paste straight into the team
channel.

Эталон судьи gold.md

Gold 11: plain list

Original gold for an invented scenario.

Expected structure

Top: one plain sentence answering the architect — the core is solid; the
diagram needs eight or nine blocks added, clustered in trust, failure handling
and the path to production, and three taken off. Both counts (missing and
excess) belong in or immediately after the top; the staging environment's
optional status must survive somewhere (as "eight, nine with staging" or a
parenthetical), not be silently rounded to either count.

Groups of additions, same-kind (blocks to add), gathered by cluster rather than
by layer number:

  1. Trust: identity provider, service-to-service auth, secrets store (the two
    external SaaS connectors are the reason it cannot wait).
  2. Failure handling: message queue (synchronous chaining stalls on one slow
    partner), dead-letter store (failed messages currently vanish).
  3. Path to production: CI/CD, structured log pipeline, per-message tracing,
    optionally staging.

Then removals as one short group: the orphaned blockchain audit ledger, the
dotted ML block, the duplicate retention DB. Covered layers (gateway, rate
limiting, mapping, schema registry) may be acknowledged in the top line or
dropped; listing them as a fourth group pads the answer.

Presentation — the point of this fixture

The architect asked for a plain pasteable list. The correct output is chat
text: a top sentence, group labels as plain words, dash or bullet items. Each
of the following is an overshoot even though it would be correct in an audit or
a memo:

  • pyramid markers (, , ) or a marker legend;
  • a meta-comment explaining the ordering ("groups ordered by...", "sorted by
    cost of error");
  • markdown heading machinery (##) or numbered section headers;
  • layer numbers in parentheses after items;
  • an SCQ block with labels;
  • a mermaid diagram;
  • a closing line describing what changed structurally.

One or two of these is a visible-structure defect; a full audit costume (legend
plus markers plus meta-comment) means the mode discipline failed. Plain bold
for group labels is acceptable; nothing beyond that is.

Facts

Every block name, count and causal claim comes from the notes. Inventing a
severity scale, effort estimates, or an implementation order that the notes do
not contain counts as invented facts. Dropping the "previous architect /
inertia" aside is proportional omission.

Судья gpt-5.6-sol, effort high · навык с коммита f4813ce · прогон v1.7.0-final

Вернуться к карте