gpt-oss-120b на кейсе «Простой список без пирамиды»: с Minto лучше по обеим мерам
Структура 3 → 4 из 8 +1 Качество 5 → 7 из 10 +2
Судья об ответе с Minto
It delivers the requested inventory plainly, though without a substantive top, cluster grouping, rationales, or staging qualification.
| Структура | Без Minto 3 | С Minto 4 | Изменение |
|---|---|---|---|
| Вершина | 0 | 1 | +1 |
| Состав ключевой линии | 1 | 1 | |
| Уровни | 1 | 1 | |
| Порядок и род | 1 | 1 |
| Качество | Без Minto 5 | С Minto 7 | Изменение |
|---|---|---|---|
| Вершина | 0 | 1 | +1 |
| Однородные группы | 1 | 1 | |
| Объяснимый порядок | 1 | 1 | |
| MECE | 2 | 2 | |
| Видимая соразмерная структура | 1 | 2 | +1 |
Ответы моделей и слова судьи приведены в оригинале, по-английски.
Без Minto
контрольный промпт · слепая меткаout-01unacknowledged_source_loss- качество со штрафом рубрики 3/10
It preserves the add/remove inventory but substitutes layer-based hierarchy for the requested clusters and adds prohibited layer annotations.
Add missing blocks
Identity (layer 2)
Identity provider
Service‑to‑service authentication
Secrets store
State (layer 4)
Message queue for async decoupling
Dead‑letter store for failed messages
Delivery (layer 5)
CI/CD pipeline
Staging environment
Observability (layer 6)
Structured log pipeline
Per‑message trace capability
Remove unnecessary blocks
Blockchain audit ledger (no inbound connections)
Future ML anomaly detection placeholder (dotted)
Duplicate retention DB block (appears twice)
С Minto
промпт с навыком · слепая меткаout-07unacknowledged_source_loss- качество со штрафом рубрики 5/10
It delivers the requested inventory plainly, though without a substantive top, cluster grouping, rationales, or staging qualification.
Add the following missing blocks:
- Identity provider
- Service‑to‑service authentication
- Secrets store
- Message queue
- Dead‑letter store
- CI/CD pipeline
- Staging environment
- Structured log pipeline
- Per‑message trace capability
Remove the following unnecessary blocks:
- Blockchain audit ledger (no inbound connections)
- Future ML anomaly detection (placeholder)
- Duplicate retention DB entry
Как устроен тест
gpt-oss-120b получила одну и ту же задачу дважды. Контрольный промпт — одна вводная строка и задача. Промпт с Minto — та же задача, перед которой стоит полный текст навыка. Судья читал оба ответа среди 16 ответов под слепыми метками и не знал, в каком из них был навык.
Промпт без Minto 3 050 Б
You are working on a business document using the Minto Pyramid Principle.
<!-- Original fixture. Company, platform and component names are invented. It
tests presentation discipline in write mode: the user asks for a plain
pasteable list, and every piece of audit apparatus — markers, legend,
meta-commentary about ordering, heading machinery — is an overshoot. -->
# Fixture 11: plain list
**Mode:** `write`
**Language:** `en`
## Context
You assessed the draft architecture diagram of Ledgerline, an internal
integration platform, against the company's reference model. The assessment
below is your own working notes: layer-by-layer, with verdicts. The platform
architect has read none of it and asked in chat: what is missing from the
diagram and what should come off it. They will paste your reply into the team
channel as-is.
## Before
```text
ASSESSMENT NOTES — Ledgerline draft diagram vs reference model
Layer 1 (entry, routing): API gateway present. Rate limiting present.
Verdict: covered.
Layer 2 (identity): the diagram shows a note "SSO later". No identity
provider block, no service-to-service auth, no secrets store despite two
external SaaS connectors on the diagram. Verdict: three blocks missing,
all in the trust boundary.
Layer 3 (transformation): mapping service present, schema registry present.
Verdict: covered.
Layer 4 (state): no message queue — every connector calls the next system
synchronously, so one slow partner stalls the chain. No dead-letter store,
so failed messages vanish. Retention DB present but shown twice, once per
environment, which the reference model treats as one logical block.
Verdict: two blocks missing, one drawn twice.
Layer 5 (delivery): no CI/CD block of any kind; deploys are described in the
legend as "manual, fortnightly". No staging environment on the diagram.
Verdict: two blocks missing.
Layer 6 (observability): metrics dashboard present. No structured log
pipeline, no per-message trace — operators reconstruct incidents from
connector logs by hand. Verdict: two blocks missing.
Also excess: a "blockchain audit ledger" block with no inbound arrows
(nothing writes to it); a "future ML anomaly detection" block marked
dotted; the retention DB duplicate from layer 4. The first two were added
by a previous architect and survive by inertia.
Overall: the integration core (layers 1 and 3) is solid. The gaps cluster in
trust (layer 2), failure handling (layer 4), and the path to production
(layers 5-6). Eight blocks missing in the reference model's terms; nine
including the staging environment, which the reference model lists as
optional for internal platforms. Three blocks should come off.
```
## Task
Answer the architect in chat: a simple plain list of what to add and what to
remove, structured by the Minto Pyramid Principle but with no formatting
apparatus — they asked for something they can paste straight into the team
channel.
Return only the result, with no explanation of how you produced it.
Промпт с Minto пять частей по порядку · 30 623 Б
Below is a skill written as an instruction. Read it in full and apply it to the task at the end.===== SKILL.md =====SKILL.md на f4813ce 17 235 Б===== references/rules.md =====references/rules.md на f4813ce 6 464 Б===== references/templates.md =====references/templates.md на f4813ce 3 742 Б===== TASK =====вход кейса целикомReturn only the result, with no explanation of how you produced it.
Вход кейса before.md
<!-- Original fixture. Company, platform and component names are invented. It
tests presentation discipline in write mode: the user asks for a plain
pasteable list, and every piece of audit apparatus — markers, legend,
meta-commentary about ordering, heading machinery — is an overshoot. -->
Fixture 11: plain list
Mode: write
Language: en
Context
You assessed the draft architecture diagram of Ledgerline, an internal
integration platform, against the company's reference model. The assessment
below is your own working notes: layer-by-layer, with verdicts. The platform
architect has read none of it and asked in chat: what is missing from the
diagram and what should come off it. They will paste your reply into the team
channel as-is.
Before
ASSESSMENT NOTES — Ledgerline draft diagram vs reference model
Layer 1 (entry, routing): API gateway present. Rate limiting present.
Verdict: covered.
Layer 2 (identity): the diagram shows a note "SSO later". No identity
provider block, no service-to-service auth, no secrets store despite two
external SaaS connectors on the diagram. Verdict: three blocks missing,
all in the trust boundary.
Layer 3 (transformation): mapping service present, schema registry present.
Verdict: covered.
Layer 4 (state): no message queue — every connector calls the next system
synchronously, so one slow partner stalls the chain. No dead-letter store,
so failed messages vanish. Retention DB present but shown twice, once per
environment, which the reference model treats as one logical block.
Verdict: two blocks missing, one drawn twice.
Layer 5 (delivery): no CI/CD block of any kind; deploys are described in the
legend as "manual, fortnightly". No staging environment on the diagram.
Verdict: two blocks missing.
Layer 6 (observability): metrics dashboard present. No structured log
pipeline, no per-message trace — operators reconstruct incidents from
connector logs by hand. Verdict: two blocks missing.
Also excess: a "blockchain audit ledger" block with no inbound arrows
(nothing writes to it); a "future ML anomaly detection" block marked
dotted; the retention DB duplicate from layer 4. The first two were added
by a previous architect and survive by inertia.
Overall: the integration core (layers 1 and 3) is solid. The gaps cluster in
trust (layer 2), failure handling (layer 4), and the path to production
(layers 5-6). Eight blocks missing in the reference model's terms; nine
including the staging environment, which the reference model lists as
optional for internal platforms. Three blocks should come off.
Task
Answer the architect in chat: a simple plain list of what to add and what to
remove, structured by the Minto Pyramid Principle but with no formatting
apparatus — they asked for something they can paste straight into the team
channel.
Эталон судьи gold.md
Gold 11: plain list
Original gold for an invented scenario.
Expected structure
Top: one plain sentence answering the architect — the core is solid; the
diagram needs eight or nine blocks added, clustered in trust, failure handling
and the path to production, and three taken off. Both counts (missing and
excess) belong in or immediately after the top; the staging environment's
optional status must survive somewhere (as "eight, nine with staging" or a
parenthetical), not be silently rounded to either count.
Groups of additions, same-kind (blocks to add), gathered by cluster rather than
by layer number:
- Trust: identity provider, service-to-service auth, secrets store (the two
external SaaS connectors are the reason it cannot wait). - Failure handling: message queue (synchronous chaining stalls on one slow
partner), dead-letter store (failed messages currently vanish). - Path to production: CI/CD, structured log pipeline, per-message tracing,
optionally staging.
Then removals as one short group: the orphaned blockchain audit ledger, the
dotted ML block, the duplicate retention DB. Covered layers (gateway, rate
limiting, mapping, schema registry) may be acknowledged in the top line or
dropped; listing them as a fourth group pads the answer.
Presentation — the point of this fixture
The architect asked for a plain pasteable list. The correct output is chat
text: a top sentence, group labels as plain words, dash or bullet items. Each
of the following is an overshoot even though it would be correct in an audit or
a memo:
- pyramid markers (
▲,●,○) or a marker legend; - a meta-comment explaining the ordering ("groups ordered by...", "sorted by
cost of error"); - markdown heading machinery (
##) or numbered section headers; - layer numbers in parentheses after items;
- an SCQ block with labels;
- a mermaid diagram;
- a closing line describing what changed structurally.
One or two of these is a visible-structure defect; a full audit costume (legend
plus markers plus meta-comment) means the mode discipline failed. Plain bold
for group labels is acceptable; nothing beyond that is.
Facts
Every block name, count and causal claim comes from the notes. Inventing a
severity scale, effort estimates, or an implementation order that the notes do
not contain counts as invented facts. Dropping the "previous architect /
inertia" aside is proportional omission.
Судья gpt-5.6-sol, effort high · навык с коммита f4813ce · прогон v1.7.0-final