Записка о наборных издержках
Режим write · Создать и переработать текст
Восемь моделей на этом кейсе
| Модель | Структура, из 8 | Качество, из 10 |
|---|---|---|
| Claude Opus 5 | 4 → 7 +3 | 6 → 8 +2 |
| Claude Sonnet 5 | 6 → 5 −1 | 7 → 7 0 |
| Claude Haiku 4.5 | 5 → 5 0 | 8 → 7 −1 |
| Codex gpt-5.6-terra | 6 → 5 −1 | 7 → 8 +1 |
| gpt-oss-120b | 4 → 5 +1 | 4 → 7 +3 |
| qwen3.6-35b-a3b | 5 → 6 +1 | 8 → 7 −1 |
| qwen3.6-fp8 | 6 → 5 −1 | 7 → 7 0 |
| qwen3.8-27b | нет пары. Исключена. В базовом прогоне модель так и не закончила ответ с навыком, поэтому пару убрали из обоих прогонов, чтобы их можно было сравнивать клетка в клетку. | |
Что судья увидел во всех ответах
The strongest output is distinguished by promoting the low-pay → staffing shortage → delay and overtime chain into a first-level action and nesting the evidence beneath it. The weaker outputs consistently preserve the source’s overlapping simplification and methods-study branches, leaving the genuine wage lever buried, misplaced, or omitted.
О чём судья спрашивал каждый ответ
- Were process simplification and method/productivity change merged as one branch?
- Was raising wages promoted to the first level?
- How many first-level elements are actions?
- Are benchmark, process, staffing, delay, and overtime observations below the actions?
- Was irrelevant comparison/opinion material omitted with an explicit note?
- Is the reader's question stated literally?
Где ответы не дотянули
| Повторяющийся дефект | Раздел навыка | Причина | У кого |
|---|---|---|---|
| Process simplification and method/productivity change remain duplicate branches | Core loop §6, MECE duplication test | правило в навыке есть, модель его не выполнила | Без Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8С Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8 |
| The wage action remains buried in observations or disappears | Core loop §§4 and 6, promotion of the real branch from observations | правило в навыке есть, модель его не выполнила | Без Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8С Minto · 6 Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8 |
| Staffing evidence is placed in context, attached to the methods branch, or omitted | Core loop, vertical support and levels | правило есть, но спрятано глубоко | Без Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8С Minto · 6 Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8 |
| The literal reader question is absent | Core loop §§1 and 3 | правило в навыке есть, модель его не выполнила | Без Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8С Minto · 6 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, qwen3.6-35b-a3b, qwen3.6-fp8 |
| Unsupported certainty, causal claims, or implementation details are introduced | Mode 3, “Invent nothing” | правило в навыке есть, модель его не выполнила | Без Minto · 5 Claude Opus 5, Claude Sonnet 5, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8С Minto · 4 Claude Sonnet 5, Claude Haiku 4.5, qwen3.6-35b-a3b, qwen3.6-fp8 |
| Omitted source material is not fully acknowledged | Mode 3, preservation and dropped-fact rule | правило в навыке есть, модель его не выполнила | Без Minto · 2 Claude Haiku 4.5, gpt-oss-120bС Minto · 3 Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b |
| No output names one of the prescribed order types | Core loop §5, obscured by Mode 3’s clean-text guidance | правило есть, но спрятано глубоко | Без Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8С Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8 |
Как устроен тест
Промпт без Minto 2 516 Б
You are working on a business document using the Minto Pyramid Principle.
<!-- Faithful paraphrase of Minto (2010), Chapter 3, the TTW composing-room
memorandum before restructuring. Original wording for this fixture. -->
# Fixture 02: TTW composing costs
**Mode:** `write`
**Language:** `en`
## Context
TTW is a printing company in Aylesbury. A junior consultant spent two weeks
examining composing-room costs. The engagement manager already knows that
composition accounts for roughly 40% of hardback cost and 50-55% of paperback
cost, that TTW is uncompetitive on simple work, and that TTW does not know
whether its composing costs are actually excessive.
## Before
```text
To:
From:
Subject: TTW
This note records the information gathered during the past two weeks.
Composition is the largest production cost. Industry comparisons suggest weak
productivity and high cost at TTW. Every title, whether a complex reference
book or a simple novel, passes through essentially the same quality-control
stages.
People in Aylesbury want to understand the causes. I spoke with Roy Walter,
Brian Thompson, and George Kennedy. Kennedy is willing to test whether some
stages can be removed and why measured productivity trails the benchmark.
The department is overloaded. Manual composition is especially short of
capacity. TTW pays below local market rates, struggles to hire and retain
compositors, has just lost two employees, and now runs most jobs late.
Overtime is more than 50% over budget, while the union is pressing a new claim.
Conclusions
1. Costs may be reduced by:
a. simplifying the process for inexpensive titles;
b. increasing productivity by changing the composing method.
2. To simplify the process, TTW should test selected jobs and measure the
effect of fewer or differently timed checks on quality and customers. The
saving might reach 10% of total composing cost. A separate methods study
could identify further productivity improvements because TTW trails the
benchmark by 20-50%.
3. Comparisons with three other printers might provide useful context, although
I doubt a simple comparison will settle the issue.
4. Managers disagree about whether costs are high, but all are willing to
investigate.
```
## Task
Rewrite the report using the Minto Pyramid Principle. Keep the useful facts,
remove or demote material that does not support the recommendation, and state
what was omitted.
Return only the result, with no explanation of how you produced it.
Промпт с Minto пять частей по порядку · 30 089 Б
Below is a skill written as an instruction. Read it in full and apply it to the task at the end.===== SKILL.md =====SKILL.md на f4813ce 17 235 Б===== references/rules.md =====references/rules.md на f4813ce 6 464 Б===== references/templates.md =====references/templates.md на f4813ce 3 742 Б===== TASK =====вход кейса целикомReturn only the result, with no explanation of how you produced it.
Вход кейса before.md
<!-- Faithful paraphrase of Minto (2010), Chapter 3, the TTW composing-room
memorandum before restructuring. Original wording for this fixture. -->
Fixture 02: TTW composing costs
Mode: write
Language: en
Context
TTW is a printing company in Aylesbury. A junior consultant spent two weeks
examining composing-room costs. The engagement manager already knows that
composition accounts for roughly 40% of hardback cost and 50-55% of paperback
cost, that TTW is uncompetitive on simple work, and that TTW does not know
whether its composing costs are actually excessive.
Before
To:
From:
Subject: TTW
This note records the information gathered during the past two weeks.
Composition is the largest production cost. Industry comparisons suggest weak
productivity and high cost at TTW. Every title, whether a complex reference
book or a simple novel, passes through essentially the same quality-control
stages.
People in Aylesbury want to understand the causes. I spoke with Roy Walter,
Brian Thompson, and George Kennedy. Kennedy is willing to test whether some
stages can be removed and why measured productivity trails the benchmark.
The department is overloaded. Manual composition is especially short of
capacity. TTW pays below local market rates, struggles to hire and retain
compositors, has just lost two employees, and now runs most jobs late.
Overtime is more than 50% over budget, while the union is pressing a new claim.
Conclusions
1. Costs may be reduced by:
a. simplifying the process for inexpensive titles;
b. increasing productivity by changing the composing method.
2. To simplify the process, TTW should test selected jobs and measure the
effect of fewer or differently timed checks on quality and customers. The
saving might reach 10% of total composing cost. A separate methods study
could identify further productivity improvements because TTW trails the
benchmark by 20-50%.
3. Comparisons with three other printers might provide useful context, although
I doubt a simple comparison will settle the issue.
4. Managers disagree about whether costs are high, but all are willing to
investigate.
Task
Rewrite the report using the Minto Pyramid Principle. Keep the useful facts,
remove or demote material that does not support the recommendation, and state
what was omitted.
Эталон судьи gold.md
Gold 02: TTW composing costs
Source structure: Minto (2010), Chapter 3, TTW analysis and rewritten memo.
Expected structure
| Element | Expected value |
|---|---|
| Reader question | How can TTW reduce composing cost? |
| Top | TTW can reduce composing cost substantially |
| Key-line kind | Actions |
| Order | Two independent changes, each followed by its causal support |
Key line
- Eliminate unnecessary stages from the composing process for simple work.
- Raise wages to competitive levels.
The source's "simplify the process" and "increase productivity by changing the
method" are overlapping descriptions of the same process intervention. They
must not survive as two separate first-level branches. The missing second
branch is embedded in the observations: low wages cause vacancies, delays, and
overtime cost.
Levels
Under eliminating stages: every job follows the same process; TTW trails the
benchmark by 20-50%; run a controlled experiment; potential saving up to 10%.
Under raising wages: below-market pay; hiring and retention difficulty; two
departures; understaffing; late work; overtime more than 50% over budget; union
pressure.
Names, conflicting opinions, and generic competitor comparisons do not answer
the reader's question. They may be omitted with one explicit note.
Судья gpt-5.6-sol, effort high · навык с коммита f4813ce · прогон v1.7.0-final