gpt-oss-120b на кейсе «Набор заголовков»: с Minto лучше по качеству и хуже по структуре

Структура 5 → 4 из 8 −1 Качество 5 → 6 из 10 +1

Судья об ответе с Minto

Correctly diagnoses the opening and six-item display, but its own perfect same-kind and MECE scores conflict with the mixed flat setout it describes.

Режим audit · кейс 07-headings-audit · прогон v1.7.0-final

Структура из 8, по 2 балла на ось
Структура	Без Minto 5	С Minto 4	Изменение
Вершина	2	2
Состав ключевой линии	1	1
Уровни	1	0	−1
Порядок и род	1	1

Качество из 10, по 2 балла на ось
Качество	Без Minto 5	С Minto 6	Изменение
Вершина	2	2
Однородные группы	1	1
Объяснимый порядок	0	1	+1
MECE	1	0	−1
Видимая соразмерная структура	1	2	+1

Ответы моделей и слова судьи приведены в оригинале, по-английски.

Без Minto С Minto

Без Minto

контрольный промпт · слепая метка out-05

Жёстких провалов нет

Finds the missing answer and flat hierarchy, but omits the six-item limit, order defect, Background level, and required audit apparatus.

Audit Findings

Missing top‑level message – The opening does not state a clear conclusion, recommendation, or key insight; it merely describes the memorandum’s purpose, which violates the Minto rule that the first paragraph should present the answer to the reader’s question.
Purpose‑statement instead of answer – “This memorandum describes …” is a purpose statement, not the answer that the reader is seeking. The opening should articulate what the reader should take away, not just that the document exists.
Absence of a logical hierarchy – The list of six sections is presented as a flat enumeration rather than a hierarchical structure that shows how the sections support the top‑level message.
Parallelism and MECE issues in headings
- The headings are not uniformly phrased (e.g., “Background” vs. “Principles of the project‑team approach” vs. “Conditions required for success”).
- Potential overlap between “Distinctive benefits and results” and “Conditions required for success” may breach the MECE principle (mutually exclusive, collectively exhaustive).
No preview of the argument flow – The opening does not indicate the logical progression (e.g., situation → complication → question → answer) that should guide the reader through the memorandum.

С Minto

промпт с навыком · слепая метка out-04

Жёстких провалов нет

Correctly diagnoses the opening and six-item display, but its own perfect same-kind and MECE scores conflict with the mixed flat setout it describes.

Legend

Marker	Meaning
`▲`	top / answer
`●`	first‑level group
`⚠`	violation (mixed kinds, unexplainable order, false grouping)
`↑`	conclusion buried lower than it belongs
`⇄`	wrong order type here
`⊗`	overlap (ME broken)
`⊕`	gap (CE broken)
`✂`	cut / move to appendix

Annotated opening

▲⚠ This memorandum describes how project teams identify and deliver significant profit improvements.
                ⚠ It is organized in six sections:
                
                ● Background
                ● Principles of the project‑team approach
                ● Definition of project work
                ● Organization of the program
                ● Distinctive benefits and results
                ● Conditions required for success

Findings

Location	Violation	Suggested fix
Top line	Does not answer a reader’s question (topic label, not answer)	Rewrite as an answer‑first statement that directly answers the implied decision/question.
First two sentences	Missing SCQ intro (no Situation, Complication, Question)	Add a short SCQ story that sets the common ground and the need for action.
List of sections	Presented as flat TOC, not a hierarchical pyramid	Re‑organize into a pyramid: top answer → 3‑4 same‑kind groups → supporting details.
Order of sections	No explicit ordering principle disclosed	State the order type (e.g., “structure”) and justify the sequence.
Overall display	Hierarchy not visible; headings/indentation absent	Use headings or indentation (or a mermaid diagram) to make the pyramid instantly readable.

Score (0–2 per axis, max 10)

Top: 0
Same‑kind groups: 2
Order: 0
MECE: 2
Display: 0

Total: 4 / 10 (baseline workable)

Top three fixes to improve the memo

Replace the opening sentence with an answer‑first line that answers the reader’s decision question.
Insert a concise SCQ intro (Situation → Complication → Question) before the answer.
Restructure the six sections into a clear pyramid (top answer → 3‑4 same‑kind groups → supports) and display it with headings or a diagram.

Судья

Повторяющийся дефект	без	с Minto
Both likely nesting relationships were missedправило сформулировано слишком размыто · Mode 2, Vertical and MECE checks	есть	есть
Required markers, findings table, score, and three fixes were omittedправило в навыке есть, модель его не выполнила · Mode 2, delivery order	есть	нет
The six-first-level-item limit was not flaggedправило в навыке есть, модель его не выполнила · Core loop §4 and Limits	есть	нет
Background’s distinct abstraction level was not isolatedправило есть, но спрятано глубоко · Canon, introduction anti-patterns	есть	есть

Решающие вопросы

Did the audit say that the setout contains topics rather than ideas and therefore has no key line?
без No
с Minto No
Did it identify the first sentence as a topic statement rather than an answer?
без Yes
с Minto Yes
Did it identify the abstraction problem with `Background`?
без No
с Minto No
Did it identify both likely nesting errors named in the gold?
без No
с Minto No
Did it flag six first-level topics?
без No
с Minto Yes
Which findings rely on the unavailable body or merely ask for more content?
без The proposed benefits/conditions overlap is not established by the headings.
с Minto None material; proposed additions remain generic.
Were quotations, line numbers, page numbers, or section contents invented?
без No
с Minto No
Did the output remain diagnosis-only and include the required audit apparatus?
без Diagnosis-only; apparatus missing.
с Minto Diagnosis-only; apparatus complete.

Проверка судьи	без	с Minto
Род первого уровня как в эталоне	да	да
Выдуманные факты	нет	нет
Потеря материала без пометки	нет	нет
Режим соблюдён	да	да
Тип порядка назван	нет	да

Ещё 4 проверки судьи не применяются к режиму audit

Как устроен тест

gpt-oss-120b получила одну и ту же задачу дважды. Контрольный промпт — одна вводная строка и задача. Промпт с Minto — та же задача, перед которой стоит полный текст навыка. Судья читал оба ответа среди 16 ответов под слепыми метками и не знал, в каком из них был навык.

Промпт без Minto 998 Б

You are working on a business document using the Minto Pyramid Principle.

<!-- Faithful paraphrase of Minto (2010), Chapter 4, the six-heading
     project-team example. Original wording for this fixture. -->

# Fixture 07: headings audit

**Mode:** `audit`
**Language:** `en`

## Context

Only the opening and setout below are available. The body of the memorandum is
not available and must not be inferred.

## Before

```text
This memorandum describes how project teams identify and deliver significant
profit improvements. It is organized in six sections:

    Background
    Principles of the project-team approach
    Definition of project work
    Organization of the program
    Distinctive benefits and results
    Conditions required for success
```

## Task

Audit the visible opening against the Minto Pyramid Principle. Report what is
wrong without rewriting the document or inventing facts about the missing body.

Return only the result, with no explanation of how you produced it.

Промпт с Minto пять частей по порядку · 28 571 Б

Below is a skill written as an instruction. Read it in full and apply it to the task at the end.
===== SKILL.md ===== SKILL.md на f4813ce 17 235 Б
===== references/rules.md ===== references/rules.md на f4813ce 6 464 Б
===== references/templates.md ===== references/templates.md на f4813ce 3 742 Б
===== TASK ===== вход кейса целиком
Return only the result, with no explanation of how you produced it.

Вход кейса before.md

Fixture 07: headings audit

Mode: audit
Language: en

Context

Only the opening and setout below are available. The body of the memorandum is
not available and must not be inferred.

Before

This memorandum describes how project teams identify and deliver significant
                profit improvements. It is organized in six sections:
                
                    Background
                    Principles of the project-team approach
                    Definition of project work
                    Organization of the program
                    Distinctive benefits and results
                    Conditions required for success

Task

Audit the visible opening against the Minto Pyramid Principle. Report what is
wrong without rewriting the document or inventing facts about the missing body.

Эталон судьи gold.md

Gold 07: headings audit

Source structure: Minto (2010), Chapter 4.

Visible defects

Defect	Severity
The setout contains topics, not ideas, so it conveys no message or key line	Critical
The first sentence describes a subject instead of answering a reader question	Critical
`Background` cannot sit at the same abstraction level as the other points	Medium
Benefits/results likely belong under principles; success conditions likely belong under program organization	Medium
Six first-level topics exceed the skill's normal display limit	Medium
The topics do not form a visible inductive or deductive argument	Medium

The output must stay within the visible evidence. Claims about missing data,
metrics, proof, vertical support inside sections, or exact content of the body
are false positives. Invented page numbers, line numbers, or quotations are
hard failures.

Судья gpt-5.6-sol, effort high · навык с коммита f4813ce · прогон v1.7.0-final