The role of the board
Mode write · Create and rework text
Eight models on this case
| Model | Structure, of 8 | Quality, of 10 |
|---|---|---|
| Claude Opus 5 | 1 → 4 +3 | 7 → 8 +1 |
| Claude Sonnet 5 | 1 → 0 −1 | 7 → 6 −1 |
| Claude Haiku 4.5 | 0 → 2 +2 | 5 → 7 +2 |
| Codex gpt-5.6-terra | 1 → 1 0 | 7 → 7 0 |
| gpt-oss-120b | 1 → 3 +2 | 6 → 7 +1 |
| qwen3.6-35b-a3b | 0 → 4 +4 | 6 → 7 +1 |
| qwen3.6-fp8 | 5 → 5 0 | 8 → 8 0 |
| qwen3.8-27b | 1 → 0 −1 | 7 → 6 −1 |
What the judge saw across all outputs
Stronger outputs derive a small set of visible actions from the agenda and place composition, tenure, role boundaries, and outside participation underneath them. Weaker outputs merely shorten, merge, or rephrase the five original topics, usually leaving transition as another branch rather than treating it as a document-wide implementation concern.
What the judge asked of every output
- Is the first-level kind
actions,mixed, ortopics/reasons? - How many first-level elements survive unchanged from the five source topics?
- Did "alternative ways to get there" survive as a branch?
- Did the purpose statement disappear?
- Does the top answer what the Board must change?
- Does the SCQ contradict the supplied context?
- Were all source topics retained as support or explicitly omitted?
- Which facts were invented?
Where the outputs fell short
| Recurring defect | Skill section | Cause | Where it appeared |
|---|---|---|---|
| Original agenda topics remain at the first level instead of becoming support for actions | Core loop §4, Groups | the skill has the rule; the model did not follow it | Without Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.8-27bWith Minto · 4 Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, qwen3.8-27b |
| “Alternative ways to get there” remains a substantive branch | Core loop §6, MECE | the skill has the rule; the model did not follow it | Without Minto · 7 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27bWith Minto · 7 Claude Opus 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27b |
| The required action to formalize Board operation through policies and procedures is absent | Core loop §4, deriving actions from source mechanics | the skill has no such rule | Without Minto · 8 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27bWith Minto · 8 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27b |
| The answer does not appear in the first sentence | Core loop §3 and Mode 3 answer-first rendering | the rule is there but buried | Without Minto · 3 Claude Opus 5, Claude Haiku 4.5, qwen3.6-35b-a3bWith Minto · 8 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27b |
| The ordering principle is never named | Core loop §5 versus Mode 3 delivery guidance | the rule is there but buried | Without Minto · 8 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27bWith Minto · 8 Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27b |
| Supplied context or a source topic disappears without acknowledgement | Mode 3, “Keep every fact from the source” | the skill has the rule; the model did not follow it | Without Minto · 7 Claude Sonnet 5, Claude Haiku 4.5, Codex gpt-5.6-terra, gpt-oss-120b, qwen3.6-35b-a3b, qwen3.6-fp8, qwen3.8-27bWith Minto · 2 Codex gpt-5.6-terra, qwen3.6-fp8 |
How the test was built
Prompt without Minto 1 403 B
You are working on a business document using the Minto Pyramid Principle.
<!-- Faithful paraphrase of Minto (2010), Chapter 2, the board-role
introduction. Original wording for this fixture. -->
# Fixture 05: role of the board
**Mode:** `write`
**Language:** `en`
## Context
In October, a reorganization gave divisional managers full authority for
day-to-day operations. Policy and planning remain the Board's responsibility.
The Board has nevertheless spent years dealing with short-term operational
problems.
Open questions include Board size and membership, the division of work between
the Board and Executive Committee, effective participation by outside members,
and selection and tenure.
## Before
```text
The purpose of this memorandum is to collect ideas for discussion about:
1. Board composition and optimum size.
2. The broad roles of the Board and Executive Committee, the responsibilities
of each, and their relationship.
3. How to make an outside director an effective participant.
4. Principles for selecting directors and setting their tenure.
5. Alternative ways to move from the present Board and Executive Committee
arrangement to the desired one.
```
## Task
Rewrite the opening using the Minto Pyramid Principle. Return the introduction
and the points it sets out, not the full memorandum.
Return only the result, with no explanation of how you produced it.
Prompt with Minto five parts, in order · 28 976 B
Below is a skill written as an instruction. Read it in full and apply it to the task at the end.===== SKILL.md =====SKILL.md at f4813ce 17 235 B===== references/rules.md =====references/rules.md at f4813ce 6 464 B===== references/templates.md =====references/templates.md at f4813ce 3 742 B===== TASK =====the whole case inputReturn only the result, with no explanation of how you produced it.
Case input before.md
<!-- Faithful paraphrase of Minto (2010), Chapter 2, the board-role
introduction. Original wording for this fixture. -->
Fixture 05: role of the board
Mode: write
Language: en
Context
In October, a reorganization gave divisional managers full authority for
day-to-day operations. Policy and planning remain the Board's responsibility.
The Board has nevertheless spent years dealing with short-term operational
problems.
Open questions include Board size and membership, the division of work between
the Board and Executive Committee, effective participation by outside members,
and selection and tenure.
Before
The purpose of this memorandum is to collect ideas for discussion about:
1. Board composition and optimum size.
2. The broad roles of the Board and Executive Committee, the responsibilities
of each, and their relationship.
3. How to make an outside director an effective participant.
4. Principles for selecting directors and setting their tenure.
5. Alternative ways to move from the present Board and Executive Committee
arrangement to the desired one.Task
Rewrite the opening using the Minto Pyramid Principle. Return the introduction
and the points it sets out, not the full memorandum.
The judge’s gold gold.md
Gold 05: role of the board
Source structure: Minto (2010), Chapter 2.
Expected structure
| Element | Expected value |
|---|---|
| Reader question | What must the Board change to focus on long-range strategy? |
| Top | The Board should make the changes that let it concentrate on strategy |
| Key-line kind | Actions |
Key line:
- Hand day-to-day operating responsibility to the Executive Committee.
- Broaden the Board to include outside directors.
- Formalize internal operation through policies and procedures.
The October reorganization is the Situation. The Board's long habit of handling
short-term issues is the Complication.
None of the five source topics should survive unchanged as a first-level point.
Composition, roles, outside participation, and tenure become evidence beneath
the three actions. "Alternative ways to get there" describes the entire
document and is not a branch. The purpose statement disappears because it is
neither Situation nor answer.
Judge gpt-5.6-sol, effort high · skill from commit f4813ce · run v1.7.0-final