gpt-oss-120b on “A set of monthly reports”: with Minto, better on quality and worse on structure

Structure 4 → 3 of 8 −1 Quality 4 → 6 of 10 +2

The judge on the output with Minto

The blanket use of ⚠ destroys hierarchy, and its suggested replacement mixes reasons, actions, and risks.

Mode audit · case 03-period-graph-books · run v1.7.0-final

Structure of 8, 2 points per axis
Structure	Without Minto 4	With Minto 3	Change
Top	1	1
Key-line composition	1	1
Levels	1	0	−1
Order and kind	1	1

Quality of 10, 2 points per axis
Quality	Without Minto 4	With Minto 6	Change
Top	1	1
Same-kind grouping	1	1
Explainable order	1	1
MECE	0	1	+1
Visible, proportionate structure	1	2	+1

Outputs and the judge’s words are quoted as written, in English.

Without Minto With Minto

Without Minto

control prompt · blind label out-11

invented_facts

The table is readable, but numerous peripheral or fabricated defects overwhelm the source-grounded Minto diagnosis.

Audit of the “Period Graph Books” memo against the Minto Pyramid Principle

Minto Principle Element	Defect(s) Identified	Why It Violates the Principle	What Must Change
Top‑Down Structure	The memo opens with a narrative description (“Since Period 5…”) rather than a clear, single overarching recommendation or conclusion.	Minto requires the first sentence (or paragraph) to state the key message (the answer to the decision‑maker’s question). The current opening leaves the reader guessing what action is being sought.	Begin with a concise Executive Summary that states the recommended course (e.g., “We should redesign the graph‑book production process to eliminate duplication, assign end‑to‑end ownership, and automate calculations”). All subsequent sections must support this statement.
Logical Flow (Situation‑Complication‑Question‑Answer)	The memo mixes “Production” steps with “Issues” without first establishing the situation (current state) and the complication (why it is problematic).	The Situation‑Complication‑Question‑Answer (SCQA) framework is not evident; the reader cannot see the logical progression from “what is happening” to “why it matters” to “what we need to decide.”	Re‑order content: 1. Situation – brief factual description of the current process (who does what, how many steps). 2. Complication – enumerate the pain points (duplication, errors, lack of ownership). 3. Question – what should we do? 4. Answer – the recommended solution (streamlining, central control, automation).
MECE Grouping	The “Issues” list mixes different problem types (data duplication, calculation errors, unexplained changes, re‑entry of prior‑period data) without clear grouping. Some items overlap (e.g., “inconsistent data” and “incorrect variable‑cost calculations”).	Minto demands that supporting points be Mutually Exclusive and Collectively Exhaustive. Overlapping or redundant bullet points dilute clarity and make it harder to prioritize actions.	Re‑group issues into MECE categories, e.g.: • Process Redundancy – multiple data entries, re‑entry of prior‑period data. • Data Integrity – inconsistent data, unexplained changes, incorrect calculations. • Ownership & Control – no end‑to‑end responsibility, fragmented hand‑offs.
Hierarchy of Detail	The “Production” steps are presented as a flat list, then the “Issues” list follows, but there is no hierarchy linking each issue to the specific step that generates it.	In a pyramid, each lower‑level point should be a logical sub‑point of a higher‑level heading. Without explicit linkage, the reader cannot see which step causes which problem.	For each production step, attach the relevant issue(s) as sub‑bullets, or create a two‑column table mapping Step → Problem. This creates a clear cause‑effect hierarchy.
Clarity of the Call‑to‑Action	The memo ends with “Could we discuss how best to proceed?” – an open‑ended question that does not specify the decision needed.	Minto’s “Answer” should be a concrete recommendation, not a vague invitation to discuss. The decision‑maker needs a clear next step.	Replace the open question with a specific Decision Request (e.g., “Approve the formation of a cross‑functional task force to redesign the graph‑book workflow and implement an automated data pipeline by Q3”).
Parallelism & Consistency	Bullet phrasing is inconsistent (some start with verbs, others with nouns; mixed tenses).	Parallel structure aids readability and reinforces the hierarchy. Inconsistent phrasing weakens the logical flow.	Standardize bullet style (e.g., all start with a verb phrase: “Gather data…”, “Calculate points…”, “Transcribe points…”, “Check data…”).
Length & Focus	The memo includes narrative filler (“the transfer inserted another person…”) that does not directly support the key recommendation.	Minto advises brevity; every sentence should serve the top‑down argument. Extraneous context dilutes focus.	Trim or relocate background information to an appendix; keep the main memo to the essential argument and supporting facts.
Use of Headings	Only “Production,” “Issues,” and “Overall assessment” headings are used, but they do not reflect the SCQA hierarchy and are not ordered logically.	Headings should mirror the pyramid levels (Situation, Complication, Question, Answer). Misaligned headings confuse the reader’s mental map.	Rename and reorder headings to match the SCQA flow (e.g., “Current Process (Situation)”, “Pain Points (Complication)”, “Decision Needed (Question)”, “Proposed Solution (Answer)”).
Data Presentation	The memo mentions “inconsistent data between periods and regions” but provides no quantitative evidence or examples.	Supporting arguments should be backed by concrete data to make the case compelling.	Insert a brief data snapshot or a chart (or reference an appendix) that quantifies the error rate or duplication cost.
Audience Orientation	The memo assumes the reader knows the history of the transfer and the names of internal groups (e.g., “Color Graphics”) without explaining their relevance.	Minto stresses tailoring the message to the audience’s needs; unnecessary jargon can obscure the argument.	Provide a one‑sentence context for each stakeholder group or move detailed background to a footnote.

Summary of Required Changes

Insert a clear executive summary that states the recommended action.
Re‑structure the memo using SCQA (Situation → Complication → Question → Answer).
Group issues into MECE categories and link each to the specific production step that generates it.
Replace the open‑ended discussion request with a concrete decision request.
Standardize bullet phrasing and ensure parallelism.
Trim non‑essential narrative and relocate detailed background to an appendix.
Rename headings to reflect the logical hierarchy and order them accordingly.
Add concise data evidence to substantiate the pain points.
Clarify stakeholder references for the audience’s understanding.

Implementing these changes will align the memo with the Minto Pyramid Principle, making the argument more persuasive, easier to follow, and actionable for senior management.

With Minto

prompt with the skill · blind label out-04

No hard failures

The blanket use of ⚠ destroys hierarchy, and its suggested replacement mixes reasons, actions, and risks.

Legend
▲ top / answer ● first‑level group ○ supporting detail ⚠ violation (mixed kinds, missing top, no order, etc.) ↑ conclusion buried lower ⇄ wrong order type ⊗ overlap (ME broken) ⊕ gap (CE broken) ✂ cut / move to appendix

⚠ Subject: Period Graph Books
                
                ⚠ Since Period 5, Financial Analysis has been responsible for producing four
                ⚠ graph‑presentation books previously handled by Corporate Planning. This memo
                ⚠ describes issues that appeared after the transfer.
                
                ⚠ Production
                ⚠ 
                ⚠ 1. Gather data from divisional forms, internal documents, and telephone calls.
                ⚠ 2. Calculate graph points manually or on the corporate computer, including
                ⚠    rolling values for revenue, cost, and percentages.
                ⚠ 3. Transcribe the points to input sheets. Analysts update the current period,
                ⚠    then Color Graphics re‑enters the points into its own database and generates
                ⚠    the graphs.
                ⚠ 4. Check the data and calculations.
                
                ⚠ Issues
                ⚠ 
                ⚠ The transfer inserted another person into an already fragmented process.
                ⚠ Divisional staff and three corporate departments participate, and the same
                ⚠ information may be entered into computer systems three times.
                ⚠ 
                ⚠ For one book, divisional data are loaded into a dedicated corporate program,
                ⚠ calculated, transferred to Color Graphics input sheets, entered again, and
                ⚠ returned for updates. Problems include inconsistent data between periods and
                ⚠ regions, incorrect variable‑cost calculations, unexplained changes to points
                ⚠ that were previously correct, and prior‑period data that must be re‑entered.
                
                ⚠ Overall assessment
                ⚠ 
                ⚠ Most problems arise from a cumbersome fragmented process. No one controls the
                ⚠ data from beginning to end, responsibility is unclear, and errors can fall
                ⚠ between stages. The production process badly needs streamlining, both for each
                ⚠ book and for common calculations. The current team cannot manage the required
                ⚠ changes as well as ongoing production.
                
                ⚠ Could we discuss how best to proceed?

Findings table

Location	Violation	Suggested fix
First line (“Subject: …”)	Missing top answer; title only	Insert a one‑sentence answer that states the recommended action (e.g., “We should redesign the graph‑book production process to eliminate duplication and improve data control”).
Sections “Production”, “Issues”, “Overall assessment”	Mixed‑kind first‑level groups (process steps, problem list, assessment) → violates same‑kind rule	Re‑structure into same‑kind groups that directly support the answer (e.g., Reasons for change, Actions to implement, Risks).
Entire memo	No explicit order (time, structure, ranking, etc.)	Choose an order type (e.g., Structure: parts of the new streamlined process) and label it in the intro.
“Issues” paragraph & “Overall assessment”	Overlap between problem description and assessment; gaps (no recommendations, no responsibility matrix)	Separate problems (facts) from analysis (reasons) and add a dedicated Recommendations group; fill gaps with clear ownership and next steps.
“Could we discuss …?”	Question placed at very end, not reflected in a top answer	Move the question to the SCQ intro and answer it up front; the memo should close the question, not open it.
Overall layout	No visible hierarchy (headings not nested, no indentation)	Use clear headings and indentation to show pyramid levels; optionally add a simple outline (Top → Group 1 → Support 1.1, etc.).
“Production” step list	Process described but not linked to the answer; buried conclusion that “process needs streamlining” appears only in assessment	Promote the conclusion to the top answer and use the production steps as supporting detail under a Current Process group.

(7 findings listed; all major defects captured.)

Score (0–2 per axis)

Axis	Score	Comment
Top	0	No answer; only a subject line.
Same‑kind groups	0	Groups are mixed (steps, issues, assessment).
Order	0	No explicit ordering principle.
MECE	0	Overlaps between problems and assessment; gaps (no recommendations).
Display	0	Hierarchy not visible; flat memo.

Total: 0 / 10 (not structured).

Three highest‑impact fixes

Add a clear top answer that states the recommended action and closes the implied question.
Re‑organize the memo into same‑kind groups (e.g., Reasons, Actions, Risks) that directly support the top answer.
Apply a consistent order type (e.g., Structure) and ensure the hierarchy is visible with headings/indentation, eliminating overlaps and filling the missing recommendation gap.

The judge

Recurring defect	without	with Minto
Missing answer is not distinguished from the buried streamlining conclusionthe skill has the rule; the model did not follow it · Mode 2 — audit, check 1: Top	present	present
Incomplete process logic is not separated into graph-generation and later-change failure pointsthe skill has no such rule · Mode 2 — audit, check 4: MECE	present	present
Suggested first-level structure uses causes, symptoms, constraints, or topics instead of actionsthe skill has the rule; the model did not follow it · Core loop, §4 Groups	present	present
Inline annotation, capped findings, five-axis score, or three highest-value fixes are absentthe skill has the rule; the model did not follow it · Mode 2 — audit, delivery contract	present	absent
Unsupported evidence requirements, process claims, or implementation specifics are introducedthe skill has the rule; the model did not follow it · Mode 2 finding rule and Core loop §8	present	present

Decisive questions

Did the audit state precisely that the required changes are absent?
without yes
with Minto yes
Did it distinguish a missing answer from a merely buried answer?
without no
with Minto no—the buried diagnosis is treated too much like the missing answer
Did it identify streamlining as an undeveloped conclusion?
without no
with Minto yes
Did it name incomplete process logic and unseparated failure points?
without partial—suggests mapping steps to problems but misses the distinct failure classes
with Minto partial
If a replacement structure was suggested, is its key line made of actions?
without no
with Minto no—the proposed first level mixes reasons, actions, and risks
What false positives or invented details appear?
without invents automation, a task force, Q3 timing, and defects in parallelism, audience knowledge, and data presentation
with Minto invents a responsibility-matrix requirement and wrongly asks for the order label in the memo
Did the output remain diagnosis-only and include the required audit apparatus?
without diagnosis-only, but lacks annotation, scoring, and a three-fix limit
with Minto yes

Judge’s check	without	with Minto
First-level kind matches the gold	no	no
Invented facts	yes	no
Source lost without a note	no	no
Mode respected	yes	yes
Order type named	no	yes

4 more judge checks do not apply to mode audit

How the test was built

gpt-oss-120b received the same task twice. The control prompt is one framing line and the task. The prompt with Minto is the same task preceded by the full text of the skill. The judge read both outputs among 16 under blind labels and did not know which one had the skill.

Prompt without Minto 2 424 B

You are working on a business document using the Minto Pyramid Principle.

<!-- Faithful paraphrase of Minto (2010), Appendix B, Exhibits B-5 and B-6.
     Original wording for this fixture. -->

# Fixture 03: Period Graph Books

**Mode:** `audit`
**Language:** `en`

## Context

Corporate Financial Analysis recently took over production of four monthly
graph books used in presentations to senior management. Management knows that
errors and inconsistencies have appeared and asked for corrective proposals.

## Before

```text
Subject: Period Graph Books

Since Period 5, Financial Analysis has been responsible for producing four
graph-presentation books previously handled by Corporate Planning. This memo
describes issues that appeared after the transfer.

Production

1. Gather data from divisional forms, internal documents, and telephone calls.
2. Calculate graph points manually or on the corporate computer, including
   rolling values for revenue, cost, and percentages.
3. Transcribe the points to input sheets. Analysts update the current period,
   then Color Graphics re-enters the points into its own database and generates
   the graphs.
4. Check the data and calculations.

Issues

The transfer inserted another person into an already fragmented process.
Divisional staff and three corporate departments participate, and the same
information may be entered into computer systems three times.

For one book, divisional data are loaded into a dedicated corporate program,
calculated, transferred to Color Graphics input sheets, entered again, and
returned for updates. Problems include inconsistent data between periods and
regions, incorrect variable-cost calculations, unexplained changes to points
that were previously correct, and prior-period data that must be re-entered.

Overall assessment

Most problems arise from a cumbersome fragmented process. No one controls the
data from beginning to end, responsibility is unclear, and errors can fall
between stages. The production process badly needs streamlining, both for each
book and for common calculations. The current team cannot manage the required
changes as well as ongoing production.

Could we discuss how best to proceed?
```

## Task

Audit this document against the Minto Pyramid Principle. Diagnose the defects
and explain what must change. Do not rewrite the document.

Return only the result, with no explanation of how you produced it.

Prompt with Minto five parts, in order · 29 997 B

Below is a skill written as an instruction. Read it in full and apply it to the task at the end.
===== SKILL.md ===== SKILL.md at f4813ce 17 235 B
===== references/rules.md ===== references/rules.md at f4813ce 6 464 B
===== references/templates.md ===== references/templates.md at f4813ce 3 742 B
===== TASK ===== the whole case input
Return only the result, with no explanation of how you produced it.

Case input before.md

Fixture 03: Period Graph Books

Mode: audit
Language: en

Context

Corporate Financial Analysis recently took over production of four monthly
graph books used in presentations to senior management. Management knows that
errors and inconsistencies have appeared and asked for corrective proposals.

Before

Subject: Period Graph Books
                
                Since Period 5, Financial Analysis has been responsible for producing four
                graph-presentation books previously handled by Corporate Planning. This memo
                describes issues that appeared after the transfer.
                
                Production
                
                1. Gather data from divisional forms, internal documents, and telephone calls.
                2. Calculate graph points manually or on the corporate computer, including
                   rolling values for revenue, cost, and percentages.
                3. Transcribe the points to input sheets. Analysts update the current period,
                   then Color Graphics re-enters the points into its own database and generates
                   the graphs.
                4. Check the data and calculations.
                
                Issues
                
                The transfer inserted another person into an already fragmented process.
                Divisional staff and three corporate departments participate, and the same
                information may be entered into computer systems three times.
                
                For one book, divisional data are loaded into a dedicated corporate program,
                calculated, transferred to Color Graphics input sheets, entered again, and
                returned for updates. Problems include inconsistent data between periods and
                regions, incorrect variable-cost calculations, unexplained changes to points
                that were previously correct, and prior-period data that must be re-entered.
                
                Overall assessment
                
                Most problems arise from a cumbersome fragmented process. No one controls the
                data from beginning to end, responsibility is unclear, and errors can fall
                between stages. The production process badly needs streamlining, both for each
                book and for common calculations. The current team cannot manage the required
                changes as well as ongoing production.
                
                Could we discuss how best to proceed?

Task

Audit this document against the Minto Pyramid Principle. Diagnose the defects
and explain what must change. Do not rewrite the document.

The judge’s gold gold.md

Gold 03: Period Graph Books

Source structure: Minto (2010), Appendix B, Exhibits B-5 and B-6.

Book-derived defects

Defect	Severity
The document never states the changes required; it therefore does not answer management's question	Critical
It describes the current process instead of presenting a top and key line	Critical
"The purpose is to describe issues" is a topic, not an answer	Critical
The process description is incomplete and does not isolate each system's failure points	Medium
Problems in graph-point generation and later presenter changes are not clearly separated	Medium

The buried conclusion is that the process needs streamlining, but this is not
turned into an actionable answer.

Expected replacement structure

Reader question: What changes should we make to prevent unreliable graphs?

Top: Remove the opportunities for error across data transfer, graph-point
generation, and final presentation changes.

Key-line kind: changes/actions.

Create a direct data link to the corporate computer.
Create a reliable computerized routine for graph-point generation.
Require presenter changes to be revalidated before use.

Audit outputs should name the missing answer, distinguish it from a conclusion
that is merely buried, annotate only visible evidence, provide no rewritten
memo, and avoid inventing defects in unseen material.

Judge gpt-5.6-sol, effort high · skill from commit f4813ce · run v1.7.0-final