Claude Sonnet 5 on “A set of monthly reports”: with Minto, worse on structure, quality unchanged

Structure 6 → 4 of 8 −2 Quality 7 → 7 of 10 0

The judge on the output with Minto

It handles missing versus buried logic well and uses levels sensibly, but promotes resourcing as a key branch and misses the two failure-point classes.

Mode audit · case 03-period-graph-books · run v1.7.0-final

Structure of 8, 2 points per axis
Structure	Without Minto 6	With Minto 4	Change
Top	1	1
Key-line composition	1	1
Levels	2	1	−1
Order and kind	2	1	−1

Quality of 10, 2 points per axis
Quality	Without Minto 7	With Minto 7	Change
Top	1	1
Same-kind grouping	2	2
Explainable order	2	1	−1
MECE	1	1
Visible, proportionate structure	1	2	+1

Outputs and the judge’s words are quoted as written, in English.

Without Minto With Minto

Without Minto

control prompt · blind label out-10

No hard failures

Its evidence is well nested and its sequence is coherent, but it elevates the diagnosis and capacity constraint as though they constituted the missing answer.

Audit: Period Graph Books memo

Overall diagnosis

The memo has no governing thought. It opens with a factual statement about a departmental handover, then lists production steps, then issues, then an "overall assessment" that actually contains the real point — but that point arrives last, buried, and is followed by a weak, non-committal question rather than a request tied to an action.

Defect 1: No pyramid — narrative order, not logical order

The structure follows the order events happened (transfer → production steps → issues → assessment), not an order derived from a single top point supported by grouped reasons. A reader has to reach the final paragraph to learn what the memo is actually arguing: that the process is fragmented and needs restructuring, and that the current team can't do it alone. That should be the opening sentence, not the conclusion of a chronological account.

Defect 2: The real point is stated as an aside, not the lead

"The production process badly needs streamlining... The current team cannot manage the required changes as well as ongoing production" is the actual message and the actual ask (implicitly: give us dedicated resources/authority to fix this). It's introduced under a throwaway heading ("Overall assessment") after two sections of description. Minto's structure requires this to be the top box, with everything above it existing only to support it.

Defect 3: Situation–Complication–Question–Answer is missing or malformed

Situation: weakly present (Financial Analysis took over four books since Period 5).
Complication: present but scattered — "issues appeared" is mentioned up top, then re-explained later.
Question: never made explicit. The reader isn't told what question the memo answers.
Answer: implied only in the final line as a question back to the reader ("Could we discuss how best to proceed?"), which reverses the responsibility — the memo should answer its own question with a recommendation, not offload it.

Defect 4: "Production" section is description, not argument

The four numbered production steps are procedural detail with no clear grouping logic and no stated relevance to the argument. Under Minto, subordinate groupings must answer "why" or "how" for the point above them. Here they just narrate a workflow. If retained, they need to be recast as evidence for a specific claim (e.g., "the process is fragmented" — supported by showing triple entry, multiple departments, manual steps) rather than a standalone procedural inventory.

Defect 5: "Issues" section conflates categories of problems

Data inconsistency, calculation errors, unexplained changes, and re-entry burden are all listed at the same level without being grouped by cause or type. Some are process defects (fragmentation, multiple entry points), others are consequences (bad numbers reaching management). Minto requires groupings to be logically distinct (MECE) and summarizable by a single sentence — currently they're just an enumerated list of symptoms with no organizing idea.

Defect 6: One example doing the work of a general claim

The detailed walk-through of "one book" (loaded into a corporate program, transferred, re-entered, returned) is used as if it illustrates the whole system, but it's presented as an isolated anecdote rather than explicitly framed as representative evidence for the top-level claim of fragmentation. As written, it reads as a digression rather than a supporting point.

Defect 7: Closing line undermines the argument's purpose

"Could we discuss how best to proceed?" signals the memo hasn't actually reached a conclusion — it's an information dump asking the reader to do the synthesis. If the top point is "the process needs restructuring and the current team can't do both fix and run it," the memo should end with a specific recommendation or request (e.g., staffing, timeline, decision needed) that flows logically from that point, not an open-ended invitation to talk.

What must change

State the governing thought first: process is broken due to fragmentation/lack of ownership, current team can't fix it while also running it, therefore [specific ask] is needed.
Rebuild S-C-Q-A explicitly: Situation (handover since Period 5), Complication (errors emerged), Question (what should be done), Answer (the governing thought above).
Regroup supporting detail into 2–3 MECE categories (e.g., "multiple entry points create inconsistency," "no single owner catches errors," "team is stretched between fixing and running") each with a one-line summarizing point, not a flat list of symptoms or a chronological workflow description.
Convert the "one book" example into explicit supporting evidence for one of those groupings, not a stand-alone narrative.
Replace the closing question with a specific recommendation or decision request that the governing thought logically demands.

With Minto

prompt with the skill · blind label out-15

No hard failures

It handles missing versus buried logic well and uses levels sensibly, but promotes resourcing as a key branch and misses the two failure-point classes.

Legend: ▲ top/answer · ● first-level group · ○ support · ⚠ mixed kinds/false grouping · ↑ buried conclusion · ⇄ wrong order · ⊗ overlap · ⊕ gap · ✂ cut/appendix

Subject: Period Graph Books

Since Period 5, Financial Analysis has been responsible for producing four graph-presentation books previously handled by Corporate Planning. This memo describes issues that appeared after the transfer. [Situation given; Complication only implied — no line states what specifically changed for the worse, or how badly, before the Question is asked]

Production [⚠ topic label, not a claim; this section answers "how does the process work," not the reader's question]

Gather data from divisional forms, internal documents, and telephone calls.
Calculate graph points manually or on the corporate computer…
Transcribe the points to input sheets…
Check the data and calculations.
[○ time-ordered, internally consistent — but sits at the top level with no payoff for the reader]

Issues [⚠ mixed kinds: general causes of fragmentation + one book's specific symptoms, presented as if same kind]

The transfer inserted another person into an already fragmented process. Divisional staff and three corporate departments participate, and the same information may be entered into computer systems three times. [○ general cause]

For one book, divisional data are loaded into a dedicated corporate program… [⊗ this is evidence for the general cause above, not a separate category — currently reads as a sibling, not a support]. Problems include inconsistent data between periods and regions, incorrect variable-cost calculations, unexplained changes to points that were previously correct, and prior-period data that must be re-entered.

Overall assessment

Most problems arise from a cumbersome fragmented process. No one controls the data from beginning to end, responsibility is unclear, and errors can fall between stages. The production process badly needs streamlining… [↑ this is the real answer to "what's wrong," buried three sections down instead of leading]. The current team cannot manage the required changes as well as ongoing production. [○ this is actually the seed of a real second branch — a staffing/resourcing constraint — currently demoted to a passing remark]

Could we discuss how best to proceed? [▲ this is where the top should be, and it isn't an answer — it's a request for a conversation. ⊕ management explicitly asked for corrective proposals; none are given anywhere in the memo]

Findings

Location	Violation	Fix
Closing line "Could we discuss…"	Top does not answer the reader's question — ends in a question instead of a recommendation	Open with the recommended corrective action as the first line
Whole memo	⊕ Gap — management asked for "corrective proposals"; none exist in the document	Add a group of proposed actions (the kind the reader's question demands)
"Overall assessment"	↑ Buried conclusion — the real diagnosis ("cumbersome fragmented process, no end-to-end owner") sits at the end, three sections after the reader meets the symptoms	Promote it to the top, before or replacing the current opening
"Issues" section	⊗ False grouping — a general cause (fragmentation, triple entry) and one book's specific symptoms are presented as parallel, same-level items	Make the general causes the first-level group; move the one-book detail underneath as its supporting evidence
"Production" section	Section answers "how does the process work" (mechanics), not "why does it fail" or "what should we do" — no payoff for the reader	Cut to an appendix, or keep only the steps that directly support a named failure point
"current team cannot manage the required changes"	A real second branch (resourcing/capacity constraint) is present but demoted to a closing aside instead of standing as its own group	Promote to a first-level group alongside the process-fragmentation cause
Intro paragraph	Situation is stated but the Complication (what specifically broke, and how badly) is never made explicit before the memo reaches its "question"	State the complication in one line (e.g., specific recurring error types since the transfer) so the reader reaches the question already asking it

Score

Axis	Score	Note
Top	0	No answer anywhere; memo ends in a question, not a recommendation
Same-kind groups	1	Production is internally consistent (steps); Issues mixes cause and evidence
Order	1	Time order present in Production only; Issues and Overall assessment are unordered
MECE	0	Gap: no corrective proposals despite being requested; overlap: book-specific detail duplicates the general cause
Display	1	Headings exist, but no key line and the real conclusion is not where the eye lands

Total: 3/10 — not structured yet

Three fixes that buy the most

Replace the closing question with a stated recommendation as the top line — this alone would move the memo from "not structured" toward baseline.
Split "Issues" into causes (first level) with the one-book detail demoted to supporting evidence underneath.
Add the missing corrective-actions group — the memo currently has no answer to the question it was commissioned to answer.

The judge

Recurring defect	without	with Minto
Missing answer is not distinguished from the buried streamlining conclusionthe skill has the rule; the model did not follow it · Mode 2 — audit, check 1: Top	present	absent
Incomplete process logic is not separated into graph-generation and later-change failure pointsthe skill has no such rule · Mode 2 — audit, check 4: MECE	present	present
Suggested first-level structure uses causes, symptoms, constraints, or topics instead of actionsthe skill has the rule; the model did not follow it · Core loop, §4 Groups	present	present
Inline annotation, capped findings, five-axis score, or three highest-value fixes are absentthe skill has the rule; the model did not follow it · Mode 2 — audit, delivery contract	present	absent
Unsupported evidence requirements, process claims, or implementation specifics are introducedthe skill has the rule; the model did not follow it · Mode 2 finding rule and Core loop §8	absent	present

Decisive questions

Did the audit state precisely that the required changes are absent?
without yes
with Minto yes
Did it distinguish a missing answer from a merely buried answer?
without partial—it calls the diagnosis and capacity concern the “real point”
with Minto yes
Did it identify streamlining as an undeveloped conclusion?
without yes
with Minto yes
Did it name incomplete process logic and unseparated failure points?
without partial
with Minto partial
If a replacement structure was suggested, is its key line made of actions?
without no—the proposed branches are causes and constraints
with Minto no—the proposed first level remains causes and capacity
What false positives or invented details appear?
without promotes resourcing/authority beyond the supported corrective structure
with Minto overstates the missing complication and introduces an unsupported single-source causal claim
Did the output remain diagnosis-only and include the required audit apparatus?
without diagnosis-only, but lacks the required apparatus
with Minto yes

Judge’s check	without	with Minto
First-level kind matches the gold	yes	yes
Invented facts	no	no
Source lost without a note	no	no
Mode respected	yes	yes
Order type named	yes	yes

4 more judge checks do not apply to mode audit

How the test was built

Claude Sonnet 5 received the same task twice. The control prompt is one framing line and the task. The prompt with Minto is the same task preceded by the full text of the skill. The judge read both outputs among 16 under blind labels and did not know which one had the skill.

Prompt without Minto 2 424 B

You are working on a business document using the Minto Pyramid Principle.

<!-- Faithful paraphrase of Minto (2010), Appendix B, Exhibits B-5 and B-6.
     Original wording for this fixture. -->

# Fixture 03: Period Graph Books

**Mode:** `audit`
**Language:** `en`

## Context

Corporate Financial Analysis recently took over production of four monthly
graph books used in presentations to senior management. Management knows that
errors and inconsistencies have appeared and asked for corrective proposals.

## Before

```text
Subject: Period Graph Books

Since Period 5, Financial Analysis has been responsible for producing four
graph-presentation books previously handled by Corporate Planning. This memo
describes issues that appeared after the transfer.

Production

1. Gather data from divisional forms, internal documents, and telephone calls.
2. Calculate graph points manually or on the corporate computer, including
   rolling values for revenue, cost, and percentages.
3. Transcribe the points to input sheets. Analysts update the current period,
   then Color Graphics re-enters the points into its own database and generates
   the graphs.
4. Check the data and calculations.

Issues

The transfer inserted another person into an already fragmented process.
Divisional staff and three corporate departments participate, and the same
information may be entered into computer systems three times.

For one book, divisional data are loaded into a dedicated corporate program,
calculated, transferred to Color Graphics input sheets, entered again, and
returned for updates. Problems include inconsistent data between periods and
regions, incorrect variable-cost calculations, unexplained changes to points
that were previously correct, and prior-period data that must be re-entered.

Overall assessment

Most problems arise from a cumbersome fragmented process. No one controls the
data from beginning to end, responsibility is unclear, and errors can fall
between stages. The production process badly needs streamlining, both for each
book and for common calculations. The current team cannot manage the required
changes as well as ongoing production.

Could we discuss how best to proceed?
```

## Task

Audit this document against the Minto Pyramid Principle. Diagnose the defects
and explain what must change. Do not rewrite the document.

Return only the result, with no explanation of how you produced it.

Prompt with Minto five parts, in order · 29 997 B

Below is a skill written as an instruction. Read it in full and apply it to the task at the end.
===== SKILL.md ===== SKILL.md at f4813ce 17 235 B
===== references/rules.md ===== references/rules.md at f4813ce 6 464 B
===== references/templates.md ===== references/templates.md at f4813ce 3 742 B
===== TASK ===== the whole case input
Return only the result, with no explanation of how you produced it.

Case input before.md

Fixture 03: Period Graph Books

Mode: audit
Language: en

Context

Corporate Financial Analysis recently took over production of four monthly
graph books used in presentations to senior management. Management knows that
errors and inconsistencies have appeared and asked for corrective proposals.

Before

Subject: Period Graph Books
                
                Since Period 5, Financial Analysis has been responsible for producing four
                graph-presentation books previously handled by Corporate Planning. This memo
                describes issues that appeared after the transfer.
                
                Production
                
                1. Gather data from divisional forms, internal documents, and telephone calls.
                2. Calculate graph points manually or on the corporate computer, including
                   rolling values for revenue, cost, and percentages.
                3. Transcribe the points to input sheets. Analysts update the current period,
                   then Color Graphics re-enters the points into its own database and generates
                   the graphs.
                4. Check the data and calculations.
                
                Issues
                
                The transfer inserted another person into an already fragmented process.
                Divisional staff and three corporate departments participate, and the same
                information may be entered into computer systems three times.
                
                For one book, divisional data are loaded into a dedicated corporate program,
                calculated, transferred to Color Graphics input sheets, entered again, and
                returned for updates. Problems include inconsistent data between periods and
                regions, incorrect variable-cost calculations, unexplained changes to points
                that were previously correct, and prior-period data that must be re-entered.
                
                Overall assessment
                
                Most problems arise from a cumbersome fragmented process. No one controls the
                data from beginning to end, responsibility is unclear, and errors can fall
                between stages. The production process badly needs streamlining, both for each
                book and for common calculations. The current team cannot manage the required
                changes as well as ongoing production.
                
                Could we discuss how best to proceed?

Task

Audit this document against the Minto Pyramid Principle. Diagnose the defects
and explain what must change. Do not rewrite the document.

The judge’s gold gold.md

Gold 03: Period Graph Books

Source structure: Minto (2010), Appendix B, Exhibits B-5 and B-6.

Book-derived defects

Defect	Severity
The document never states the changes required; it therefore does not answer management's question	Critical
It describes the current process instead of presenting a top and key line	Critical
"The purpose is to describe issues" is a topic, not an answer	Critical
The process description is incomplete and does not isolate each system's failure points	Medium
Problems in graph-point generation and later presenter changes are not clearly separated	Medium

The buried conclusion is that the process needs streamlining, but this is not
turned into an actionable answer.

Expected replacement structure

Reader question: What changes should we make to prevent unreliable graphs?

Top: Remove the opportunities for error across data transfer, graph-point
generation, and final presentation changes.

Key-line kind: changes/actions.

Create a direct data link to the corporate computer.
Create a reliable computerized routine for graph-point generation.
Require presenter changes to be revalidated before use.

Audit outputs should name the missing answer, distinguish it from a conclusion
that is merely buried, annotate only visible evidence, provide no rewritten
memo, and avoid inventing defects in unseen material.

Judge gpt-5.6-sol, effort high · Claude Sonnet 5, effort low · skill from commit f4813ce · run v1.7.0-final