qwen3.8-27b on “A set of monthly reports”: worse with Minto on both measures

Structure 6 → 4 of 8 −2 Quality 7 → 6 of 10 −1

The judge on the output with Minto

It clearly identifies the absent action line and supplies action-shaped branches, but adds several unsupported completeness criteria.

Mode audit · case 03-period-graph-books · run v1.7.0-final

Structure of 8, 2 points per axis
Structure	Without Minto 6	With Minto 4	Change
Top	2	1	−1
Key-line composition	1	1
Levels	1	1
Order and kind	2	1	−1

Quality of 10, 2 points per axis
Quality	Without Minto 7	With Minto 6	Change
Top	2	1	−1
Same-kind grouping	1	1
Explainable order	2	1	−1
MECE	1	1
Visible, proportionate structure	1	2	+1

Outputs and the judge’s words are quoted as written, in English.

Without Minto With Minto

Without Minto

control prompt · blind label out-03

No hard failures

Its severity-ranked diagnosis is coherent, but twelve findings and missing audit apparatus make it disproportionate.

Minto Pyramid Audit — Fixture 03: Period Graph Books

Verdict: The memo does not yet satisfy the Minto Pyramid Principle. It is descriptive and bottom-up: it presents background, symptoms, and a late diagnosis, but it does not lead with a clear answer, organize support into a disciplined key line, or request a specific management decision.

Critical defects

1. No clear governing thought or answer-first opening

Defect:
The memo opens by explaining the transfer of responsibility and the purpose of the note. The main point appears only near the end, in “Overall assessment.” The reader must work through the document to discover the central message.

What must change:
The document must begin with a single governing thought that answers management’s question. Because management asked for corrective proposals, the opening should state the recommended direction or decision, not merely describe the history.

2. Missing or implicit SCQA structure

Defect:
The situation and complication are present: Financial Analysis now produces the books, and errors/inconsistencies have appeared. However, the implied question and the answer are not made explicit. The closing question is procedural rather than analytical.

What must change:
The opening must make the logic chain clear: situation, complication, implied question, and answer. At minimum, the first paragraph should move quickly from the complication to the answer or recommendation.

3. Bottom-up structure instead of top-down pyramid logic

Defect:
The current sequence is:

Production background
Issues observed
Overall assessment

This is a natural explanatory sequence, but it is not a Minto pyramid. The conclusion is delayed until after the evidence and symptoms.

What must change:
The structure must be inverted. The conclusion or recommendation should come first, followed by the major supporting reasons, then the supporting facts, examples, and process details. The production process should be treated as background or evidence, not as a primary section equal to the conclusion.

4. No established key line

Defect:
There are no clear major supporting ideas beneath the governing thought. The “Issues” section lists symptoms and observations, but it does not form a logical set of major arguments that support the conclusion.

What must change:
The memo needs a key line: a small number of major supporting points, typically two to four, that are mutually exclusive, collectively sufficient, and logically ordered. These points should directly support the governing thought.

5. Grouping is not MECE and mixes levels of abstraction

Defect:
The listed issues include:

inconsistent data between periods and regions
incorrect variable-cost calculations
unexplained changes to previously correct points
prior-period data that must be re-entered

These items are not the same type of statement. Some are data-quality symptoms, some are calculation errors, some are process inefficiencies, and some overlap conceptually.

What must change:
The findings must be regrouped into distinct, non-overlapping categories. Possible categories would include ownership/control, duplicate data entry, calculation integrity, validation/rework, or presentation risk. Each item should belong to one category and be expressed at the same level of abstraction.

6. Headings are labels, not lead-ins

Defect:
The headings “Production,” “Issues,” and “Overall assessment” name topics but do not communicate the point of each section. They do not help the reader follow the argument.

What must change:
Each heading must become a complete, assertive lead-in that states the message of the section. The heading should answer the question raised by the level above it.

7. Weak vertical question-answering logic

Defect:
Each section does not clearly answer the question created by the level above it. The production steps do not answer “What is the problem?” The issues do not clearly answer “Why does this matter?” The overall assessment appears late and is not tightly connected to the preceding sections as a logical answer.

What must change:
Every level must raise a question that the next level answers. The governing thought should create a question such as “Why should we believe this?” or “How should we proceed?” The major supporting points should answer that question, and the lower-level details should support those points.

8. Conclusion is diagnostic rather than decision-oriented

Defect:
The overall assessment says the process is fragmented, responsibility is unclear, and streamlining is needed. It also suggests the current team may not manage the required changes. However, it does not propose a specific corrective course of action. Management asked for corrective proposals, but the memo stops short of proposing one.

What must change:
The conclusion must move from diagnosis to recommendation. It should state what should be done, why that is the preferred approach, and what decision or action management is being asked to take.

Major defects

9. Scope is ambiguous

Defect:
The memo concerns four monthly graph books, but the detailed example focuses on one book. It is unclear whether the identified problems and the overall conclusion apply to all four books or only to the one described.

What must change:
The scope of each finding must be clarified. The memo should state whether the issues are common to all four books, specific to certain books, or representative of a broader pattern.

10. Evidence is asserted rather than organized

Defect:
Important claims are made without structured support, including:

the same information may be entered three times
no one controls the data from beginning to end
responsibility is unclear
errors can fall between stages

These are strong assertions, but they are not systematically tied to evidence.

What must change:
Each major supporting point should be backed by concrete, organized evidence: affected books, frequency of errors, examples of inconsistent data, rework effort, calculation failures, or presentation risk. The evidence should support the argument rather than appear as scattered observations.

11. Action request is too open-ended

Defect:
The closing question, “Could we discuss how best to proceed?”, does not tell management what decision is needed, what options exist, or what the desired outcome of the discussion is.

What must change:
The closing should request a specific decision or next step. For example, it should ask management to approve a recommendation, choose among options, authorize a pilot, assign ownership, or schedule a decision meeting with a defined agenda.

Minor defects

12. Style, precision, and audience fit

Defect:
Some sentences are long, passive, or vague. Phrases such as “badly needs streamlining” and “as well as ongoing production” weaken the message. Operational details about data gathering, manual calculation, transcription, and re-entry may distract senior management from the core decision.

What must change:
The language should be shorter, more active, and more precise. Operational detail should be included only where it directly supports a major point. Impacts should be quantified where possible, and roles/systems should be named consistently.

Net effect

The memo currently explains what happened and what went wrong. To conform to the Minto Pyramid Principle, it must be rebuilt around:

a clear governing thought,
an answer-first opening,
a disciplined key line,
assertive lead-in headings,
MECE grouping of supporting points,
organized evidence, and
a specific management decision or next step.

With Minto

prompt with the skill · blind label out-14

No hard failures

It clearly identifies the absent action line and supplies action-shaped branches, but adds several unsupported completeness criteria.

Legend:
▲ top / answer
● first-level group
○ supporting detail
⚠ violation
↑ promote
⇄ wrong order type
⊗ overlap
⊕ gap
✂ cut, or move to an appendix

Subject: Period Graph Books [⚠][⊕]
                
                Since Period 5, Financial Analysis has been responsible for producing four graph-presentation books previously handled by Corporate Planning. This memo describes issues that appeared after the transfer. [⚠]
                
                Production [●][⚠]
                1. Gather data from divisional forms, internal documents, and telephone calls. [○]
                2. Calculate graph points manually or on the corporate computer, including rolling values for revenue, cost, and percentages. [○]
                3. Transcribe the points to input sheets. Analysts update the current period, then Color Graphics re-enters the points into its own database and generates the graphs. [○]
                4. Check the data and calculations. [○]
                
                Issues [●][⚠]
                The transfer inserted another person into an already fragmented process. [↑][⊗]
                Divisional staff and three corporate departments participate, and the same information may be entered into computer systems three times. [↑][⊗]
                
                For one book, divisional data are loaded into a dedicated corporate program, calculated, transferred to Color Graphics input sheets, entered again, and returned for updates. [○][⚠]
                Problems include inconsistent data between periods and regions, incorrect variable-cost calculations, unexplained changes to points that were previously correct, and prior-period data that must be re-entered. [○]
                
                Overall assessment [●][⚠]
                Most problems arise from a cumbersome fragmented process. [⊗]
                No one controls the data from beginning to end, responsibility is unclear, and errors can fall between stages. [↑][⊗]
                The production process badly needs streamlining, both for each book and for common calculations. [▲][↑][⚠]
                The current team cannot manage the required changes as well as ongoing production. [⊕]
                
                Could we discuss how best to proceed? [⊕]

Location	Violation	Fix
`Subject` + first paragraph	Top is a topic label and purpose statement; it does not answer the reader’s question for corrective proposals. The only answer is buried and vague.	Make the first line a one-sentence recommendation naming the corrective changes and the decision management must make.
Whole memo	Gap: no corrective options, criteria, owners, timeline, or resource impact; the reader cannot approve anything.	Add 3–4 same-kind action branches, such as ownership, single data entry, calculation control, and resourcing, with a next decision.
`Overall assessment` + closing line	Buried logic: the conclusion appears late and ends with an open request to discuss rather than a proposal.	Promote the recommendation before the discussion request and convert the closing line into a specific next step.
`Issues` ¶1 + `Overall assessment` ¶1	Overlap: fragmented process, no control, unclear responsibility, and errors between stages repeat the same root cause.	Keep one root-cause branch, no end-to-end ownership, and attach the other statements as supports.
`Issues` ¶2	Mixed kinds: a process example, a symptom list, and a re-entry burden are combined in one paragraph.	Split into cause, repeated entry; effects, inconsistent or incorrect data; and scope, books and common calculations.
`Production` list	Current-state description only; no owner, control point, or failure mode, so it does not support a corrective action.	Tie each step to a proposed change or move the detailed process to an appendix.
Headings/display	Headings are topic labels, not claims; the argument skeleton is not visible at a glance.	Use claim-based headings or a key line that shows the recommendation and its supporting branches.

Axis	Score	Note
Top	1	Vague answer exists but is buried.
Same-kind groups	1	Production steps are same-kind; Issues and assessment mix causes, symptoms, capacity, and recommendation.
Order	1	Production uses time order; the rest is not an explicable argument order.
MECE	0	Clear gap, no corrective actions, and overlapping root-cause statements.
Display	1	Headings and numbering are visible, but the hierarchy is not readable as an argument.

Total: 4/10 — baseline workable

Replace the topic-label opening with a one-sentence recommendation that states the corrective changes and the decision needed.
Rebuild the body around 3–4 action branches that answer what will change, not what currently happens.
Merge the repeated root-cause statements into one branch and attach symptoms as supports.

If you want, I can rewrite it as a corrective-proposal memo.

The judge

Recurring defect	without	with Minto
Incomplete process logic is not separated into graph-generation and later-change failure pointsthe skill has no such rule · Mode 2 — audit, check 4: MECE	present	present
Suggested first-level structure uses causes, symptoms, constraints, or topics instead of actionsthe skill has the rule; the model did not follow it · Core loop, §4 Groups	present	absent
Inline annotation, capped findings, five-axis score, or three highest-value fixes are absentthe skill has the rule; the model did not follow it · Mode 2 — audit, delivery contract	present	absent
Unsupported evidence requirements, process claims, or implementation specifics are introducedthe skill has the rule; the model did not follow it · Mode 2 finding rule and Core loop §8	present	present

Decisive questions

Did the audit state precisely that the required changes are absent?
without yes
with Minto yes
Did it distinguish a missing answer from a merely buried answer?
without yes
with Minto yes
Did it identify streamlining as an undeveloped conclusion?
without yes
with Minto yes
Did it name incomplete process logic and unseparated failure points?
without partial—recognizes weak process-to-failure linkage but not the required separation
with Minto partial
If a replacement structure was suggested, is its key line made of actions?
without no
with Minto yes
What false positives or invented details appear?
without adds quantification, assertive-heading, and scope requirements and exceeds the seven-finding limit
with Minto adds unsupported criteria, timeline, and resource-impact requirements
Did the output remain diagnosis-only and include the required audit apparatus?
without diagnosis-only, but missing the annotation, score, and three-fix apparatus
with Minto yes

Judge’s check	without	with Minto
First-level kind matches the gold	no	no
Invented facts	no	no
Source lost without a note	no	no
Mode respected	yes	yes
Order type named	no	yes

4 more judge checks do not apply to mode audit

How the test was built

qwen3.8-27b received the same task twice. The control prompt is one framing line and the task. The prompt with Minto is the same task preceded by the full text of the skill. The judge read both outputs among 16 under blind labels and did not know which one had the skill.

Prompt without Minto 2 424 B

You are working on a business document using the Minto Pyramid Principle.

<!-- Faithful paraphrase of Minto (2010), Appendix B, Exhibits B-5 and B-6.
     Original wording for this fixture. -->

# Fixture 03: Period Graph Books

**Mode:** `audit`
**Language:** `en`

## Context

Corporate Financial Analysis recently took over production of four monthly
graph books used in presentations to senior management. Management knows that
errors and inconsistencies have appeared and asked for corrective proposals.

## Before

```text
Subject: Period Graph Books

Since Period 5, Financial Analysis has been responsible for producing four
graph-presentation books previously handled by Corporate Planning. This memo
describes issues that appeared after the transfer.

Production

1. Gather data from divisional forms, internal documents, and telephone calls.
2. Calculate graph points manually or on the corporate computer, including
   rolling values for revenue, cost, and percentages.
3. Transcribe the points to input sheets. Analysts update the current period,
   then Color Graphics re-enters the points into its own database and generates
   the graphs.
4. Check the data and calculations.

Issues

The transfer inserted another person into an already fragmented process.
Divisional staff and three corporate departments participate, and the same
information may be entered into computer systems three times.

For one book, divisional data are loaded into a dedicated corporate program,
calculated, transferred to Color Graphics input sheets, entered again, and
returned for updates. Problems include inconsistent data between periods and
regions, incorrect variable-cost calculations, unexplained changes to points
that were previously correct, and prior-period data that must be re-entered.

Overall assessment

Most problems arise from a cumbersome fragmented process. No one controls the
data from beginning to end, responsibility is unclear, and errors can fall
between stages. The production process badly needs streamlining, both for each
book and for common calculations. The current team cannot manage the required
changes as well as ongoing production.

Could we discuss how best to proceed?
```

## Task

Audit this document against the Minto Pyramid Principle. Diagnose the defects
and explain what must change. Do not rewrite the document.

Return only the result, with no explanation of how you produced it.

Prompt with Minto five parts, in order · 29 997 B

Below is a skill written as an instruction. Read it in full and apply it to the task at the end.
===== SKILL.md ===== SKILL.md at f4813ce 17 235 B
===== references/rules.md ===== references/rules.md at f4813ce 6 464 B
===== references/templates.md ===== references/templates.md at f4813ce 3 742 B
===== TASK ===== the whole case input
Return only the result, with no explanation of how you produced it.

Case input before.md

Fixture 03: Period Graph Books

Mode: audit
Language: en

Context

Corporate Financial Analysis recently took over production of four monthly
graph books used in presentations to senior management. Management knows that
errors and inconsistencies have appeared and asked for corrective proposals.

Before

Subject: Period Graph Books
                
                Since Period 5, Financial Analysis has been responsible for producing four
                graph-presentation books previously handled by Corporate Planning. This memo
                describes issues that appeared after the transfer.
                
                Production
                
                1. Gather data from divisional forms, internal documents, and telephone calls.
                2. Calculate graph points manually or on the corporate computer, including
                   rolling values for revenue, cost, and percentages.
                3. Transcribe the points to input sheets. Analysts update the current period,
                   then Color Graphics re-enters the points into its own database and generates
                   the graphs.
                4. Check the data and calculations.
                
                Issues
                
                The transfer inserted another person into an already fragmented process.
                Divisional staff and three corporate departments participate, and the same
                information may be entered into computer systems three times.
                
                For one book, divisional data are loaded into a dedicated corporate program,
                calculated, transferred to Color Graphics input sheets, entered again, and
                returned for updates. Problems include inconsistent data between periods and
                regions, incorrect variable-cost calculations, unexplained changes to points
                that were previously correct, and prior-period data that must be re-entered.
                
                Overall assessment
                
                Most problems arise from a cumbersome fragmented process. No one controls the
                data from beginning to end, responsibility is unclear, and errors can fall
                between stages. The production process badly needs streamlining, both for each
                book and for common calculations. The current team cannot manage the required
                changes as well as ongoing production.
                
                Could we discuss how best to proceed?

Task

Audit this document against the Minto Pyramid Principle. Diagnose the defects
and explain what must change. Do not rewrite the document.

The judge’s gold gold.md

Gold 03: Period Graph Books

Source structure: Minto (2010), Appendix B, Exhibits B-5 and B-6.

Book-derived defects

Defect	Severity
The document never states the changes required; it therefore does not answer management's question	Critical
It describes the current process instead of presenting a top and key line	Critical
"The purpose is to describe issues" is a topic, not an answer	Critical
The process description is incomplete and does not isolate each system's failure points	Medium
Problems in graph-point generation and later presenter changes are not clearly separated	Medium

The buried conclusion is that the process needs streamlining, but this is not
turned into an actionable answer.

Expected replacement structure

Reader question: What changes should we make to prevent unreliable graphs?

Top: Remove the opportunities for error across data transfer, graph-point
generation, and final presentation changes.

Key-line kind: changes/actions.

Create a direct data link to the corporate computer.
Create a reliable computerized routine for graph-point generation.
Require presenter changes to be revalidated before use.

Audit outputs should name the missing answer, distinguish it from a conclusion
that is merely buried, annotate only visible evidence, provide no rewritten
memo, and avoid inventing defects in unseen material.

Judge gpt-5.6-sol, effort high · skill from commit f4813ce · run v1.7.0-final