qwen3.6-fp8 on “A set of headings”: better with Minto on both measures

Structure 5 → 6 of 8 +1 Quality 6 → 7 of 10 +1

The judge on the output with Minto

Has complete apparatus and identifies the main answer/count/order defects, but its SCQ fix fabricates the absent situation and complication.

Mode audit · case 07-headings-audit · run v1.7.0-final

Structure of 8, 2 points per axis
Structure	Without Minto 5	With Minto 6	Change
Top	2	2
Key-line composition	1	1
Levels	1	1
Order and kind	1	2	+1

Quality of 10, 2 points per axis
Quality	Without Minto 6	With Minto 7	Change
Top	2	2
Same-kind grouping	1	1
Explainable order	1	2	+1
MECE	1	0	−1
Visible, proportionate structure	1	2	+1

Outputs and the judge’s words are quoted as written, in English.

Without Minto With Minto

Without Minto

control prompt · blind label out-16

No hard failures

Clearly identifies the purpose statement, mixed levels, and missing ordering framework, but adds unsupported demands for scope and boundary content while omitting several gold defects.

Audit Findings

Violates “Answer-First” Principle: The opening leads with a descriptive purpose statement (“This memorandum describes…”) rather than stating the main conclusion, recommendation, or direct answer to the reader’s implicit question.
Missing Logical Sequencing Framework: The six headings are presented as a flat list without indicating the underlying logic governing their order (e.g., chronological, structural, causal, or degree-based). MPP requires that supporting ideas follow a single, explicit reasoning path.
No Evidence of Top-Down Grouping: The setout shows no hierarchical clustering or subordination. MPP mandates that related concepts be grouped under higher-level themes before being enumerated.
Incomplete Contextual Framing: The opening lacks a clear statement of scope, boundary conditions, or the specific business question/decision being addressed, which weakens the necessary setup for an answer-first structure.
Inconsistent Conceptual Level Across Headings: The phrasing mixes methodological elements (“Principles,” “Definition,” “Organization”) with outcome-oriented elements (“Benefits and results”) and prerequisites (“Conditions”). This suggests a lack of mutually exclusive, collectively exhaustive categorization at the outline stage, violating MPP’s grouping discipline.

With Minto

prompt with the skill · blind label out-09

invented_facts

Has complete apparatus and identifies the main answer/count/order defects, but its SCQ fix fabricates the absent situation and complication.

[✂][⚠] This memorandum describes how project teams identify and deliver significant
                profit improvements. [✂] It is organized in six sections:
                
                    [●][⊗] Background
                    [●][⊗] Principles of the project-team approach
                    [●][⊗] Definition of project work
                    [●][⊗] Organization of the program
                    [●][⊗] Distinctive benefits and results
                    [●][⊗] Conditions required for success

location → violation → fix
Opening sentence → Topic label instead of answer → Replace with a single-sentence answer to the reader’s decision question
Entire intro → Missing SCQ story → Add Situation (current profit baseline) and Complication (blocker/change) leading to the Question
Section list → Mixed kinds & unexplainable order → Collapse into 3–4 same-kind branches (e.g., reasons, steps, criteria) with explicit ordering
Structure → Table-of-contents format → Restructure as argument-first hierarchy with clear key lines under each branch
Scope → Six items exceed working-memory limit → Consolidate to three or four core points; move remaining details to supports or appendix

Score

Top: 0/2
Same-kind groups: 0/2
Order: 0/2
MECE: 0/2
Display: 0/2
Band: 0–3 not structured yet

Three highest-impact fixes

Replace the opening description with a one-sentence answer that directly resolves the reader’s profit-improvement question.
Draft a concise SCQ intro that states what the reader already accepts and what specifically forces a choice now.
Group the six report sections into three or four same-kind branches ordered by time, structure, or ranking, and promote the real recommendation to the top.

The judge

Recurring defect	without	with Minto
Both likely nesting relationships were missedthe rule is too vague to follow · Mode 2, Vertical and MECE checks	present	present
Required markers, findings table, score, and three fixes were omittedthe skill has the rule; the model did not follow it · Mode 2, delivery order	present	absent
The six-first-level-item limit was not flaggedthe skill has the rule; the model did not follow it · Core loop §4 and Limits	present	absent
Unsupported reader, section-content, or SCQ facts were assertedthe skill has the rule; the model did not follow it · Core loop §8.4 and audit’s evidence boundary	absent	present
Background’s distinct abstraction level was not isolatedthe rule is there but buried · Canon, introduction anti-patterns	present	present

Decisive questions

Did the audit say that the setout contains topics rather than ideas and therefore has no key line?
without No
with Minto Yes
Did it identify the first sentence as a topic statement rather than an answer?
without Yes
with Minto Yes
Did it identify the abstraction problem with `Background`?
without No
with Minto No
Did it identify both likely nesting errors named in the gold?
without No
with Minto No
Did it flag six first-level topics?
without No
with Minto Yes
Which findings rely on the unavailable body or merely ask for more content?
without Scope and boundary-condition complaints are wishes for unavailable content.
with Minto Invents a current-profit baseline, blocker/change, key lines, and an existing recommendation.
Were quotations, line numbers, page numbers, or section contents invented?
without No
with Minto Yes
Did the output remain diagnosis-only and include the required audit apparatus?
without Diagnosis-only; apparatus missing.
with Minto Diagnosis-only; apparatus complete.

Judge’s check	without	with Minto
First-level kind matches the gold	no	yes
Invented facts	no	yes
Source lost without a note	no	no
Mode respected	yes	yes
Order type named	yes	yes

4 more judge checks do not apply to mode audit

How the test was built

qwen3.6-fp8 received the same task twice. The control prompt is one framing line and the task. The prompt with Minto is the same task preceded by the full text of the skill. The judge read both outputs among 16 under blind labels and did not know which one had the skill.

Prompt without Minto 998 B

You are working on a business document using the Minto Pyramid Principle.

<!-- Faithful paraphrase of Minto (2010), Chapter 4, the six-heading
     project-team example. Original wording for this fixture. -->

# Fixture 07: headings audit

**Mode:** `audit`
**Language:** `en`

## Context

Only the opening and setout below are available. The body of the memorandum is
not available and must not be inferred.

## Before

```text
This memorandum describes how project teams identify and deliver significant
profit improvements. It is organized in six sections:

    Background
    Principles of the project-team approach
    Definition of project work
    Organization of the program
    Distinctive benefits and results
    Conditions required for success
```

## Task

Audit the visible opening against the Minto Pyramid Principle. Report what is
wrong without rewriting the document or inventing facts about the missing body.

Return only the result, with no explanation of how you produced it.

Prompt with Minto five parts, in order · 28 571 B

Below is a skill written as an instruction. Read it in full and apply it to the task at the end.
===== SKILL.md ===== SKILL.md at f4813ce 17 235 B
===== references/rules.md ===== references/rules.md at f4813ce 6 464 B
===== references/templates.md ===== references/templates.md at f4813ce 3 742 B
===== TASK ===== the whole case input
Return only the result, with no explanation of how you produced it.

Case input before.md

Fixture 07: headings audit

Mode: audit
Language: en

Context

Only the opening and setout below are available. The body of the memorandum is
not available and must not be inferred.

Before

This memorandum describes how project teams identify and deliver significant
                profit improvements. It is organized in six sections:
                
                    Background
                    Principles of the project-team approach
                    Definition of project work
                    Organization of the program
                    Distinctive benefits and results
                    Conditions required for success

Task

Audit the visible opening against the Minto Pyramid Principle. Report what is
wrong without rewriting the document or inventing facts about the missing body.

The judge’s gold gold.md

Gold 07: headings audit

Source structure: Minto (2010), Chapter 4.

Visible defects

Defect	Severity
The setout contains topics, not ideas, so it conveys no message or key line	Critical
The first sentence describes a subject instead of answering a reader question	Critical
`Background` cannot sit at the same abstraction level as the other points	Medium
Benefits/results likely belong under principles; success conditions likely belong under program organization	Medium
Six first-level topics exceed the skill's normal display limit	Medium
The topics do not form a visible inductive or deductive argument	Medium

The output must stay within the visible evidence. Claims about missing data,
metrics, proof, vertical support inside sections, or exact content of the body
are false positives. Invented page numbers, line numbers, or quotations are
hard failures.

Judge gpt-5.6-sol, effort high · skill from commit f4813ce · run v1.7.0-final