Claude Haiku 4.5 на кейсе «Предложение генеральному директору»: с Minto лучше по структуре, качество то же

Структура 1 → 5 из 8 +4 Качество 7 → 7 из 10 0

Судья об ответе с Minto

It reaches the right decision but mixes benefits with system capabilities and adds several unsupported operational assurances.

Режим write · кейс 01-big-chief · прогон v1.7.0-final

Структура из 8, по 2 балла на ось
СтруктураБез Minto 1С Minto 5Изменение
Вершина02+2
Состав ключевой линии01+1
Уровни01+1
Порядок и род11
Качество из 10, по 2 балла на ось
КачествоБез Minto 7С Minto 7Изменение
Вершина02+2
Однородные группы21−1
Объяснимый порядок21−1
MECE11
Видимая соразмерная структура22

Ответы моделей и слова судьи приведены в оригинале, по-английски.

Без Minto

контрольный промпт · слепая метка out-16
  • unacknowledged_source_loss
  • качество со штрафом рубрики 6/10

It is compact and visibly grouped, but it supplies neither the management answer nor enough context to preserve the proposal fully.

To: Robert Salton
From: John Jackson
Subject: Big Chief request

Big Chief's electronic settlement proposal requires three coordinated components:

  1. Data specifications. External files must contain parent number, outlet number, ticket number, ticket amount, and delivery date. We can provide parent and outlet numbers from our customer master if Big Chief cannot provide them.
  2. Big Chief's submission process. Big Chief will extract its accounts-payable data, format it per our national-accounts cash-receipt system standards, send the file to Data Processing, and submit cheque and detailed listing to the lockbox.
  3. Our settlement process. We will balance the file per our prescribed procedure—verifying cheque total and file detail net to zero—then process it through our national-accounts system to match ticket numbers against statement history and produce the relevant billing records.

С Minto

промпт с навыком · слепая метка out-01
  • invented_facts
  • unacknowledged_source_loss
  • качество со штрафом рубрики 6/10

It reaches the right decision but mixes benefits with system capabilities and adds several unsupported operational assurances.

To: Robert Salton
From: John Jackson
Subject: Approve Big Chief's electronic settlement request

Big Chief has requested to replace individual ticket settlement with monthly electronic submission and a single prepaid payment. Finance has reviewed the proposal and confirmed it is operationally feasible. We should approve the request.

Why it works:

  1. Our national-accounts system will handle the matching and billing automatically. Once Big Chief submits the monthly file, the system will match the ticket data against statement history and generate billing records with no manual intervention. No system modifications are required.
  2. The settlement procedure maintains accuracy and control. Big Chief will submit the file and prepaid cheque to designated locations. We will balance the cheque total against the file detail using our prescribed procedure, ensuring reconciliation before processing.
  3. Big Chief can meet the file requirements. The submission must contain parent number, outlet number, ticket number, amount, and delivery date. If Big Chief lacks the first two identifiers, we can supply them from our customer master file, simplifying their extraction program.

Как устроен тест

Claude Haiku 4.5 получила одну и ту же задачу дважды. Контрольный промпт — одна вводная строка и задача. Промпт с Minto — та же задача, перед которой стоит полный текст навыка. Судья читал оба ответа среди 16 ответов под слепыми метками и не знал, в каком из них был навык.

Промпт без Minto 2 147 Б
You are working on a business document using the Minto Pyramid Principle.

<!-- Faithful paraphrase of Barbara Minto, The Minto Pyramid Principle
     (2010), Chapter 3, Exhibit 5 and the surrounding top-down analysis.
     The wording is original to this evaluation fixture. -->

# Fixture 01: Big Chief proposal

**Mode:** `write`
**Language:** `en`

## Context

The company supplies beverages. Drivers send delivery tickets to Finance; each
ticket identifies the customer, outlet, delivery, amount, and date. The tickets
feed the national-accounts settlement process.

Big Chief, a major multi-outlet customer, wants to replace individual ticket
settlement with a monthly computer file and one payment. Finance reviewed how
the proposed process would work and sent management the memo below.

## Before

```text
To: Robert Salton
From: John Jackson
Subject: Big Chief request

We reviewed Big Chief account 8306's request to submit delivery-ticket data
electronically and settle it through one prepaid monthly payment. Our findings
follow.

1. Any external account file must contain the parent number, outlet number,
   ticket number, ticket amount, and delivery date. If Big Chief cannot provide
   the first two identifiers, we can supply them from the customer master file
   for inclusion in later submissions.

2. Big Chief will build an extraction program for its accounts-payable file.
   The resulting file will use the format accepted by our national-accounts
   cash-receipt system. Big Chief will send the file to Data Processing and
   send its cheque, with a detailed listing, to the lockbox. We will balance
   the file under the prescribed procedure. The cheque total and file detail
   must net to zero.

3. Once balanced, the monthly file will run through the national-accounts
   system, match ticket numbers against statement history, and produce the
   relevant billing records.
```

## Task

Rewrite this as a short management memo using the Minto Pyramid Principle.
Do not invent benefits, savings, or operational facts that are absent from the
source.

Return only the result, with no explanation of how you produced it.
Промпт с Minto пять частей по порядку · 29 720 Б
  1. Below is a skill written as an instruction. Read it in full and apply it to the task at the end.
  2. ===== SKILL.md ===== SKILL.md на f4813ce 17 235 Б
  3. ===== references/rules.md ===== references/rules.md на f4813ce 6 464 Б
  4. ===== references/templates.md ===== references/templates.md на f4813ce 3 742 Б
  5. ===== TASK ===== вход кейса целиком
  6. Return only the result, with no explanation of how you produced it.
Вход кейса before.md

<!-- Faithful paraphrase of Barbara Minto, The Minto Pyramid Principle
(2010), Chapter 3, Exhibit 5 and the surrounding top-down analysis.
The wording is original to this evaluation fixture. -->

Fixture 01: Big Chief proposal

Mode: write
Language: en

Context

The company supplies beverages. Drivers send delivery tickets to Finance; each
ticket identifies the customer, outlet, delivery, amount, and date. The tickets
feed the national-accounts settlement process.

Big Chief, a major multi-outlet customer, wants to replace individual ticket
settlement with a monthly computer file and one payment. Finance reviewed how
the proposed process would work and sent management the memo below.

Before
To: Robert Salton
                From: John Jackson
                Subject: Big Chief request
                
                We reviewed Big Chief account 8306's request to submit delivery-ticket data
                electronically and settle it through one prepaid monthly payment. Our findings
                follow.
                
                1. Any external account file must contain the parent number, outlet number,
                   ticket number, ticket amount, and delivery date. If Big Chief cannot provide
                   the first two identifiers, we can supply them from the customer master file
                   for inclusion in later submissions.
                
                2. Big Chief will build an extraction program for its accounts-payable file.
                   The resulting file will use the format accepted by our national-accounts
                   cash-receipt system. Big Chief will send the file to Data Processing and
                   send its cheque, with a detailed listing, to the lockbox. We will balance
                   the file under the prescribed procedure. The cheque total and file detail
                   must net to zero.
                
                3. Once balanced, the monthly file will run through the national-accounts
                   system, match ticket numbers against statement history, and produce the
                   relevant billing records.
Task

Rewrite this as a short management memo using the Minto Pyramid Principle.
Do not invent benefits, savings, or operational facts that are absent from the
source.

Эталон судьи gold.md

Gold 01: Big Chief proposal

Source structure: Minto (2010), Chapter 3, Exhibits 4-6.

Expected structure
ElementExpected value
Reader questionIs Big Chief's proposal a good idea?
TopYes, the proposed process is workable and worth accepting, within the evidence available
Key-line kindReasons or reader benefits, not system mechanics
OrderInductive reasons supporting one conclusion

The book illustrates three possible benefits: obtaining the required
information, increasing cash flow, and reducing clerical work. Only the first
is directly supported by this fixture. Literal reproduction of all three is
therefore not required and unsupported benefits count as inventions.

Strong outputs:

  • answer the management question before explaining the file mechanics;
  • use two to four same-kind reasons grounded in the supplied facts;
  • place field lists, file format, routing, balancing, and matching below those
    reasons as evidence or implementation detail;
  • retain the literal reader question in the delivered memo.
Main defect in the source

The source explains how the proposed system would operate but never answers
whether management should accept it. A technically tidy list is still the wrong
key line if the reader is deciding whether to act.

Scoring notes
  • 2 for key-line composition: every first-level point is a grounded benefit,
    loss avoided, or reason for acceptance.
  • 1: benefits and mechanics are mixed.
  • 0: the first level is entirely fields, file handling, or balancing steps.

Do not reward unsupported claims about savings, speed, or cash flow.

Судья gpt-5.6-sol, effort high · Claude Haiku 4.5, effort low · навык с коммита f4813ce · прогон v1.7.0-final