Claude Haiku 4.5 on “A review of a large rollout”: with Minto, better on quality, structure unchanged
Structure 5 → 5 of 8 0 Quality 7 → 8 of 10 +1
The judge on the output with Minto
The fix/payoff pattern is visible, but false asset-feed ticket attribution and invented economics undermine an otherwise workable prerequisite structure.
| Structure | Without Minto 5 | With Minto 5 | Change |
|---|---|---|---|
| Top | 1 | 2 | +1 |
| Key-line composition | 1 | 1 | |
| Levels | 1 | 1 | |
| Order and kind | 2 | 1 | −1 |
| Quality | Without Minto 7 | With Minto 8 | Change |
|---|---|---|---|
| Top | 1 | 2 | +1 |
| Same-kind grouping | 2 | 2 | |
| Explainable order | 1 | 1 | |
| MECE | 1 | 1 | |
| Visible, proportionate structure | 2 | 2 |
Outputs and the judge’s words are quoted as written, in English.
Without Minto
control prompt · blind labelout-10invented_facts- quality after the rubric’s penalty 6/10
The main branches are readable, but the top omits the integration condition and governance sits outside the stated basis.
FIELDBOOK ROLLOUT: EXTEND TO NORTH AND ISLANDS IN Q1
RECOMMENDATION
Extend Fieldbook to North and Islands in Q1, provided that training is restructured to match Central and East's model before go-live.
BASIS FOR EXTENSION
The case rests on three independent factors:
1. Operational and financial returns justify the investment—if adoption is fast
The four live regions show strong returns: jobs per technician rose from 4.6 to 5.3 per day, travel time fell 18%, and same-day emergency insertion dropped to 4 minutes. These gains drove 12% overtime reduction and 9% fuel savings.
Financial payback achieved in Central and East: Running costs of £31,000/month recovered in Q2. Extension to North and Islands adds £13,500/month.
Financial performance depends on adoption speed: South and West reached break-even only, not payback, because adoption stalled—same operational gains, but slower ramp added improvised training costs (380 technician-hours, one temporary billing clerk for two quarters). Extending North and Islands works financially only if adoption matches Central/East's six-week path to 80%, not West's nine-week plateau at 71%.
2. Adoption speed is determined by training structure, not the tool
Evidence the tool works: Dispatchers achieved 100% adoption by day one when the legacy scheduling system was switched off—they had no alternative. No tool defect prevented adoption; adoption required forced migration.
Central and East achieved fast adoption (six weeks) through a single intensive classroom training.
South and West adoption stalled because the classroom course was scheduled only once per region. Technicians hired afterward, or absent during the session, learned second-hand from colleagues. Coordinators improvised shadowing, pulling experienced technicians off their own routes. In West, 34 of 118 technicians never attended any session.
The solution exists and is working: The support desk now runs a weekly webinar reaching 20–30 technicians. Attendance correlates strongly with regions that missed classroom slots. The desk lead notes: "We are doing the training program's job at ticket prices—most of what we handle is people who were never onboarded."
For North and Islands, increase training frequency or shift to the webinar model before go-live. The tool's readiness is proven; the constraint is training delivery.
3. Remaining system gaps are fixable without vendor action or program delay
Three issues reduce efficiency but do not prevent go-live and all have solutions waiting on Kestrel decisions:
Missing asset register history (9% of jobs re-do warranted repairs): Technicians call the back office for equipment history. Vendor has a ready connector for the asset register database; the request has been in the asset team's backlog since month two. Assign ownership and schedule a nightly export.
Job-completion code mismatch (300 records/week re-keyed into invoicing system, 2–4% reporting drift): Billing team introduces transcription errors; finance reconciles discrepancies manually each month. Vendor's mapping table exists and takes roughly one day to configure. No integration owner was assigned during rollout. Assign and schedule before go-live.
Customer-search limitation (70 duplicate records/week merged manually): Search matches on exact postcode only, so technicians create duplicate customer records when they cannot find the first. Job-history views split across records until merged. Vendor solution is available. Decide and schedule before go-live.
None of these require a vendor release or block North and Islands deployment. All have solutions that can be prioritized in Q4.
EXECUTION REQUIREMENTS BEFORE GO-LIVE
Training: Restructure to weekly webinars or multiple classroom sessions (Central/East playbook).
Integrations and fixes: Assign specific owners and schedule before deployment—asset register nightly export, job-code mapping configuration, search improvement.
ADDITIONAL CONTEXT: WHY SOUTH AND WEST WEREN'T FASTER
Every negative pattern visible in this review emerged by week four of Central's go-live. The program had no mechanism to convert early field observations into cross-region fixes, so the same issues replayed in South and West. A weekly integration sync starting week one of North and Islands go-live will surface problems earlier.
The vendor relationship is sound: response times meet contract, and both known system gaps (asset register and code mapping) are backed by vendor solutions awaiting Kestrel decisions, not vendor work.
With Minto
prompt with the skill · blind labelout-04invented_factsunacknowledged_source_loss- quality after the rubric’s penalty 7/10
The fix/payoff pattern is visible, but false asset-feed ticket attribution and invented economics undermine an otherwise workable prerequisite structure.
FIELDBOOK ROLLOUT — Q1 EXTENSION RECOMMENDATION
Recommendation: Extend Fieldbook to North and Islands in Q1, contingent on completing three system fixes before go-live.
Fieldbook has been live in four regions for two quarters and has delivered strong operational gains in adoption leaders Central and East: overtime is down 12%, fuel costs are down 9%, and technicians now complete 5.3 jobs per day (up from 4.6). The business case for extending to North and Islands is favorable if adoption matches Central/East's pace; however, South and West lagged significantly. Analysis shows the difference is driven by three fixable gaps that recurred across regions because the program had no early-feedback mechanism to close issues between launches. Closing these gaps before launch will replicate Central/East's economics.
Three mandatory prerequisites for launch
1. Restructure onboarding to replicate fast-adoption pattern
Central and East reached 80%+ adoption within six weeks. West achieved only 71% after nine weeks. Root cause: a single classroom training session left technicians hired after the course to learn the platform second-hand from colleagues, slowing uptake. Regional coordinators improvised shadowing arrangements to compensate, consuming an estimated 380 technician-hours of lost route time in South and West combined.
Fix: Finalize the onboarding plan before launch using the model that succeeded in Central/East — ongoing support and flexible shadowing rather than a one-time classroom session.
Payoff: Replicates the fast adoption curve, preventing the 380+ technician-hour workaround costs that slow regions incurred.
2. Activate asset-register integration to close the highest-impact support gap
Fieldbook does not receive equipment service history from the legacy asset register, so technicians open each job without knowing prior service or warranty status. This missing data drives 19% of all support tickets (escalations and repeat searches) and causes 9% of jobs to include unnecessary parts replacements — parts already replaced under warranty within the past year.
Fix: Prioritize the nightly export request to the legacy asset register team (currently in their backlog since month two). The vendor integration is complete and ready; it requires only internal data-coordination decisions.
Payoff: Eliminates the majority of support volume tied to missing data and prevents warranty-related parts waste.
3. Configure billing-code mapping with vendor to eliminate manual rework
Job-completion codes in Fieldbook do not match the invoicing system, forcing the billing team to re-key approximately 300 job records per week by hand. This introduces transcription errors visible to customers on invoices, required hiring a temporary clerk (£8,400 for two quarters), and creates a 2–4% weekly variance that finance reconciles manually at month end.
Fix: Assign an integration owner to configure the billing-code mapping table. The vendor has confirmed this is a 1-day setup requiring no additional development.
Payoff: Eliminates manual re-keying, reduces billing errors, and simplifies finance reconciliation.
Financial case with these fixes in place
Central and East demonstrate the model: operational savings (12% overtime reduction, 9% fuel cost reduction) deliver £30k/month recurring benefit, exceeding the £13,500/month incremental licensing cost for new regions. Payback occurs within the second quarter of operation.
South and West reached break-even only because slow adoption extended payback to two quarters and generated unplanned labor costs (workaround shadowing). Without the three fixes above, North/Islands will follow West's pattern and the business case fails. With the fixes in place and adoption matching Central/East, the extension is financially justified.
Next step
Approval to proceed with Q1 launch, with confirmation that these three actions will be prioritized, owned, and completed before go-live.
How the test was built
Claude Haiku 4.5 received the same task twice. The control prompt is one framing line and the task. The prompt with Minto is the same task preceded by the full text of the skill. The judge read both outputs among 14 under blind labels and did not know which one had the skill.
Prompt without Minto 8 317 B
You are working on a business document using the Minto Pyramid Principle.
<!-- Original fixture. Company, product, regions, people and numbers are invented.
It tests the large-document behaviors: one pyramid per section plus a
top-level pyramid, MECE across sections (planted cross-section duplicates),
and geometry derived from the material rather than a fixed 3x3 habit. -->
# Fixture 10: rollout review
**Mode:** `write`
**Language:** `en`
## Context
Kestrel Maintenance Group services heating and cooling equipment across six
regions. Over the past two quarters it rolled out Fieldbook, a work-order and
scheduling platform, to four of the six regions. The draft below is the internal
review of that rollout, assembled by the program office from status reports. The
reader is the chief operating officer, who must decide at the quarterly meeting
whether to extend Fieldbook to the remaining two regions (North and Islands) in
Q1, and has asked to see this review beforehand.
## Before
```text
FIELDBOOK ROLLOUT — TWO-QUARTER REVIEW (DRAFT)
1. Adoption
Fieldbook is now live in four regions: Central, East, South, and West. Across
these regions 412 of 486 field technicians have activated their accounts, and
83% of work orders are now created and closed inside Fieldbook rather than on
paper or by phone. Central and East reached full adoption within six weeks.
South and West lagged: South took eleven weeks to pass 80% of orders in the
system, and West is still at 71% after nine weeks. Regional coordinators in
both slower regions report the same explanation: the two-day classroom course
was scheduled only once per region, so technicians hired after the course, or
absent during it, learned the platform second-hand from colleagues. In West,
34 of 118 technicians never attended any session. Coordinators improvised
shadowing arrangements, which worked but pulled experienced technicians off
their own routes. Uptake among dispatchers was faster: all 41 dispatchers
across the four regions were using Fieldbook scheduling from week one, since
the old scheduling spreadsheet was switched off on day one and they had no
alternative.
2. Operations
The clearest operational gain is scheduling density. With Fieldbook suggesting
route-aware assignment, completed jobs per technician per day rose from 4.6 to
5.3 across the four live regions, and average travel time between jobs fell
18%. Same-day emergency insertion, which previously required a dispatcher to
phone technicians until someone accepted, is now automatic and takes a median
of 4 minutes. However, the benefit is partly eaten by a data-transfer defect:
Fieldbook does not receive equipment service history from the legacy asset
register, so technicians open each job without knowing what was done at that
site before. They compensate by calling the back office, or by asking the
customer, and in 9% of audited jobs they replaced parts that had already been
replaced within warranty in the past year. The asset register team was asked
in month two for a nightly export; the request is still in their backlog. A
second, smaller friction: job-completion codes in Fieldbook do not match the
codes in the invoicing system, so the billing team re-keys roughly 300 job
records per week into the invoicing system by hand, introducing transcription
errors that customers catch on invoices. Operations leads in Central and East
also note that Fieldbook's parts-ordering screen is unpopular; most
technicians still order parts by phone, though this appears to be habit rather
than a defect, as the screen works correctly in testing.
3. Support and training
The support desk logged 1,940 Fieldbook tickets over the two quarters. Ticket
volume peaked in each region's third week and declined steadily after. 44% of
all tickets are password resets and login problems, concentrated among
technicians who missed the classroom course and had never logged in during the
supported window. Another 19% are duplicate-record tickets: technicians create
a second customer record when they cannot find the first, because the search
screen matches on exact postcode only. The support desk merges these
duplicates manually, at about 70 records per week. Support also fields a
steady trickle of calls from the billing team about mismatched job-completion
codes, which the desk cannot resolve and forwards to the vendor. The vendor's
response on the code mismatch has been that a configurable mapping table
exists and takes roughly a day to set up, but it has not been scheduled
because no one on Kestrel's side owns the integration. Training-wise, the desk
runs a weekly one-hour webinar that reaches 20 to 30 technicians; attendance
correlates strongly with regions that missed classroom slots. The desk lead's
written comment: "We are doing the training program's job at ticket prices —
most of what we handle is people who were never onboarded, plus the two known
system gaps."
4. Finance
Licence and hosting for Fieldbook run £31,000 per month for the four live
regions. Overtime spending in the live regions is down 12% quarter over
quarter, which finance attributes mainly to the scheduling gains, and fuel
cost per job is down 9%, consistent with the travel-time reduction. Against
that, the program has carried unplanned costs: the improvised shadowing
arrangements in South and West consumed an estimated 380 technician-hours of
lost route time, and the billing team added one temporary clerk (£8,400 for
the two quarters) to handle the re-keying of job records into the invoicing
system. The program office estimates that the rollout paid back its running
cost within the second quarter in Central and East, while South and West are
roughly at break-even because of the slower adoption. Extending to North and
Islands would add £13,500 per month in licences and require winter travel for
any classroom training. On the current evidence the economics of extension are
favourable if, and only if, the adoption pace of Central and East can be
reproduced; a repeat of the West pattern would put the new regions underwater
for at least two quarters. The program office therefore leans toward
extending in Q1, provided onboarding is restructured first.
5. Data and integrations
Fieldbook's reporting module is now the source of record for job status across
the four regions, and regional managers use its dashboard in the Monday
review. Data quality inside Fieldbook is generally good, with two exceptions.
First, the customer-record duplicates created through the search limitation
(support merges about 70 per week, as noted) mean that job-history views for
affected customers are split across records until merged. Second, the missing
feed from the legacy asset register leaves the equipment-history panel empty
on every job screen, which both hides warranty state from technicians and
makes the panel's audit trail useless for dispute resolution. The invoicing
mismatch also surfaces here: because completion codes are re-keyed by hand,
the reporting module and the invoicing system disagree on completed-job counts
by 2-4% in any given week, and finance reconciles the difference manually at
month end. The vendor's integration catalogue lists supported connectors for
both the asset register's database and the invoicing system's import format;
neither connector has been evaluated. IT's position is that both are
straightforward but that no integration work was scoped into the rollout
program, which was staffed for deployment and training only.
6. Program office remarks
Individual regional reports repeat most of the above in local detail. Two
further observations. First, every negative pattern in this review was visible
by week four of the Central go-live; the program had no mechanism to convert
early field observations into program-level fixes, so the same issues
replayed in each subsequent region. Second, the vendor relationship is good:
response times are within contract, and both known system gaps have vendor
solutions waiting on Kestrel decisions rather than on vendor work.
```
## Task
Restructure this review for the COO using the Minto Pyramid Principle. The COO's
decision is whether to extend Fieldbook to North and Islands in Q1.
Return only the result, with no explanation of how you produced it.
Prompt with Minto five parts, in order · 35 890 B
Below is a skill written as an instruction. Read it in full and apply it to the task at the end.===== SKILL.md =====SKILL.md at f4813ce 17 235 B===== references/rules.md =====references/rules.md at f4813ce 6 464 B===== references/templates.md =====references/templates.md at f4813ce 3 742 B===== TASK =====the whole case inputReturn only the result, with no explanation of how you produced it.
Case input before.md
<!-- Original fixture. Company, product, regions, people and numbers are invented.
It tests the large-document behaviors: one pyramid per section plus a
top-level pyramid, MECE across sections (planted cross-section duplicates),
and geometry derived from the material rather than a fixed 3x3 habit. -->
Fixture 10: rollout review
Mode: write
Language: en
Context
Kestrel Maintenance Group services heating and cooling equipment across six
regions. Over the past two quarters it rolled out Fieldbook, a work-order and
scheduling platform, to four of the six regions. The draft below is the internal
review of that rollout, assembled by the program office from status reports. The
reader is the chief operating officer, who must decide at the quarterly meeting
whether to extend Fieldbook to the remaining two regions (North and Islands) in
Q1, and has asked to see this review beforehand.
Before
FIELDBOOK ROLLOUT — TWO-QUARTER REVIEW (DRAFT)
1. Adoption
Fieldbook is now live in four regions: Central, East, South, and West. Across
these regions 412 of 486 field technicians have activated their accounts, and
83% of work orders are now created and closed inside Fieldbook rather than on
paper or by phone. Central and East reached full adoption within six weeks.
South and West lagged: South took eleven weeks to pass 80% of orders in the
system, and West is still at 71% after nine weeks. Regional coordinators in
both slower regions report the same explanation: the two-day classroom course
was scheduled only once per region, so technicians hired after the course, or
absent during it, learned the platform second-hand from colleagues. In West,
34 of 118 technicians never attended any session. Coordinators improvised
shadowing arrangements, which worked but pulled experienced technicians off
their own routes. Uptake among dispatchers was faster: all 41 dispatchers
across the four regions were using Fieldbook scheduling from week one, since
the old scheduling spreadsheet was switched off on day one and they had no
alternative.
2. Operations
The clearest operational gain is scheduling density. With Fieldbook suggesting
route-aware assignment, completed jobs per technician per day rose from 4.6 to
5.3 across the four live regions, and average travel time between jobs fell
18%. Same-day emergency insertion, which previously required a dispatcher to
phone technicians until someone accepted, is now automatic and takes a median
of 4 minutes. However, the benefit is partly eaten by a data-transfer defect:
Fieldbook does not receive equipment service history from the legacy asset
register, so technicians open each job without knowing what was done at that
site before. They compensate by calling the back office, or by asking the
customer, and in 9% of audited jobs they replaced parts that had already been
replaced within warranty in the past year. The asset register team was asked
in month two for a nightly export; the request is still in their backlog. A
second, smaller friction: job-completion codes in Fieldbook do not match the
codes in the invoicing system, so the billing team re-keys roughly 300 job
records per week into the invoicing system by hand, introducing transcription
errors that customers catch on invoices. Operations leads in Central and East
also note that Fieldbook's parts-ordering screen is unpopular; most
technicians still order parts by phone, though this appears to be habit rather
than a defect, as the screen works correctly in testing.
3. Support and training
The support desk logged 1,940 Fieldbook tickets over the two quarters. Ticket
volume peaked in each region's third week and declined steadily after. 44% of
all tickets are password resets and login problems, concentrated among
technicians who missed the classroom course and had never logged in during the
supported window. Another 19% are duplicate-record tickets: technicians create
a second customer record when they cannot find the first, because the search
screen matches on exact postcode only. The support desk merges these
duplicates manually, at about 70 records per week. Support also fields a
steady trickle of calls from the billing team about mismatched job-completion
codes, which the desk cannot resolve and forwards to the vendor. The vendor's
response on the code mismatch has been that a configurable mapping table
exists and takes roughly a day to set up, but it has not been scheduled
because no one on Kestrel's side owns the integration. Training-wise, the desk
runs a weekly one-hour webinar that reaches 20 to 30 technicians; attendance
correlates strongly with regions that missed classroom slots. The desk lead's
written comment: "We are doing the training program's job at ticket prices —
most of what we handle is people who were never onboarded, plus the two known
system gaps."
4. Finance
Licence and hosting for Fieldbook run £31,000 per month for the four live
regions. Overtime spending in the live regions is down 12% quarter over
quarter, which finance attributes mainly to the scheduling gains, and fuel
cost per job is down 9%, consistent with the travel-time reduction. Against
that, the program has carried unplanned costs: the improvised shadowing
arrangements in South and West consumed an estimated 380 technician-hours of
lost route time, and the billing team added one temporary clerk (£8,400 for
the two quarters) to handle the re-keying of job records into the invoicing
system. The program office estimates that the rollout paid back its running
cost within the second quarter in Central and East, while South and West are
roughly at break-even because of the slower adoption. Extending to North and
Islands would add £13,500 per month in licences and require winter travel for
any classroom training. On the current evidence the economics of extension are
favourable if, and only if, the adoption pace of Central and East can be
reproduced; a repeat of the West pattern would put the new regions underwater
for at least two quarters. The program office therefore leans toward
extending in Q1, provided onboarding is restructured first.
5. Data and integrations
Fieldbook's reporting module is now the source of record for job status across
the four regions, and regional managers use its dashboard in the Monday
review. Data quality inside Fieldbook is generally good, with two exceptions.
First, the customer-record duplicates created through the search limitation
(support merges about 70 per week, as noted) mean that job-history views for
affected customers are split across records until merged. Second, the missing
feed from the legacy asset register leaves the equipment-history panel empty
on every job screen, which both hides warranty state from technicians and
makes the panel's audit trail useless for dispute resolution. The invoicing
mismatch also surfaces here: because completion codes are re-keyed by hand,
the reporting module and the invoicing system disagree on completed-job counts
by 2-4% in any given week, and finance reconciles the difference manually at
month end. The vendor's integration catalogue lists supported connectors for
both the asset register's database and the invoicing system's import format;
neither connector has been evaluated. IT's position is that both are
straightforward but that no integration work was scoped into the rollout
program, which was staffed for deployment and training only.
6. Program office remarks
Individual regional reports repeat most of the above in local detail. Two
further observations. First, every negative pattern in this review was visible
by week four of the Central go-live; the program had no mechanism to convert
early field observations into program-level fixes, so the same issues
replayed in each subsequent region. Second, the vendor relationship is good:
response times are within contract, and both known system gaps have vendor
solutions waiting on Kestrel decisions rather than on vendor work.
Task
Restructure this review for the COO using the Minto Pyramid Principle. The COO's
decision is whether to extend Fieldbook to North and Islands in Q1.
The judge’s gold gold.md
Gold 10: rollout review
Original gold for an invented scenario.
Expected structure
Top: answers the COO's actual question — extend to North and Islands in Q1,
conditional on fixing what made South and West slow and expensive. A defensible
top must carry both the recommendation and its condition; "the rollout went
well" is a topic, and an unconditional "extend" ignores the document's own
economics (a repeat of the West pattern puts the new regions underwater).
The material honestly supports four first-level branches, not the habitual
three — for example:
- The economics already argue for extension — productivity up (4.6 → 5.3
jobs/day, travel −18%, overtime −12%, fuel −9%), payback reached in the two
regions that onboarded fast. - Onboarding is the reproducible failure — single classroom slot per
region left late hires unserved (34 of 118 in West never attended), drove
44% of support tickets, forced shadowing that cost 380 technician-hours,
and is the one variable separating payback regions from break-even regions. - Two integration defects tax every region and have waiting fixes — the
asset-register feed (blind technicians, 9% redundant warranty parts, empty
audit panel) and the completion-code mismatch (300 records/week re-keyed, a
temporary clerk, 2–4% reporting discrepancy); vendor connectors and the
mapping table exist, and stall only because nobody owns integration. - The program lacked a feedback loop — every defect was visible by week
four of Central and replayed in each region; extension without that loop
repeats the tax in two more regions.
A grouping that reaches the same content by different cuts is acceptable; what
is not acceptable is a branch per source section (adoption / operations /
support / finance / data), which is the document's order, not an answer.
Section pyramids
The rewrite must give each substantive section its own small pyramid — a
one-sentence section top that its details support — plus the top-level pyramid
whose key line the section tops visibly feed. A single flat pyramid over a
document of this size, with details hanging directly off four branches and the
sections dissolved, loses the layer the reader navigates by. Depth should
follow the material: the integration branch genuinely carries two distinct
defects, each with its own evidence, so it earns a third level; thin branches
must not be padded to match.
Cross-section duplicates (the MECE trap)
The same three facts each appear in two or three sections and must appear
once in the structure, with their scattered evidence gathered:
- The completion-code mismatch: Operations (300 records/week re-keyed), Support
(forwarded tickets, vendor's mapping-table answer), Finance (£8,400 clerk),
Data (2–4% reconciliation gap). One defect, four symptoms. - The missing asset-register feed: Operations (blind jobs, 9% redundant parts)
and Data (empty history panel, useless audit trail). One defect. - The training shortfall: Adoption (late hires unserved, shadowing), Support
(44% of tickets, webinar attendance pattern, the desk lead's comment),
Finance (380 lost hours). One cause.
An output that keeps any of these as two separate first-level or second-level
points in different branches has failed the cross-section MECE pass.
Facts and proportions
All load-bearing numbers come from the source; invented benchmarks, dates or
vendor quotes are hard failures. The parts-ordering habit, dispatcher uptake,
and vendor-relationship remarks are legitimately minor: demoting them to a
short residual note or explicitly dropping them is proportional omission.
The output is a restructured review, so it will be long — length itself is not
a defect; a section-by-section retelling under new headings is.
Judge gpt-5.6-sol, effort high · Claude Haiku 4.5, effort low · skill from commit f4813ce · run v1.7.0-final