qwen3.6-35b-a3b on “A review of a large rollout”: better with Minto on both measures
Structure 5 → 6 of 8 +1 Quality 7 → 8 of 10 +1
The judge on the output with Minto
The three-branch memo consolidates the duplicated defects well, but loses the program feedback mechanism and overstates dispatcher evidence.
| Structure | Without Minto 5 | With Minto 6 | Change |
|---|---|---|---|
| Top | 2 | 2 | |
| Key-line composition | 1 | 1 | |
| Levels | 1 | 2 | +1 |
| Order and kind | 1 | 1 |
| Quality | Without Minto 7 | With Minto 8 | Change |
|---|---|---|---|
| Top | 2 | 2 | |
| Same-kind grouping | 1 | 2 | +1 |
| Explainable order | 1 | 1 | |
| MECE | 1 | 1 | |
| Visible, proportionate structure | 2 | 2 |
Outputs and the judge’s words are quoted as written, in English.
Without Minto
control prompt · blind labelout-01invented_factsunacknowledged_source_loss- quality after the rubric’s penalty 6/10
The recommendation is clean, but the missing feedback-loop branch and compressed technical layer leave the key line incomplete.
RECOMMENDATION
Extend Fieldbook to North and Islands in Q1, provided onboarding is restructured to guarantee technician coverage and data integrations are resolved prior to launch.
1. The extension is financially sound, supported by proven cost savings and productivity gains.
- 1.1 Operational efficiency has improved significantly across live regions.
- Completed jobs per technician rose from 4.6 to 5.3, and average travel time between jobs fell 18%.
- Same-day emergency insertion is now automatic with a median response time of 4 minutes.
- Dispatcher adoption reached 100% immediately due to the removal of the legacy scheduling spreadsheet.
- 1.2 Return on investment validates the economics of expansion.
- Overtime spending decreased by 12% quarter-over-quarter, and fuel cost per job dropped 9%.
- Central and East regions paid back running costs within the second quarter; South and West are at break-even.
- Extension adds £13,500 per month in licence costs, but the economics remain favourable if the adoption pace of Central and East is replicated.
2. Profitability risks from slow adoption can be eliminated by reforming the training program.
- 2.1 Adoption delays in South and West stemmed from insufficient training access.
- Classroom courses were scheduled only once per region, excluding technicians hired after the session or absent during it.
- Reliance on improvised shadowing consumed an estimated 380 technician-hours of lost route time.
- 44% of support tickets were password resets linked to technicians who missed the classroom window.
- 2.2 Full onboarding coverage is essential to replicate Central and East performance.
- A repeat of the West adoption pattern would render the extension unprofitable for at least two quarters.
- Restructuring onboarding to ensure every technician attends a session will prevent delays and unplanned costs.
- Weekly webinars provide supplementary support but cannot replace mandatory initial training for new hires.
3. Technical defects impacting operations are resolvable through vendor-supported integrations requiring only internal action.
- 3.1 Critical data gaps cause rework but have vendor-ready fixes.
- Missing asset register feed leaves technicians without service history, leading to unnecessary part replacements in 9% of audited jobs.
- Mismatched job-completion codes force the billing team to manually re-key 300 records weekly, introducing transcription errors.
- Search limitations create duplicate customer records, splitting job history and requiring manual merging.
- The vendor maintains connectors for both the asset register and invoicing system, plus a configurable mapping table for codes; all solutions are available and require approximately one day to configure.
- 3.2 Implementation requires assigning ownership and scoping integration tasks.
- IT confirms both integrations are straightforward, but no work was scoped during the rollout phase.
- Assigning internal owners to schedule the integration work and patch the search function will eliminate defects before Q1.
- The vendor relationship is strong, with response times within contract and solutions waiting on Kestrel decisions rather than vendor development.
With Minto
prompt with the skill · blind labelout-13invented_factsunacknowledged_source_loss- quality after the rubric’s penalty 7/10
The three-branch memo consolidates the duplicated defects well, but loses the program feedback mechanism and overstates dispatcher evidence.
Subject: Recommendation: Extend Fieldbook to North and Islands in Q1
Context
Fieldbook is live in four regions, delivering strong operational gains and financial payback in Central and East. However, rollout performance in South and West revealed adoption delays driven by training bottlenecks, while persistent integration gaps continue to generate manual work and data errors. The vendor has solutions ready for these gaps, but they require Kestrel action.
Recommendation
Extend Fieldbook to North and Islands in Q1, contingent on restructuring onboarding to ensure rapid adoption and scoping integration fixes to protect margins.
Justification
- Extension delivers strong economic returns when adoption pace matches high-performing regions.
- Operational gains are proven: jobs per technician rose from 4.6 to 5.3, travel time fell 18%, and emergency insertion is now automatic (4-minute median).
- Financial benefits are clear: overtime spending dropped 12% and fuel cost per job fell 9%.
- Payback is confirmed in Central and East, where the rollout covered its running costs within the second quarter. South and West are at break-even despite slower adoption.
- Extension economics remain favourable if adoption replicates the Central/East pattern, even accounting for £13,500/month in licence costs and winter travel expenses for classroom training.
- Adoption delays in South and West resulted from training capacity constraints, not platform limitations.
- Root cause was structural: the two-day classroom course ran only once per region, leaving late hires and absentees to learn second-hand. In West, 34 of 118 technicians never attended a session.
- Remediation proved effective but costly: improvised shadowing improved uptake but consumed 380 technician-hours of lost route time, driving unplanned costs.
- Platform usability is validated by dispatcher adoption: all 41 dispatchers used Fieldbook scheduling from week one, as the legacy spreadsheet was disabled immediately, demonstrating that forced transition eliminates resistance.
- Password-reset tickets (44% of support volume) correlate directly with missed training sessions, confirming that onboarding gaps drive support load.
- Critical system gaps require immediate action to prevent margin erosion and data integrity issues.
- Asset register integration: Missing service history forces technicians to call the back office or replace parts unnecessarily; audited jobs showed a 9% error rate in warranty replacements. The asset register team has the request in backlog.
- Invoicing code mismatch: Job-completion codes do not match the billing system, requiring the finance team to re-key ~300 records weekly, introducing transcription errors and manual reconciliation.
- Customer record duplicates: Exact-postcode search creates duplicate records, splitting job history and generating 70 merge requests per week in support.
- Vendor readiness: Both integration gaps have vendor solutions (mapping table, database connectors) awaiting Kestrel decisions; response times are within contract.
- IT scoping requirement: IT confirms both integrations are straightforward, but neither was scoped into the rollout programme. Scoping must occur now to avoid delaying extension or repeating manual-work burdens in new regions.
Required Actions
- Restructure onboarding for North and Islands to allow multiple classroom sessions and eliminate reliance on shadowing.
- Authorise IT to scope and implement the asset register export and invoicing code mapping immediately.
- Confirm winter travel logistics for classroom training in the northern regions.
How the test was built
qwen3.6-35b-a3b received the same task twice. The control prompt is one framing line and the task. The prompt with Minto is the same task preceded by the full text of the skill. The judge read both outputs among 14 under blind labels and did not know which one had the skill.
Prompt without Minto 8 317 B
You are working on a business document using the Minto Pyramid Principle.
<!-- Original fixture. Company, product, regions, people and numbers are invented.
It tests the large-document behaviors: one pyramid per section plus a
top-level pyramid, MECE across sections (planted cross-section duplicates),
and geometry derived from the material rather than a fixed 3x3 habit. -->
# Fixture 10: rollout review
**Mode:** `write`
**Language:** `en`
## Context
Kestrel Maintenance Group services heating and cooling equipment across six
regions. Over the past two quarters it rolled out Fieldbook, a work-order and
scheduling platform, to four of the six regions. The draft below is the internal
review of that rollout, assembled by the program office from status reports. The
reader is the chief operating officer, who must decide at the quarterly meeting
whether to extend Fieldbook to the remaining two regions (North and Islands) in
Q1, and has asked to see this review beforehand.
## Before
```text
FIELDBOOK ROLLOUT — TWO-QUARTER REVIEW (DRAFT)
1. Adoption
Fieldbook is now live in four regions: Central, East, South, and West. Across
these regions 412 of 486 field technicians have activated their accounts, and
83% of work orders are now created and closed inside Fieldbook rather than on
paper or by phone. Central and East reached full adoption within six weeks.
South and West lagged: South took eleven weeks to pass 80% of orders in the
system, and West is still at 71% after nine weeks. Regional coordinators in
both slower regions report the same explanation: the two-day classroom course
was scheduled only once per region, so technicians hired after the course, or
absent during it, learned the platform second-hand from colleagues. In West,
34 of 118 technicians never attended any session. Coordinators improvised
shadowing arrangements, which worked but pulled experienced technicians off
their own routes. Uptake among dispatchers was faster: all 41 dispatchers
across the four regions were using Fieldbook scheduling from week one, since
the old scheduling spreadsheet was switched off on day one and they had no
alternative.
2. Operations
The clearest operational gain is scheduling density. With Fieldbook suggesting
route-aware assignment, completed jobs per technician per day rose from 4.6 to
5.3 across the four live regions, and average travel time between jobs fell
18%. Same-day emergency insertion, which previously required a dispatcher to
phone technicians until someone accepted, is now automatic and takes a median
of 4 minutes. However, the benefit is partly eaten by a data-transfer defect:
Fieldbook does not receive equipment service history from the legacy asset
register, so technicians open each job without knowing what was done at that
site before. They compensate by calling the back office, or by asking the
customer, and in 9% of audited jobs they replaced parts that had already been
replaced within warranty in the past year. The asset register team was asked
in month two for a nightly export; the request is still in their backlog. A
second, smaller friction: job-completion codes in Fieldbook do not match the
codes in the invoicing system, so the billing team re-keys roughly 300 job
records per week into the invoicing system by hand, introducing transcription
errors that customers catch on invoices. Operations leads in Central and East
also note that Fieldbook's parts-ordering screen is unpopular; most
technicians still order parts by phone, though this appears to be habit rather
than a defect, as the screen works correctly in testing.
3. Support and training
The support desk logged 1,940 Fieldbook tickets over the two quarters. Ticket
volume peaked in each region's third week and declined steadily after. 44% of
all tickets are password resets and login problems, concentrated among
technicians who missed the classroom course and had never logged in during the
supported window. Another 19% are duplicate-record tickets: technicians create
a second customer record when they cannot find the first, because the search
screen matches on exact postcode only. The support desk merges these
duplicates manually, at about 70 records per week. Support also fields a
steady trickle of calls from the billing team about mismatched job-completion
codes, which the desk cannot resolve and forwards to the vendor. The vendor's
response on the code mismatch has been that a configurable mapping table
exists and takes roughly a day to set up, but it has not been scheduled
because no one on Kestrel's side owns the integration. Training-wise, the desk
runs a weekly one-hour webinar that reaches 20 to 30 technicians; attendance
correlates strongly with regions that missed classroom slots. The desk lead's
written comment: "We are doing the training program's job at ticket prices —
most of what we handle is people who were never onboarded, plus the two known
system gaps."
4. Finance
Licence and hosting for Fieldbook run £31,000 per month for the four live
regions. Overtime spending in the live regions is down 12% quarter over
quarter, which finance attributes mainly to the scheduling gains, and fuel
cost per job is down 9%, consistent with the travel-time reduction. Against
that, the program has carried unplanned costs: the improvised shadowing
arrangements in South and West consumed an estimated 380 technician-hours of
lost route time, and the billing team added one temporary clerk (£8,400 for
the two quarters) to handle the re-keying of job records into the invoicing
system. The program office estimates that the rollout paid back its running
cost within the second quarter in Central and East, while South and West are
roughly at break-even because of the slower adoption. Extending to North and
Islands would add £13,500 per month in licences and require winter travel for
any classroom training. On the current evidence the economics of extension are
favourable if, and only if, the adoption pace of Central and East can be
reproduced; a repeat of the West pattern would put the new regions underwater
for at least two quarters. The program office therefore leans toward
extending in Q1, provided onboarding is restructured first.
5. Data and integrations
Fieldbook's reporting module is now the source of record for job status across
the four regions, and regional managers use its dashboard in the Monday
review. Data quality inside Fieldbook is generally good, with two exceptions.
First, the customer-record duplicates created through the search limitation
(support merges about 70 per week, as noted) mean that job-history views for
affected customers are split across records until merged. Second, the missing
feed from the legacy asset register leaves the equipment-history panel empty
on every job screen, which both hides warranty state from technicians and
makes the panel's audit trail useless for dispute resolution. The invoicing
mismatch also surfaces here: because completion codes are re-keyed by hand,
the reporting module and the invoicing system disagree on completed-job counts
by 2-4% in any given week, and finance reconciles the difference manually at
month end. The vendor's integration catalogue lists supported connectors for
both the asset register's database and the invoicing system's import format;
neither connector has been evaluated. IT's position is that both are
straightforward but that no integration work was scoped into the rollout
program, which was staffed for deployment and training only.
6. Program office remarks
Individual regional reports repeat most of the above in local detail. Two
further observations. First, every negative pattern in this review was visible
by week four of the Central go-live; the program had no mechanism to convert
early field observations into program-level fixes, so the same issues
replayed in each subsequent region. Second, the vendor relationship is good:
response times are within contract, and both known system gaps have vendor
solutions waiting on Kestrel decisions rather than on vendor work.
```
## Task
Restructure this review for the COO using the Minto Pyramid Principle. The COO's
decision is whether to extend Fieldbook to North and Islands in Q1.
Return only the result, with no explanation of how you produced it.
Prompt with Minto five parts, in order · 35 890 B
Below is a skill written as an instruction. Read it in full and apply it to the task at the end.===== SKILL.md =====SKILL.md at f4813ce 17 235 B===== references/rules.md =====references/rules.md at f4813ce 6 464 B===== references/templates.md =====references/templates.md at f4813ce 3 742 B===== TASK =====the whole case inputReturn only the result, with no explanation of how you produced it.
Case input before.md
<!-- Original fixture. Company, product, regions, people and numbers are invented.
It tests the large-document behaviors: one pyramid per section plus a
top-level pyramid, MECE across sections (planted cross-section duplicates),
and geometry derived from the material rather than a fixed 3x3 habit. -->
Fixture 10: rollout review
Mode: write
Language: en
Context
Kestrel Maintenance Group services heating and cooling equipment across six
regions. Over the past two quarters it rolled out Fieldbook, a work-order and
scheduling platform, to four of the six regions. The draft below is the internal
review of that rollout, assembled by the program office from status reports. The
reader is the chief operating officer, who must decide at the quarterly meeting
whether to extend Fieldbook to the remaining two regions (North and Islands) in
Q1, and has asked to see this review beforehand.
Before
FIELDBOOK ROLLOUT — TWO-QUARTER REVIEW (DRAFT)
1. Adoption
Fieldbook is now live in four regions: Central, East, South, and West. Across
these regions 412 of 486 field technicians have activated their accounts, and
83% of work orders are now created and closed inside Fieldbook rather than on
paper or by phone. Central and East reached full adoption within six weeks.
South and West lagged: South took eleven weeks to pass 80% of orders in the
system, and West is still at 71% after nine weeks. Regional coordinators in
both slower regions report the same explanation: the two-day classroom course
was scheduled only once per region, so technicians hired after the course, or
absent during it, learned the platform second-hand from colleagues. In West,
34 of 118 technicians never attended any session. Coordinators improvised
shadowing arrangements, which worked but pulled experienced technicians off
their own routes. Uptake among dispatchers was faster: all 41 dispatchers
across the four regions were using Fieldbook scheduling from week one, since
the old scheduling spreadsheet was switched off on day one and they had no
alternative.
2. Operations
The clearest operational gain is scheduling density. With Fieldbook suggesting
route-aware assignment, completed jobs per technician per day rose from 4.6 to
5.3 across the four live regions, and average travel time between jobs fell
18%. Same-day emergency insertion, which previously required a dispatcher to
phone technicians until someone accepted, is now automatic and takes a median
of 4 minutes. However, the benefit is partly eaten by a data-transfer defect:
Fieldbook does not receive equipment service history from the legacy asset
register, so technicians open each job without knowing what was done at that
site before. They compensate by calling the back office, or by asking the
customer, and in 9% of audited jobs they replaced parts that had already been
replaced within warranty in the past year. The asset register team was asked
in month two for a nightly export; the request is still in their backlog. A
second, smaller friction: job-completion codes in Fieldbook do not match the
codes in the invoicing system, so the billing team re-keys roughly 300 job
records per week into the invoicing system by hand, introducing transcription
errors that customers catch on invoices. Operations leads in Central and East
also note that Fieldbook's parts-ordering screen is unpopular; most
technicians still order parts by phone, though this appears to be habit rather
than a defect, as the screen works correctly in testing.
3. Support and training
The support desk logged 1,940 Fieldbook tickets over the two quarters. Ticket
volume peaked in each region's third week and declined steadily after. 44% of
all tickets are password resets and login problems, concentrated among
technicians who missed the classroom course and had never logged in during the
supported window. Another 19% are duplicate-record tickets: technicians create
a second customer record when they cannot find the first, because the search
screen matches on exact postcode only. The support desk merges these
duplicates manually, at about 70 records per week. Support also fields a
steady trickle of calls from the billing team about mismatched job-completion
codes, which the desk cannot resolve and forwards to the vendor. The vendor's
response on the code mismatch has been that a configurable mapping table
exists and takes roughly a day to set up, but it has not been scheduled
because no one on Kestrel's side owns the integration. Training-wise, the desk
runs a weekly one-hour webinar that reaches 20 to 30 technicians; attendance
correlates strongly with regions that missed classroom slots. The desk lead's
written comment: "We are doing the training program's job at ticket prices —
most of what we handle is people who were never onboarded, plus the two known
system gaps."
4. Finance
Licence and hosting for Fieldbook run £31,000 per month for the four live
regions. Overtime spending in the live regions is down 12% quarter over
quarter, which finance attributes mainly to the scheduling gains, and fuel
cost per job is down 9%, consistent with the travel-time reduction. Against
that, the program has carried unplanned costs: the improvised shadowing
arrangements in South and West consumed an estimated 380 technician-hours of
lost route time, and the billing team added one temporary clerk (£8,400 for
the two quarters) to handle the re-keying of job records into the invoicing
system. The program office estimates that the rollout paid back its running
cost within the second quarter in Central and East, while South and West are
roughly at break-even because of the slower adoption. Extending to North and
Islands would add £13,500 per month in licences and require winter travel for
any classroom training. On the current evidence the economics of extension are
favourable if, and only if, the adoption pace of Central and East can be
reproduced; a repeat of the West pattern would put the new regions underwater
for at least two quarters. The program office therefore leans toward
extending in Q1, provided onboarding is restructured first.
5. Data and integrations
Fieldbook's reporting module is now the source of record for job status across
the four regions, and regional managers use its dashboard in the Monday
review. Data quality inside Fieldbook is generally good, with two exceptions.
First, the customer-record duplicates created through the search limitation
(support merges about 70 per week, as noted) mean that job-history views for
affected customers are split across records until merged. Second, the missing
feed from the legacy asset register leaves the equipment-history panel empty
on every job screen, which both hides warranty state from technicians and
makes the panel's audit trail useless for dispute resolution. The invoicing
mismatch also surfaces here: because completion codes are re-keyed by hand,
the reporting module and the invoicing system disagree on completed-job counts
by 2-4% in any given week, and finance reconciles the difference manually at
month end. The vendor's integration catalogue lists supported connectors for
both the asset register's database and the invoicing system's import format;
neither connector has been evaluated. IT's position is that both are
straightforward but that no integration work was scoped into the rollout
program, which was staffed for deployment and training only.
6. Program office remarks
Individual regional reports repeat most of the above in local detail. Two
further observations. First, every negative pattern in this review was visible
by week four of the Central go-live; the program had no mechanism to convert
early field observations into program-level fixes, so the same issues
replayed in each subsequent region. Second, the vendor relationship is good:
response times are within contract, and both known system gaps have vendor
solutions waiting on Kestrel decisions rather than on vendor work.
Task
Restructure this review for the COO using the Minto Pyramid Principle. The COO's
decision is whether to extend Fieldbook to North and Islands in Q1.
The judge’s gold gold.md
Gold 10: rollout review
Original gold for an invented scenario.
Expected structure
Top: answers the COO's actual question — extend to North and Islands in Q1,
conditional on fixing what made South and West slow and expensive. A defensible
top must carry both the recommendation and its condition; "the rollout went
well" is a topic, and an unconditional "extend" ignores the document's own
economics (a repeat of the West pattern puts the new regions underwater).
The material honestly supports four first-level branches, not the habitual
three — for example:
- The economics already argue for extension — productivity up (4.6 → 5.3
jobs/day, travel −18%, overtime −12%, fuel −9%), payback reached in the two
regions that onboarded fast. - Onboarding is the reproducible failure — single classroom slot per
region left late hires unserved (34 of 118 in West never attended), drove
44% of support tickets, forced shadowing that cost 380 technician-hours,
and is the one variable separating payback regions from break-even regions. - Two integration defects tax every region and have waiting fixes — the
asset-register feed (blind technicians, 9% redundant warranty parts, empty
audit panel) and the completion-code mismatch (300 records/week re-keyed, a
temporary clerk, 2–4% reporting discrepancy); vendor connectors and the
mapping table exist, and stall only because nobody owns integration. - The program lacked a feedback loop — every defect was visible by week
four of Central and replayed in each region; extension without that loop
repeats the tax in two more regions.
A grouping that reaches the same content by different cuts is acceptable; what
is not acceptable is a branch per source section (adoption / operations /
support / finance / data), which is the document's order, not an answer.
Section pyramids
The rewrite must give each substantive section its own small pyramid — a
one-sentence section top that its details support — plus the top-level pyramid
whose key line the section tops visibly feed. A single flat pyramid over a
document of this size, with details hanging directly off four branches and the
sections dissolved, loses the layer the reader navigates by. Depth should
follow the material: the integration branch genuinely carries two distinct
defects, each with its own evidence, so it earns a third level; thin branches
must not be padded to match.
Cross-section duplicates (the MECE trap)
The same three facts each appear in two or three sections and must appear
once in the structure, with their scattered evidence gathered:
- The completion-code mismatch: Operations (300 records/week re-keyed), Support
(forwarded tickets, vendor's mapping-table answer), Finance (£8,400 clerk),
Data (2–4% reconciliation gap). One defect, four symptoms. - The missing asset-register feed: Operations (blind jobs, 9% redundant parts)
and Data (empty history panel, useless audit trail). One defect. - The training shortfall: Adoption (late hires unserved, shadowing), Support
(44% of tickets, webinar attendance pattern, the desk lead's comment),
Finance (380 lost hours). One cause.
An output that keeps any of these as two separate first-level or second-level
points in different branches has failed the cross-section MECE pass.
Facts and proportions
All load-bearing numbers come from the source; invented benchmarks, dates or
vendor quotes are hard failures. The parts-ordering habit, dispatcher uptake,
and vendor-relationship remarks are legitimately minor: demoting them to a
short residual note or explicitly dropping them is proportional omission.
The output is a restructured review, so it will be long — length itself is not
a defect; a section-by-section retelling under new headings is.
Judge gpt-5.6-sol, effort high · skill from commit f4813ce · run v1.7.0-final