qwen3.6-35b-a3b на кейсе «Разбор большого запуска»: с Minto лучше по обеим мерам

Структура 5 → 6 из 8 +1 Качество 7 → 8 из 10 +1

Судья об ответе с Minto

The three-branch memo consolidates the duplicated defects well, but loses the program feedback mechanism and overstates dispatcher evidence.

Режим write · кейс 10-rollout-review · прогон v1.7.0-final

Структура из 8, по 2 балла на ось
СтруктураБез Minto 5С Minto 6Изменение
Вершина22
Состав ключевой линии11
Уровни12+1
Порядок и род11
Качество из 10, по 2 балла на ось
КачествоБез Minto 7С Minto 8Изменение
Вершина22
Однородные группы12+1
Объяснимый порядок11
MECE11
Видимая соразмерная структура22

Ответы моделей и слова судьи приведены в оригинале, по-английски.

Без Minto

контрольный промпт · слепая метка out-01
  • invented_facts
  • unacknowledged_source_loss
  • качество со штрафом рубрики 6/10

The recommendation is clean, but the missing feedback-loop branch and compressed technical layer leave the key line incomplete.

RECOMMENDATION

Extend Fieldbook to North and Islands in Q1, provided onboarding is restructured to guarantee technician coverage and data integrations are resolved prior to launch.

1. The extension is financially sound, supported by proven cost savings and productivity gains.

  • 1.1 Operational efficiency has improved significantly across live regions.
    • Completed jobs per technician rose from 4.6 to 5.3, and average travel time between jobs fell 18%.
    • Same-day emergency insertion is now automatic with a median response time of 4 minutes.
    • Dispatcher adoption reached 100% immediately due to the removal of the legacy scheduling spreadsheet.
  • 1.2 Return on investment validates the economics of expansion.
    • Overtime spending decreased by 12% quarter-over-quarter, and fuel cost per job dropped 9%.
    • Central and East regions paid back running costs within the second quarter; South and West are at break-even.
    • Extension adds £13,500 per month in licence costs, but the economics remain favourable if the adoption pace of Central and East is replicated.

2. Profitability risks from slow adoption can be eliminated by reforming the training program.

  • 2.1 Adoption delays in South and West stemmed from insufficient training access.
    • Classroom courses were scheduled only once per region, excluding technicians hired after the session or absent during it.
    • Reliance on improvised shadowing consumed an estimated 380 technician-hours of lost route time.
    • 44% of support tickets were password resets linked to technicians who missed the classroom window.
  • 2.2 Full onboarding coverage is essential to replicate Central and East performance.
    • A repeat of the West adoption pattern would render the extension unprofitable for at least two quarters.
    • Restructuring onboarding to ensure every technician attends a session will prevent delays and unplanned costs.
    • Weekly webinars provide supplementary support but cannot replace mandatory initial training for new hires.

3. Technical defects impacting operations are resolvable through vendor-supported integrations requiring only internal action.

  • 3.1 Critical data gaps cause rework but have vendor-ready fixes.
    • Missing asset register feed leaves technicians without service history, leading to unnecessary part replacements in 9% of audited jobs.
    • Mismatched job-completion codes force the billing team to manually re-key 300 records weekly, introducing transcription errors.
    • Search limitations create duplicate customer records, splitting job history and requiring manual merging.
    • The vendor maintains connectors for both the asset register and invoicing system, plus a configurable mapping table for codes; all solutions are available and require approximately one day to configure.
  • 3.2 Implementation requires assigning ownership and scoping integration tasks.
    • IT confirms both integrations are straightforward, but no work was scoped during the rollout phase.
    • Assigning internal owners to schedule the integration work and patch the search function will eliminate defects before Q1.
    • The vendor relationship is strong, with response times within contract and solutions waiting on Kestrel decisions rather than vendor development.

С Minto

промпт с навыком · слепая метка out-13
  • invented_facts
  • unacknowledged_source_loss
  • качество со штрафом рубрики 7/10

The three-branch memo consolidates the duplicated defects well, but loses the program feedback mechanism and overstates dispatcher evidence.

Subject: Recommendation: Extend Fieldbook to North and Islands in Q1

Context
Fieldbook is live in four regions, delivering strong operational gains and financial payback in Central and East. However, rollout performance in South and West revealed adoption delays driven by training bottlenecks, while persistent integration gaps continue to generate manual work and data errors. The vendor has solutions ready for these gaps, but they require Kestrel action.

Recommendation
Extend Fieldbook to North and Islands in Q1, contingent on restructuring onboarding to ensure rapid adoption and scoping integration fixes to protect margins.

Justification

  1. Extension delivers strong economic returns when adoption pace matches high-performing regions.
    • Operational gains are proven: jobs per technician rose from 4.6 to 5.3, travel time fell 18%, and emergency insertion is now automatic (4-minute median).
    • Financial benefits are clear: overtime spending dropped 12% and fuel cost per job fell 9%.
    • Payback is confirmed in Central and East, where the rollout covered its running costs within the second quarter. South and West are at break-even despite slower adoption.
    • Extension economics remain favourable if adoption replicates the Central/East pattern, even accounting for £13,500/month in licence costs and winter travel expenses for classroom training.
  2. Adoption delays in South and West resulted from training capacity constraints, not platform limitations.
    • Root cause was structural: the two-day classroom course ran only once per region, leaving late hires and absentees to learn second-hand. In West, 34 of 118 technicians never attended a session.
    • Remediation proved effective but costly: improvised shadowing improved uptake but consumed 380 technician-hours of lost route time, driving unplanned costs.
    • Platform usability is validated by dispatcher adoption: all 41 dispatchers used Fieldbook scheduling from week one, as the legacy spreadsheet was disabled immediately, demonstrating that forced transition eliminates resistance.
    • Password-reset tickets (44% of support volume) correlate directly with missed training sessions, confirming that onboarding gaps drive support load.
  3. Critical system gaps require immediate action to prevent margin erosion and data integrity issues.
    • Asset register integration: Missing service history forces technicians to call the back office or replace parts unnecessarily; audited jobs showed a 9% error rate in warranty replacements. The asset register team has the request in backlog.
    • Invoicing code mismatch: Job-completion codes do not match the billing system, requiring the finance team to re-key ~300 records weekly, introducing transcription errors and manual reconciliation.
    • Customer record duplicates: Exact-postcode search creates duplicate records, splitting job history and generating 70 merge requests per week in support.
    • Vendor readiness: Both integration gaps have vendor solutions (mapping table, database connectors) awaiting Kestrel decisions; response times are within contract.
    • IT scoping requirement: IT confirms both integrations are straightforward, but neither was scoped into the rollout programme. Scoping must occur now to avoid delaying extension or repeating manual-work burdens in new regions.

Required Actions

  • Restructure onboarding for North and Islands to allow multiple classroom sessions and eliminate reliance on shadowing.
  • Authorise IT to scope and implement the asset register export and invoicing code mapping immediately.
  • Confirm winter travel logistics for classroom training in the northern regions.

Как устроен тест

qwen3.6-35b-a3b получила одну и ту же задачу дважды. Контрольный промпт — одна вводная строка и задача. Промпт с Minto — та же задача, перед которой стоит полный текст навыка. Судья читал оба ответа среди 14 ответов под слепыми метками и не знал, в каком из них был навык.

Промпт без Minto 8 317 Б
You are working on a business document using the Minto Pyramid Principle.

<!-- Original fixture. Company, product, regions, people and numbers are invented.
     It tests the large-document behaviors: one pyramid per section plus a
     top-level pyramid, MECE across sections (planted cross-section duplicates),
     and geometry derived from the material rather than a fixed 3x3 habit. -->

# Fixture 10: rollout review

**Mode:** `write`
**Language:** `en`

## Context

Kestrel Maintenance Group services heating and cooling equipment across six
regions. Over the past two quarters it rolled out Fieldbook, a work-order and
scheduling platform, to four of the six regions. The draft below is the internal
review of that rollout, assembled by the program office from status reports. The
reader is the chief operating officer, who must decide at the quarterly meeting
whether to extend Fieldbook to the remaining two regions (North and Islands) in
Q1, and has asked to see this review beforehand.

## Before

```text
FIELDBOOK ROLLOUT — TWO-QUARTER REVIEW (DRAFT)

1. Adoption

Fieldbook is now live in four regions: Central, East, South, and West. Across
these regions 412 of 486 field technicians have activated their accounts, and
83% of work orders are now created and closed inside Fieldbook rather than on
paper or by phone. Central and East reached full adoption within six weeks.
South and West lagged: South took eleven weeks to pass 80% of orders in the
system, and West is still at 71% after nine weeks. Regional coordinators in
both slower regions report the same explanation: the two-day classroom course
was scheduled only once per region, so technicians hired after the course, or
absent during it, learned the platform second-hand from colleagues. In West,
34 of 118 technicians never attended any session. Coordinators improvised
shadowing arrangements, which worked but pulled experienced technicians off
their own routes. Uptake among dispatchers was faster: all 41 dispatchers
across the four regions were using Fieldbook scheduling from week one, since
the old scheduling spreadsheet was switched off on day one and they had no
alternative.

2. Operations

The clearest operational gain is scheduling density. With Fieldbook suggesting
route-aware assignment, completed jobs per technician per day rose from 4.6 to
5.3 across the four live regions, and average travel time between jobs fell
18%. Same-day emergency insertion, which previously required a dispatcher to
phone technicians until someone accepted, is now automatic and takes a median
of 4 minutes. However, the benefit is partly eaten by a data-transfer defect:
Fieldbook does not receive equipment service history from the legacy asset
register, so technicians open each job without knowing what was done at that
site before. They compensate by calling the back office, or by asking the
customer, and in 9% of audited jobs they replaced parts that had already been
replaced within warranty in the past year. The asset register team was asked
in month two for a nightly export; the request is still in their backlog. A
second, smaller friction: job-completion codes in Fieldbook do not match the
codes in the invoicing system, so the billing team re-keys roughly 300 job
records per week into the invoicing system by hand, introducing transcription
errors that customers catch on invoices. Operations leads in Central and East
also note that Fieldbook's parts-ordering screen is unpopular; most
technicians still order parts by phone, though this appears to be habit rather
than a defect, as the screen works correctly in testing.

3. Support and training

The support desk logged 1,940 Fieldbook tickets over the two quarters. Ticket
volume peaked in each region's third week and declined steadily after. 44% of
all tickets are password resets and login problems, concentrated among
technicians who missed the classroom course and had never logged in during the
supported window. Another 19% are duplicate-record tickets: technicians create
a second customer record when they cannot find the first, because the search
screen matches on exact postcode only. The support desk merges these
duplicates manually, at about 70 records per week. Support also fields a
steady trickle of calls from the billing team about mismatched job-completion
codes, which the desk cannot resolve and forwards to the vendor. The vendor's
response on the code mismatch has been that a configurable mapping table
exists and takes roughly a day to set up, but it has not been scheduled
because no one on Kestrel's side owns the integration. Training-wise, the desk
runs a weekly one-hour webinar that reaches 20 to 30 technicians; attendance
correlates strongly with regions that missed classroom slots. The desk lead's
written comment: "We are doing the training program's job at ticket prices —
most of what we handle is people who were never onboarded, plus the two known
system gaps."

4. Finance

Licence and hosting for Fieldbook run £31,000 per month for the four live
regions. Overtime spending in the live regions is down 12% quarter over
quarter, which finance attributes mainly to the scheduling gains, and fuel
cost per job is down 9%, consistent with the travel-time reduction. Against
that, the program has carried unplanned costs: the improvised shadowing
arrangements in South and West consumed an estimated 380 technician-hours of
lost route time, and the billing team added one temporary clerk (£8,400 for
the two quarters) to handle the re-keying of job records into the invoicing
system. The program office estimates that the rollout paid back its running
cost within the second quarter in Central and East, while South and West are
roughly at break-even because of the slower adoption. Extending to North and
Islands would add £13,500 per month in licences and require winter travel for
any classroom training. On the current evidence the economics of extension are
favourable if, and only if, the adoption pace of Central and East can be
reproduced; a repeat of the West pattern would put the new regions underwater
for at least two quarters. The program office therefore leans toward
extending in Q1, provided onboarding is restructured first.

5. Data and integrations

Fieldbook's reporting module is now the source of record for job status across
the four regions, and regional managers use its dashboard in the Monday
review. Data quality inside Fieldbook is generally good, with two exceptions.
First, the customer-record duplicates created through the search limitation
(support merges about 70 per week, as noted) mean that job-history views for
affected customers are split across records until merged. Second, the missing
feed from the legacy asset register leaves the equipment-history panel empty
on every job screen, which both hides warranty state from technicians and
makes the panel's audit trail useless for dispute resolution. The invoicing
mismatch also surfaces here: because completion codes are re-keyed by hand,
the reporting module and the invoicing system disagree on completed-job counts
by 2-4% in any given week, and finance reconciles the difference manually at
month end. The vendor's integration catalogue lists supported connectors for
both the asset register's database and the invoicing system's import format;
neither connector has been evaluated. IT's position is that both are
straightforward but that no integration work was scoped into the rollout
program, which was staffed for deployment and training only.

6. Program office remarks

Individual regional reports repeat most of the above in local detail. Two
further observations. First, every negative pattern in this review was visible
by week four of the Central go-live; the program had no mechanism to convert
early field observations into program-level fixes, so the same issues
replayed in each subsequent region. Second, the vendor relationship is good:
response times are within contract, and both known system gaps have vendor
solutions waiting on Kestrel decisions rather than on vendor work.
```

## Task

Restructure this review for the COO using the Minto Pyramid Principle. The COO's
decision is whether to extend Fieldbook to North and Islands in Q1.

Return only the result, with no explanation of how you produced it.
Промпт с Minto пять частей по порядку · 35 890 Б
  1. Below is a skill written as an instruction. Read it in full and apply it to the task at the end.
  2. ===== SKILL.md ===== SKILL.md на f4813ce 17 235 Б
  3. ===== references/rules.md ===== references/rules.md на f4813ce 6 464 Б
  4. ===== references/templates.md ===== references/templates.md на f4813ce 3 742 Б
  5. ===== TASK ===== вход кейса целиком
  6. Return only the result, with no explanation of how you produced it.
Вход кейса before.md

<!-- Original fixture. Company, product, regions, people and numbers are invented.
It tests the large-document behaviors: one pyramid per section plus a
top-level pyramid, MECE across sections (planted cross-section duplicates),
and geometry derived from the material rather than a fixed 3x3 habit. -->

Fixture 10: rollout review

Mode: write
Language: en

Context

Kestrel Maintenance Group services heating and cooling equipment across six
regions. Over the past two quarters it rolled out Fieldbook, a work-order and
scheduling platform, to four of the six regions. The draft below is the internal
review of that rollout, assembled by the program office from status reports. The
reader is the chief operating officer, who must decide at the quarterly meeting
whether to extend Fieldbook to the remaining two regions (North and Islands) in
Q1, and has asked to see this review beforehand.

Before
FIELDBOOK ROLLOUT — TWO-QUARTER REVIEW (DRAFT)
                
                1. Adoption
                
                Fieldbook is now live in four regions: Central, East, South, and West. Across
                these regions 412 of 486 field technicians have activated their accounts, and
                83% of work orders are now created and closed inside Fieldbook rather than on
                paper or by phone. Central and East reached full adoption within six weeks.
                South and West lagged: South took eleven weeks to pass 80% of orders in the
                system, and West is still at 71% after nine weeks. Regional coordinators in
                both slower regions report the same explanation: the two-day classroom course
                was scheduled only once per region, so technicians hired after the course, or
                absent during it, learned the platform second-hand from colleagues. In West,
                34 of 118 technicians never attended any session. Coordinators improvised
                shadowing arrangements, which worked but pulled experienced technicians off
                their own routes. Uptake among dispatchers was faster: all 41 dispatchers
                across the four regions were using Fieldbook scheduling from week one, since
                the old scheduling spreadsheet was switched off on day one and they had no
                alternative.
                
                2. Operations
                
                The clearest operational gain is scheduling density. With Fieldbook suggesting
                route-aware assignment, completed jobs per technician per day rose from 4.6 to
                5.3 across the four live regions, and average travel time between jobs fell
                18%. Same-day emergency insertion, which previously required a dispatcher to
                phone technicians until someone accepted, is now automatic and takes a median
                of 4 minutes. However, the benefit is partly eaten by a data-transfer defect:
                Fieldbook does not receive equipment service history from the legacy asset
                register, so technicians open each job without knowing what was done at that
                site before. They compensate by calling the back office, or by asking the
                customer, and in 9% of audited jobs they replaced parts that had already been
                replaced within warranty in the past year. The asset register team was asked
                in month two for a nightly export; the request is still in their backlog. A
                second, smaller friction: job-completion codes in Fieldbook do not match the
                codes in the invoicing system, so the billing team re-keys roughly 300 job
                records per week into the invoicing system by hand, introducing transcription
                errors that customers catch on invoices. Operations leads in Central and East
                also note that Fieldbook's parts-ordering screen is unpopular; most
                technicians still order parts by phone, though this appears to be habit rather
                than a defect, as the screen works correctly in testing.
                
                3. Support and training
                
                The support desk logged 1,940 Fieldbook tickets over the two quarters. Ticket
                volume peaked in each region's third week and declined steadily after. 44% of
                all tickets are password resets and login problems, concentrated among
                technicians who missed the classroom course and had never logged in during the
                supported window. Another 19% are duplicate-record tickets: technicians create
                a second customer record when they cannot find the first, because the search
                screen matches on exact postcode only. The support desk merges these
                duplicates manually, at about 70 records per week. Support also fields a
                steady trickle of calls from the billing team about mismatched job-completion
                codes, which the desk cannot resolve and forwards to the vendor. The vendor's
                response on the code mismatch has been that a configurable mapping table
                exists and takes roughly a day to set up, but it has not been scheduled
                because no one on Kestrel's side owns the integration. Training-wise, the desk
                runs a weekly one-hour webinar that reaches 20 to 30 technicians; attendance
                correlates strongly with regions that missed classroom slots. The desk lead's
                written comment: "We are doing the training program's job at ticket prices —
                most of what we handle is people who were never onboarded, plus the two known
                system gaps."
                
                4. Finance
                
                Licence and hosting for Fieldbook run £31,000 per month for the four live
                regions. Overtime spending in the live regions is down 12% quarter over
                quarter, which finance attributes mainly to the scheduling gains, and fuel
                cost per job is down 9%, consistent with the travel-time reduction. Against
                that, the program has carried unplanned costs: the improvised shadowing
                arrangements in South and West consumed an estimated 380 technician-hours of
                lost route time, and the billing team added one temporary clerk (£8,400 for
                the two quarters) to handle the re-keying of job records into the invoicing
                system. The program office estimates that the rollout paid back its running
                cost within the second quarter in Central and East, while South and West are
                roughly at break-even because of the slower adoption. Extending to North and
                Islands would add £13,500 per month in licences and require winter travel for
                any classroom training. On the current evidence the economics of extension are
                favourable if, and only if, the adoption pace of Central and East can be
                reproduced; a repeat of the West pattern would put the new regions underwater
                for at least two quarters. The program office therefore leans toward
                extending in Q1, provided onboarding is restructured first.
                
                5. Data and integrations
                
                Fieldbook's reporting module is now the source of record for job status across
                the four regions, and regional managers use its dashboard in the Monday
                review. Data quality inside Fieldbook is generally good, with two exceptions.
                First, the customer-record duplicates created through the search limitation
                (support merges about 70 per week, as noted) mean that job-history views for
                affected customers are split across records until merged. Second, the missing
                feed from the legacy asset register leaves the equipment-history panel empty
                on every job screen, which both hides warranty state from technicians and
                makes the panel's audit trail useless for dispute resolution. The invoicing
                mismatch also surfaces here: because completion codes are re-keyed by hand,
                the reporting module and the invoicing system disagree on completed-job counts
                by 2-4% in any given week, and finance reconciles the difference manually at
                month end. The vendor's integration catalogue lists supported connectors for
                both the asset register's database and the invoicing system's import format;
                neither connector has been evaluated. IT's position is that both are
                straightforward but that no integration work was scoped into the rollout
                program, which was staffed for deployment and training only.
                
                6. Program office remarks
                
                Individual regional reports repeat most of the above in local detail. Two
                further observations. First, every negative pattern in this review was visible
                by week four of the Central go-live; the program had no mechanism to convert
                early field observations into program-level fixes, so the same issues
                replayed in each subsequent region. Second, the vendor relationship is good:
                response times are within contract, and both known system gaps have vendor
                solutions waiting on Kestrel decisions rather than on vendor work.
Task

Restructure this review for the COO using the Minto Pyramid Principle. The COO's
decision is whether to extend Fieldbook to North and Islands in Q1.

Эталон судьи gold.md

Gold 10: rollout review

Original gold for an invented scenario.

Expected structure

Top: answers the COO's actual question — extend to North and Islands in Q1,
conditional on fixing what made South and West slow and expensive. A defensible
top must carry both the recommendation and its condition; "the rollout went
well" is a topic, and an unconditional "extend" ignores the document's own
economics (a repeat of the West pattern puts the new regions underwater).

The material honestly supports four first-level branches, not the habitual
three — for example:

  1. The economics already argue for extension — productivity up (4.6 → 5.3
    jobs/day, travel −18%, overtime −12%, fuel −9%), payback reached in the two
    regions that onboarded fast.
  2. Onboarding is the reproducible failure — single classroom slot per
    region left late hires unserved (34 of 118 in West never attended), drove
    44% of support tickets, forced shadowing that cost 380 technician-hours,
    and is the one variable separating payback regions from break-even regions.
  3. Two integration defects tax every region and have waiting fixes — the
    asset-register feed (blind technicians, 9% redundant warranty parts, empty
    audit panel) and the completion-code mismatch (300 records/week re-keyed, a
    temporary clerk, 2–4% reporting discrepancy); vendor connectors and the
    mapping table exist, and stall only because nobody owns integration.
  4. The program lacked a feedback loop — every defect was visible by week
    four of Central and replayed in each region; extension without that loop
    repeats the tax in two more regions.

A grouping that reaches the same content by different cuts is acceptable; what
is not acceptable is a branch per source section (adoption / operations /
support / finance / data), which is the document's order, not an answer.

Section pyramids

The rewrite must give each substantive section its own small pyramid — a
one-sentence section top that its details support — plus the top-level pyramid
whose key line the section tops visibly feed. A single flat pyramid over a
document of this size, with details hanging directly off four branches and the
sections dissolved, loses the layer the reader navigates by. Depth should
follow the material: the integration branch genuinely carries two distinct
defects, each with its own evidence, so it earns a third level; thin branches
must not be padded to match.

Cross-section duplicates (the MECE trap)

The same three facts each appear in two or three sections and must appear
once in the structure, with their scattered evidence gathered:

  • The completion-code mismatch: Operations (300 records/week re-keyed), Support
    (forwarded tickets, vendor's mapping-table answer), Finance (£8,400 clerk),
    Data (2–4% reconciliation gap). One defect, four symptoms.
  • The missing asset-register feed: Operations (blind jobs, 9% redundant parts)
    and Data (empty history panel, useless audit trail). One defect.
  • The training shortfall: Adoption (late hires unserved, shadowing), Support
    (44% of tickets, webinar attendance pattern, the desk lead's comment),
    Finance (380 lost hours). One cause.

An output that keeps any of these as two separate first-level or second-level
points in different branches has failed the cross-section MECE pass.

Facts and proportions

All load-bearing numbers come from the source; invented benchmarks, dates or
vendor quotes are hard failures. The parts-ordering habit, dispatcher uptake,
and vendor-relationship remarks are legitimately minor: demoting them to a
short residual note or explicitly dropping them is proportional omission.
The output is a restructured review, so it will be long — length itself is not
a defect; a section-by-section retelling under new headings is.

Судья gpt-5.6-sol, effort high · навык с коммита f4813ce · прогон v1.7.0-final