qwen3.8-27b на кейсе «Выжимка из нескольких источников»: с Minto лучше по структуре, качество то же
Структура 7 → 8 из 8 +1 Качество 10 → 10 из 10 0
Судья об ответе с Minto
It combines the cross-source facts, ownership split, source locators, and today’s actions without unnecessary apparatus.
| Структура | Без Minto 7 | С Minto 8 | Изменение |
|---|---|---|---|
| Вершина | 2 | 2 | |
| Состав ключевой линии | 1 | 2 | +1 |
| Уровни | 2 | 2 | |
| Порядок и род | 2 | 2 |
| Качество | Без Minto 10 | С Minto 10 | Изменение |
|---|---|---|---|
| Вершина | 2 | 2 | |
| Однородные группы | 2 | 2 | |
| Объяснимый порядок | 2 | 2 | |
| MECE | 2 | 2 | |
| Видимая соразмерная структура | 2 | 2 |
Ответы моделей и слова судьи приведены в оригинале, по-английски.
Без Minto
контрольный промпт · слепая меткаout-03Жёстких провалов нет
Strong action-oriented synthesis with correct levels; missing source locators are its principal weakness.
Compliance digest — Aldergate Bank support assistant (as of 18 Aug)
Bottom line: Build-phase pricing is still blocked because security requirements are not fully confirmed. The outstanding Aldergate responses are the approved-package list, the architecture-dependent data-storage/encryption/logging answer, and Lisbon staffing confirmation. A direct route to S. Okafor is also needed to close the track. Fernway can immediately unblock the data-storage item by sending the draft architecture diagram.
Already answered
- Security review: Mandatory before production. Whether a penetration test is also required will be decided during the review by the assigned reviewer.
- Security baseline: Aldergate has an internal hardening framework, but a review is still required regardless of conformance.
- Open-source dependencies: A policy exists. Dependency usage, including critical dependencies, is assessed during the security review. There is no standard approval turnaround; it is case by case.
- Staff location: Work from Ireland is acceptable. No position is recorded for other countries.
Still to obtain from Aldergate
- Approved-package list
- Promised 12 May by D. Mercer and re-promised 9 July by R. Whitcombe.
- Not received as of 18 Aug; open 98 days.
- Need current owner and a firm delivery date.
- Data storage, encryption, and logging requirements
- Gate review Q7 was declined until Aldergate sees an architecture diagram and design walkthrough.
- Requirements exist, but the answer is architecture-specific.
- Fernway’s draft architecture diagram has existed since 11 Aug but has not been sent.
- Need to send the diagram and obtain the formal answer.
- Lisbon-based Fernway engineers
- Two Fernway engineers based in Lisbon were raised at the 9 July sync.
- No Aldergate response is recorded.
- Need explicit confirmation whether they may work on the project.
- Direct compliance route
- S. Okafor is the named information security owner, but has not attended a joint meeting; questions are currently passed through R. Whitcombe.
- Need a direct call/meeting or named response path with Okafor to close the remaining security items.
Do today
- Send the draft architecture diagram to Whitcombe/Okafor and request a design walkthrough plus the storage/encryption/logging answer.
- Chase the approved-package list for a date and confirm whether Mercer or Whitcombe owns it.
- Ask for written confirmation on the Lisbon-based engineers.
- Request a direct compliance session with S. Okafor so the remaining security requirements can be closed before build-phase pricing is fixed.
С Minto
промпт с навыком · слепая меткаout-11Жёстких провалов нет
It combines the cross-source facts, ownership split, source locators, and today’s actions without unnecessary apparatus.
Bottom line: the confirmed items define the security-review path, but build-phase pricing is blocked until you send the architecture diagram and obtain the package list, data-storage answer, and Lisbon ruling (gate-review Q1–Q7; sync-0709; risks R-07, R-11, R-14).
Already answered
- Security review is mandatory before production; whether a penetration test is needed is decided by the assigned reviewer during the review (gate-review Q1).
- An internal hardening framework exists, but a review is still required even if the build conforms to it (gate-review Q2).
- An open-source policy exists; dependency usage is assessed during the security review, with no standard individual-approval turnaround (gate-review Q3–Q5).
- Work from Ireland is acceptable; no other country was confirmed (gate-review Q6).
Still needed from Aldergate
- Customer-data storage, encryption, and logging requirements, declined until an architecture diagram and design walkthrough are provided (gate-review Q7).
- The approved-package list, promised by Mercer on 12 May and re-promised by Whitcombe on 9 July, still not received and open 98 days (gate-review Q3; sync-0709; risk R-11).
- A dated commitment for the package list and a scheduled design walkthrough; S. Okafor is the named information-security owner but has not attended a joint meeting (sync-0709; gate-review Q7).
- Confirmation for the two Fernway engineers based in Lisbon, raised on 9 July with no Aldergate response (sync-0709; gate-review Q6).
Do today
- Send the draft architecture diagram, existing since 11 August and agreed at the 9 July sync, to unblock the data-storage answer (risk R-14; sync-0709).
- Ask Whitcombe/Okafor for a dated package-list delivery, a walkthrough slot, and written confirmation on Lisbon (sync-0709; risk R-11; gate-review Q6).
Как устроен тест
qwen3.8-27b получила одну и ту же задачу дважды. Контрольный промпт — одна вводная строка и задача. Промпт с Minto — та же задача, перед которой стоит полный текст навыка. Судья читал оба ответа среди 16 ответов под слепыми метками и не знал, в каком из них был навык.
Промпт без Minto 3 667 Б
You are working on a business document using the Minto Pyramid Principle.
<!-- Original fixture. The scenario, company, people, systems and numbers are
invented for this benchmark. The defect pattern it tests — a digest of several
read sources delivered answer-first into chat — is modeled on real usage. -->
# Fixture 09: sources digest
**Mode:** `digest`
**Language:** `en`
## Context
You are assisting the tech lead of Fernway Systems, a vendor building a customer
support assistant for Aldergate Bank. Before Fernway can price the build phase, the
bank's compliance office must confirm its security requirements. You have just read
the three sources below in full. The tech lead has not read them and will not; they
are deciding what to do about the compliance track today.
## Before
Source 1 — `gate-review.xlsx`, sheet "Security questions", filled in by Aldergate's
compliance office on 12 May:
```text
Q1 Is a security review required before production?
A: Yes. A review is mandatory; whether a penetration test is also needed is
decided during the review itself, by the assigned reviewer.
Q2 Is there a security standard or checklist we can build against?
A: An internal hardening framework exists; a review is still required
regardless of conformance.
Q3 What is the policy on open-source dependencies?
A: A policy exists. The approved-package list will be sent to Fernway.
Owner: D. Mercer. Date: 12 May.
Q4 Must critical dependencies be approved individually?
A: Dependency usage is assessed during the security review.
Q5 What is the turnaround time for dependency approval?
A: There is no standard turnaround; it is case by case.
Q6 Can Fernway staff work from outside the UK?
A: Work from Ireland is acceptable. (No other country is mentioned.)
Q7 Where may customer data be stored, and with what encryption and logging?
A: Declined — "cannot be answered without an architecture diagram and a
design walkthrough". Encryption and logging requirements exist but are
not applicable outside an architectural context.
```
Source 2 — follow-up note from the 9 July delivery sync, `sync-0709.md`:
```text
Attendees: R. Whitcombe (Aldergate, delivery), tech lead (Fernway), PM (Fernway).
- Whitcombe repeated that the approved-package list "is being prepared" and will
reach Fernway "shortly". Action on Whitcombe, no date attached.
- Compliance contact named for the first time: S. Okafor, information security
owner. Okafor has not attended any joint meeting so far; Whitcombe passes
questions along.
- Agreed: Fernway sends the draft architecture diagram before the next sync so
the storage and encryption questions can get a real answer.
- Fernway's PM noted two Fernway engineers are based in Lisbon. Nobody from
Aldergate responded to the remark in the meeting.
```
Source 3 — Fernway's internal risk register, rows touching compliance, as of
18 August:
```text
R-07 Security requirements unconfirmed; build-phase price cannot be fixed.
Owner: tech lead. Status: open since 3 May.
R-11 Approved-package list not received. First promised 12 May (Mercer, per
gate review), promised again 9 July (Whitcombe). Status: open, 98 days.
R-14 Architecture diagram not yet sent to Aldergate; blocks the data-storage
answer (gate review Q7). Draft exists since 11 August. Status: open.
```
## Task
Give the tech lead a digest of these three sources in chat: what is already
answered on the compliance track, and what still has to be obtained from
Aldergate. The tech lead will act on it today.
Return only the result, with no explanation of how you produced it.
Промпт с Minto пять частей по порядку · 31 240 Б
Below is a skill written as an instruction. Read it in full and apply it to the task at the end.===== SKILL.md =====SKILL.md на f4813ce 17 235 Б===== references/rules.md =====references/rules.md на f4813ce 6 464 Б===== references/templates.md =====references/templates.md на f4813ce 3 742 Б===== TASK =====вход кейса целикомReturn only the result, with no explanation of how you produced it.
Вход кейса before.md
<!-- Original fixture. The scenario, company, people, systems and numbers are
invented for this benchmark. The defect pattern it tests — a digest of several
read sources delivered answer-first into chat — is modeled on real usage. -->
Fixture 09: sources digest
Mode: digest
Language: en
Context
You are assisting the tech lead of Fernway Systems, a vendor building a customer
support assistant for Aldergate Bank. Before Fernway can price the build phase, the
bank's compliance office must confirm its security requirements. You have just read
the three sources below in full. The tech lead has not read them and will not; they
are deciding what to do about the compliance track today.
Before
Source 1 — gate-review.xlsx, sheet "Security questions", filled in by Aldergate's
compliance office on 12 May:
Q1 Is a security review required before production?
A: Yes. A review is mandatory; whether a penetration test is also needed is
decided during the review itself, by the assigned reviewer.
Q2 Is there a security standard or checklist we can build against?
A: An internal hardening framework exists; a review is still required
regardless of conformance.
Q3 What is the policy on open-source dependencies?
A: A policy exists. The approved-package list will be sent to Fernway.
Owner: D. Mercer. Date: 12 May.
Q4 Must critical dependencies be approved individually?
A: Dependency usage is assessed during the security review.
Q5 What is the turnaround time for dependency approval?
A: There is no standard turnaround; it is case by case.
Q6 Can Fernway staff work from outside the UK?
A: Work from Ireland is acceptable. (No other country is mentioned.)
Q7 Where may customer data be stored, and with what encryption and logging?
A: Declined — "cannot be answered without an architecture diagram and a
design walkthrough". Encryption and logging requirements exist but are
not applicable outside an architectural context.
Source 2 — follow-up note from the 9 July delivery sync, sync-0709.md:
Attendees: R. Whitcombe (Aldergate, delivery), tech lead (Fernway), PM (Fernway).
- Whitcombe repeated that the approved-package list "is being prepared" and will
reach Fernway "shortly". Action on Whitcombe, no date attached.
- Compliance contact named for the first time: S. Okafor, information security
owner. Okafor has not attended any joint meeting so far; Whitcombe passes
questions along.
- Agreed: Fernway sends the draft architecture diagram before the next sync so
the storage and encryption questions can get a real answer.
- Fernway's PM noted two Fernway engineers are based in Lisbon. Nobody from
Aldergate responded to the remark in the meeting.
Source 3 — Fernway's internal risk register, rows touching compliance, as of
18 August:
R-07 Security requirements unconfirmed; build-phase price cannot be fixed.
Owner: tech lead. Status: open since 3 May.
R-11 Approved-package list not received. First promised 12 May (Mercer, per
gate review), promised again 9 July (Whitcombe). Status: open, 98 days.
R-14 Architecture diagram not yet sent to Aldergate; blocks the data-storage
answer (gate review Q7). Draft exists since 11 August. Status: open.
Task
Give the tech lead a digest of these three sources in chat: what is already
answered on the compliance track, and what still has to be obtained from
Aldergate. The tech lead will act on it today.
Эталон судьи gold.md
Gold 09: sources digest
Original gold for an invented scenario.
Expected structure
Top: an answer to the tech lead's question, not a description of the sources.
The strongest available top is an action with its ground: the blocking questions
cannot be answered by more reading — send the architecture diagram (drafted and
sitting unsent since 11 August) and escalate the approved-package list as an
obligation now 98 days old. A top that merely states "five of seven questions are
answered" is weaker but acceptable if the two actions lead the groups.
Three same-kind groups the material supports, in any defensible order:
- Settled — do not re-ask. Review is mandatory (pen test decided during it);
a hardening framework exists but exempts nothing; dependencies are assessed in
the review with no standard turnaround; Ireland is cleared for remote work. - Blocked on Fernway's own move. The storage/encryption/logging answer (Q7)
was declined pending an architecture diagram; the draft has existed since
11 August and has not been sent (R-14). This is Fernway's blocker, not
Aldergate's. - Owed by Aldergate and overdue. The approved-package list, first promised
12 May by Mercer per the gate review — not 9 July by Whitcombe, so 98 days
overdue, which makes the ask an escalation of an old commitment rather than a
reminder. Direct access to Okafor, who has never attended and is reachable
only through Whitcombe.
Cross-source synthesis the digest must make
- The package-list promise appears in two sources with two dates; the digest
must date it from 12 May (the older commitment), not 9 July. - Q7's "declined" plus R-14's "draft exists since 11 August" must combine into
"the ball is in Fernway's court"; either fact alone misses the point. - The Lisbon remark: Q6 clears Ireland only. Two engineers in Lisbon are an
unanswered exposure, not a settled fact. Presenting Lisbon as approved is an
invention; omitting it entirely loses the one new risk the sources surface.
Presentation
Chat prose for one reader: answer first, grouped support, a reference to which
source each load-bearing claim comes from (sheet, note, or register row —
exact citation format is free). No marker legend, no mermaid diagram, no
findings table, no meta-commentary about how the groups are ordered. Headings or
bold group labels are acceptable; a full document apparatus (SCQ labels as
headings, scoring, appendix) is excessive. Length: roughly one chat screen —
condensing the six settled answers into one or two lines each is proportional
omission, not source loss.
Судья gpt-5.6-sol, effort high · навык с коммита f4813ce · прогон v1.7.0-final