Guideline Assist, replayed.
A live assist for a support agent that reads the chat so far and the company's written guidelines, and says what the intent is, which step comes next with the values to enter, and which guideline section it is following, and a post-call QA that grades a finished chat against the same guidelines. Below are six recorded chats played back one turn at a time, with each suggestion shown before the turn it was about and what the agent actually did shown after.
Nothing on this page is live. Every suggestion is copied from evals/results/assist-2026-09-24.json in the repo (Sonnet 5 with the whole guideline library in a cached prompt), and every QA status from qa-2026-09-24.json. The chats are from ABCD, ASAPP Research, role-played conversations with trained crowdworkers, no real customers. ABCD is MIT licensed, Copyright (c) 2021 ASAPP Research.
In plain terms: it works like a colleague reading over a support agent's shoulder who has memorized the rulebook and whispers the next step, and then checks the finished conversation for steps that were skipped or done wrong.
01Pick a chat, watch what it would have told the agent
the suggestion shows before its turn, and what the agent really did shows after02One chat, three planted defects, and what the QA said about each
every defect is made in code, so the right answer is known before the QA runs03Caching the whole guideline library beat looking up one section
Arm A puts all 55 guideline sections in a cached prompt prefix on every turn, and arm B asks for the intent first and then passes in only that section. A cached read costs a tenth of fresh input, and arm B pays for a second call and still gets the intent wrong more often, so arm A came out cheaper and faster on both models, and Sonnet on arm A is the version replayed above.
In plain terms: handing the model the whole rulebook every time, from a cache, turned out cheaper and quicker than looking up the one page it needed first.