Skip to content

Commit 1fa7d65

Browse files
authored
fix(router): scope hasToolResult to the current turn (multi-turn replay) (#316)
## Problem `match.hasToolResult` is a whole-conversation predicate — "does ANY message have role `tool`". In a multi-turn chat session that pins it to `true` forever after the first tool call, so a turn's **leg-1** fixture (`hasToolResult: false`, the tool-call leg) can never match again: every 2nd+ user turn carries an earlier turn's tool result. Symptom in the CopilotKit showcase (Beautiful Chat): the first pill/tool in a thread works, but every pill after it returns **"No fixture matched"** (schedule-meeting, excalidraw, calculator, etc.). Reproduced + verified fixed against the live showcase. ## Fix Scope `hasToolResult` to the **current turn** — messages after the last `user` message. For a single-turn request this is identical to the old check (so existing fixtures are unaffected); multi-turn leg-1 fixtures match again. ## Tests Updated the `hasToolResult` unit tests to use realistic leg-2 shapes (the tool result is the last message of the turn, not followed by a fresh user message — the old tests appended a trailing user message, which is actually the multi-turn leg-1 case) and added a multi-turn regression test. Full suite green (4448 tests). ## Consumer note CopilotKit showcase needs the released version (npm + ghcr) to fix multi-turn manual testing; CI single-turn probes are unaffected either way.
2 parents 4584be6 + d3037d9 commit 1fa7d65

2 files changed

Lines changed: 65 additions & 11 deletions

File tree

src/__tests__/router.test.ts

Lines changed: 44 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1222,14 +1222,22 @@ describe("matchFixture — turnIndex", () => {
12221222
// ---------------------------------------------------------------------------
12231223

12241224
describe("matchFixture — hasToolResult", () => {
1225-
it("matches hasToolResult: true when tool messages present", () => {
1225+
// hasToolResult is TURN-SCOPED: it asks whether the CURRENT turn (messages
1226+
// after the last user message) contains a tool result — not whether the whole
1227+
// conversation ever did. This is what lets the leg-1 (tool call) / leg-2
1228+
// (post-tool narration) fixture pair keep discriminating across MULTI-TURN
1229+
// sessions: on the 2nd+ user turn the request still carries earlier turns'
1230+
// tool results, and a whole-conversation check would pin hasToolResult=true
1231+
// forever so the turn's leg-1 fixture (false) could never match again.
1232+
1233+
it("matches hasToolResult: true when the current turn has a tool result", () => {
12261234
const fixture = makeFixture({ userMessage: "hello", hasToolResult: true });
1235+
// Real leg-2 shape: the tool result is the LAST message (no new user turn).
12271236
const req = makeReq({
12281237
messages: [
12291238
{ role: "user", content: "hello" },
12301239
{ role: "assistant", content: "calling tool" },
12311240
{ role: "tool", content: "tool output" },
1232-
{ role: "user", content: "hello" },
12331241
],
12341242
});
12351243
expect(matchFixture([fixture], req)).toBe(fixture);
@@ -1255,20 +1263,36 @@ describe("matchFixture — hasToolResult", () => {
12551263
expect(matchFixture([fixture], req)).toBe(fixture);
12561264
});
12571265

1258-
it("skips hasToolResult: false when tool messages present", () => {
1266+
it("skips hasToolResult: false when the current turn has a tool result", () => {
12591267
const fixture = makeFixture({ userMessage: "hello", hasToolResult: false });
1268+
// Real leg-2 shape: tool result is the last message in the current turn.
12601269
const req = makeReq({
12611270
messages: [
12621271
{ role: "user", content: "hello" },
12631272
{ role: "assistant", content: "calling tool" },
12641273
{ role: "tool", content: "tool output" },
1265-
{ role: "user", content: "hello" },
12661274
],
12671275
});
12681276
expect(matchFixture([fixture], req)).toBeNull();
12691277
});
12701278

1271-
it("discriminates 2-step HITL flow with hasToolResult", () => {
1279+
it("matches hasToolResult: false on a new turn even when an EARLIER turn had a tool result (multi-turn)", () => {
1280+
// The regression this scoping fixes: a 2nd pill click in the same chat
1281+
// thread. The request carries turn-1's tool result, but the CURRENT turn
1282+
// (after the last user message) has none, so leg-1 (false) must still match.
1283+
const fixture = makeFixture({ userMessage: "hello", hasToolResult: false });
1284+
const req = makeReq({
1285+
messages: [
1286+
{ role: "user", content: "hello" },
1287+
{ role: "assistant", content: "calling tool" },
1288+
{ role: "tool", content: "prior-turn tool output" },
1289+
{ role: "user", content: "hello" },
1290+
],
1291+
});
1292+
expect(matchFixture([fixture], req)).toBe(fixture);
1293+
});
1294+
1295+
it("discriminates 2-step HITL flow with hasToolResult across turns", () => {
12721296
const beforeTool = makeFixture(
12731297
{ userMessage: "hello", hasToolResult: false },
12741298
{ content: "before-tool" },
@@ -1278,20 +1302,34 @@ describe("matchFixture — hasToolResult", () => {
12781302
{ content: "after-tool" },
12791303
);
12801304

1305+
// Turn 1, leg 1: fresh user message, no tool yet.
12811306
const reqBefore = makeReq({
12821307
messages: [{ role: "user", content: "hello" }],
12831308
});
12841309
expect(matchFixture([beforeTool, afterTool], reqBefore)).toBe(beforeTool);
12851310

1311+
// Turn 1, leg 2: the tool result is the last message in the current turn.
12861312
const reqAfter = makeReq({
12871313
messages: [
12881314
{ role: "user", content: "hello" },
12891315
{ role: "assistant", content: "calling tool" },
12901316
{ role: "tool", content: "result" },
1291-
{ role: "user", content: "hello" },
12921317
],
12931318
});
12941319
expect(matchFixture([beforeTool, afterTool], reqAfter)).toBe(afterTool);
1320+
1321+
// Turn 2, leg 1: a new user message after turn-1's completed tool flow —
1322+
// must fall back to the leg-1 fixture, not re-serve leg-2's narration.
1323+
const reqNextTurn = makeReq({
1324+
messages: [
1325+
{ role: "user", content: "hello" },
1326+
{ role: "assistant", content: "calling tool" },
1327+
{ role: "tool", content: "result" },
1328+
{ role: "assistant", content: "before-tool" },
1329+
{ role: "user", content: "hello" },
1330+
],
1331+
});
1332+
expect(matchFixture([beforeTool, afterTool], reqNextTurn)).toBe(beforeTool);
12951333
});
12961334
});
12971335

src/router.ts

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -461,12 +461,28 @@ export function matchFixtureDiagnostic(
461461
}
462462
}
463463

464-
// hasToolResult — request-SHAPE predicate: does the conversation contain a
465-
// tool result message? Must be evaluated with the other shape predicates
466-
// ABOVE the sequence/turn state gates so that a fixture whose shape never
467-
// matched is not miscounted as "skipped by sequence/turn state".
464+
// hasToolResult — request-SHAPE predicate: does the CURRENT turn contain a
465+
// tool result message? "Current turn" = messages after the last user
466+
// message, so this is scoped to the turn being matched rather than the
467+
// whole conversation. This is what makes leg-1 (tool call, hasToolResult
468+
// false) vs leg-2 (narration, hasToolResult true) fixtures keep working
469+
// across MULTI-TURN sessions: on the 2nd+ user turn the request still
470+
// carries earlier turns' tool results, and a whole-conversation check would
471+
// force hasToolResult=true forever, so the turn's leg-1 fixture (false)
472+
// could never match again ("No fixture matched" on every pill after the
473+
// first). For a single-turn request this is identical to the old
474+
// whole-conversation check. Must be evaluated with the other shape
475+
// predicates ABOVE the sequence/turn state gates so that a fixture whose
476+
// shape never matched is not miscounted as "skipped by sequence/turn state".
468477
if (match.hasToolResult !== undefined) {
469-
const hasTool = effective.messages.some((m) => m.role === "tool");
478+
let lastUserIdx = -1;
479+
for (let i = effective.messages.length - 1; i >= 0; i--) {
480+
if (effective.messages[i].role === "user") {
481+
lastUserIdx = i;
482+
break;
483+
}
484+
}
485+
const hasTool = effective.messages.slice(lastUserIdx + 1).some((m) => m.role === "tool");
470486
if (hasTool !== match.hasToolResult) continue;
471487
}
472488

0 commit comments

Comments
 (0)