diff --git a/.claude/skills/write-docs/SKILL.md b/.claude/skills/write-docs/SKILL.md index 518cae5ab3..ff54332047 100644 --- a/.claude/skills/write-docs/SKILL.md +++ b/.claude/skills/write-docs/SKILL.md @@ -358,6 +358,17 @@ Spelling to enforce on every page: - **"realtime", one word.** Never "real time" or "real-time" when describing Ably delivery ("sees the same conversation in realtime", "realtime messaging"). This is Ably house spelling. Sweep with `grep -rn "real[- ]time" "$P"` and expect zero hits. +### Stay inside the product vocabulary + +Use the terms the product actually defines. Do not introduce vocabulary that is not part of the AI Transport domain — not internal-codebase jargon, and not generic AI/ML terms the docs have chosen not to use. A reader who has read the concept pages should never hit a word the docs never taught them. + +- **Prefer the named primitives.** The concept pages define the vocabulary: [Session](/docs/ai-transport/concepts/sessions), [Connection](/docs/ai-transport/concepts/connections), [Run](/docs/ai-transport/concepts/runs), [Step](/docs/ai-transport/concepts/steps), [Invocation](/docs/ai-transport/concepts/invocations), [Codec](/docs/ai-transport/concepts/codecs), [conversation tree](/docs/ai-transport/concepts/conversation-tree), and view. Reach for these before inventing a synonym. +- **Do not leak internal jargon.** Words that name SDK internals but never appear in the concept or API docs (for example `projection`, `RunNode`, or other type names not documented for readers) read as jargon. Use the documented term instead: "the conversation" or "the Run's view", not "the projection"; "the Run" or "the conversation tree", not "`RunNode`". +- **Do not smuggle in generic AI/ML vocab the docs avoid.** Describe a loop cycle as an "iteration", not a "pass". Describe a model invocation as a "model call" or "response", not an "inference pass". The verb "pass" (to pass an argument, hook, or signal) is fine; the noun "pass" (meaning a loop cycle) is not. +- **Respect the Run/turn split.** "Run" is the only primitive the docs explain. Use "turn" only in jobs-to-be-done framing (titles, intros, the "concurrent turns" feature name), never as a defined primitive alongside Run. + +When a word is genuinely new and needed, define it on first use or link to the concept page that does. If you cannot point to where it is defined, it does not belong on the page. + ## Per-page workflow 1. **Identify the page type.** Pick the matching template above. @@ -401,6 +412,11 @@ grep -rn "—" "$P" # "realtime" is one word (no "real time" / "real-time") grep -rn "real[- ]time" "$P" +# No off-vocabulary terms. Noun "pass" (loop cycle), "inference pass", and internal +# jargon that never appears in the concept/API docs. The verb "pass" (an argument, +# hook, signal) is fine, so this targets the noun forms and known jargon only. +grep -rniE "inference pass|\b(this|that|next|current|each|one|its|a|the|per|another) (pass|passes)\b|\bprojection\b|\bRunNode\b" "$P" + # No bold-prefix bullets grep -rnE "^\s*[\*\-]\s+\*\*[^*]+:\*\*" "$P" @@ -481,7 +497,7 @@ When reviewing someone else's docs PR (or your own at PR time), walk this list. **Writing standards** 9. Page complies with the rules in [`writing-style-guide.md`](../../../writing-style-guide.md). Use the verification greps above to spot the common violations (em dashes, bold-prefix bullets, Latin abbreviations, subjective phrases, vague modals, "we" as the subject, AI-fingerprint patterns). -10. Author ran `/deslop` (or equivalent AI-pattern check). +10. Page stays inside the product vocabulary — no internal-codebase jargon (`projection`, `RunNode`) and no generic AI/ML terms the docs avoid (noun "pass", "inference pass"). Every non-obvious term is defined on first use or links to the concept page that defines it. Use the off-vocabulary grep above. **Code accuracy** diff --git a/src/data/nav/aitransport.ts b/src/data/nav/aitransport.ts index ebabea2a1a..823fd14135 100644 --- a/src/data/nav/aitransport.ts +++ b/src/data/nav/aitransport.ts @@ -132,76 +132,76 @@ export default { name: 'Features', pages: [ { - name: 'Token streaming', - link: '/docs/ai-transport/features/token-streaming', + name: 'Agent presence', + link: '/docs/ai-transport/features/agent-presence', }, { - name: 'Cancellation', - link: '/docs/ai-transport/features/cancellation', + name: 'Branching, edit, and regenerate', + link: '/docs/ai-transport/features/branching', }, { - name: 'Reconnection and recovery', - link: '/docs/ai-transport/features/reconnection-and-recovery', + name: 'Cancellation', + link: '/docs/ai-transport/features/cancellation', }, { - name: 'Multi-device sessions', - link: '/docs/ai-transport/features/multi-device', + name: 'Chain of thought', + link: '/docs/ai-transport/features/chain-of-thought', }, { - name: 'History and replay', - link: '/docs/ai-transport/features/history', + name: 'Concurrent turns', + link: '/docs/ai-transport/features/concurrent-turns', }, { name: 'Database hydration', link: '/docs/ai-transport/features/database-hydration', }, { - name: 'Branching, edit, and regenerate', - link: '/docs/ai-transport/features/branching', - }, - { - name: 'Interruption', - link: '/docs/ai-transport/features/interruption', - }, - { - name: 'Concurrent turns', - link: '/docs/ai-transport/features/concurrent-turns', - }, - { - name: 'Tool calling', - link: '/docs/ai-transport/features/tool-calling', + name: 'Double texting', + link: '/docs/ai-transport/features/double-texting', }, { name: 'Durable execution', link: '/docs/ai-transport/features/durable-execution', }, { - name: 'Human-in-the-loop', - link: '/docs/ai-transport/features/human-in-the-loop', + name: 'History and replay', + link: '/docs/ai-transport/features/history', }, { - name: 'Optimistic updates', - link: '/docs/ai-transport/features/optimistic-updates', + name: 'Human-in-the-loop', + link: '/docs/ai-transport/features/human-in-the-loop', }, { - name: 'Agent presence', - link: '/docs/ai-transport/features/agent-presence', + name: 'Interruption and steering', + link: '/docs/ai-transport/features/interruption-and-steering', }, { name: 'LiveObjects State', link: '/docs/ai-transport/features/liveobjects', }, + { + name: 'Multi-device sessions', + link: '/docs/ai-transport/features/multi-device', + }, + { + name: 'Optimistic updates', + link: '/docs/ai-transport/features/optimistic-updates', + }, { name: 'Push notifications', link: '/docs/ai-transport/features/push-notifications', }, { - name: 'Chain of thought', - link: '/docs/ai-transport/features/chain-of-thought', + name: 'Reconnection and recovery', + link: '/docs/ai-transport/features/reconnection-and-recovery', }, { - name: 'Double texting', - link: '/docs/ai-transport/features/double-texting', + name: 'Token streaming', + link: '/docs/ai-transport/features/token-streaming', + }, + { + name: 'Tool calling', + link: '/docs/ai-transport/features/tool-calling', }, ], }, diff --git a/src/pages/docs/ai-transport/concepts/runs.mdx b/src/pages/docs/ai-transport/concepts/runs.mdx index d931915399..7948d6ed3e 100644 --- a/src/pages/docs/ai-transport/concepts/runs.mdx +++ b/src/pages/docs/ai-transport/concepts/runs.mdx @@ -91,6 +91,37 @@ await clientRun.cancel(); On the agent side, the symmetric primitive is [`AgentSession.createRun(invocation, runtime?)`](/docs/ai-transport/api/javascript/core/agent-session#create-run). The agent mints `runId` (for a fresh run) or reads the existing `runId` off the triggering input event (for a continuation), and stamps it on every event it publishes; the client reads it from `clientRun.runId` once `clientRun.started` resolves. +## Steer a Run + +While a Run is active, a client can send a follow-up message into it. This is called *steering* the Run. The agent picks up the new message on the next loop iteration; no new Run starts. + +The follow-up has the same shape as the input that started the Run. It joins the conversation tree under the same `RunNode` and every connected client sees it. Unlike a fresh `view.send`, it carries the active Run's `runId`, which tells the agent to add it to the existing Run. + +Steering messages don't change how a Run ends. The terminal `RunEndReason` reports how the Run finished. Each steering message also gets its own outcome, so the client can tell which ones the agent saw before the Run ended and which it didn't. + + +```javascript +const activeRun = await session.view.send(UIMessageCodec.createUserMessage({ + id: crypto.randomUUID(), + role: 'user', + parts: [{ type: 'text', text: 'Plan a 3-day trip to Lisbon.' }], +})); + +const { published, outcome } = activeRun.steer(UIMessageCodec.createUserMessage({ + id: crypto.randomUUID(), + role: 'user', + parts: [{ type: 'text', text: 'Make it 5 days, and keep it under £800.' }], +})); + +await published; // The steering message has been published to the channel. + +const { consumed } = await outcome; // Resolves when the Run ends. +console.log('Reached the agent:', consumed); +``` + + +See [interruption and steering](/docs/ai-transport/features/interruption-and-steering#steer) for how to call `steer()`, what `outcome` contains, and how to write the agent's loop. + ## Run concurrent Runs A session can hold multiple Runs in flight at the same time. They share the channel; they don't share state. See [concurrent turns](/docs/ai-transport/features/concurrent-turns) for the patterns that arise when more than one Run is `'active'`. diff --git a/src/pages/docs/ai-transport/features/cancellation.mdx b/src/pages/docs/ai-transport/features/cancellation.mdx index df2deddeb9..98c279ea87 100644 --- a/src/pages/docs/ai-transport/features/cancellation.mdx +++ b/src/pages/docs/ai-transport/features/cancellation.mdx @@ -64,11 +64,16 @@ Every run exposes an `abortSignal` that fires when the run is cancelled. Pass it ```javascript const run = session.createRun(invocation, { signal: req.signal }); + +// Drain run.view for the full conversation to feed the model. run.messages +// is only this Run's own turn. +while (run.view.hasOlder()) await run.view.loadOlder(); await run.start(); +const conversation = run.view.getMessages().map(({ message }) => message); const result = streamText({ model: anthropic('claude-sonnet-4-20250514'), - messages: run.messages, + messages: await convertToModelMessages(conversation), abortSignal: run.abortSignal, }); @@ -155,6 +160,6 @@ The cancel signal is a published message on the channel. See [pricing](/docs/pla ## Related features -- [Interruption](/docs/ai-transport/features/interruption): cancel and immediately send a new message. +- [Interruption and steering](/docs/ai-transport/features/interruption-and-steering): steer the active Run, cancel and re-prompt, or send alongside. - [Concurrent turns](/docs/ai-transport/features/concurrent-turns): multiple turns with independent cancel handles. - [Token streaming](/docs/ai-transport/features/token-streaming): what gets cancelled when the abort fires. diff --git a/src/pages/docs/ai-transport/features/concurrent-turns.mdx b/src/pages/docs/ai-transport/features/concurrent-turns.mdx index bdb5fc7bb4..0c937eb270 100644 --- a/src/pages/docs/ai-transport/features/concurrent-turns.mdx +++ b/src/pages/docs/ai-transport/features/concurrent-turns.mdx @@ -158,7 +158,7 @@ const newRun = await view.send(UIMessageCodec.createUserMessage({ ``` -See [Interruption](/docs/ai-transport/features/interruption) for the full pattern. +See [Interruption and steering](/docs/ai-transport/features/interruption-and-steering) for the full pattern. ### Multi-user sessions @@ -239,5 +239,5 @@ A Run is one unit of agent work that produces multiple messages. The `runId` gro ## Related features - [Cancellation](/docs/ai-transport/features/cancellation): scoped cancel signals and server-side abort handling. -- [Interruption](/docs/ai-transport/features/interruption): cancel and immediately send a new message. +- [Interruption and steering](/docs/ai-transport/features/interruption-and-steering): steer the active Run, cancel and re-prompt, or send alongside. - [Multi-device sessions](/docs/ai-transport/features/multi-device): concurrent turns across devices. diff --git a/src/pages/docs/ai-transport/features/double-texting.mdx b/src/pages/docs/ai-transport/features/double-texting.mdx index f7653c9553..5c18cde0a2 100644 --- a/src/pages/docs/ai-transport/features/double-texting.mdx +++ b/src/pages/docs/ai-transport/features/double-texting.mdx @@ -76,7 +76,7 @@ This gives users feedback that their message was received and will be processed ## Cancel and re-prompt -To cancel the current Run before sending the new message, use the [interruption](/docs/ai-transport/features/interruption) pattern: +To cancel the current Run before sending the new message, use the [interruption and steering](/docs/ai-transport/features/interruption-and-steering) pattern: ```javascript @@ -141,5 +141,5 @@ No. Each user message is a sibling under the same parent. The tree records both ## Related features - [Concurrent turns](/docs/ai-transport/features/concurrent-turns): the underlying mechanism for parallel turns. -- [Interruption](/docs/ai-transport/features/interruption): the cancel-then-send pattern in detail. +- [Interruption and steering](/docs/ai-transport/features/interruption-and-steering): steer, cancel-then-send, and send-alongside in detail. - [Cancellation](/docs/ai-transport/features/cancellation): cancel signals and scoped cancellation. diff --git a/src/pages/docs/ai-transport/features/interruption-and-steering.mdx b/src/pages/docs/ai-transport/features/interruption-and-steering.mdx new file mode 100644 index 0000000000..2e6310c75c --- /dev/null +++ b/src/pages/docs/ai-transport/features/interruption-and-steering.mdx @@ -0,0 +1,277 @@ +--- +title: "Interruption and steering" +meta_description: "Let users change direction mid-response in Ably AI Transport. Three patterns: steer the active Run with a follow-up prompt, cancel and re-prompt, or send alongside as a concurrent Run." +meta_keywords: "AI interruption, interrupt, steering, steer, activeRun.steer, follow-up prompt, cancel and send, concurrent turns, AI Transport, Ably" +intro: "Your users can change direction mid-response. AI Transport gives you three patterns: steer the active Run with a follow-up prompt, cancel and re-prompt, or send alongside as a concurrent Run." +redirect_from: + - /docs/ai-transport/features/interruption +--- + +A user changes direction while the agent is still working: they narrow the scope, add a constraint, or press Stop and start over. The session is bidirectional, so the client can send a follow-up into the same [active Run](/docs/ai-transport/concepts/runs), cancel the Run and start a new one, or open a parallel Run on the same session. + +![Diagram showing session continuity during interruption](../../../../images/content/diagrams/ait-session-continuity.png) + +Start a Run with `view.send()`, then steer it by calling `activeRun.steer()` with a follow-up message: + + +```javascript +const activeRun = await session.view.send(UIMessageCodec.createUserMessage({ + id: crypto.randomUUID(), + role: 'user', + parts: [{ type: 'text', text: 'Plan a 3-day trip to Lisbon.' }], +})); + +const { published, outcome } = activeRun.steer(UIMessageCodec.createUserMessage({ + id: crypto.randomUUID(), + role: 'user', + parts: [{ type: 'text', text: 'Make it 5 days, and keep it under £800.' }], +})); +``` + + +## How it works + +Each turn becomes a [Run](/docs/ai-transport/concepts/runs) on the session. While a Run is active, the client can do one of three things: + +| Pattern | What happens | When to use it | +| --- | --- | --- | +| Steer the active Run | A new user message is published to the active Run; the agent picks it up on the next loop iteration. | Quick correction or scope clarification while the agent is still working. | +| Cancel and re-prompt | The active Run aborts; a new Run starts. | The previous direction is wrong; the user wants to start over. | +| Send alongside | A second Run starts; both Runs stream in parallel. | Quick follow-up that doesn't replace the in-flight reply. | + +Steering is the least destructive: it doesn't end the Run, start a new Run, or break the stream. + +## Steer the active Run + +Call `activeRun.steer(input)` to send a steering message to the agent. The agent picks it up on the next loop iteration. A steering message will not start a new Run. + +`steer()` returns two promises: + + +```javascript +const { published, outcome } = activeRun.steer(UIMessageCodec.createUserMessage({ + id: crypto.randomUUID(), + role: 'user', + parts: [{ type: 'text', text: 'Also include vegan options.' }], +})); + +const { serial } = await published; +const { consumed, runTerminalReason } = await outcome; +``` + + +- `published` resolves once the steering message has been published to the channel, carrying the Ably-assigned `serial`. +- `outcome` resolves to `{ consumed: boolean, runTerminalReason?: RunEndReason }` once the agent consumes the steering message, or once the Run ends without consuming it: + - `consumed: true`: the agent saw the steering message before it sent its response. The `pipe()` that records the message resolves `outcome`, so a consumed message resolves even when the Run later suspends rather than ends. + - `consumed: false`: the Run ended before the agent saw the message. + - `runTerminalReason`: how the Run ended (`'complete'`, `'cancelled'`, or `'error'`). Absent when the agent consumes the message at a `run-suspend`, since the Run has not ended. + +On the agent side, `Run.hasInput()` controls the loop. It returns `true` while there's input the agent hasn't responded to (the original prompt, or a steering message from the previous iteration), and `false` once the agent has caught up: + + +```javascript +const run = session.createRun(invocation, { signal: req.signal }); + +// Drain run.view for the full conversation before starting. run.messages is +// only this Run's own turn, so feed run.view to the model instead. +while (run.view.hasOlder()) await run.view.loadOlder(); +await run.start(); + +while (run.hasInput()) { + const conversation = run.view.getMessages().map(({ message }) => message); + const result = streamText({ + model: anthropic('claude-sonnet-4-20250514'), + messages: await convertToModelMessages(conversation), + abortSignal: run.abortSignal, + }); + await run.pipe(result.toUIMessageStream()); +} + +await run.end({ reason: 'complete' }); +await session.end(); +``` + + +Each `pipe()` records which steering messages the agent had seen before producing that response. The client uses those records to resolve each message's outcome when the Run ends. + +### Cancel the in-flight model call + +The loop above only sees a steering message once the current model call finishes and `hasInput()` is checked again, so the steer takes effect on the *next* response. To react immediately, pass the optional `onSteer` hook to `createRun()`. It fires the moment a steering message folds into the Run. The SDK does not interrupt the model call for you, so wire `onSteer` to abort the current model call; the loop then restarts with the steering message already in the conversation. + +Give each model call its own `AbortController`, abort it from `onSteer`, and pass a signal that fires on either run cancellation or a steer. The subtlety is that `onSteer` must abort *only* the model call, never the Run, and the loop has to tell a steer-interruption apart from a genuine cancel before deciding how to end: + + +```javascript +let stepAbort = new AbortController(); + +const run = session.createRun(invocation, { + signal: req.signal, + // Abort only the current model call, never run.abortSignal, so a steer + // stops the response without cancelling the Run. + onSteer: () => stepAbort.abort(), +}); + +while (run.view.hasOlder()) await run.view.loadOlder(); +await run.start(); + +let outcome; +while (run.hasInput()) { + stepAbort = new AbortController(); + const conversation = run.view.getMessages().map(({ message }) => message); + const result = streamText({ + model: anthropic('claude-sonnet-4-20250514'), + messages: await convertToModelMessages(conversation), + // Abort on a genuine cancel (run.abortSignal) OR a steer (stepAbort). + abortSignal: AbortSignal.any([run.abortSignal, stepAbort.signal]), + }); + + const pipeResult = await run.pipe(result.toUIMessageStream()); + + // run.pipe races only run.abortSignal, so a steer abort ends the stream + // cleanly (pipe reason 'complete') while a real cancel ends it 'cancelled'. + // A steer interrupted this iteration if the stream ended clean, this + // iteration's stepAbort fired, and the Run itself was not cancelled. + const steerInterrupted = + pipeResult.reason === 'complete' && + stepAbort.signal.aborted && + !run.abortSignal.aborted; + + if (steerInterrupted) { + // Don't pass an interrupted iteration to vercelRunOutcome: an aborted + // streamText rejects finishReason abort-like, which maps to 'cancelled' and + // would end the Run. Swallow the rejection so it isn't unhandled, then loop. + // hasInput() is now true, so the next iteration re-infers with the steer at + // the tail of the conversation. + result.finishReason.catch(() => {}); + continue; + } + + outcome = await vercelRunOutcome(pipeResult, result.finishReason); + if (outcome.reason !== 'complete') break; +} + +if (outcome?.reason === 'suspend') { + await run.suspend(); +} else { + await run.end(outcome ?? { reason: 'complete' }); +} +await session.end(); +``` + + +When `onSteer` aborts the model call, `run.pipe()` returns with the partial output published, `run.hasInput()` returns `true` because the steer has folded into the conversation, and the next iteration streams a fresh response that includes it. `onSteer` takes no arguments and returns `void`; keep it synchronous and cheap, since it runs on the message-handling path. + + + +## Implement cancel and re-prompt + +To replace the in-flight Run with a fresh one, cancel active Runs and then send the new message: + + +```javascript +import { useClientSession, useView } from '@ably/ai-transport/react'; + +function Chat() { + const { session } = useClientSession(); + const view = useView(); + + const handleSend = async (text) => { + const active = view.runs().filter((run) => run.status === 'active'); + await Promise.all(active.map((run) => session.cancel(run.runId))); + + await view.send(UIMessageCodec.createUserMessage({ + id: crypto.randomUUID(), + role: 'user', + parts: [{ type: 'text', text }], + })); + }; +} +``` + + +`session.cancel(runId)` publishes a cancel signal. The agent's `Run.abortSignal` fires, the LLM stream stops, and the Run ends with reason `'cancelled'`. The new message starts a clean Run. + +## Implement send-alongside + +Send a new message without cancelling. Both Runs stream concurrently; each `ClientRun` carries its own `runId` and `cancel()`, and both Runs' outputs land on the same channel where they fold into the conversation tree by `runId`: + + +```javascript +const followUp = await session.view.send(UIMessageCodec.createUserMessage({ + id: crypto.randomUUID(), + role: 'user', + parts: [{ type: 'text', text: 'Also include vegan options.' }], +})); +``` + + +Cancel them independently with `session.cancel(specificRunId)` or `activeRun.cancel()`. + +## Drive the Stop / Send toggle + +`session.view.runs()` returns the visible `RunInfo` snapshots. Filter by `status === 'active'` to drive the Send / Stop toggle: + + +```javascript +const { session } = useClientSession(); +const isStreaming = session.view.runs().some((run) => run.status === 'active'); + +return isStreaming + ? cancelAll()} /> + : ; +``` + + +## Edge cases and unhappy paths + +- If a steering message arrives just as the agent finishes its last `pipe()`, `outcome` resolves to `consumed: false`. This is normal under load. Show the user that the message didn't reach the agent rather than dropping it silently. +- Once a Run has ended, `activeRun.steer()` rejects both promises immediately and publishes nothing. Only allow steering while `RunInfo.status === 'active'`. +- If a Run is `'suspended'` (waiting on a tool approval, for example), a steering message the agent has not yet consumed keeps its `outcome` pending until the Run resumes and consumes it or ends. A message the agent already consumed has resolved as `consumed: true`. Don't block UI updates on a pending outcome. +- `consumed: true` means the agent saw the steering message before it sent its response. It doesn't mean the LLM acted on it; that depends on the model, the system prompt, and what the agent passed in. +- Cancel is asynchronous. A small tail of tokens arrives after `session.cancel(runId)` returns and before the agent's `abortSignal` fires. The view emits them on the cancelled Run; treat them as belonging to that Run, not the new one. +- A Run cancelled while awaiting a tool result ends with `'cancelled'`. Tool calls triggered before the cancel may still run on your server unless your handler honours the same `AbortSignal`. +- A network drop on the client cancels nothing. The agent keeps streaming into the session. When the client reconnects, the response is still there. Check `session.view.runs()` on reconnect to decide whether to show Stop. +- Send-alongside is rate-limited by your channel and any server-side concurrency you enforce. See [concurrent turns](/docs/ai-transport/features/concurrent-turns). +- `'cancelled'` is reported on the Run's terminal `RunEndReason`. Don't infer cancellation from the absence of further tokens; subscribe to `view.on('run', ...)` and read the lifecycle event. + +## FAQ + +### When should I steer instead of cancel? + +Steer when the existing direction is still useful and the user wants to refine it ("make it 5 days", "also include vegan options"). The agent keeps its tokens, its tool results, and its reasoning so far; the new input joins the conversation. Cancel when the existing direction is wrong and the user wants to start over. + +### What does `consumed: false` mean for the user? + +The Run ended before the agent saw the steering message. Two common causes: the message arrived after the agent's final `pipe()`, or the Run was cancelled or errored first. It's up to your application whether to re-submit the steering message as a new user prompt for a new Run, or to drop it. + +### Can the agent ignore a steering message? + +Yes. `consumed: true` only means the agent saw the message, not that the LLM acted on it. Whether the model heeds new mid-flight input depends on the model, the system prompt, and what the agent passed in. + +### Can I cancel one of several concurrent Runs without touching the others? + +Yes. `session.cancel(runId)` only cancels the matching Run. Pull `runId`s from `session.view.runs()` to target the right one. + +### Does cancel work from a different device? + +Yes. Cancel is a signal on the shared channel, so any client with publish capability can cancel a Run, not just the device that started it. The agent's `onCancel` hook decides whether to honour it; the default accepts every request. Scope it to the Run owner if a user should only cancel their own turns. See [cancellation](/docs/ai-transport/features/cancellation#authorization) for the authorisation pattern. + +### Do steering messages persist in the conversation? + +Yes. A steering message joins the conversation tree under the Run it steers, so every connected client sees it live and it survives reload. On rehydration `run.view.getMessages()` returns it in order alongside the original prompt and the agent's output, so the model sees the full exchange on the next turn. + +### What happens if I send several steering messages at once? + +The next `run.hasInput()` call drains all pending steers together, and the following `pipe()` stamps every one of their codec-message-ids, resolving each message's `outcome` as `consumed`. The agent folds them all into a single model call rather than replying to each separately. + +### What if the agent endpoint is down when I send the follow-up? + +The follow-up is published to the channel before your agent endpoint is called, so it is durable regardless of the endpoint's health. For a steering message that is all that is needed: the already-running agent picks it up on its next `hasInput()` check. For a new Run (cancel-and-reprompt or send-alongside) the client also POSTs to wake the agent; if that POST fails, the user input still sits on the channel, so retry the POST once the endpoint recovers and the agent drains history to find the trigger and start the Run. + +## Related features + +- [Cancellation](/docs/ai-transport/features/cancellation): cancel signals and agent-side authorisation. +- [Concurrent turns](/docs/ai-transport/features/concurrent-turns): multiple Runs in flight on the same session. +- [Runs](/docs/ai-transport/concepts/runs#steer): how steering fits the Run model. diff --git a/src/pages/docs/ai-transport/features/interruption.mdx b/src/pages/docs/ai-transport/features/interruption.mdx deleted file mode 100644 index 2594beb68c..0000000000 --- a/src/pages/docs/ai-transport/features/interruption.mdx +++ /dev/null @@ -1,133 +0,0 @@ ---- -title: "Interruption" -meta_description: "Let users interrupt AI agents mid-stream with Ably AI Transport. Cancel-then-send and send-alongside patterns for responsive AI interactions." -meta_keywords: "AI interruption, interrupt, cancel and send, concurrent turns, AI Transport, Ably" -intro: "Your users can change direction mid-response. AI Transport's session layer lets a client cancel the in-progress Run and start a new one without breaking the stream." ---- - -Interruption is the pattern where a user sends a new message while the agent is still streaming a response. The new message creates a new turn on the [session](/docs/ai-transport/concepts/sessions); whether the existing turn is cancelled first or runs in parallel is up to you. - -![Diagram showing session continuity during interruption](../../../../images/content/diagrams/ait-session-continuity.png) - -A minimal interruption cancels the active Run and sends a new message: - - -```javascript -const active = session.view.runs().filter((run) => run.status === 'active'); -await Promise.all(active.map((run) => session.cancel(run.runId))); - -await session.view.send(UIMessageCodec.createUserMessage({ - id: crypto.randomUUID(), - role: 'user', - parts: [{ type: 'text', text: 'Wait, make it 5 days.' }], -})); -``` - - -## How it works - -Each turn becomes a [Run](/docs/ai-transport/concepts/runs) on the session. The session is bidirectional, so a client can publish a cancel signal at any time, even while the response is mid-flight. A new `send` doesn't have to wait for the previous Run to finish. - -Two patterns: - -| Pattern | Behaviour | When to use it | -| --- | --- | --- | -| Cancel-then-send | Cancel the active Run, then send the new message. | A Stop button followed by a new prompt. | -| Send-alongside | Send the new message while the active Run continues. | Quick follow-up that doesn't invalidate the in-flight response. | - -With cancel-then-send, the active Run aborts before the new message dispatches. The agent stops generating, ends the Run with reason `'cancelled'`, and the new Run starts clean. With send-alongside, both Runs run concurrently (each with its own stream and its own cancel handle). See [concurrent turns](/docs/ai-transport/features/concurrent-turns) for the multiplexing model. - -## Implement cancel-then-send - -Detect whether a Run is active, cancel it, then send the new message: - - -```javascript -import { useClientSession, useView } from '@ably/ai-transport/react'; - -function Chat() { - const { session } = useClientSession(); - const view = useView(); - - const handleSend = async (text) => { - const active = view.runs().filter((run) => run.status === 'active'); - await Promise.all(active.map((run) => session.cancel(run.runId))); - - await view.send(UIMessageCodec.createUserMessage({ - id: crypto.randomUUID(), - role: 'user', - parts: [{ type: 'text', text }], - })); - }; -} -``` - - -`session.cancel(runId)` publishes a cancel signal. The agent's `Run.abortSignal` fires, the LLM stream stops, and the Run ends with reason `'cancelled'`. The new message starts a clean Run. - -## Implement send-alongside - -Send a new message without cancelling. Both Runs stream concurrently; each `ClientRun` carries its own `runId` and `cancel()`, and both Runs' outputs land on the same channel where they fold into the conversation tree by `runId`: - - -```javascript -const followUp = await session.view.send(UIMessageCodec.createUserMessage({ - id: crypto.randomUUID(), - role: 'user', - parts: [{ type: 'text', text: 'Also include vegan options.' }], -})); -``` - - -Cancel them independently with `session.cancel(specificRunId)` or `clientRun.cancel()`. - -## Drive the Stop / Send toggle - -`session.view.runs()` returns the visible `RunInfo` snapshots. Filter by `status === 'active'` to drive the Send / Stop toggle: - - -```javascript -const { session } = useClientSession(); -const isStreaming = session.view.runs().some((run) => run.status === 'active'); - -return isStreaming - ? cancelAll()} /> - : ; -``` - - -## Edge cases and unhappy paths - -- Cancel is asynchronous. A small tail of tokens arrives after `session.cancel(runId)` returns and before the agent's `abortSignal` fires. The view emits them on the cancelled Run; treat them as belonging to that Run, not the new one. -- A Run cancelled while awaiting a tool result ends with `'cancelled'`. Tool calls triggered before the cancel may still run on your server unless your handler honours the same `AbortSignal`. -- A network drop on the client cancels nothing. The agent keeps streaming into the session. When the client reconnects, the response is still there. Check `session.view.runs()` on reconnect to decide whether to show Stop. -- Send-alongside is rate-limited by your channel and any server-side concurrency you enforce. See [concurrent turns](/docs/ai-transport/features/concurrent-turns). -- `'cancelled'` is reported on the Run's terminal `RunEndReason`. Don't infer cancellation from the absence of further tokens; subscribe to `view.on('run', ...)` and read the lifecycle event. - -## FAQ - -### What happens to tokens already in flight when I cancel? - -The agent keeps publishing until its `abortSignal` fires. A small tail of tokens arrives after `cancel()` returns. The view reflects them on the cancelled Run; the new Run is unaffected. - -### Can I cancel one of several concurrent Runs without touching the others? - -Yes. `session.cancel(runId)` only cancels the matching Run. Pull `runId`s from `session.view.runs()` to target the right one. - -### Does cancel work from a different device? - -Yes. Any client connected to the same session can publish a cancel. The agent receives it through its own subscription regardless of which client started the Run. See [cancellation](/docs/ai-transport/features/cancellation) for the authorisation model. - -### What if the agent endpoint is down when I send the follow-up? - -The Ably channel publishes still happen (the cancel and the new user message both land on the channel). The new POST retries the agent endpoint; the new Run starts as soon as the endpoint is reachable. The session and existing messages are unaffected. - -### How is interruption different from concurrent turns? - -Interruption is the user-facing pattern (Stop and re-prompt, or quick follow-up). [Concurrent turns](/docs/ai-transport/features/concurrent-turns) is the underlying mechanism that makes both possible: multiple Runs coexisting on the same session. - -## Related features - -- [Cancellation](/docs/ai-transport/features/cancellation): cancel signals and agent-side authorisation. -- [Concurrent turns](/docs/ai-transport/features/concurrent-turns): multiple Runs in flight on the same session. -- [Double texting](/docs/ai-transport/features/double-texting): handling multiple user messages in quick succession. diff --git a/src/pages/docs/ai-transport/roadmap.mdx b/src/pages/docs/ai-transport/roadmap.mdx index ddbbad96ae..af65cc5931 100644 --- a/src/pages/docs/ai-transport/roadmap.mdx +++ b/src/pages/docs/ai-transport/roadmap.mdx @@ -17,7 +17,7 @@ Available today. | [Connection resilience](/docs/ai-transport/features/reconnection-and-recovery) | Realtime delivery over WebSocket with automatic fallback to HTTP streaming and long-polling, so streams survive proxies, firewalls, and unreliable networks. | | [Multi-device session continuity](/docs/ai-transport/features/multi-device) | The same session opens on any device the user picks up, with the full conversation and current state. | | [Persistent history and replay](/docs/ai-transport/features/history) | Conversations survive disconnects and stay available as history, for resumption, audit, and review. | -| Bidirectional control | Any client signals the agent through the session, so a user can [cancel generation](/docs/ai-transport/features/cancellation), [interrupt](/docs/ai-transport/features/interruption), [send again while the agent responds](/docs/ai-transport/features/double-texting), and keep interacting rather than only consuming a one-way stream. | +| Bidirectional control | Any client signals the agent through the session, so a user can [cancel generation](/docs/ai-transport/features/cancellation), [interrupt](/docs/ai-transport/features/interruption-and-steering), [send again while the agent responds](/docs/ai-transport/features/double-texting), and keep interacting rather than only consuming a one-way stream. | | [Conversation branching](/docs/ai-transport/features/branching) | Edit and regenerate messages, with each branch kept in the session tree. | | Client-side tool calls and approval gates | Call [client tools](/docs/ai-transport/features/tool-calling) over the session, with [human-in-the-loop](/docs/ai-transport/features/human-in-the-loop) approval where you need it. | | Drop-in framework integration | A drop-in transport for the [Vercel AI SDK](/docs/ai-transport/getting-started/vercel-ai-sdk), plus the [Core SDK](/docs/ai-transport/getting-started/core-sdk) for everything else. | diff --git a/src/pages/docs/ai-transport/why/index.mdx b/src/pages/docs/ai-transport/why/index.mdx index 4758060f5b..d91be0c375 100644 --- a/src/pages/docs/ai-transport/why/index.mdx +++ b/src/pages/docs/ai-transport/why/index.mdx @@ -48,7 +48,7 @@ Three capabilities define what the layer is for: - Durable streaming: token streams persist, accumulate, and resume. Reconnecting clients receive assembled state, not a replay of every token. Disconnects become a non-event for your users. See [token streaming](/docs/ai-transport/features/token-streaming) and [reconnection and recovery](/docs/ai-transport/features/reconnection-and-recovery). - Session continuity: the session follows the user, not the connection. Multi-device, multi-user, multi-surface. Users switch devices; the session continues with full state. Agents hand off to humans without losing context. See [multi-device sessions](/docs/ai-transport/features/multi-device) and [human-in-the-loop](/docs/ai-transport/features/human-in-the-loop). -- Visibility and control: bidirectional. Cancel, interrupt, and steer mid-response. Push to users when they are offline. The session is stateful, not just a message pipe. See [agent presence](/docs/ai-transport/features/agent-presence), [interruption](/docs/ai-transport/features/interruption), and [push notifications](/docs/ai-transport/features/push-notifications). +- Visibility and control: bidirectional. Cancel, interrupt, and steer mid-response. Push to users when they are offline. The session is stateful, not just a message pipe. See [agent presence](/docs/ai-transport/features/agent-presence), [interruption and steering](/docs/ai-transport/features/interruption-and-steering), and [push notifications](/docs/ai-transport/features/push-notifications). The shape this changes for you: