AI Module Deep Dive

Read Core Mental Model first if you are new to RED’s AI module. This page is the deeper reference for engineers who need to understand how the engine behaves.

What the Module Provides

The AI module turns chat into persisted application state:

• aiThreads store conversations.
• aiMessages and aiMessageParts store the UI transcript.
• aiRuns store active run state.
• aiCheckpointParts store provider-shaped model history.
• tasks and taskEvents store tool-backed work.
• Boundaries provide shared workspace memory, members, and default agent selection.

The browser subscribes through Convex reactive queries. The engine writes rows as tokens, tool calls, tool results, task progress, and errors arrive.

Engine Flow

sendMessage
  -> append user message
  -> create or wake aiRun
  -> schedule driveRun

driveRun
  -> prepareRun: queued -> running
  -> resolve agent by agentKey
  -> resolve agent context through resolveContext
  -> load checkpoint messages
  -> stream the ToolLoopAgent
  -> drain stream into aiMessageParts
  -> write new checkpoint parts
  -> complete, requeue, or suspend for async tasks

pollWaitingTasks
  -> read taskEvents since the last cursor
  -> publish progress as status parts
  -> settle terminal tasks
  -> requeue driveRun when needed

The core engine is intentionally agent-agnostic. It knows about threads, runs, checkpoints, boundaries, tasks, and messages. Product-specific IDs flow through agent context.

Two Storage Planes

RED keeps UI transcript and model checkpoint separate.

UI transcript

• aiMessages
• aiMessageParts

This is what the frontend renders. It can include token chunks, tool calls, tool results, status parts, and errors.

Provider checkpoint

• aiCheckpointParts
• aiRuns.checkpointIndex

This is what the next driveRun rehydrates into model messages. Async settlements rewrite sentinel rows in aiCheckpointParts, not in aiRuns.

That separation lets RED display live progress while keeping the model’s next turn clean.

Agent Definitions

Agents live in packages/backend/src/convex/modules/ai/agents/.

An agent declares:

• key
• instructions
• modelDefault
• tools
• optional resolveContext
• optional stopWhen
• optional composeUserAttachments
• optional composeSystemPrompt
• optional postCompletionHook
• optional maxStepsPerTurn
• optional showReasoning

The default assistant is assistantAgent. It resolves the brief bound to the current thread and exposes it to tools as ToolContext.extras.briefId.

The four optional hooks at the bottom of the list are the engine’s extension surface — see Engine Extension Hooks below.

Engine Extension Hooks

The engine stays agent-agnostic by routing per-turn decisions through optional hooks on the AgentDefinition. Each hook is pure — the engine owns persistence, retries, and safety caps. Agents that omit a hook get the engine default.

composeSystemPrompt

Owns final system-prompt assembly. The engine gathers per-turn context (boundary memory, pending tasks, extras, agent state, the full run document) and the agent returns { stable, volatile? }.

• stable is the system block. Keep it byte-identical across turns to stay in the Anthropic prompt cache.
• volatile is injected as a <context_update> user message AFTER the committed-history checkpoint, so per-turn freshness (pending tasks, TODO state, surface snapshot) doesn’t bust the cache.

import { defineAgent, renderTaskSummary } from "../core/types"

composeSystemPrompt: ({ baseInstructions, boundaryMemory, pendingTasks, agentState }) => ({
  stable: `${baseInstructions}${boundaryMemory ? `\n\nWorkspace memory:\n${boundaryMemory}` : ""}`,
  volatile: pendingTasks.length > 0
    ? `Active background tasks:\n${renderTaskSummary(pendingTasks)}`
    : undefined,
})

When unset, the engine falls back to defaultComposeSystemPrompt, which renders instructions + boundary + tasks + extras in a fixed order.

postCompletionHook

Fires after a turn cleanly closes (finishReason of stop or length). The hook decides whether to inject a hidden “supervisor reminder” and requeue the run. Return null to leave the run completed.

postCompletionHook: ({ run, finishReason, turnText, turnToolCallCount }) => {
  const state = run.agentState as { pendingTodos?: string[] } | undefined
  if (!state?.pendingTodos?.length) return null
  return {
    directive: `You still have ${state.pendingTodos.length} unfinished TODO(s). Continue.`,
    counterKey: "todoResumptions",
    maxRetries: 4,
  }
}

When the hook returns a directive the engine:

1. Inserts a hidden aiMessages row (hidden: true) with the directive as user content.
2. Increments the named counter (forcedContinuations or todoResumptions).
3. Flips status back to queued and schedules driveRun.
4. Clamps total uses at min(maxRetries, 16) — ABSOLUTE_MAX_CONTINUATION_RETRIES.

Hidden messages are filtered from listMessages (the UI transcript) but replayed to the model via checkpoint. Both counters reset to 0 on every real user sendMessage, so the retry budget is per user-turn, not per thread.

maxStepsPerTurn

Per-turn cap on the inner ToolLoopAgent.stream. Defaults to 8; the engine clamps any request to a hard ceiling of 24 (ABSOLUTE_MAX_AGENT_STEPS_PER_TURN). When the cap fires mid-thought (finishReason === "tool-calls") the engine routes through requeueAfterStepCap — the run goes back to queued instead of completed, so the next drive can finish the thought. Independent of MAX_RUN_LOOP_STEPS (also 24), which bounds total prepareRun invocations per run.

showReasoning

When true, provider reasoning/thinking parts are forwarded to the UI transcript. Defaults to false. Observability captures reasoning regardless of this flag.

agentState

aiRuns.agentState is an opaque v.any() slot the engine never inspects. Agents persist whatever they need between turns (TODO lists, planning scratchpads, conversation state machines) and read it back through composeSystemPrompt and postCompletionHook. Source-of-truth product data still belongs in your own tables.

Tool Context

Tool bodies receive the AI SDK execution options. RED’s context is in options.experimental_context:

const toolContext = options.experimental_context as ToolContext<MyContext> | undefined
if (!toolContext) throw new Error("Missing tool context")

ToolContext includes:

• ctx — Convex action context.
• runId
• threadId
• organizationId
• startedBy
• agentKey
• attachments
• attachmentsByUserMessageId
• extras — the agent-specific payload from resolveContext.

Feature-specific values belong in extras. Do not add product fields to the core engine.

Tool Modes

RED has three helper APIs and four common modes.

Direct Sync

defineSyncTool() wraps the AI SDK tool helper and adds a per-tool timeout. Use it for fast work with no task row.

Sync Internal Task

defineSyncInternalTaskTool() runs an internal task node inline and optionally persists task history. The node must be internal and set inlineExecution: "sync".

Async Internal Task

defineAsyncTool() starts a task with startTaskForAi. If the node is internal, dispatchTask calls the node’s run(ctx, input, { task }).

The Brief export is an async internal task:

exportBrief
  -> startTaskForAi("tasks.briefs.export")
  -> briefExportNodeDefinition.kind = "internal"
  -> runBriefExport schedules progress events
  -> finalizeBriefExport writes the Markdown artifact

It returns blocking: false, so completion arrives later as a task_event.

Async External Task

External nodes implement trigger(ctx, task, { handle, handleUrl }). The trigger calls a remote service and gives it handleUrl.

The remote service posts events to:

POST /api/tasks/{handle}/event

The HTTP route validates the handle, appends the task event, and terminal events finalize the task.

Blocking and Non-Blocking Async Tools

Async tools return a sentinel with:

• taskIds
• summary
• pollIntervalMs
• blocking

blocking: true means the agent needs the result before it can continue. The run waits in waiting_tasks. When the task settles, the checkpoint is rewritten with the real tool result and the run is queued again.

blocking: false means the agent can keep talking while work continues. The run moves to awaiting_background. When the task settles, RED creates a task_event message and queues the run so the agent can react.

This policy is independent of node kind. Internal tasks can be non-blocking, as the Brief export demonstrates.

Task Nodes

Task node definitions live under packages/backend/src/convex/modules/tasks/node_types/.

Internal nodes use run:

export const myNodeDefinition = {
  key: "tasks.example.my_node",
  name: "Example Node",
  description: "Does internal work.",
  kind: "internal" as const,
  scope: "member" as const,
  inputSchema,
  outputSchema,
  run,
}

External nodes use trigger:

export const myExternalNodeDefinition = {
  key: "tasks.example.remote",
  name: "Remote Node",
  description: "Delegates work to a remote service.",
  kind: "external" as const,
  scope: "member" as const,
  inputSchema,
  outputSchema,
  trigger,
}

Register built-in product nodes in packages/backend/src/convex/modules/tasks/node_types/registry.ts. The engine uses tasks_engine.ts to look up and dispatch those definitions.

Task Events

Supported event types:

• started
• progress
• heartbeat
• success
• error
• cancelled
• custom

Progress events are rendered as status parts in the originating assistant bubble. Success and error events settle the async sentinel. For non-blocking tasks, terminal settlement also creates a task_event message.

Brief Assistant Reference

The included assistant uses these real files:

• modules/ai/agents/assistant.agent.ts — instructions, tools, resolveContext, and attachment composition.
• modules/ai/tools/getBriefSections.tool.ts — reads current sections.
• modules/ai/tools/patchBrief.tool.ts — edits one section.
• modules/ai/tools/addBriefSection.tool.ts — creates a section.
• modules/ai/tools/exportBrief.tool.ts — starts the non-blocking Markdown export task.
• modules/tasks/node_types/brief_export_node.ts — internal node that emits progress and writes the artifact.
• modules/tasks/node_types/registry.ts — registers the Brief export node.
• http.ts — exposes the external task callback route for external nodes.

Add an Agent

1. Add a file in modules/ai/agents/.
2. Add the key to packages/backend-contract/src/ai/agents.ts.
3. Register the agent in modules/ai/agents/agents.registry.ts.
4. Create threads with that agentKey.

Agent selection is locked at thread creation because the checkpoint stores tool names from that agent’s tool map.

Add a Tool

1. Pick the right helper: defineSyncTool, defineSyncInternalTaskTool, or defineAsyncTool.
2. Read RED context from options.experimental_context.
3. Keep org-scoped checks close to document reads and writes.
4. For async work, start a task with tasks_internal.startTaskForAi.

Add a Task Node

1. Add a node file under modules/tasks/node_types/.
2. Export input and output schemas.
3. Export an internal run or external trigger.
4. Register it in node_types/registry.ts.
5. Run bunx convex dev --once after backend changes.

Observability

The engine captures per-step and per-tool-call traces for every run into three tables under packages/backend/src/convex/modules/ai/ai_observability_*:

• aiObservabilitySteps — one row per LLM step: model, finish reason, duration, token usage, cost.
• aiObservabilityToolCalls — one row per tool call: name, input/output preview (truncated to 8 kB), status, duration, watched task IDs.
• aiObservabilitySummaries — one row per run with rollups (token totals, cost totals, error code, step/tool counts).

Step and tool-call rows expire 30 days after creation via expiresAt. Summaries persist with the run.

Per-run capture mode

aiRuns.observabilityCaptureMode toggles capture verbosity:

• "off" (default) — rollup summary only; step and tool-call rows are skipped.
• "debug" — full per-step + per-tool-call capture.

Set it when creating the run for sessions you want to inspect later (admin tooling, support escalations).

Cost & token tracking

Cost is parsed from the Vercel AI Gateway providerMetadata.gateway block on every step:

• totalCost, marketCost, gatewayCost, surchargeCost, inferenceCost — USD floats.
• generationId — pass to gateway.getGenerationInfo(id) for after-the-fact reconciliation.

Token usage tracks input/output/total plus cache reads/writes and reasoning tokens.

Structured run trace

traceRun(runId, phase, event, data) emits a grep-friendly line per lifecycle event:

[ai-run=jh7d…2k] phase=drive event=stream.start loopStep=1 messages=4 model=anthropic/claude-sonnet-4-6
[ai-run=jh7d…2k] phase=suspend event=waiting_tasks pendingAsync=1 hasBlocking=true
[ai-run=jh7d…2k] phase=poll event=events.published taskId=… appended=3
[ai-run=jh7d…2k] phase=settle event=queued completedTools=1 remainingPendingAsync=0

Tail with bunx convex logs. Set AI_TRACE=off in the Convex env to silence all trace lines. traceTask(taskId, phase, event, data) is the sibling helper for the tasks engine.

Operational Notes

• aiRuns.pendingAsync is the poll state for async work.
• sentinelResults tracks checkpoint tool-call IDs that need settlement.
• aiRuns.checkpointIndex is the commit marker for visible checkpoint parts.
• activeAssistantMessageId is a progress anchor, not a lock.
• task_event messages re-enter the model as user messages on the next turn.
• awaiting_approval is a non-terminal status: the run is paused on a tool-approval-request and resumes when every outstanding approval has been answered via respondToToolApproval. See the approval recipe in Building AI Features.
• aiRuns.agentState is an opaque per-agent slot — the engine never reads it. Agents own its shape via Engine Extension Hooks.
• aiMessages.hidden: true rows are filtered from listMessages but still replayed to the model via checkpoint. Used by the postCompletionHook continuation pattern.
• The engine repairs Haiku’s occasional stringified tool-call.input payloads on both the write path (normalizeToolCallInput in streamSink.ts) and the read path (sanitizeToolCallPart in checkpoint.ts). The fallback to {} emits a trace event so the quirk stays visible in production logs.

For implementation recipes, use Building AI Features.