Agentic Loops

This page documents the runtime and automation boundaries behind UserTold.ai. Core Concepts is the authoritative entity and lifecycle model. Exact REST contracts are published at /api/openapi; model IDs come from the runtime model registry and are described here by capability rather than by provider snapshot.

1. Interview

How an interview runs

  1. Participant opens the widget (embedded on your product or via an intake link).
  2. Widget connects to the interview endpoint. Conductor Durable Object initializes.
  3. Each segment runs in sequence: mode switches, prompts fire, screen/audio records.
  4. For observe segments, deterministic checks (timeout, URL change, participant action, or user Done) advance the script without live coaching. Goals guide analysis and planned debriefs, not observe-mode advancement.
  5. The interview ends when all segments complete or participant exits.
  6. The interview record is written with status completed.

STS model

Voice interviews use the configured realtime speech-to-speech models. The system prompt is constructed per-segment from the study script. The model speaks with the participant, captures audio, and the transcript is assembled from message events.

Recording

Audio chunks stream from the browser during the interview. Screen recordings are captured and merged server-side after completion. Both are stored in R2 with signed URL access.

2. Evidence Extraction

After an interview completes, a queue message (signals.extract) triggers the extraction pipeline.

Extraction steps

  1. Transcription — Audio is processed by the configured transcription models and returned as timed segments.
  2. Enriched timeline — Transcript segments are merged with interview events (mode changes, navigation, observed actions, and deterministic advancement events) into a unified enriched timeline.
  3. Evidence extraction — The configured extraction models analyze the transcript against the study goals and segment context. For each evidence card found, they output:
    • signal_type — one of the values in the canonical evidence vocabulary
    • quote — verbatim text from the transcript
    • confidence — float 0–1
    • intensity — float 0–1 (how strongly expressed)
    • timestamp_ms — position in the interview recording
    • page_url — page URL at the time (when available for observe segments)

The canonical evidence vocabulary is defined in Core Concepts and is shared by the dashboard, REST, CLI, and MCP surfaces.

3. Work Automation

The product-model boundary remains important in automation: grouping creates review packets, not delivery instructions. A project-aware human or agent verifies a packet before promoting it to work.

From evidence to review packets

The review packet creation process:

  1. Groups related evidence by user experience pattern.
  2. Generates a neutral title and description grounded in the evidence quotes.
  3. Links back to each contributing evidence card (and its interview).
  4. Stores the packet for review with status: backlog.
  5. Leaves promotion to delivery work to a project-aware verification step.

Verified work item → Tracker Issue

The intended push workflow:

  1. Is intended for verified work items, not unreviewed packets.
  2. Authenticates with the configured provider.
  3. Creates an issue with a structured body based on the work item description and linked evidence; provider-specific bodies may add review prompts, confidence summaries, source links, and related-task context.
  4. Stores the issue URL back on the work item record.
  5. Returns the issue URL in the JSON response.

4. Linear Sync

The Linear integration stores the provider issue URL on the work item. Completion sync updates the work item and resolves its current linked evidence; extraction continues to process later interviews so similar evidence can be reviewed as possible recurrence. GitHub handoff creates a linked issue but does not provide the Linear-specific completion sync.

Continue with the Quickstart for a first interview or For Agents to choose an agent surface. CLI and MCP describe their current surfaces through built-in discovery.