Agentic Loops

UserTold.ai is a structured research pipeline. This page documents the full technical flow — from interview setup through evidence extraction, evidence review packet creation, verified work handoff, and recurrence review. API contracts are published at /api/openapi; model IDs are loaded from the runtime model registry and are described here by capability rather than by provider snapshot.

1. Interview

Interview modes

Each study defines a sequence of segments, and each segment has a mode:

• talk — AI-conducted conversational interview. The conductor asks questions, listens, and follows up based on participant responses.
• speak — AI delivers a scripted transition message out loud (TTS). Used for introductions, participant task instructions, transitions, and thanks.
• observe — Screen recording + voice capture while the participant completes a task. The conductor stays silent and preserves stuck moments as evidence for later interpretation and debrief.

How an interview runs

1. Participant opens the widget (embedded on your product or via an intake link).
2. Widget connects to the interview endpoint. Conductor Durable Object initializes.
3. Each segment runs in sequence: mode switches, prompts fire, screen/audio records.
4. For observe segments, deterministic checks (timeout, URL change, participant action, or user Done) advance the script without live coaching. Goals guide analysis and planned debriefs, not observe-mode advancement.
5. The interview ends when all segments complete or participant exits.
6. The interview record is written with status completed.

STS model

Voice interviews use the configured realtime speech-to-speech models. The system prompt is constructed per-segment from the study script. The model speaks with the participant, captures audio, and the transcript is assembled from message events.

Recording

Audio chunks stream from the browser during the interview. Screen recordings are captured and merged server-side after completion. Both are stored in R2 with signed URL access.

---

2. Evidence Extraction

After an interview completes, a queue message (signals.extract) triggers the extraction pipeline.

Extraction steps

1. Transcription — Audio is processed by the configured transcription models and returned as timed segments.
2. Enriched timeline — Transcript segments are merged with interview events (mode changes, navigation, observed actions, and deterministic advancement events) into a unified enriched timeline.
3. Evidence extraction — The configured extraction models analyze the transcript against the study goals and segment context. For each evidence card found, they output:
   • signal_type — one of the values in the table below
   • quote — verbatim text from the transcript
   • confidence — float 0–1
   • intensity — float 0–1 (how strongly expressed)
   • timestamp_ms — position in the interview recording
   • page_url — page URL at the time (when available for observe segments)

Evidence types

signal_type	Definition
struggling_moment	User hits friction, fails a task, or expresses confusion
desired_outcome	User states what they want to accomplish
hiring_criteria	Why the user chose your product (or a competitor)
firing_moment	What would make the user stop using your product
workaround	User invents a substitute behavior to work around a gap
emotional_response	Strong positive or negative reaction
critical_error	Blocking in-product failure observed during the interview
recovery_success	User got unstuck — marks where the product already helps
smooth_completion	User completed a task with no friction (positive evidence)
no_issue_found	Analyzer ran but found no extractable evidence in this window
decision_point	User weighed alternatives or hesitated before committing

This is the canonical enum; the same list is used by REST (?type=), CLI (evidence list --type), and MCP (evidence.list).

Evidence JSON shape

{
  "id": "sig_abc123",
  "signal_type": "struggling_moment",
  "quote": "I tried this flow three times and still cannot find where to change billing settings.",
  "confidence": 0.91,
  "intensity": 0.8,
  "session_id": "ses_xyz789",
  "timestamp_ms": 142300,
  "page_url": "/checkout/step-3"
}

---

3. Evidence Review Packets

Evidence review packets are grouped from evidence clusters. They preserve source context for triage; they are not delivery instructions until a project-aware human or agent verifies the packet and promotes it to work.

From evidence to review packets

usertold work create-from-evidence <projectRef> --title "Users cannot finish onboarding" --evidence sig_1,sig_2 --json

Or via MCP:

work.create_from_evidence

The review packet creation process:

1. Groups related evidence by user experience pattern.
2. Generates a neutral title and description grounded in the evidence quotes.
3. Links back to each contributing evidence card (and its interview).
4. Stores the packet for review with status: backlog.
5. Leaves promotion to delivery work to a project-aware verification step.

Verified work item → Tracker Issue

usertold work push <projectRef> <workId> --json

Or via MCP:

work.push

The intended push workflow:

1. Is intended for verified work items, not unreviewed packets.
2. Authenticates with the configured provider.
3. Creates an issue with a structured body based on the work item description and linked evidence; provider-specific bodies may add review prompts, confidence summaries, source links, and related-task context.
4. Stores the issue URL back on the work item record.
5. Returns the issue URL in the JSON response.

---

4. Linear Completion and Recurrence

After your agent ships a fix, keep the linked delivery issue connected to the evidence that justified it.

How the completion loop works

1. Push an evidence-backed work item to Linear.
2. Complete the Linear issue when the team ships or decides the work is done.
3. UserTold syncs the completion state back to the work item and resolves the current linked evidence.
4. Future interviews are still processed for similar evidence.
5. Similar future evidence can resurface as possible recurrence or a related issue with confidence.

This does not prove a shipped fix worked or attribute business impact. It keeps the current evidence cleanly resolved while preserving a path for recurrence review.

---

5. Full Data Model

Key entities and their relationships:

Project → has many Studies, Interviews, Evidence cards, Work items

Study → defines the interview structure: goals plus ordered Talk, Observe, and Speak segments.

Interview → one participant interview. Lifecycle status: pending, active, completed, abandoned, or error; processing status is tracked separately for result generation and recovery. Linked to a Study.

Evidence card → extracted observation from an Interview. Linked to an Interview, timestamped, typed.

Work item → evidence-backed. Linked to one or more evidence cards. It may start as an evidence review packet and should only be pushed to GitHub Issues or Linear after project-aware verification.

IntakeLink → public entry point that routes participants into a Study.

---

CLI Quick Reference

# Setup
usertold init --org acme --json --yes

# Studies
usertold study list <projectRef> --json
usertold study create <projectRef> --title "Checkout study" --activate --json

# Interviews
usertold interview list <projectRef> --json
usertold interview reprocess <projectRef> <interviewId>

# Evidence
usertold evidence list <projectRef> --json

# Review packets and verified work
usertold work create-from-evidence <projectRef> --title "Users cannot finish onboarding" --evidence sig_1,sig_2 --json
usertold work push <projectRef> <workId> --json

---

MCP Tool Reference

Base endpoint: POST https://mcp.usertold.ai/mcp

Tool domains: studies.*, interviews.*, evidence.*, work.*, projects.*, intake.*

Key tools:

• projects.get_context — Project planning snapshot: evidence distribution, work item status, active studies, coverage gaps
• studies.create — Create a study idempotently from a script
• evidence.list — List evidence with filters (type, interview, confidence threshold)
• work.create_from_evidence — Create evidence review packets from evidence clusters
• work.push — Push work items to GitHub Issues or Linear after verification
• Linear completion sync — Resolve current linked evidence and watch future interviews for possible recurrence

---

Get started · For agents · CLI reference