Skip to content

Interview System

The interview system enables deep self-exploration through AI-conducted voice interviews. It manages structured prompts, stores transcripts, tracks domain coverage, and extracts identity-building artifacts like quotes and observations.

Why This Exists

Traditional introspection methods (journals, surveys, questionnaires) rarely go deep. Conversational AI interviewers (primarily Grok voice mode) can push back, ask follow-ups, and draw out insights that surface-level reflection misses.

Two distinct AI roles are at play, and a reimplementer should keep them separate:

Role Who Where
Prompt authoring (generate/revise/suggest) Claude, via ClaudeRunnerService.run_prompt() (scripts.claude.run_batch → local claude CLI / daemon) api/src/services/claude_runner.py
Conducting the interview External voice model (Grok voice mode) Manual — outside the system

The system solves three problems: 1. Preparation friction: Without good prompts, interviews stay shallow. Claude-generated prompts with personalized context produce better conversations. 2. Knowledge loss: Interview insights evaporate unless captured. The system extracts quotes, stories, and observations for permanent storage. 3. Coverage blindness: Without tracking, you revisit familiar topics while neglecting others. Domain coverage visualization reveals gaps.

System Architecture

+------------------+      +-------------------+      +-------------------+
|   Web UI         |      |   CLI             |      |   External AI     |
|   /interviews    |      |   /interview      |      |   (Grok Voice)    |
+--------+---------+      +--------+----------+      +--------+----------+
         |                         |                          |
         v                         v                          |
+--------+--------------------------+                         |
|           FastAPI Interviews Router                         |
|           /api/v1/interviews/*                              |
+--------+--------------------------+                         |
         |                                                    |
         v                                                    |
+--------+--------------------------+                         |
|           InterviewsService                                 |
|  - Prompt CRUD & generation                                 |
|  - Coverage calculation                                     |
|  - Transcript/quote/observation retrieval                   |
+--------+--------------------------+                         |
         |                                                    |
         v                                                    v
+--------+--------------------------+      +------------------+----------+
|        Data Layer                 |<-----|  Manual transcript import  |
|  data/interviews/                 |      |  (copy-paste from Grok)    |
|   prompts/*.json                  |      +-----------------------------+
|   transcripts/*.md                |
|   summaries/*.json                |
|  data/identity/                   |
|   interviews.jsonl (metadata)     |
|   quotes.jsonl                    |
|   observations.jsonl              |
|   stories.json                    |
+-----------------------------------+

Two Modes: Prepare and Review

The web UI (/interviews) has two modes:

Prepare Mode

  • View domain coverage heatmap (9 domains, sorted by gaps)
  • Request AI-suggested topics based on coverage gaps
  • Create new prompts from natural language ideas
  • Browse, expand, revise, and delete existing prompts
  • Copy rich prompt text to clipboard for pasting into Grok

Review Mode

  • Browse interview transcripts with chat-style reader
  • Filter and explore extracted quotes
  • View narrative stories
  • Browse identity observations by type

Prompt Schema

Each prompt is a JSON file in data/interviews/prompts/ with this structure:

{
  "v": 2,
  "id": "PHI-001",
  "title": "Your Relationship with Risk",
  "opener": "You've built a company, you skate hard at NN, you seem to have a non-standard relationship with risk. Walk me through how you actually think about risk...",
  "probes": [
    "What's the biggest risk you've ever taken where the expected value was probably negative?",
    "How has your risk tolerance changed over time?",
    "Is there a risk you didn't take that still haunts you?"
  ],
  "mode": "philosophical",
  "domain": "values_beliefs",
  "duration_min": 25,
  "duration_max": 50,
  "settings": ["walk", "desk"],
  "moods": ["contemplative", "curious"],
  "challenge_level": 3,
  "created": "2026-01-01T00:00:00Z",
  "last_explored": "2026-01-02T12:00:00Z",
  "status": "explored"
}

Field Reference

Field Type Description
v int Schema version. On-disk prompts are v: 2 (all four seed files), but the Pydantic PromptDetail.v default is 1 (api/src/schemas/interviews.py) — the model default lags the data. Harmless, but a reimplementer should match the on-disk value, not the default.
id string Unique ID: {MODE_PREFIX}-{NNN} (e.g., PHI-001, BIO-002)
title string Short title (5-10 words)
opener string Opening question to start the conversation
probes string[] Follow-up directions (typically 3-5)
mode enum philosophical, biographical, challenge, exploratory
domain enum One of 9 life domains (see below)
duration_min int Minimum recommended duration in minutes
duration_max int Maximum recommended duration in minutes
settings string[] Suitable contexts: walk, drive, desk
moods string[] Suitable mental states: reflective, contemplative, energized, curious
challenge_level int Difficulty 1-5 (higher = more emotionally demanding)
created ISO-8601 When prompt was created
last_explored ISO-8601 or null When prompt was last used
status enum ready (unused), explored (used), archived (retired)

Modes

Mode ID Prefix Description
philosophical PHI Value and belief examination
biographical BIO Memory-focused exploration of past experiences
challenge CHL Stress-test existing beliefs, steelman opposing views
exploratory EXP Open-ended curiosity about unexplored topics

Note: The prompt library is small and seeded — only four prompts ship on disk (BIO-001, CHL-001, EXP-001, PHI-001), one per mode. The library grows as new prompts are created via the API or CLI.

Domains (9 Total)

The nine domains are defined as the InterviewDomain enum in api/src/schemas/interviews.py:

Domain Focus Area
values_beliefs Core principles, philosophy, decision frameworks
childhood_family Formative years, family dynamics, early memories
self_knowledge Identity, personality, psychology
work_purpose Career, business, leadership, meaning in work
relationships Connection patterns, friendship, partnership
challenges_resilience Failure, risk, adversity, growth
legacy_mortality Death, meaning, what you leave behind
spirituality Religion, flow states, transcendence
creativity_passions Hobbies, passions, creative expression

Note: epistemology is not a domain — it is a tag in the TAG_TO_DOMAIN map that resolves to values_beliefs. Tags and domains are different layers (see Coverage Tracking).

Rich Prompt Generation

When a prompt is selected or expanded, the system generates a rich "full prompt text" designed to be copied into Grok voice mode. This is controlled by the interview context configuration at config/interview_context.json.

How Rich Prompts Work

The service dynamically selects background facts based on relevance to the interview topic:

  1. Fact scoring: Each background fact in the config has domains and keywords. The system scores facts by:
  2. Domain match: +3 points
  3. Keyword match in title: +2 points per keyword

  4. Word overlap with title: +1 point per matching word

  5. Context assembly: The top 8 most relevant facts are selected and assembled with:

  6. Background facts about the user
  7. The interview topic and opening question
  8. Follow-up probes with instructions for natural flow
  9. Style guidance (e.g., "aggressive pushback preferred")
  10. Domain-specific goals
  11. Duration and challenge level notes

Example Generated Prompt

**You're conducting a deep interview with me about: Your Relationship with Risk**

**Background on me:**
- I'm ****, ** years old, CEO of **** (** employees)
- Big into **** (still **** at **) - **** is the "deepest core of my identity"
- I believe "****"
- I view myself as a "****" that shifts high-risk ventures to medium-risk

**Interview topic: Your Relationship with Risk**

**Your opening question:**
"You've built a company, you **** at **, you seem to have a non-standard relationship with risk..."

**Follow-up probes (let the conversation flow naturally):**
1. What's the biggest risk you've ever taken where the expected value was probably negative?
2. How has your risk tolerance changed over time?
3. Is there a risk you didn't take that still haunts you?

**Interview style guidance:**
- I prefer aggressive pushback over gentle probing
- Use Socratic questioning
- I think in systems and models
- Target 25-50 minutes of conversation
- This is a challenge level 3 interview - push harder than usual

**What I'm hoping to get out of this:**
I want to examine and stress-test my philosophical positions...

**Just respond with 'OK' and I'll tell you when I'm ready to begin.**

API Endpoints

Endpoint Method Description
/api/v1/interviews/prompts GET List all prompts, optionally filter by status
/api/v1/interviews/prompts/{id} GET Get single prompt with full details and generated text
/api/v1/interviews/prompts POST Create prompt from a natural language idea (Claude-generated JSON)
/api/v1/interviews/prompts/{id} PUT Update prompt fields directly
/api/v1/interviews/prompts/{id}/revise POST Revise prompt via natural language feedback (Claude-generated JSON)
/api/v1/interviews/prompts/{id} DELETE Delete a prompt
/api/v1/interviews/prompts/suggest POST Get Claude suggestions for new topics based on coverage gaps
/api/v1/interviews/transcripts GET List all transcript metadata
/api/v1/interviews/transcripts/{filename} GET Get transcript with full markdown content
/api/v1/interviews/quotes GET List quotes, filter by tag or theme
/api/v1/interviews/stories GET List extracted stories
/api/v1/interviews/observations GET List observations, filter by type
/api/v1/interviews/coverage GET Get domain coverage statistics

The three generative endpoints (POST /prompts, POST /prompts/{id}/revise, POST /prompts/suggest) each build a system + user prompt and call ClaudeRunnerService.run_prompt(), which routes through scripts.claude.run_batch to the local claude CLI/daemon and expects raw JSON back (a JSONDecodeError surfaces as HTTP 500). For model-routing details, see Model Guidance.

Creating a Prompt via API

curl -X POST http://localhost:33800/api/v1/interviews/prompts \
  -H "Content-Type: application/json" \
  -d '{"idea": "exploring my relationship with failure and how I process setbacks"}'

Response: 

{
  "prompt": {
    "id": "PHI-002",
    "title": "Processing Failure and Setbacks",
    "opener": "Tell me about your relationship with failure...",
    "probes": [...],
    "mode": "philosophical",
    "domain": "challenges_resilience",
    "full_prompt_text": "**You're conducting a deep interview...**"
  },
  "message": "Created new philosophical prompt: Processing Failure and Setbacks"
}

Revising a Prompt via AI Feedback

curl -X POST http://localhost:33800/api/v1/interviews/prompts/PHI-002/revise \
  -H "Content-Type: application/json" \
  -d '{"feedback": "Make it more confrontational, add probes about public failures"}'

Getting Topic Suggestions

curl -X POST http://localhost:33800/api/v1/interviews/prompts/suggest

Returns AI-generated suggestions based on coverage gaps: 

{
  "suggestions": [
    {
      "title": "The Role of Luck in Your Success",
      "rationale": "Your achievements might be attributed to skill, but luck plays a role worth examining",
      "domain": "self_knowledge",
      "mode": "challenge",
      "challenge_level": 4
    }
  ],
  "coverage": {
    "domains": [...],
    "lowest_coverage_domain": "legacy_mortality"
  }
}

Coverage Tracking

The system calculates coverage across all 9 domains based on:

Source Points per Item
Explored prompts +10 per domain
Extracted quotes +3 per quote tagged to domain
Identity observations +5 per observation matching domain

Coverage is displayed as a heatmap in the web UI, sorted by lowest coverage first to highlight gaps. The /api/v1/interviews/prompts/suggest endpoint uses this to recommend new topics.

Coverage Calculation Logic

The service maps tags/themes to domains using keyword matching in api/src/services/interviews.py. The TAG_TO_DOMAIN dict has ~46 entries:

TAG_TO_DOMAIN = {
    "philosophy": InterviewDomain.VALUES_BELIEFS,
    "childhood": InterviewDomain.CHILDHOOD_FAMILY,
    "skateboarding": InterviewDomain.CREATIVITY_PASSIONS,
    # ... ~46 mappings total
}

Coverage classification works on two levels: explored prompts contribute via their declared domain field, while quotes and observations are classified by scanning their tags/text for the first matching TAG_TO_DOMAIN keyword.

Note: The real TAG_TO_DOMAIN in the repo hard-codes personal keywords — proper names and family-role words that resolve to relationships / childhood_family. If you reimplement this, genericize the tag map for your own vocabulary rather than copying it verbatim; the keyword list is what tailors classification to a specific person.

Transcript Management

File Naming Convention

Transcripts are stored in data/interviews/transcripts/ with this naming pattern: 

YYYY-MM-DD_platform_type.md

Examples: - 2025-12-29_grok_background.md - 2026-01-02_grok_risk.md - 2025-12-30_grok_philosophy.md

Metadata Storage

Transcript metadata is stored in data/identity/interviews.jsonl:

{
  "id": "3a5498b8-d346-4b28-9cf6-11f58735a776",
  "ts": "2026-01-02T16:08:50.299058",
  "session": 4,
  "type": "philosophy_risk",
  "interviewer": "grok",
  "topics": ["local_maxima_trap", "business_risk_tolerance", "minimize_future_regret"],
  "source_file": "data/interviews/transcripts/2026-01-02_grok_risk.md",
  "note": "User enjoyed high pushback style - got to deeper places",
  "v": 1
}

Transcript Reader

The web UI renders transcripts in a chat-style format: - Parses markdown based on ## Grok Replied: sections - Detects speakers using heuristics (word count, personal pronouns, question patterns) - Displays as conversation bubbles with avatar icons - System prompts are collapsible

Voice Corpus Integration

Interview transcripts also feed into the voice analysis pipeline (scripts/voice/corpus.py). The parse_interviews() function extracts bio-Zack's spoken responses from transcript files, using NBSP markers to identify his segments vs the interviewer's. These are normalized as modality: "spoken" messages, enabling the voice system to capture spoken communication patterns distinct from written ones. The corpus is built from data/writing/corpus/interviews/ (typically symlinked to data/interviews/transcripts/).

Data Extraction

After interviews, the following artifacts can be extracted:

Quotes (data/identity/quotes.jsonl)

{
  "id": "q-001",
  "quote": "I believe failure is a choice - the only way you fail is by choosing to quit",
  "context": "Discussing philosophy around risk and resilience",
  "tags": ["resilience", "philosophy", "failure"],
  "theme": "persistence",
  "source": "2026-01-02_grok_risk.md",
  "ts": "2026-01-02T17:00:00Z"
}

Stories (data/identity/stories.json)

{
  "id": "story-001",
  "title": "An Early Software Origin Story",
  "themes": ["entrepreneurship", "childhood", "programming"],
  "people_mentioned": ["a parent"],
  "time_period": "early teens (~1990s)",
  "narrative": "As a kid I built a small software utility and distributed it locally...",
  "key_moments": ["First distribution", "First revenue"]
}

Observations (data/identity/observations.jsonl)

{
  "id": "obs-001",
  "ts": "2026-01-02T17:05:00Z",
  "type": "value",
  "obs": "Views failure as a choice rather than an outcome - the only failure is quitting",
  "evidence": ["Direct statement in risk interview", "Pattern across multiple topics"],
  "confidence": "high",
  "source": "2026-01-02_grok_risk.md"
}

Observation types: value, personality, preference, communication

Web UI Components

The interviews page (/interviews) includes these components:

Component Purpose
CoverageHeatmap Interactive bar chart showing domain coverage with tooltips
PromptCard Expandable card with copy-to-clipboard, revise, delete actions
TranscriptViewer Browsable list/summary view of transcripts with metadata
TranscriptReader Full-screen modal with chat-style rendering
QuoteGrid Filterable grid of quotes with tag/theme faceting
StoryCard Display narrative stories with themes and people
ObservationList Type-grouped list of identity observations

Note: The actual component files in web/src/components/interviews/ are CoverageHeatmap.tsx, PromptCard.tsx, TranscriptViewer.tsx, TranscriptReader.tsx, QuoteGrid.tsx, StoryCard.tsx, and ObservationList.tsx. There is no TranscriptList component.

CLI Commands

# Show prompts matching filters (suggest mode)
/interview [--time=N] [--mode=M] [--setting=S] [--mood=M] [--pick=ID]

# Select a specific prompt (pick mode)
/interview --pick=PHI-001

The /interview command only has two modes: suggest (default) and pick (--pick=ID). There are no process or coverage subcommands.

  • Processing a transcript is triggered by natural language ("I just did an interview" / "process this interview"), which auto-applies the interview-processing skill (.claude/skills/interview-processing/).
  • Coverage gaps are surfaced via the API endpoint GET /api/v1/interviews/coverage and the CoverageHeatmap component in the web UI, not via a CLI subcommand.

File Locations Summary

Path Description
data/interviews/prompts/*.json Prompt definitions
data/interviews/transcripts/*.md Interview transcripts (markdown)
data/interviews/summaries/*.json Transcript summaries
data/identity/interviews.jsonl Transcript metadata
data/identity/quotes.jsonl Extracted quotes
data/identity/observations.jsonl Identity observations
data/identity/stories.json Narrative stories
config/interview_context.json Rich prompt generation config
api/src/routers/interviews.py API endpoints
api/src/services/interviews.py Business logic and prompt management
api/src/schemas/interviews.py Pydantic models
web/src/routes/interviews/index.tsx Main page with Prepare/Review modes
web/src/components/interviews/*.tsx UI components
web/src/hooks/useInterviews.ts React Query hooks
.claude/skills/interview-processing/ CLI skill for transcript processing

Known Limitations

  1. Manual transcript import: Getting transcripts out of Grok requires manual copy-paste. No API integration exists.

  2. Crude coverage calculation: The keyword-matching approach for domain coverage misses nuance. A quote about "risk in relationships" might get tagged to challenges_resilience when relationships is more appropriate.

  3. No follow-up scheduling: The system tracks when prompts were explored but doesn't suggest when to revisit. Some topics benefit from periodic re-exploration.

  4. Single interviewer assumed: The system is optimized for Grok voice mode. Different AI interviewers might need different briefing formats.

  5. No audio storage: Only text transcripts are stored. Voice nuance is lost.

  6. Transcript parsing is heuristic: The speaker detection in the transcript reader uses word count and pronoun patterns, which occasionally misattributes.

Improvement Opportunities

These are potential enhancements for the interview system:

High Value

  • Transcript auto-import: Watch a folder for new transcripts and auto-process
  • Domain inference via LLM: Use Claude to properly classify quotes/observations instead of keyword matching
  • Follow-up scheduling: Suggest revisiting topics based on time since last exploration and new life events

Medium Value

  • Interviewer profiles: Support different AI interviewers with tailored briefing formats (GPT voice, Claude, etc.)
  • Interview chains: Link related prompts into multi-session exploration sequences
  • Rich transcript parsing: Use LLM to parse transcripts and extract speaker turns more accurately

Lower Priority

  • Audio archival: Store voice recordings if Grok eventually supports audio export
  • Real-time transcript capture: Browser extension to capture transcripts during conversation
  • Prompt effectiveness scoring: Track which prompts lead to the most quotes/observations