Living Briefing

The Living Briefing is a real-time intelligence surface that consolidates signals from across all of vita's data sources into a single, prioritized view. It surfaces what matters before bio-Zack asks — actions to take, decisions to make, things to be aware of, opportunities to pursue, signals to watch, and wins to celebrate.

Unlike the VITA Daily Brief (VDB) which is a point-in-time email snapshot, the Living Briefing is continuously updated throughout the day as new data arrives. Items flow through category-specific state machines, get automatically linked across sources via cross-source clustering, and decay through staleness rules when no longer relevant.

Architecture

Data Sources                  Processor              Database              API / UI
┌──────────────┐
│ commitments  │──┐
│ email        │──┤
│ calendar     │──┤        ┌──────────┐        ┌───────────┐        ┌──────────┐
│ slack_saved  │──┼──────▶│ processor │──────▶│ SQLite DB │──────▶│ FastAPI  │
│ stream_caps  │──┤        │          │        │           │        │ Router   │
│ health       │──┤        │ 1. collect│        │ items     │        │          │
│ research     │──┤        │ 2. upsert │        │ events    │        │ /summary │
│ forge        │──┘        │ 3. cluster│        │ links     │        │ /items   │
                           │ 4. decay  │        │ fts5      │        │ /debug/* │
                           └──────────┘        │ cursors   │        └────┬─────┘
                                                └───────────┘             │
                                                                          ▼
                                                                    ┌──────────┐
                                                                    │ React UI │
                                                                    │          │
                                                                    │ Hero     │
                                                                    │ Sections │
                                                                    │ Clusters │
                                                                    │ Cards    │
                                                                    └──────────┘

The processor runs on the scheduler (every 15 minutes, but only during waking hours — 5am–11pm PT) and can be triggered manually via POST /api/v1/briefing/process.

Engine liveness monitoring

The nightly /sleep data-source-health leg treats the processor itself as a synthetic source named briefing.engine. It reads briefing.db in SQLite read-only mode and compares the newest briefing_items.created_at or updated_at value with two thresholds:

• older than 48 hours: stale;
• older than 72 hours: likely dark, with an explicit warning.

Missing databases, empty tables, query failures, and malformed timestamps are reported as health rows rather than allowed to break /sleep. This is detection-only: the check does not restart or mutate the briefing engine.

Data Sources (Adapters)

Each adapter implements collect() -> list[BriefingItemCandidate]. All mapping from source data to briefing categories is deterministic — no LLM calls in adapters.

Adapter	Source Type	File	Categories	What It Surfaces
CommitmentsAdapter	commitment	adapters/commitments.py	action	Open commitments from /commitments tracking
EmailAdapter	email	adapters/email.py	action, decision, awareness	Important flagged emails, threads needing response
CalendarAdapter	calendar	adapters/calendar.py	action, awareness	Upcoming meetings, prep items, schedule conflicts
SlackSavedAdapter	slack_saved	adapters/slack_saved.py	action, signal	Slack saved-for-later messages (inbox queue)
StreamCapturesAdapter	stream_capture	adapters/stream_captures.py	action, signal, awareness	Voice memo captures, file-this tags, follow-ups
HealthAdapter	health	adapters/health.py	awareness, signal	Sleep gate status, health alerts, CGM data
ResearchAdapter	research	adapters/research.py	opportunity, signal	Research findings, web discoveries from /sleep
ForgeAdapter	forge	adapters/forge.py	opportunity, signal	Active forge ideas (sparks, smelts) needing attention
BookmarksAdapter	bookmark	adapters/bookmarks.py	signal	High-relevance X bookmarks (score ≥ 0.7)
CoachingAdapter	coaching	adapters/coaching.py	action, win	Coaching nudges from working memory
AgentContractsAdapter	agent	adapters/agent_contracts.py	varies	Briefing contributions declared by autonomous agents
DataHygieneAdapter	data_hygiene	adapters/data_hygiene.py	awareness	Data quality alerts and stale-source warnings

All adapters live in scripts/briefing/adapters/. The registry at adapters/__init__.py defines ALL_ADAPTERS — the processor iterates through this list.

Categories & State Machines

Each briefing item belongs to exactly one category. Categories have independent state machines reflecting different lifecycle patterns.

Action

Things bio-Zack needs to do.

pending ──(168h)──▶ dropped
   │
   ▼
in_progress ──(168h)──▶ dropped
   │
   ▼
  done (terminal)

Decision

Choices that need to be made.

surfaced ──(96h)──▶ [archived]
   │
   ▼
decided
   │
   ▼
communicated (terminal)

Awareness

Things to know about (low-friction, high-decay).

new ──(24h)──▶ archived (terminal)
 │
 ▼
acknowledged ──(72h)──▶ archived (terminal)

Opportunity

Things that could be pursued.

spotted ──(168h)──▶ [archived]
   │
   ▼
evaluating ──(336h)──▶ [archived]
   │
   ▼
pursued (terminal)
   │
   ▼
passed (terminal)

Signal

Patterns and trends emerging from data.

emerging ──(72h)──▶ faded ──(48h)──▶ [archived]
   │
   ▼
strengthening
   │
   ▼
crystallized (terminal)

Win

Accomplishments to celebrate.

achieved ──(72h)──▶ archived (terminal)
   │
   ▼
celebrated ──(48h)──▶ archived (terminal)

Terminal states: done, dropped, communicated, archived, pursued, passed, crystallized, faded, celebrated.

Staleness Rules

Items automatically transition through their state machines via staleness decay. The processor runs apply_staleness_decay() after each collection cycle.

Category	State	Hours	Next State
action	pending	168	dropped
action	in_progress	168	dropped
decision	surfaced	96	archived
awareness	new	24	archived
awareness	acknowledged	72	archived
opportunity	spotted	168	archived
opportunity	evaluating	336	archived
signal	emerging	72	faded
signal	faded	48	archived
win	achieved	72	archived
win	celebrated	48	archived

Rules are defined in scripts/briefing/models.py as STALENESS_RULES.

Cluster-Aware Decay

Related items in a cluster are protected from premature decay. When a related item hits its staleness threshold, the system checks whether the cluster's primary item is still in a non-terminal state. If so, the related item skips decay — it will decay naturally after the primary does.

This prevents clusters from shrinking incoherently. The primary item controls the cluster's lifecycle.

If the cluster map fails to build for any reason, decay falls back to per-item behavior (no regression).

Cross-Source Clustering

The clustering engine links related items from different sources into clusters. This is the core intelligence: a commitment about "follow up with a colleague on financials", an email thread about the same topic, and a voice memo mentioning it should all appear together.

The pipeline has 3 levels, run in order. Matches from earlier levels suppress further matching for those items.

Level 1: Source-Specific Dedup

Deterministic, zero-cost. Catches obvious duplicates.

• Title substring (same source type only): If both items share a source_type and one title contains the other (both > 10 chars, different lengths) → duplicate. Cross-source pairs are handled by the dedicated commitment ↔ stream_capture rule below, not by this generic check.
• Email thread ID: Same thread_id in metadata → same_thread
• Cross-source commitment ↔ stream_capture: Strict substring match (shorter must be ≥20 chars and ≤80% of longer's length) → same_thread

File: clustering.py → _check_source_dedup()

Level 2: Entity Overlap

Deterministic, uses an entity dictionary built from profile data.

Entity Types and Weights:

Type	Weight	Source	Examples
person_full	1.5	profile/relationships.json	"Alex Rivera", "Jordan Lee"
medication	1.5	Hardcoded list	(medications loaded from config)
project	1.0	data/projects/registry.json	"Ask-Ride", "Cadence"
key_term	0.8	Hardcoded list	"restructuring deal", "sleep gate"
person_first	0.5	profile/relationships.json	"Alex", "Jordan", "Sam"

Match threshold: Weighted overlap score ≥ 1.5 → entity_match

Entity extraction results are cached in each item's metadata (entities field) to avoid re-extraction on subsequent cycles.

File: entities.py → extract_entities(), compute_entity_score()

Level 3: LLM Semantic Matching

Only runs if levels 1–2 found no matches. Uses FTS5 to find candidate items, then calls a small, fast LLM via OpenRouter (the running DEFAULT_MODEL is google/gemini-3.1-flash-lite-preview) to determine if they're about the exact same topic.

Note: classifier.py's module docstring and clustering.py's comments still mention "Haiku" (this code was adapted from an earlier pipeline that used a Claude model). Those are legacy comments — the live DEFAULT_MODEL constant in classifier.py is authoritative. For model-routing conventions across vita, see Model Guidance.

Pipeline: 1. Build query from new item's title + summary 2. FTS5 search for candidate matches (top 10, excluding the new item) 3. Format candidates with their entities into a structured prompt 4. Call LLM with temperature 0.1 for deterministic matching 5. Validate: returned item ID must exist in candidate set (hallucination guard) 6. Confidence threshold: ≥ 0.9 required to create a link

Relationship type: llm_match

File: classifier.py → match_item_to_candidates()

OpenRouter config: ~/.config/vita/openrouter.json

Cluster Formation

After matching, clusters are formed via union-find on the link graph. Each connected component becomes a cluster.

Primary selection (from _select_primary() in clustering.py): 1. Highest importance score 2. Highest category priority (action=6, decision=5, awareness=4, opportunity=3, signal=2, win=1) 3. Most recently updated

Cross-source importance boost: Items corroborated by multiple source types get +0.1 per additional source. A 3-source cluster gets +0.2 boost to the primary's importance.

API Reference

All endpoints are under /api/v1/briefing. Router file: api/src/routers/briefing.py.

Core Endpoints

Method	Path	Description
GET	/items	List active items. Query params: category, state, limit (default 100)
GET	/items/{item_id}	Single item with event history
GET	/summary	Home page summary: items by category, clusters, counts, pinned. Query param: min_importance (default 0.25)
GET	/stats	System stats: total/active/archived counts, by-category, processor state
POST	/items/{item_id}/pin	Pin/unpin an item. Body: {"pinned": true}
POST	/items/{item_id}/archive	Archive an item. Query param: cascade=true archives cluster-related items
POST	/items/{item_id}/state	Transition state. Body: {"state": "done", "reason": "completed"}. Query param: cascade=true
GET	/items/{item_id}/links	All links for an item
DELETE	/links/{link_id}	Delete a single link (manual cleanup)
POST	/process	Manually trigger the processor

Debug Endpoints

Method	Path	Description
GET	/debug/clusters	Full cluster graph. Multi-item clusters only. Each: primary, related, links (with relationship types/scores/entities), source types, size
GET	/debug/staleness	Items approaching auto-transition. Query param: hours_ahead (default 24). Returns: id, title, category, current/next state, hours_remaining, source_type
GET	/debug/adapters	Per-adapter health: source_type, total/active/archived counts, last_run, cursor_value

Frontend

Component Hierarchy

Dashboard (routes/index.tsx)
└── BriefingHero          — headline, category counts, pinned items
└── BriefingSection       — one per category, collapsible
    ├── ClusterCard       — grouped related items under primary
    │   └── BriefingCard  — individual item (pin/archive/done actions)
    └── BriefingCard      — standalone items not in clusters

Key Files

File	Purpose
web/src/routes/index.tsx	Dashboard page, calls useBriefingSummary()
web/src/components/briefing/BriefingHero.tsx	Hero card with headline + counts
web/src/components/briefing/BriefingSection.tsx	Category section with cluster/flat display
web/src/components/briefing/BriefingCard.tsx	Individual item card
web/src/components/briefing/ClusterCard.tsx	Cluster grouping with expandable related items
web/src/hooks/useBriefing.ts	React Query hooks for all briefing API operations

Hooks

• useBriefingSummary() — fetches categorized + clustered items for dashboard
• useBriefingStats() — system statistics
• useBriefingItems(category?, state?) — filtered item list
• useBriefingItem(id) — single item with events
• usePinItem() — pin/unpin mutation
• useArchiveItem() — archive mutation (cascades to cluster)
• useTransitionState() — state transition mutation (cascades to cluster)
• useTriggerProcess() — manually trigger processor

Database Schema

SQLite database at data/briefing/briefing.db. WAL mode enabled.

briefing_items

Column	Type	Notes
id	INTEGER	Primary key, autoincrement
category	TEXT	action, decision, awareness, opportunity, signal, win
state	TEXT	Category-specific state
title	TEXT	Item title
summary	TEXT	Optional summary
importance_score	REAL	0.0–1.0, default 0.5
source_type	TEXT	Adapter source (e.g. "commitment", "email")
source_id	TEXT	Unique within source_type
is_pinned	INTEGER	0 or 1
is_archived	INTEGER	0 or 1
metadata	TEXT	JSON blob (entities, thread_id, etc.)
created_at	TEXT	ISO datetime
updated_at	TEXT	ISO datetime
state_changed_at	TEXT	ISO datetime, resets on state transition

Unique constraint: (source_type, source_id)

Indexes: category, state, source, archived

briefing_item_events

Audit trail for all state transitions.

Column	Type	Notes
id	INTEGER	Primary key
item_id	INTEGER	FK → briefing_items
old_state	TEXT	Nullable (null for creation)
new_state	TEXT
reason	TEXT	"created", "adapter_update", "manual", "staleness_decay", "notification_sent"
created_at	TEXT	ISO datetime

briefing_processor_state

Per-adapter cursor tracking.

Column	Type	Notes
source_type	TEXT	Primary key
cursor_value	TEXT	ISO datetime of last run
last_run	TEXT	ISO datetime

briefing_item_links

Cross-source relationships between items.

Column	Type	Notes
id	INTEGER	Primary key
item_a_id	INTEGER	FK → briefing_items (smaller ID)
item_b_id	INTEGER	FK → briefing_items (larger ID)
relationship	TEXT	"duplicate", "same_thread", "entity_match", "llm_match"
score	REAL	Match confidence
matched_entities	TEXT	Comma-separated entity strings
created_at	TEXT	ISO datetime

Unique constraint: (item_a_id, item_b_id, relationship)

briefing_items_fts

FTS5 virtual table on title + summary. Used by Level 3 clustering to find candidate matches.

Processor Pipeline

The processor (scripts/briefing/processor.py) runs a 4-step pipeline:

1. Collect

Iterate through ALL_ADAPTERS. Each adapter's collect() returns list[BriefingItemCandidate].

2. Upsert

For each candidate, upsert by (source_type, source_id): - New item: Insert + log "created" event + index in FTS5 - Existing item: Update fields. If state changed, log event + update state_changed_at. Archived items only un-archive if state explicitly changes to non-terminal.

3. Cluster

For newly created items only (not updates), run run_clustering_for_new_items(): 1. Load entity dictionary from relationships.json + registry.json 2. Build entity index for all active items (cached in metadata) 3. For each new item, run 3-level matching pipeline 4. Create links for matches

4. Decay

Run apply_staleness_decay(): 1. Build cluster primary map (BFS on link graph) 2. For each active item past its staleness threshold: - If in a cluster and not primary: skip if primary is non-terminal - Otherwise: transition to next state or archive

Concurrency: A threading lock prevents overlapping runs. If the processor is already running, subsequent calls skip silently.

Timing: The scheduler's _briefing_loop ticks every 15 minutes, but only fires the processor during waking hours — between 5am and 11pm PT (gated by is_time_in_window_service(start_hour=5, end_hour=23) in api/src/scheduler.py). Outside that window the loop still wakes but skips the run. Manual trigger via POST /api/v1/briefing/process works at any hour.

Manual Operations

Run the processor

# Via API
curl -X POST http://localhost:33801/api/v1/briefing/process

# Via CLI
cd /home/zack/work/vita
PYTHONPATH=scripts uv run python -m briefing.processor

Inspect clusters

# Full cluster graph
curl http://localhost:33801/api/v1/briefing/debug/clusters | python -m json.tool

# What's about to decay
curl "http://localhost:33801/api/v1/briefing/debug/staleness?hours_ahead=48" | python -m json.tool

# Adapter health
curl http://localhost:33801/api/v1/briefing/debug/adapters | python -m json.tool

Delete a bad link

# Find the link ID from cluster debug output or item links
curl http://localhost:33801/api/v1/briefing/items/42/links | python -m json.tool

# Delete it
curl -X DELETE http://localhost:33801/api/v1/briefing/links/17

Manual state transition

# Mark an action as done
curl -X POST http://localhost:33801/api/v1/briefing/items/42/state \
  -H "Content-Type: application/json" \
  -d '{"state": "done", "reason": "completed manually"}'

# Pin an item
curl -X POST http://localhost:33801/api/v1/briefing/items/42/pin \
  -H "Content-Type: application/json" \
  -d '{"pinned": true}'

# Archive with cascade to cluster
curl -X POST "http://localhost:33801/api/v1/briefing/items/42/archive?cascade=true"

Backfill entities

cd /home/zack/work/vita
PYTHONPATH=scripts uv run python -m briefing.backfill_entities

Direct SQLite queries

sqlite3 data/briefing/briefing.db

-- Active items by category
SELECT category, state, COUNT(*) FROM briefing_items
WHERE is_archived = 0 GROUP BY category, state;

-- Items in clusters (have links)
SELECT DISTINCT i.id, i.title, i.source_type
FROM briefing_items i
JOIN briefing_item_links l ON i.id = l.item_a_id OR i.id = l.item_b_id
WHERE i.is_archived = 0;

-- Link graph
SELECT l.id, a.title as item_a, b.title as item_b, l.relationship, l.score
FROM briefing_item_links l
JOIN briefing_items a ON l.item_a_id = a.id
JOIN briefing_items b ON l.item_b_id = b.id
WHERE a.is_archived = 0 AND b.is_archived = 0
ORDER BY l.score DESC;

Troubleshooting

Items not appearing on dashboard

Check min_importance filter: The summary endpoint filters items below min_importance (default 0.25). Low-importance items won't show.

# List all active items regardless of importance
curl "http://localhost:33801/api/v1/briefing/items?limit=200" | python -m json.tool | grep -c '"id"'

Check if archived: Items with is_archived = 1 are hidden.

sqlite3 data/briefing/briefing.db "SELECT COUNT(*) FROM briefing_items WHERE is_archived = 1"

Check per-category caps: The /summary endpoint caps how many items each high-volume category returns — signal=15, opportunity=10, win=10 (defined in _CATEGORY_CAPS in db.py → get_summary()). Items already sorted by importance fill the cap first; lower-importance items in a capped category silently drop off the dashboard (pinned items are exempt). If an item is missing but /items?category=signal shows it, the cap is the cause — pin it or raise the cap.

Clusters look wrong (bad link)

1. Identify the bad link via /debug/clusters
2. Delete it: DELETE /api/v1/briefing/links/{link_id}
3. Re-run processor to rebuild clusters

Adapter not producing items

1. Check /debug/adapters for the adapter's last_run and item counts
2. Check the source data file exists and has content
3. Run processor with logging: PYTHONPATH=scripts uv run python -m briefing.processor
4. Check if the adapter's source data has changed since the cursor

Items decaying too fast

1. Check /debug/staleness for what's about to transition
2. Pin critical items to exempt them from staleness decay
3. Check if items are cluster-related (related items should be protected by primary)
4. Review staleness rules in models.py — STALENESS_RULES

LLM matching not working

1. Check OpenRouter config exists: cat ~/.config/vita/openrouter.json
2. Check API key is valid (try a test call)
3. LLM matching only runs when levels 1–2 find no matches
4. Confidence threshold is 0.9 — marginal matches are intentionally dropped

Processor stuck / overlapping runs

The processor uses a threading lock. If a previous run crashed without releasing the lock, restart the API server.

Configuration

Setting	Location	Value
Database path	scripts/briefing/db.py → DB_PATH	data/briefing/briefing.db
OpenRouter API key	~/.config/vita/openrouter.json	{"api_key": "..."}
LLM model	scripts/briefing/classifier.py	google/gemini-3.1-flash-lite-preview
LLM confidence threshold	scripts/briefing/classifier.py	0.9
Entity match threshold	scripts/briefing/entities.py	1.5
Min importance (UI)	scripts/briefing/db.py → get_summary()	0.25
Category caps (UI)	scripts/briefing/db.py → get_summary()	signal=15, opportunity=10, win=10
Processor schedule	api/src/scheduler.py → _briefing_loop	Every 15 minutes, waking hours only (5am–11pm PT)
Entity dictionary sources	profile/relationships.json, data/projects/registry.json	Auto-loaded

Notifications

scripts/briefing/notifications.py → check_and_notify()

Sends Telegram alerts for: - High-importance actions (≥0.85) in pending state (max once per 24h per item) - Sleep gate awareness items that are blocked

Uses enqueue_message() from the proactive queue. Notification events are logged as notification_sent in the event audit trail.

Relationship to VDB

The Living Briefing and VITA Daily Brief (VDB) are complementary:

	Living Briefing	VDB
Update frequency	Continuous (every 15 min, waking hours)	Once daily (5:30 AM)
Format	Interactive dashboard	Email newsletter
Data model	Structured items with state machines	Freeform HTML sections
Clustering	Automatic cross-source linking	Manual section organization
Staleness	Automatic decay + archival	One-shot, no lifecycle
Primary use	Throughout the day	Morning orientation

The VDB may eventually source its action items and highlights from the Living Briefing rather than re-querying raw data directly.