VITA Daily Brief

The VDB (VITA Daily Brief) is an automated morning email that aggregates ~15 data sources, has an LLM compose a personalized newsletter, and delivers it via Fastmail JMAP each morning. It runs as a two-pass stepwise flow: a draft is composed before wake-up without fresh sleep data, a poll executor waits for the night's sleep data to sync, then the sleep section is revised and the email is sent. A spoken-word podcast episode is generated from the same data after the email goes out.

Note: The VDB is designed for graceful degradation. Missing or stale data sources don't break the brief — the LLM is told not to render sections for absent data, so they simply don't appear. Every external sync runs in a try/except that returns None on failure.

What it does

Capability	Mechanism
Parallel data sync	All external sources sync concurrently in a `ThreadPoolExecutor(max_workers=9)`; 30s timeout each, RWGPS gets 300s
LLM-composed newsletter	Full email (subject + HTML + text) composed in one LLM call from the gathered context plus a voice/structure prompt — not a template
Two-pass sleep revision	Draft generated ~4:45 AM PT before sleep data is available; a poll executor watches for fresh Garmin sleep, then the sleep section is revised and sent
Podcast generation	After send, a conversational spoken-word transcript is generated from the same context and synthesized to audio
Feedback loop	Reply to the email; the reply is matched by header, parsed, and folded into tomorrow's context (and optionally distilled into permanent guidelines)
Idempotent	A `data/briefs/YYYY-MM-DD.json` `sent` flag prevents duplicate sends (bypassed by `--force`/`--preview`/`--no-send`)
Web surface	An API router exposes the brief archive, stored HTML, the captured context, and per-source health

Pipeline (the production path)

The production orchestration is a stepwise flow, not a single CLI script. The scheduler dispatches flows/vdb/FLOW.yaml; each step is a thin Python wrapper that shells into the one real workhorse, flows/vdb/generate_newsletter.py, with different flags.

       VitaScheduler._vdb_loop  (api/src/scheduler.py)
       draft target 4:45 AM PT, deadline 07:00 PT
                        │  _run_stepwise_flow("vdb", ...)
                        ▼
        ┌──────────────────────────────────────────────┐
        │            flows/vdb/FLOW.yaml                │
        └──────────────────────────────────────────────┘
                        │
   ┌────────────┬───────┴────────┬──────────────┐
   ▼            ▼                 ▼              ▼
 draft.py   poll_sleep_check.py  send.py     podcast.py
   │         (poll executor)       │             │
   │  generate_newsletter.py       │             │
   │  --no-send --skip-garmin      │             │
   │  --force                      │             │
   │                               │             │
   ▼            ▼                  ▼             ▼
 save draft   sync Garmin,     generate_       scripts/podcast/
 (no sleep,   detect fresh     newsletter.py   generate_transcript.py
  no send)    sleep OR 07:00   --revise-sleep  → transcript → ElevenLabs
              deadline         (revise + send)    audio

Step 1 — `draft.py` (pass 1)

Runs generate_newsletter.py --no-send --skip-garmin --force. This gathers all context except Garmin sleep (the watch usually hasn't synced the night yet), has the LLM compose the full newsletter, and writes a draft state file. It also snapshots whatever Garmin data was present at generation time (garmin_at_generation) so pass 2 can detect whether sleep actually changed.

Step 2 — `poll_sleep_check.py` (poll executor)

This is a stepwise poll executor, not a normal run step. The FLOW declares:

poll_garmin:
  executor: poll
  check_command: |
    cd /home/zack/work/vita && python3 flows/vdb/poll_sleep_check.py
  interval_seconds: 300          # check every 5 minutes
  after: [draft]
  decorators:
    - type: timeout
      config: { minutes: 120 }   # hard ceiling on the wait

Each tick the check script:

Reads the draft's garmin_at_generation snapshot.
Syncs Garmin (scripts/garmin-sync.py 1 --sync-activities, 120s timeout).
Reads the current data/garmin/YYYY-MM-DD.json sleep payload.
Fresh sleep (current sleep differs from the snapshot) → prints JSON {has_fresh_sleep, sleep_score, sleep_hours} and the poll step completes.
Past deadline (07:00 PT — DEADLINE_HOUR=7, DEADLINE_MINUTE=0) → prints the same JSON with has_fresh_sleep: false and completes anyway.
Neither → prints nothing; the poll repeats on the next 5-minute interval.

Warning: The deadline that actually governs the wait is the 07:00 PT constant in poll_sleep_check.py. Some surrounding doc-strings (the FLOW prompt text, a scheduler comment) still say 06:30 — those strings are stale; the operative constant is 07:00.

Step 3 — `send.py` (pass 2)

Runs generate_newsletter.py --revise-sleep. The revision logic (revise_with_sleep) is conservative:

If the draft is already sent, it does nothing.
If fresh Garmin sleep equals the draft's garmin_at_generation snapshot, it skips the LLM entirely and sends the draft as-is.
Otherwise it builds a revision prompt (existing brief + fresh Garmin) and asks the LLM to update only the body-check / sleep section and any sleep-gate references, keeping everything else identical.
On any failure (LLM error, JSON parse failure, empty HTML), it falls back to sending the original draft. The brief always ships.

The send itself uploads an RFC822 blob and submits it via Fastmail JMAP (scripts/mail/fastmail.send_email), then rewrites the state file with the sent fields.

Step 4 — `podcast.py` (post-send)

Runs scripts/podcast/generate_transcript.py with a retry decorator (max_retries: 1, exponential backoff). It re-gathers the same VDB context, has the LLM produce a single-voice spoken narrative (no headers, no URLs, no markdown), saves the transcript to data/podcast/transcripts/YYYY-MM-DD.txt, and synthesizes audio via ElevenLabs. See Podcast for the audio pipeline.

The LLM call

The composition function is named call_claude() but does not call Claude — it POSTs to OpenRouter:

httpx.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers={"Authorization": f"Bearer {api_key}", ...},
    json={
        "model": "moonshotai/kimi-k2.6",   # newsletter composer
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0.7,
        "max_tokens": 32000,
    },
    timeout=120.0,
)

Credentials load from ~/.config/vita/openrouter.json (api_key). The podcast step uses the same OpenRouter endpoint but a different model (anthropic/claude-sonnet-4-6, temperature 0.8, max_tokens 16000). Both are reachable model choices on OpenRouter; for the broader model-routing picture see Model Guidance.

Note: The function name call_claude and the brief-state constant generator: "claude-newsletter" are historical labels, not the model in use. The newsletter is composed by Kimi K2.6 via OpenRouter.

Prompt assembly and output parsing

The prompt is voice.md (the structure/voice spec) followed by the full context dict serialized to JSON, with a closing instruction to "Output ONLY a JSON object with subject, html, and text keys." There is no Jinja2 rendering — the LLM emits the finished HTML and text directly.

Because the HTML body contains many {/} from inline CSS, the response parser (extract_json_from_response) is layered:

Direct json.loads if the response starts with {.
A fenced ```json code-block match.
Strip conversational preamble before the first {.
Bracket-depth matching that ignores braces inside JSON string values (the robust path for HTML/CSS payloads).
A greedy regex fallback.

If all paths fail it raises and the run aborts (pass 1) or falls back to the draft (pass 2).

The `voice.md` section spec

The LLM is steered toward a flowing newsletter (it may reorder by relevance, and is told never to render a section for missing data). The canonical structure in flows/vdb/prompts/voice.md:

#	Section	Notes
1	Opening	1–2 sentences
2	Body check	Sleep / HRV / recovery — the section pass 2 revises
3	Day ahead	Calendar + communications
4	Signal	Optional — top media / discoveries
5	Overnight build	Optional — what was built during `/sleep`
6	Focus	What to prioritize today — part of the single composed newsletter, not a separate LLM call
7	Conversation thread	Only if `yesterday_reply_items` is present (parsed reply)
8	Sign-off	—

Context: where the data comes from

gather_context() (scripts/vdb/gather_context.py) assembles one dict. It first runs sync_all_sources() in parallel, then layers on locally-loaded context. The full dict is also persisted to data/briefs/contexts/YYYY-MM-DD.json so the API/web "what data went into this brief" surface has something to show (audited 2026-05-01: the loader existed but no writer did, so the API returned context: null).

Synced sources (parallel, `ThreadPoolExecutor`)

Key	Source	Timeout	Module
`weather`	OpenWeatherMap — temp / condition / forecast	30s	`scripts/integrations/weather.py`
`garmin`	Sleep hours, score, HRV; gate fields layered in `gather_context`	30s	`scripts/garmin-sync.py` via `scripts/vdb/data.py:sync_garmin`
`work_email`	Primary mailbox — unread / VIP / summary	30s	`scripts/integrations/google_api.py`
`personal_email`	Secondary mailbox	30s	`scripts/integrations/google_api.py`
`calendar`	Today's events	30s	`scripts/integrations/google_api.py`
`bookmarks`	X/Twitter saved bookmarks (browser scrape)	30s	`scripts/integrations/x_sync.py`
`today`	Tasks pending / in-progress, habits logged	30s	`scripts/integrations/today_api.py`
`rwgps`	Activity sync → analyze → (auto-log skipped in read-only)	300s	`scripts/rwgps_sync.py` + `scripts/rwgps-sync.py`
`media_intel`	Scored high-signal items from feeds	30s	`scripts/media/vdb_section.py`
`email_attention`	Pending replies / actions / screener items	30s	`scripts/email_triage/notifications.py`

The media_intel and email_attention futures are only submitted if their modules import cleanly (graceful HAS_* flags). Garmin's future is omitted entirely when skip_garmin=True (the draft pass).

Locally-loaded context (after sync)

Key	What it carries	Source
`garmin` (derived)	Adds `sleep_avg_7day`, `sleep_gate_active`, `sleep_gate_threshold`, bedtime/waketime	`scripts/scheduling/sleep_gate.py`
`goals`	Up to 5 active goals, progress computed from `current_value/target_value` (renders `unknown`, never a fake `0`)	`goals/active/*.json`
`training`	Weekly training-component progress vs. targets, marked authoritative over the overnight summary	`scripts/scheduling/goals.py` + `engine.py`
`scheduling`	Drift warnings, today's commitments, calendar-aware suggestions	`scripts/scheduling/engine.py:get_scheduling_section`
`commitment_triage`	Triaged commitment summary	`scripts/consolidation/commitment_triage.py`
`overnight`	What was built during the overnight `/sleep` cycle	`data/overnight/YYYY-MM-DD.json`
`discoveries`	Overnight web discoveries (X / HN browsing)	`data/discoveries/YYYY-MM-DD.json`
`yesterday_feedback`	Raw reply text from yesterday's brief	`data/briefs/YYYY-MM-DD.json`
`yesterday_reply_items`	Reply parsed into links / questions / directives / commentary	`scripts/vdb/reply_parser.py:parse_reply`
`yesterday_recap`	Yesterday's meetings, telegram, scratch notes, commitments	`scripts/vdb/data.py:load_yesterday_recap`
`briefing_summary`	Top items per category from the Living Briefing	`scripts.briefing.db:get_summary`
`agent_context`	Staleness-aware contributions from background agents	`scripts.agents.contributions:collect_surface_context("vdb")`
`guidelines`	Permanent VDB formatting/voice guidelines (feedback-distilled)	`data/vdb/guidelines.md`

RWGPS pipeline (read-only in this context)

The rwgps source runs sync → analyze → auto-log, but gather_context passes read_only=True, which skips the auto-log step (the only state-mutating sub-step). A dedicated scheduler cron owns the auto-log write path; running it here too would double-drive activity logging. The analyze step's idempotent pending writes still run so the returned read is fresh. See Activities.

Scope: RWGPS is bio-zack's employer integration; this page documents the mechanism (sync/analyze/read-only) only. None of the activity contents, work email, or work-strategy data flowing through the brief is documented here.

CLI surface

The flow wrappers call one script. You can also run it directly:

Command	Behavior
`uv run python flows/vdb/generate_newsletter.py`	Gather → compose → send → save sent state
`… --preview`	Print subject/HTML/text, don't send, don't save
`… --force`	Skip the idempotency `sent` check
`… --no-send`	Save a draft (`sent:false, draft:true`) without emailing — used by `draft.py`
`… --skip-garmin`	Skip the Garmin sync during gathering — used by the draft pass
`… --revise-sleep`	Revise the existing draft with fresh sleep and send — used by `send.py`

The whole flow runs via stepwise: stepwise run flows/vdb --name "vdb: <date>" --wait (always from ~/work/vita).

Scheduling

Setting	Value	Where
Loop	`_vdb_loop` → `_run_vdb_cycle` → `_run_stepwise_flow("vdb")`	`api/src/scheduler.py`
Task name	`vdb_daily` (gated by `_check_enabled_or_skip`)	`api/src/scheduler.py`
Draft target	4:45 AM PT (`VDB_DRAFT_HOUR=4`, `VDB_DRAFT_MINUTE=45`)	`api/src/scheduler.py`
Poll interval	5 minutes (`VDB_POLL_INTERVAL = 5*60`; FLOW `interval_seconds: 300`)	scheduler / FLOW
Send deadline	07:00 PT (`DEADLINE_HOUR=7`)	`flows/vdb/poll_sleep_check.py`
Poll timeout ceiling	120 minutes	`flows/vdb/FLOW.yaml`
Timezone	`America/Los_Angeles`	scheduler / poll check

_run_vdb_cycle is idempotent at dispatch: it reads today's brief file, and if it's already sent it sleeps until the next day's draft target instead of re-dispatching. The scheduler runs through ./run api — never systemd. Stepwise's job persistence handles recovery of an in-flight flow (a suspended poll step exits non-zero but the server-side job stays alive and resumes on its next tick).

Feedback loop

Replying to the email feeds the next day's brief — no app, just email.

Day N — the brief ships with Message-ID: <vdb-YYYY-MM-DD@…>. The vdb-<date> shape is what makes reply matching possible.
Day N — you reply; the reply carries an In-Reply-To header pointing at that Message-ID.
Every 15 min, 6 AM–10 PM PT — scripts/mail/check_replies.py (via run_replies.sh) fetches the last 24h of inbox, builds a Message-ID → date map from the brief files, and matches In-Reply-To.
On match it extracts the body (plain text preferred, html2text fallback) and writes it into the brief file under feedback ({text, received_at, email_id}). It de-dupes on email_id.
Day N+1 — gather_context loads it as yesterday_feedback (raw) and yesterday_reply_items (parsed into links/questions/directives). Optionally, process_feedback distills recurring feedback into the permanent data/vdb/guidelines.md.

API endpoints

All under /api/v1/vdb (api/src/routers/vdb.py), served on port 33800 (the web UI on 33801 reads through it; the public read-only mirror is 33802).

Method	Endpoint	Description
GET	`/api/v1/vdb/briefs`	Paginated list of briefs (date-descending, `-failed` files excluded)
GET	`/api/v1/vdb/briefs/{date}`	One brief: stored HTML/text, feedback, and the captured context dict
POST	`/api/v1/vdb/briefs/{date}/feedback`	Set/overwrite a brief's feedback string
GET	`/api/v1/vdb/sources`	Per-source health: `ok` / `stale` / `error` / `unconfigured`

Source health check

GET /sources does a lightweight liveness check per source — it does not sync. For each of weather, garmin, calendar, work_email, personal_email, today, rwgps, x it:

returns unconfigured if the config file under ~/.config/vita/ is missing;
otherwise, if the source has a data directory, finds the newest *.json, and returns stale if its mtime is older than the 24h threshold, else ok (with last_sync);
otherwise returns ok on config presence alone.

State files

Brief state lives in data/briefs/YYYY-MM-DD.json. The draft and sent shapes differ:

// draft (after pass 1)
{
  "date": "2026-01-03",
  "sent": false,
  "draft": true,
  "generated_at": "2026-01-03T04:46:10.000000",
  "generator": "claude-newsletter",
  "subject": "...",
  "html": "...",
  "text": "...",
  "garmin_at_generation": { "...": "snapshot used to detect fresh sleep" }
}

// sent (after pass 2 / send)
{
  "date": "2026-01-03",
  "sent": true,
  "sent_at": "2026-01-03T06:12:33.000000",
  "message_id": "<vdb-2026-01-03@PLACEHOLDER>",
  "generator": "claude-newsletter",
  "subject": "...",
  "html": "...",
  "text": "...",
  "feedback": {
    "text": "<sanitized reply text>",
    "received_at": "2026-01-03T08:30:00.000000",
    "email_id": "Mxxxxx"
  }
}

Note: Older briefs may also carry a sections map ({weather: true, garmin: false, …}); the API tolerates its absence. Failed runs are written with a -failed suffix and excluded from list/detail.

Configuration

All credentials live in ~/.config/vita/ (outside the repo). Each source degrades gracefully if its config is missing.

Config file	Service	Used by
`openrouter.json`	OpenRouter (`api_key`)	Newsletter + podcast LLM calls
`fastmail.json`	Fastmail JMAP (`jmap_token`, `account_id`, `from_email`, `to_email`)	Send + reply detection
`weather.json` / `secrets.toml`	OpenWeatherMap	Weather sync
`garmin.json`	Garmin Connect	Sleep / HRV sync
`google_oauth.json`	Primary mailbox + calendar	Email / calendar sync
`google_credentials_personal.json`	Secondary mailbox	Personal email sync
`x.json`	X/Twitter session	Bookmark scrape
`secrets.toml`	Today app, personal IMAP	Tasks / personal email
`rwgps.json`	RideWithGPS	Activity pipeline

File locations

Path	Purpose
`flows/vdb/FLOW.yaml`	Stepwise flow definition (draft → poll → send → podcast)
`flows/vdb/generate_newsletter.py`	The workhorse: gather, compose, parse, save, send, revise
`flows/vdb/draft.py`	Pass-1 wrapper (`--no-send --skip-garmin --force`)
`flows/vdb/poll_sleep_check.py`	Poll executor: sync Garmin, detect fresh sleep, 07:00 deadline
`flows/vdb/send.py`	Pass-2 wrapper (`--revise-sleep`)
`flows/vdb/podcast.py`	Post-send podcast wrapper
`flows/vdb/prompts/voice.md`	Voice + section-structure prompt
`scripts/vdb/gather_context.py`	Context assembly (sync + local)
`scripts/vdb/data.py`	`sync_all_sources()` + per-source loaders
`scripts/vdb/reply_parser.py`	Parse replies into structured items
`scripts/vdb/process_feedback.py`	Distill feedback into guidelines
`scripts/mail/fastmail.py`	JMAP send client; generates the `vdb-<date>` Message-ID
`scripts/mail/check_replies.py`	Reply detection / feedback capture
`scripts/podcast/generate_transcript.py`	Spoken-word transcript + ElevenLabs audio
`api/src/routers/vdb.py`	`/api/v1/vdb` router
`api/src/scheduler.py`	`_vdb_loop` / `_run_vdb_cycle` dispatch
`data/vdb/guidelines.md`	Permanent, feedback-distilled guidelines
`data/briefs/YYYY-MM-DD.json`	Brief state (draft → sent + feedback)
`data/briefs/contexts/YYYY-MM-DD.json`	Captured context for the web "why did it say X?" surface

Where to go next

Living Briefing — the real-time dashboard whose summary the VDB folds in.
Sleep Guardian — the sleep-gate logic surfaced in the body-check section.
Podcast — the audio episode generated from the same context.
Media — the signal pipeline behind media_intel.
Scheduling — drift warnings, commitments, and suggestions.
Improve / Sleep — the overnight cycle that produces the "overnight build" section.
Model Guidance — model routing across the system.