Skip to content

Insights & Patterns

The /insights command surfaces patterns, trends, and correlations from health data — what is changing, what correlates with what, and what might need attention. This page documents /insights itself plus the surrounding machinery it draws on: pre-computed memory views, drift detection, pattern learning, the coaching nudge system, and micro-grit streak tracking.

The single most important architectural fact: /insights is an LLM-executed command, not a statistics engine. It is defined as a slash command in .claude/commands/insights.md — a step-by-step prompt the model follows at runtime. There is no scripts/insights.py, no regression/std-dev module, and no deterministic "pattern detection algorithm" backing it. The "algorithms" described below are guidance the model applies by reading data files and reasoning over them. Everything in this page that is deterministic code (views, drift, pattern-learning, coaching, micro-grit) lives in scripts/ and is cited file-by-file.

How /insights runs

/insights is a command file, not a Python entry point. When invoked, the agent reads .claude/commands/insights.md and walks five steps:

Step What the agent does
1. Check pending insights Read data/scratch/daily.md (or weekly.md) for a ## Pending Insights section; surface those first.
2. Gather data by lookback Read the relevant data files per metric category (see windows below).
3. Analyze patterns Reason over the data for trends, correlations, anomalies, cyclical patterns.
4. Generate insights Produce 3–5 actionable insights, each with supporting numbers.
5. Update context Move surfaced items from ## Pending Insights to ## Recently Surfaced Insights with a timestamp; queue any new ones.

Because the analysis is done by the model rather than by code, the "thresholds" and "factor pairs" the command suggests (e.g. "anomaly at >2 std dev", "sleep duration vs next-day energy") are prompt instructions, not configured constants. A reimplementer should treat them as a checklist the LLM is asked to consider, and either keep them as prompt guidance or replace them with real statistical code if deterministic rigor is wanted.

Lookback windows (prompt guidance)

The command tells the model to use different analysis windows by metric speed. These windows are written into the prompt, not enforced anywhere in code:

Metric category Lookback Data sources
Daily (sleep, energy) 7 days check-ins, garmin
Weekly (training volume) 30 days activities
Slow (weight, body comp) 90 days weight, body-comp scans
Lab/hormone panels 180 days lab extracts

Pending-insight storage

Pending and surfaced insights are plain markdown sections inside data/scratch/daily.md:

## Pending Insights
<!-- Insights queued for next check-in -->
- Sleep down ~0.5 hrs on average this week
- Training volume up ~20% vs last month

## Recently Surfaced Insights
<!-- Insights shown in recent check-ins -->
- 2026-05-25 14:32 PT: HRV stabilized after recovery week

Note: Pre-2026-05-25 these sections lived in a file named CONTEXT_SUMMARY.md, which was retired on that date; the sections migrated to data/scratch/daily.md. A future structured-storage migration to data/insights/pending.jsonl is still queued — data/insights/ does not yet exist. See Scratch Pads for the daily/weekly markdown layering.

Pre-computed memory views (the data foundation)

/insights does not crunch raw garmin/check-in JSON from scratch. It leans on memory views — LLM-synthesized summaries rebuilt from raw data and cached as JSON. See Memory & Trends for the full view system; the views most relevant to insights are health-oriented:

View What it synthesizes
health-snapshot Body composition, cardiovascular fitness, sleep averages, lab/hormone status.
week-summary Last 7 days of sleep/steps/HRV with trends, highlights, and concerns.
training-status Recovery readiness (HRV status, sleep score), recent workouts, weekly training vs targets.
goal-progress All active goals with progress, frequency-goal tracking, attention items.
identity-snapshot Communication preferences/patterns (used for personalization, not metric analysis).

The view registry (memory/_meta/view-registry.json) also defines additional non-health views — ceo-scorecard and current-status — plus extra view JSON exists in memory/views/ (ceo-progress, relationship-health, calendar-travel-context). Those are general life/work surfaces, not part of the health-insights path, and are out of scope here.

View structure

Each view is a JSON document at memory/views/{view-id}.json:

{
  "content": {
    "summary": "Brief narrative summary...",
    "metrics": { "...": "..." },
    "highlights": ["...", "..."],
    "concerns": ["...", "..."]
  },
  "citations": [
    { "claim": "Average sleep ~7 hours", "source": "data/garmin/<date-range>" }
  ],
  "view_id": "week-summary",
  "built_at": "2026-01-02T22:19:50Z",
  "builder": "claude --print --tools",
  "prompt_hash": "<hex>"
}

The builder field is literally "claude --print --tools" — views are rebuilt by invoking Claude headlessly with the view's prompt. Freshness is tracked separately in memory/_meta/staleness.json (fresh / stale per view with last_built and stale_at).

View building

scripts/memory/build_view.py is the deterministic part: it gathers source data, fills a prompt template, and runs the builder.

  • Prompts: memory/prompts/build-*.md, one per view (e.g. build-health-snapshot.md, build-week-summary.md, build-training-status.md, build-goal-progress.md).
  • Source gatherers: gather_sources_<view>() functions in build_view.py read raw files and return a dict of template variables. gather_sources(view_id) dispatches to the right one.

Prompt templates use {variable} placeholders filled by the gatherers:

Variable Source
{garmin_data} Recent data/garmin/YYYY-MM-DD.json (rolling window computed inline in the gatherer)
{checkins_data} Recent data/checkins/YYYY-MM-DD.json
{dexa_data} Latest body-composition scan under memory/extracted/performance/
{hormone_labs} Latest lab/hormone panel under memory/extracted/medical/ (content truncated to ~10 KB)
{goals_data} All goals/active/*.json
{occurrences_data} Last ~3 months of data/scheduling/occurrences-*.jsonl

Note: There is no scripts/stats.py and no standalone rolling-statistics module. Rolling averages (sleep, HRV, steps) are computed inline inside the gather_sources_*() functions in build_view.py while assembling the prompt variables. If you reimplement, that's where the math lives.

Data sources analyzed

Source Location Contents
Garmin sync data/garmin/YYYY-MM-DD.json Sleep, HRV, steps, body battery
Check-ins data/checkins/YYYY-MM-DD.json Self-reported energy, mood, notes
Activities data/activities/YYYY-MM-DD.json Logged workouts and sessions
Goals goals/active/*.json Targets and deadlines
Occurrences data/scheduling/occurrences-YYYY-MM.jsonl Frequency-goal logging
Lab panels memory/extracted/medical/*.json Blood / hormone panel extracts
Body-comp scans memory/extracted/performance/*.json Composition + cardiovascular fitness tests

See Garmin and Check-in for how the upstream data is collected.

Garmin daily example (sanitized)

{
  "date": "2026-01-01",
  "synced_at": "2026-01-02T13:55:50Z",
  "sleep": {
    "duration_hours": 7.5,
    "deep_hours": 1.5,
    "light_hours": 4.0,
    "rem_hours": 2.0,
    "awake_hours": 0.1,
    "score": 80
  },
  "hrv": { "average_ms": 60, "status": "BALANCED" },
  "steps": 9000
}

(Values are illustrative placeholders.)

Drift detection

scripts/scheduling/drift_detector.py measures how well frequency-based goals are tracking their targets. It is real deterministic code (unlike /insights's own analysis) and feeds the briefing.

Status levels (DriftStatus enum)

Status Meaning
ON_TRACK Within target interval
WARNING Approaching deadline
OVERDUE Past target but within tolerance
CRITICAL Past tolerance window
NO_DATA No occurrences logged yet
QUIET Suppressed by quiet_mode on the goal

Goal types

  • Interval goals (check_interval_goal): compute days since the last occurrence against target_interval_days, with drift_warning_days (default target − 1) and tolerance_days (default 2) determining the WARNING / OVERDUE / CRITICAL bands.
  • Weekly goals (check_weekly_goal): count each component's occurrences inside the current week window and compare to its target. Week boundaries come from get_week_boundaries(week_starts, reset_hour) — defaults Monday start, 4 a.m. reset (4 a.m. chosen to dodge DST edges). Each component is graded independently.

Occurrence timestamps are parsed defensively by _parse_occurrence_time(), which tries occurred_at, then logged_at, then a date field, and forces every result tz-aware (TZ-localizing naive legacy timestamps so comparisons against datetime.now(TZ) don't crash).

How drift reaches the briefing

The filtering and capping happen in scripts/scheduling/engine.py (≈ lines 589–671), not in the briefing layer:

  • Interval goals with status WARNING, OVERDUE, or CRITICAL are appended to drift_warnings, then truncated to [:2].
  • Weekly-goal components in WARNING are routed to suggestions instead (not drift_warnings).
  • Calendar-based time suggestions for the top drifting goal are added, and suggestions is then capped at [:3].

So the "2 items in the brief" limit is an interval-goal-only cap applied in engine.py. See the Living Briefing for how these are displayed.

Pattern learning

scripts/scheduling/pattern_learner.py learns when activities tend to happen, so scheduling suggestions can recommend the right day and time slot.

How it works

  1. Each logged occurrence calls update_patterns(goal_id, component, occurred_at), incrementing day-of-week and time-slot counters (file-locked write).
  2. Counters are normalized to weights and stored in data/scheduling/patterns.json, keyed by goal_id (or goal_id:component for weekly goals).
  3. get_preferred_days() / get_preferred_times() only return learned values once a pattern has MIN_OCCURRENCES = 3 occurrences; otherwise they fall back to the goal's configured preferred_days / preferred_times.
  4. decay_old_patterns() applies exponential decay to any pattern not updated within days_threshold (default 90 days), using DECAY_RATE = 0.85 raised to (days_since / 30) (monthly decay), then re-normalizes.

Time slots

Slot Hours
morning 6:00 – 12:00
afternoon 12:00 – 17:00
evening 17:00 – 22:00

Hours outside these ranges default to evening.

Coaching system

scripts/coaching/ generates proactive, context-aware coaching nudges (delivered via Telegram). It turns a goal's drift/progress state into a short message that references identity values rather than raw metrics.

The scripts/coaching/ directory also contains work- and comms-calibration modules (communication.py, post_meeting_debrief.py, and others) that are unrelated to health insights and out of scope for this page. The insight-relevant core is the four files below plus identity_loader.py.

File Purpose
generator.py Picks a context and renders a templated (or LLM-generated) message.
mapper.py Maps a goal to identity domains so framing references values, not just numbers.
effectiveness.py Logs each send and enforces the per-goal cooldown.
proactive.py Orchestrates the batch: find stagnating goals, skip ones on cooldown, queue, and log.

Message generation

generator.py classifies each goal into one of five ContextType values — progress, stagnation, setback, celebration, new — via determine_context(), then renders a template. Templates come in two banks: TEMPLATES_WITH_QUOTE (which splice in a quote the user has said before) and TEMPLATES_NO_QUOTE. Templates use {goal_title}, {days}, {occurrences}, and {quote} placeholders only — no personal content is hard-coded. An optional generate_coaching_message_llm() path produces a fully LLM-written message instead of a template.

Goal → identity mapping

mapper.py extracts goal keywords, finds relevant identity observations and quotes, and resolves which identity domains a goal touches (get_or_create_mapping, cached). The map lives at data/coaching/goal-identity-map.json. Conceptually a goal like backflip maps to a physical_capability domain and date_night to relationships — the mechanism lets a nudge speak to why the goal matters, not just the streak count.

Effectiveness logging and cooldown

This is narrower than it sounds. effectiveness.py records only the send event — it does not track acknowledgement, action, or whether a nudge was ignored.

log_coaching_event() appends one JSON line to data/coaching/effectiveness.jsonl:

{
  "event_id": "<goal_id>-<UTC timestamp>",
  "ts": "2026-01-01T12:00:00+00:00",
  "goal_id": "<goal-id>",
  "message": "<truncated to 200 chars>",
  "context": "stagnation",
  "source": "proactive",
  "template_used": "<template or null>",
  "quote_used": "<truncated to 50 chars or null>"
}

It then writes the goal's last-coached time into data/coaching/cooldowns.json. is_on_cooldown(goal_id) returns true if the goal was coached within COOLDOWN_HOURS = 12 (a fixed 12-hour minimum gap per goal). get_effectiveness_stats(days=30) just tallies totals by context and by source over a window — there is no acknowledged / acted-on / ignored detection anywhere in scripts/coaching/.

Warning: Treat coaching effectiveness as observational only. The log captures what was sent, not what worked, so it cannot (yet) be used to optimize timing, framing, or frequency. A reimplementer wanting a real feedback loop must add outcome capture (e.g. correlating sends with subsequent occurrences or replies).

Proactive delivery

proactive.py's run_proactive_coaching() finds stagnating goals (check_stagnating_goals(stagnation_days=7)), drops any goal already on cooldown, builds a batch, queues it for Telegram, and calls log_coaching_event() for each. See Proactive Outreach for the delivery queue.

Micro-grit streak tracking

scripts/identity/microgrit.py runs the daily micro-grit loop: a small task (or up to a few) set each morning and checked each evening, with streak analytics. State lives at data/identity/micro-grit.json.

{
  "v": 2,
  "enabled": true,
  "current_streak": 5,
  "longest_streak": 14,
  "today": {
    "date": "2026-01-28",
    "tasks": [{"task": "10 min mobility", "completed": true}],
    "morning_sent": true,
    "evening_sent": true
  },
  "history": [],
  "milestones": {
    "7": "one week strong",
    "14": "two weeks - this is becoming you",
    "30": "a month. this IS you now",
    "100": "triple digits. unstoppable"
  }
}

Verified constants and behavior:

  • Schema v: 2 — multi-task per day, each with its own completed flag (v1 single-task auto-migrates on load).
  • MAX_DAILY_TASKS = 3 — too many tasks drives 0% completion, so the daily count is capped.
  • MAX_HISTORY = 30 — history is pruned to the last 30 archived days on day rollover.
  • _load_state() rolls the day over automatically when today.date is stale, archiving yesterday's tasks with completed_count / total_count.
  • Current/longest streak and milestone celebration text match the table above exactly.

Delivery (morning task push, evening check-in) goes out over Telegram; see Proactive Outreach.

File locations

File Purpose
.claude/commands/insights.md The /insights command — LLM-executed prompt, step-by-step instructions
scripts/memory/build_view.py Gathers sources, fills prompts, rebuilds memory views (rolling averages computed inline here)
memory/views/*.json Pre-computed view documents
memory/prompts/build-*.md Per-view builder prompts
memory/_meta/view-registry.json View definitions and dependencies
memory/_meta/staleness.json Per-view freshness state
scripts/scheduling/drift_detector.py Frequency-goal drift status (DriftStatus, interval/weekly checks)
scripts/scheduling/engine.py Filters/caps drift warnings + suggestions for the briefing ([:2]/[:3])
scripts/scheduling/pattern_learner.py Learns preferred days/time slots from occurrence history
data/scheduling/patterns.json Learned activity patterns
scripts/coaching/generator.py Coaching message context + templates
scripts/coaching/mapper.py Goal → identity-domain mapping
scripts/coaching/effectiveness.py Send-event log + 12h cooldown
scripts/coaching/proactive.py Proactive coaching orchestration
data/coaching/effectiveness.jsonl Coaching send log
data/coaching/cooldowns.json Per-goal coaching cooldown timestamps
data/coaching/goal-identity-map.json Goal → identity-domain map
scripts/identity/microgrit.py Micro-grit state + streak management
data/identity/micro-grit.json Micro-grit state and streaks
data/scratch/daily.md Pending / recently-surfaced insight sections

Known limitations

  1. No statistical rigor in /insights. Trend/correlation/anomaly detection is LLM reasoning over data, not Pearson/Spearman or regression. There are no confidence intervals or p-values; spurious correlations are possible.
  2. Suggested factor pairs and thresholds are prompt text. They cannot be added or tuned without editing the command file; nothing reads them as configuration.
  3. Fragile insight storage. Pending/surfaced insights are freeform markdown in data/scratch/daily.md. There is no queryable insight history; once surfaced, an insight is not retrievable as structured data.
  4. Coaching effectiveness is send-only. No outcome detection — it can describe what was sent, not what worked.
  5. Fixed windows and cooldowns. Lookback windows (7/30/90/180) and the 12-hour coaching cooldown are hardcoded.

Improvement opportunities

  1. Structured insight storage — migrate the markdown sections to data/insights/pending.jsonl + history.jsonl (directory not yet created) for a queryable surface and to kill the regex-over-markdown fragility.
  2. Real statistical tests — add deterministic correlation/anomaly code with significance thresholds where rigor matters.
  3. Coaching feedback loop — capture acknowledgement / subsequent-occurrence signals so effectiveness data can optimize timing and framing.
  4. Adaptive lookback — learn the right window per metric from its variance instead of hardcoding.
  5. Cross-goal correlation — detect when progress on one goal predicts another.

Where to go next