Homeostasis & Predictions

The homeostasis subsystem makes a single claim from vita's manifesto measurable: that the sync between bio-zack and silicon-zack has a deviation that can be quantified and watched, not merely asserted. It has two parts — a falsifiable-prediction ledger that scores how often silicon-zack's claims about bio-zack come true, and a deviation aggregator that rolls every instrumentable sync proxy into one dict for /sleep and the executive loop.

Why this exists

The manifesto defines the homeostasis target as measurable. From docs/pages/manifesto.md:

Deviation from equilibrium is measurable: - Questions silicon-Zack can't answer - Predictions that miss - Recommendations bio-Zack rejects as out-of-touch - Drafts that require significant revision

Of those four, "predictions that miss" is the most falsifiable — and it was uninstrumented until 2026-06-10. This package is the v1 measurement scaffold: turn a vibe ("I know him accurately") into a number (calibration drift). The governing design principle is that a half-dark instrument that still reports is better than one that crashes — every read degrades gracefully, because in vita the dominant systemic failure mode is instrumentation that lies rather than instrumentation that breaks loudly.

Note: This is self-instrumentation, not a forecasting product. The predictions are about bio-zack and the near future ("he'll skip the Thursday ride", "this nudge gets ignored", "the gate clears in two nights"). The point is the calibration curve, not any individual call.

Architecture

  silicon-zack makes a falsifiable claim
                 |
                 v
   log_prediction(statement, horizon, confidence, source, tags)
                 |
                 v
   data/homeostasis/predictions.jsonl   <-- append-only, two record types
                 |                            (prediction + resolution)
        horizon passes
                 |
                 v
   resolve_prediction(id, "hit"|"miss"|"void", note)
                 |
                 v
   calibration_summary()  -- hit rate, per-bucket rates, Brier score
                 |
   +-------------+------------------------------------------+
   |                                                        |
   v                                                        v
  aggregate_deviation_signals()  <-- also reads:           /sleep
   - proactive engagement (frequency_state.json)            preprocess
   - reminder ack rates (queue.db reminder_acks)            step 7c
   - composer voice drift (daily.md audit)                     |
                 |                                              v
                 v                              data/sleep/staging/homeostasis.json
            write_staging()  ------------------------------------>

Two modules live under scripts/homeostasis/:

Module	Role
`predictions.py`	The falsifiable-prediction ledger: log, resolve, list, score.
`deviation.py`	The aggregator: collect every sync-deviation proxy (including the ledger's calibration) into one dict and stage it for `/sleep`.
`__init__.py`	Lazy PEP-562 re-exports so `from scripts.homeostasis import log_prediction` works without triggering the `python -m homeostasis.deviation` double-import warning.

Tip: vita puts scripts/ on PYTHONPATH, so imports are written from homeostasis.predictions import log_prediction (not from scripts.homeostasis...) inside the codebase. Both spellings resolve; the module's own try/except ImportError handles either.

The prediction ledger

Data model

The ledger is a single append-only JSONL at data/homeostasis/predictions.jsonl. Two record types share the file, distinguished by type:

{"type": "prediction", "id": "ab12cd34", "ts": "2026-06-12T09:14:03",
 "statement": "<falsifiable claim, stated so a future reader can score it>",
 "horizon": "2026-06-19", "confidence": 0.7, "source": "session", "tags": ["example"]}

{"type": "resolution", "id": "ef56gh78", "prediction_id": "ab12cd34",
 "ts": "2026-06-19T08:02:11", "outcome": "hit", "note": "<short evidence>"}

Field (prediction)	Meaning
`id`	UUID from `scripts.utils.generate_uuid()`; the key resolutions point back to.
`ts`	Local-time ISO timestamp of when it was logged.
`statement`	The claim. Stored stripped; an empty statement raises `ValueError`.
`horizon`	ISO date (`YYYY-MM-DD`) by which it resolves. Non-ISO raises `ValueError`.
`confidence`	Float in `[0, 1]` — silicon-zack's probability the statement is true. Out-of-range raises `ValueError`.
`source`	Where it originated (`session`, `ooda`, `sleep`, `cli`, …). Defaults to `session` in code, `cli` from the command line.
`tags`	Optional topic tags (list).

Field (resolution)	Meaning
`prediction_id`	The id of the prediction being resolved.
`outcome`	One of `hit`, `miss`, `void`. Anything else raises `ValueError`.
`note`	Free-text evidence/explanation.

The split-record design means the file is never rewritten — both logging and resolving are pure appends via atomic_append(). The reader (_read_ledger) skips blank and corrupt lines so one malformed line never poisons the ledger.

API

from homeostasis.predictions import (
    log_prediction, resolve_prediction,
    list_open_predictions, calibration_summary,
)

# Log a falsifiable claim with a horizon and honest confidence.
pid = log_prediction(
    "bio-zack skips the Thursday ride",
    "2026-06-13",          # horizon (ISO date)
    0.7,                   # confidence in [0, 1]
    source="session",
    tags=["training"],
)

# When the horizon passes, resolve it.
resolve_prediction(pid, "miss", note="he rode after all")

Function	Behavior
`log_prediction(statement, horizon, confidence, source="session", tags=None)`	Validates inputs, appends a `prediction` record, returns the new id.
`resolve_prediction(prediction_id, outcome, note="")`	Appends a `resolution`. Raises `ValueError` on unknown id, already-resolved id, or invalid outcome — you cannot double-resolve.
`list_open_predictions(now=None)`	Returns every unresolved prediction, each annotated `overdue: bool` (horizon strictly before today, or an injected `now`).
`calibration_summary()`	The scorecard: overall hit rate, per-confidence-bucket hit rates, and Brier score.

Scoring & calibration

calibration_summary() is where prediction becomes measurement. It pairs each resolved prediction with its confidence and scores it:

Outcome encoding: hit → 1, miss → 0. void resolutions are excluded entirely — a voided prediction is one that became unscoreable, not a wrong one.
Hit rate: fraction of scored predictions that hit.
Confidence buckets: predictions are grouped by stated confidence so you can check whether the confidence is honest. Well-calibrated means the 0.7–0.9 bucket actually hits ~70–90% of the time.

Bucket	Range
`0-0.5`	`[0.0, 0.5)`
`0.5-0.7`	`[0.5, 0.7)`
`0.7-0.9`	`[0.7, 0.9)`
`0.9-1.0`	`[0.9, 1.0]` (top bucket is inclusive of 1.0)

Brier score: mean((confidence - outcome)^2), lower is better. 0.25 is the score you'd get from saying "50%" about everything — so a Brier below 0.25 means the confidences carry real information. This is the single number that quantifies calibration drift.

Example shape (illustrative values):

{
  "count": 11,
  "hit_rate": 0.7273,
  "brier": 0.1841,
  "buckets": {
    "0-0.5":   {"n": 1, "hit_rate": 0.0},
    "0.5-0.7": {"n": 3, "hit_rate": 0.6667},
    "0.7-0.9": {"n": 5, "hit_rate": 0.8},
    "0.9-1.0": {"n": 2, "hit_rate": 1.0}
  }
}

CLI

The ledger has a small argparse CLI (run with scripts/ on PYTHONPATH):

python -m homeostasis.predictions log "bio-zack skips the Thursday ride" 2026-06-13 0.7 \
    --source session --tags training
python -m homeostasis.predictions resolve <prediction_id> hit --note "..."
python -m homeostasis.predictions list       # open predictions as JSON
python -m homeostasis.predictions summary     # calibration summary as JSON

log prints the new id; resolve prints resolved <id> -> <outcome>; list and summary print indented JSON.

The deviation aggregator

deviation.py collects every sync-deviation proxy that exists in vita's data layer today into one dict. It is the layer the manifesto's four-bullet definition actually maps onto. Calibration is one input; the others are behavioral signals about whether bio-zack engages with what silicon-zack produces.

from homeostasis.deviation import aggregate_deviation_signals, write_staging
signals = aggregate_deviation_signals()   # dict, four sections
path = write_staging()                     # also writes the /sleep staging file

Sections

aggregate_deviation_signals() returns a dict with four independently-guarded sections. Every section returns {"available": False, "error": "..."} on any failure rather than raising — one dark source never blanks the whole instrument.

Section	Reads	Measures (maps to manifesto bullet)
`proactive_engagement`	`data/proactive/frequency_state.json`	Source counts, recent send/response totals, response rate, and "silent streaks" — sources with ≥5 consecutive unanswered sends. (Recommendations rejected as out-of-touch — bio-zack ignoring nudges.)
`reminder_acks`	`data/proactive/queue.db` → `reminder_acks` table (read-only URI)	Reminder ack/abandon counts overall and over a trailing 30 days, with a 30-day ack rate.
`predictions`	`data/homeostasis/predictions.jsonl`	The full `calibration_summary()` plus counts of `open` and `overdue` predictions. (Predictions that miss.)
`voice_drift`	composer voice-drift audit in `data/scratch/daily.md` (with archive fallback)	How often a blind classifier re-attributes silicon-zack's drafts back to his own voice. (Drafts that require significant revision.)

Graceful-degradation details worth knowing for a reimplementation

Read-only DB access: the reminder-ack read opens the queue with sqlite3.connect("file:...?mode=ro", uri=True) so a missing or locked DB is never created or mutated by the aggregator.
Silent-streak threshold: SILENT_STREAK_THRESHOLD = 5. The worst five streaks are surfaced, sorted descending.
Voice-drift timing fallback: the composer audit writes to daily.md during the day, but daily.md rolls over at 00:05 PT and the aggregator stages at ~00:31, so on the nightly schedule today's file has no audit section. The reader falls back to yesterday's then the day-before-yesterday's archived daily (data/scratch/archive/daily/<date>.md) and reports which file it used via source / stale_days.
Honoring the audit's own verdict (the anti-lying-instrument logic): the voice-drift audit's "overall: x/y" line counts items the blind classifier put back on the correct voice — high is healthy. drift_ratio is the complement (1 - ratio), so a 20/20 healthy audit reads drift_ratio: 0.0. But a 0/N reading is ambiguous: it could mean total drift or a classifier that simply couldn't discriminate. So:
Below _MIN_VOICE_SAMPLE = 5 posts, the section returns unavailable — too few samples to mean anything.
A "low confidence / scattered" verdict returns unavailable — that's a classifier failure, indistinguishable from drift, and rendering it as max-drift would be the instrument lying.
Only an explicit "drift detected" qualifier produces a status: "drift" verdict (with a drift_to voice if the audit names one). Otherwise status: "ok".

Staging for /sleep

write_staging() wraps the aggregate with a generated_at timestamp and atomically writes it to data/sleep/staging/homeostasis.json:

{
  "generated_at": "2026-06-21T00:31:02",
  "signals": { "proactive_engagement": {...}, "reminder_acks": {...},
               "predictions": {...}, "voice_drift": {...} }
}

This is wired into the nightly /sleep pipeline at step 7c of scripts/sleep/preprocess.py, which calls write_staging() inside a try/except (a failed aggregator logs an error but never breaks the sleep run — graceful degradation all the way up). /sleep then reads the staging file as one of its reflection inputs.

CLI

python -m homeostasis.deviation            # print the aggregate dict as JSON
python -m homeostasis.deviation --write    # also write the /sleep staging file

Reimplementation notes

Append-only, two-record JSONL is the whole storage layer — no database, no migrations. Predictions and their resolutions are separate records linked by prediction_id, which keeps logging and resolving as pure appends and makes the ledger trivially auditable by hand.
Calibration over accuracy. The Brier score and per-bucket hit rates matter more than the raw hit rate: the goal is honest confidence, not optimistic prediction. A system that's right 70% of the time and says 70% is calibrated; one that's right 90% but always says 99% is not.
The aggregator is a read-only fan-in. It mutates nothing it reads; it only writes its own staging file. Each input is behind a guard that converts any exception into {available: False, error}, so the aggregate is always well-formed even when half its sources are missing.
Tests: tests/test_homeostasis.py covers both modules (validation, double-resolve rejection, bucket edges, Brier math, and the degradation paths).

File Locations

Path	Purpose
`scripts/homeostasis/predictions.py`	Prediction ledger: log / resolve / list / score + CLI.
`scripts/homeostasis/deviation.py`	Deviation aggregator + `/sleep` staging writer + CLI.
`scripts/homeostasis/__init__.py`	Lazy re-exports.
`data/homeostasis/predictions.jsonl`	The append-only prediction ledger.
`data/sleep/staging/homeostasis.json`	Aggregated signals staged for `/sleep`.
`scripts/sleep/preprocess.py` (step 7c)	Where the aggregator is invoked nightly.
`tests/test_homeostasis.py`	Test coverage for both modules.

Where to go next

Manifesto — the homeostasis target and the four-bullet definition of measurable deviation.
Executive Loop — the OODA loop that consumes deviation signals and logs predictions from its own cycles.
Improve & Sleep — the nightly pipeline that stages and reflects on the calibration summary.
Sleep Guardian — a sibling system that also scores predictions against outcomes and self-tunes on accuracy.
Proactive Outreach — the engagement/ack data that feeds two of the aggregator's four sections.