iOS App

A native SwiftUI iPhone client for the VITA server. After the 2026-07-21 overhaul the app is voice-first and deliberately thin: its default surface is a hands-free voice mode that turns spoken utterances into messages injected into telegram topic sessions, and its remaining tabs — location, photos, podcast, and (rebuilt) meetings — are all thin views over server-side APIs. Almost no logic lives on the device: transcription, routing, storage, agent work, and every reply happen on the server, reached over a private Tailscale network. The one deliberate exception is the meetings tab's live agenda-item voice session, which connects the phone directly to OpenAI's realtime API — but even there the phone never holds a key or a prompt (both are baked server-side into an ephemeral, short-lived credential).

Framing

silicon-zack lives on the server. The iOS app is one of his embodiments — a way for bio-zack to feed him voice, photos, location, and to hold live agenda working-sessions hands-free. The design rule that follows: the phone holds no intelligence and no persistent secrets. No API keys ship to the device, no model runs on it. It records audio, captures GPS, uploads photos, and — for voice — fires-and-forgets. The live agenda-item session is the sole live-model surface, and it runs on a server-minted ephemeral credential (120s TTL) with the persona/grounding/tools baked in server-side, so the phone still never sees a raw key or a prompt.

The key framing shift in the overhaul: the voice inbox is fire-and-forget messaging, not conversation. The phone does not host a chat. A spoken utterance becomes a message dropped into a telegram topic, and silicon-zack replies there, in telegram, in the resident session that owns that topic. This replaced the old turn-based voice-session UI (record → server Claude turn → TTS → play), which is gone.

What the overhaul removed

~73 Swift files were deleted. Removed from the app entirely:

• Turn-based voice sessions — VoiceSessionManager family, the Voice chat/session-list UI, and the hands-free PCM WebSocket path.
• Clux sessions, Cadence remote-session control, and Cloud session streaming views — remote control of server sessions from the phone was unwanted.
• Feed UI, Reports tab, SystemKernel overview.
• The pre-existing meetings realtime-surrogate stack — the old Gemini Live / OpenAI Realtime surrogate that let silicon-zack "join" a live meeting, plus the record→upload meetings flow that briefly replaced it on the morning of 2026-07-21. Both are gone. The morning's record→upload flow was killed the same day (bio-zack: the phone will never capture meetings — his HiDock/recorder system owns capture); the meetings tab is now agenda meetings (below), a fundamentally different feature. Deleted iOS classes: MeetingRecorder, MeetingUploadManager / MeetingMultipartBody, and the old Views/Sync/ UI.

The corresponding server routers (voice.py, cloud.py, clux.py, reports.py, system_kernel.py, and the non-push parts of feed.py) are still registered in main.py but are now dead code from the app's perspective; feed.py's push-registration endpoints stay live. The old meetings surrogate stack is fully deleted server-side too: meetings_upload.py, the pre-existing sync_meetings.py router, and sync_voice.py are gone (not just dead). The remaining dead endpoints are flagged for later pruning.

Retained: photos (upload + tab), location, podcast player, meetings (rebuilt as agenda meetings), settings/onboarding, push notifications, debug logging, and the audio primitives (AudioRecorder / AudioPlayer / AudioSessionManager / ToneGenerator / SessionNameSpeaker).

The tab shell

ContentView deliberately does not use SwiftUI's TabView (with overflow it auto-wraps into an unwanted "More" page). Instead it uses a manual switcher — a ZStack that keeps every tab's view alive across switches — plus a custom icon-only VitaTabBar. The tab enum is the source of truth (Design/Components/VitaTabBar.swift):

Index	Tab	Controller / Manager	View
0	Voice (default)	VoiceModeController	VoiceModeView
1	Location	LocationManager	LocationTab
2	(removed)	—	old projects/clux tab retired
3	Photos	PhotosManager	PhotosTabView
4	Podcast	PodcastManager	PodcastTabView
5	Meetings	AgendaMeetingManager	MeetingsTabView

The app launches on tab 0. vitaApp.swift is the @main entry: it registers the photo-upload background task (com.vita.photo-upload), wires APNs registration through AppDelegate, and shows OnboardingView if credentials are missing (which, since network auth is Tailscale, effectively never — KeychainHelper.hasRequiredCredentials() returns true; onboarding still sets the server URL).

Voice mode (the flagship surface)

Voice mode is a hands-free, on-device wake-word listener that captures a spoken utterance and dispatches it to the server voice inbox. It is designed to run while bio-zack is rowing, biking, or driving — with a podcast playing.

On-device state machine (VoiceModeController)

off → starting → armed
armed → triggered        (wake phrase OR PTT button)
triggered → recording    (earcon, then capture) OR → armed (abort)
recording → finalizing   (silence timeout / "send it" / 120s max)
          → armed         ("never mind" / too short)
finalizing → dispatching → confirming(2s) → armed
any → suspended          (AVAudioSession interruption began, e.g. a call)
suspended → armed        (interruption ended)

Mechanics that matter:

• Wake phrase. WakeWordListener runs an on-device SFSpeechRecognizer in 50-second rolling windows (proactively rolled during silence so a wake phrase is never cut mid-word), with an RMS pre-gate so silence doesn't waste the recognition budget. Default phrase "hey vita" plus homophones (hey veda, hey rita, …). A 3-second debounce prevents double-triggers. If speech recognition is unavailable, the wake word is disabled and the app falls back to PTT-only.
• Single shared audio engine. VoiceModeEngine owns one AVAudioEngine and one input tap, fanning each buffer to the wake listener and (during capture) the recorder. There is no second engine.
• Podcast keeps playing while armed. The listen-mode session is .playAndRecord + .mixWithOthers + .allowBluetoothA2DP + .defaultToSpeaker (VoiceModeAudioSession.listenConfig). This keeps high-quality A2DP output on the headphones while capturing on the phone's built-in mic — it never enables plain .allowBluetooth (HFP), which would force both directions onto the low-quality mono Bluetooth link and tank the podcast.
• On trigger. An earcon plays; then:
• if the in-app podcast is playing, it's paused via PodcastManager.pausePlaybackForVoiceMode() and the mixable listen session is kept;
• if nothing of ours is playing, the session switches to the non-mixable capture config (.mixWithOthers dropped), which interrupts external audio (Spotify etc.) for a clean recording.
• Recording ends on 1.8s of silence (configurable 1.0–3.0s), the phrase "send it", the phrase "never mind" (cancel/discard), or a 120s hard cap. The utterance is an M4A.
• Resume. After the file closes, the session is released with .notifyOthersOnDeactivation so external audio resumes fast, and our own podcast resumes on the episode we paused.
• PTT fallback. A push-to-talk button in VoiceModeView works in every state — the always-available manual path, independent of wake-word availability.

Offline outbox (VoiceModeOutbox)

Every utterance is persisted to disk (m4a + JSON manifest) with an idempotency key BEFORE the first POST, so a crash or kill between capture and upload never loses audio. Failed/offline utterances stay queued and drain FIFO when connectivity returns (NetworkMonitor) or on foreground, with bounded retry (3 attempts at 1s/4s/16s). The idempotency key is echoed to the server so a replay returns the stored result (status: duplicate) rather than re-injecting.

Server pipeline (voice_inbox.py + voice_inbox_routing.py)

The app POSTs multipart/form-data to POST /api/v1/voice-inbox/dispatch. Fields: audio (the m4a, audio/mp4), optional transcript_hint, optional topic_hint, dry_run, idempotency_key. Server flow:

1. Idempotency replay — a seen idempotency_key returns the stored response (status: duplicate) before any effect.
2. Transcribe — Groq Whisper (scripts/integrations/groq_whisper.py); audio wins, else the transcript_hint. (The Groq mime bug is fixed: the content-type is now derived from the filename extension — iOS m4a → audio/mp4 — instead of being hardcoded to audio/ogg.) STT failure → HTTP 422.
3. Resolve target topic (resolve_topic):
4. topic_hint (client already knows the id or name), then
5. inline directive — the speaker named the topic ("send this to the education topic: …"); the directive is stripped from the injected body but the full transcript is still echoed. Aliases (ed→education, doc→doctor, …) and fuzzy matching (difflib floor 0.75, margin 0.08) apply, then
6. LLM fallback — a small openrouter model picks the best topic from the live list, then
7. default — a durable topic literally named voice, created once and persisted; a routing_note records that resolution fell through.
8. Echo — post 🎤 <transcript> into the resolved topic via vita.telegram.send (synchronous, ledger-recorded so the resident session sees it as context).
9. Inject — enqueue_prompt(topic_id=…) drops the bare transcript (with only a short (voice message from bio-zack) marker — never duplicating the echoed text) into the resident telegram-<topic_id> session, which runs it on its next scheduler tick and replies in the topic.

The response carries transcript, resolved_topic (topic_id, name, method ∈ hint/alias/directive/llm/default, confidence), candidates, and status ∈ dispatched/dry_run/duplicate/error. GET /api/v1/voice-inbox/topics?limit=N returns the live topic list for the client's hint picker.

E2E verified live 2026-07-21 (topic 13732 test): the echo appeared in the topic and the resident session replied in-topic.

Meetings (rebuilt: agenda meetings — prep → live per-item duplex voice → async processing → wrap)

The record→upload flow is gone. Meetings are now structured solo working sessions: bio-zack hands silicon-zack a goal, silicon-zack researches and lays out a grounded agenda, and they talk it through one item at a time in a live duplex voice call — with the real work happening asynchronously between items and a written summary at the end. Nothing on the phone is a meeting recorder; the phone is a live-conversation client. (Capture of actual meetings lives entirely with the HiDock recorder / recorder-service ingest path — not the phone.)

The flow

1. Compose a goal. MeetingComposerView — "what do you want to work through?" — a goal field (e.g. plan the growth WBR for tuesday's L10) + optional notes. POST /api/v1/agenda-meetings opens a per-meeting telegram topic, persists the meeting (status preparing), and launches the prep agent.
2. Prep (a full Claude agent). A full-toolset agent (agenda_prep task type, via the claude daemon / stream_inference) researches the goal — searches vita's long-term memory, reads files, pulls live data — and breaks it into 2-6 agenda items. Each item gets a title, goal, integer suggested_minutes, and a compiled grounding_prompt: a context pack (facts with sources, prior decisions, open questions, what to push bio-zack on) that arms the live voice agent. Status flips preparing → ready; the agenda posts to the telegram topic.
3. Run each item as one bounded duplex voice session. In MeetingDetailView, "start" on an item opens LiveItemView. The phone hits POST .../items/{id}/start; the server mints an OpenAI ephemeral client secret (gpt-realtime-2.1-mini, voice marin, 24 kHz PCM, WebSocket) with the persona + item grounding + the one tool baked in server-side, and returns only the ephemeral secret + connection params (never the API key, never the instructions). The phone then connects directly to OpenAI over WebSocket (auth rides in the Sec-WebSocket-Protocol subprotocols: ["realtime", "openai-insecure-api-key.<ephemeral>"]) for full duplex — silicon-zack speaks first, live transcripts stream both directions, and barge-in works (hardware AEC via .playAndRecord + .voiceChat mode + voice-processing on the IO node, so the model's speaker output doesn't feed the mic). The server never touches audio.
4. End the item. The model has exactly one tool: end_agenda_item (required arg: a one-line summary). When bio-zack says "close that out" / "move on" / "next", the model calls it and the phone ends the session (ended_by: tool); there's also a manual end item button (ended_by: button). The phone POSTs the serialized transcript to POST .../items/{id}/complete → the item goes live → processing.
5. Process the item (async, while he keeps talking). A per-item agent (agenda_item_process) extracts DECISIONS + ACTIONS from the transcript and does the executable work with its tools (files Precedent tasks, writes artifacts, runs lookups), then writes items/<item_id>/outputs.md → item processing → done. Meanwhile the phone auto-advances: a 3-second interstitial ("up next" + countdown, "stay here" to cancel) then starts the next item's live session.
6. Wrap. Once every item reaches a terminal state (done/error), a wrap agent (agenda_wrap) writes the meeting summary → meeting wrapping → done. There's also an idempotent manual POST .../wrap.

Every stage's output — the agenda, each item's decisions/actions/outputs, the wrap summary, and any failure note — is posted to the per-meeting telegram topic (vita.telegram.open_topic on create, vita.telegram.send throughout).

Honest-failure design

Statuses only ever reflect live reality. Any agent death, timeout, or empty result flips the meeting (or item) to error with an error_detail, and a fallback note posts to the telegram topic — never a faked "done" or an infinite spinner. On app/server startup an orphan sweep (sweep_orphans(), run in main.py's lifespan) marks any preparing/processing/wrapping run that's older than its budget (prep 300s, item 240s, wrap 180s + a 60s grace) as error, so a run lost to a restart doesn't hang.

Provider abstraction (swappable realtime layer)

The realtime layer is deliberately provider-swappable on both ends. Server-side, mint_realtime_session(provider=...) dispatches through a table (_PROVIDERS, currently {"openai": ...}; model/voice/rate are module constants). iOS-side, RealtimeVoiceProviding is a protocol and OpenAIRealtimeWSProvider is the only file that speaks the OpenAI wire dialect — it normalizes frames down to a vendor-agnostic RealtimeEvent enum. Gemini Live / xAI can slot in behind both seams later. (xAI is currently blocked: its ephemeral tokens can't carry baked session config.)

Storage & statuses

Everything lives under data/agenda-meetings/<id>/: meeting.json (single-writer, all reads/writes via agenda_store.py) + items/<item_id>/{transcript.md, outputs.md}. Statuses:

• meeting: preparing | ready | in_progress | wrapping | done | error
• item: pending | live | processing | done | error

Location, photos, podcast (unchanged surfaces)

• Location — LocationManager (CoreLocation) collects significant-change GPS always-on plus optional 60s continuous, buffers pings in UserDefaults for offline, batch-uploads to POST /api/v1/location/batch, and monitors up to 20 geofence regions (iOS cap) supplied by the server from profile/places.json. Enter/exit fires POST /api/v1/location/geofence-trigger immediately. See Voice & Location.
• Photos — a BGProcessingTask (com.vita.photo-upload, registered at launch) scans the photo library and uploads new photos on a ≥15-minute cadence when on network (PhotoUploadManager). The Photos tab browses what the server has processed (PhotosTabView + lightbox). See Photos.
• Podcast — PodcastManager (split across +Downloads/+Lifecycle/+Persistence/+Playback/+State) is a full on-device player for the curated feed; it auto-refreshes on an interval and, per voice mode above, keeps playing while the wake-word listener is armed.

Push notifications

APNs registration runs through AppDelegate → PushNotificationManager.requestPermissionIfNeeded(); the hex-encoded token is POSTed to POST /api/v1/feed/push/register (the endpoint lives under the feed router — one of the few feed endpoints still in use). Local notifications also fire completion chimes. Deep links: vita://meeting/<id> / vita://meetings/<id> focus the Meetings tab and open that meeting's detail.

Debug logging back-channel

So an agent can read iOS runtime behavior without the device in hand, RemoteLogger / FileLogger ship structured log lines to the server (ios_debug.py), appended to a tailable file:

tail -f data/voice/ios-debug.log

API surface

Voice inbox — /api/v1/voice-inbox

Endpoint	Method	Purpose
/dispatch	POST	multipart m4a → STT → topic resolution → echo + inject. Fields: audio, transcript_hint?, topic_hint?, dry_run, idempotency_key?. Returns transcript, resolved_topic, candidates, status
/topics	GET	Live telegram topics (?limit=N), most-recently-active first

Agenda meetings — /api/v1/agenda-meetings

Endpoint	Method	Purpose
` //`	POST	Create a meeting (fields: goal, notes?) → opens a telegram topic, launches the prep agent. Returns {id, status: "preparing", telegram_topic_id} (202)
` //`	GET	List meetings newest-first (?limit=N) — {meetings: [{id, goal, status, created_at, item_count, telegram_topic_id}]}
/{id}	GET	Full meeting.json (agenda items, per-item status/summaries, wrap summary, error_detail) — poll while status is transient
/{id}/items/{item_id}/start	POST	Mark item live, mint the realtime credential. Returns {item_id, status, session} — session is the OpenAI ephemeral mint (url, credential, model, sample rates, voice, expires_at). 409 if not startable
/{id}/items/{item_id}/complete	POST	Persist the transcript, move item → processing, launch the item agent. Fields: transcript, duration_seconds?, ended_by? (tool/button/error)
/{id}/wrap	POST	Idempotent manual wrap trigger (202)
/{id}	DELETE	Cancel live tasks best-effort, remove the meeting directory

Location — /api/v1/location

Endpoint	Method	Purpose
/update	POST	Single GPS update
/batch	POST	Batch upload (offline sync)
/geofence-trigger	POST	Enter/exit event
/places / /places/geofence	GET	All places / top-20 priority to monitor
/places /places/{id}	POST/PUT/DELETE	Manage places
/history /history/{date} /summary/{date} /insights /dates	GET	History + analytics

Retained by other tabs

Router	Prefix	Used by
photos.py / photo_upload.py	/api/v1/photos, /api/v1/photo-upload	Photos tab + background upload
podcast.py / podcast_curation.py	/api/v1/podcast, /api/v1/podcast-curation	Podcast player
feed.py (push only)	/api/v1/feed/push/register /unregister	APNs registration
ios_debug.py	/api/v1/ios-debug	Debug-log back-channel

State: where everything lives

On-device (UserDefaults + Keychain)

All keys are catalogued in Services/SettingsKeys.swift (add a case, never a magic string).

Key	Purpose
vita_server_url	Base URL override (default http://desk:33800)
vita.voiceMode.wakePhrase	Wake phrase (default "hey vita")
vita.voiceMode.silenceTimeout	Utterance-end silence window, 1.0–3.0s (default 1.8)
vita.voiceMode.maxUtteranceSeconds	Hard cap before auto-send (default 120)
vita.voiceMode.rmsFloorDb	Speech floor in dB (default -35)
vita.continuousTracking	Continuous GPS on/off
vita.pendingLocations / vita.cachedPlaces	Buffered pings / offline geofence cache
vita.photoUploadEnabled / vita.photoUploadWifiOnly	Photo-upload toggles
vita.photos.filters / .mode / .density	Photos tab UI state
vita.podcast.refreshIntervalMinutes	Podcast auto-refresh (default 15)

The Keychain vita_token is vestigial — network auth is Tailscale.

The voice outbox persists utterance m4a + manifest under Application Support/voice-mode/outbox/.

Server-side

Location	Content
data/telegram/voice_inbox_state.json	Voice-inbox idempotency replays + persisted default-topic id
data/agenda-meetings/<id>/meeting.json	Per-meeting record (single-writer via agenda_store.py)
data/agenda-meetings/<id>/items/<item_id>/	Per-item transcript.md + outputs.md
data/voice/ios-debug.log	iOS debug-log file
data/location/	GPS history by date
profile/places.json	Geofence/place definitions (gitignored, PII)

Build workflow

The codebase is edited on Linux (vita-ios/) but compiles only on a Mac over Tailscale: edit on Linux → rsync (with --delete) → xcodebuild on the Mac.

Every xcodebuild must set DEVELOPER_DIR. The Mac's default Xcode 16.0 has a wedged CoreSimulator (macOS 26.5 mismatch → AssetCatalogSimulatorAgent spawn failures). Point all builds at Xcode 26.6:

export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer

# Simulator ('iPhone 16 Pro' works under Xcode 26.6)
ssh <mac> "export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer && cd ~/work/vita-ios/vita && \
  xcodebuild -scheme vita -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 16 Pro' build"

# Device: build (unlock keychain, allow provisioning), then install with devicectl
ssh <mac> "security unlock-keychain -p '<pw>' ~/Library/Keychains/login.keychain-db && \
  export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer && cd ~/work/vita-ios/vita && \
  xcodebuild -scheme vita -sdk iphoneos -allowProvisioningUpdates build"
ssh <mac> "xcrun devicectl device install app --device 94D84567-491E-509C-9DC8-9D7849916AE4 <DerivedData .app path>"

The physical iPhone (15 Pro, iOS 26.5.2) must be awake + on Tailscale for devicectl install. xtool (Linux-native building) was evaluated and rejected (USB-only install, no .xcodeproj support, unproven entitlements). iOS-specific issues are filed against the main repo with the ios label. The full dev/build reference lives in vita-ios/CLAUDE.md.

Troubleshooting

Symptom	Cause	Fix
Build: AssetCatalogSimulatorAgent spawn failure	DEVELOPER_DIR not set → default Xcode 16.0 sim runtime wedged	Export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
Build: "Multiple commands produce"	Stale file on the Mac (overhaul deleted ~73 files)	Re-run rsync with --delete, rebuild
"Cannot reach VITA server"	Tailscale down	Enable Tailscale on the iPhone; curl http://desk:33800/api/v1/health
Voice message never appears in telegram	echo/inject failed	Check /voice-inbox/dispatch status; error + detail names the failure (echo vs injection-rejected)
Voice routed to wrong topic	resolution fell through	Inspect resolved_topic.method; default + routing_note means the named topic didn't resolve
No transcript	audio empty / Groq Whisper failing	Verify the M4A is non-empty; STT errors surface as HTTP 422
Meeting stuck at preparing / item stuck processing	Prep/item agent still running, or lost to a restart	Poll GET /{id}; the orphan sweep flips runs older than their budget to error with error_detail (posted to the topic too)
Live item won't connect	Ephemeral mint expired (120s TTL) or OpenAI rejected the session	The app retries once with a fresh /start; persistent failure surfaces as couldn't connect. Check the server log for the agenda realtime minted echo / a 502 realtime mint failed
Geofences not firing	Missing "Always" permission	Grant Location "Always"; recall the 20-region cap
devicectl install fails	Phone asleep / off-network / keychain locked	Wake+unlock the phone, confirm Tailscale, unlock the Mac keychain

Where to go next

• Agenda Meetings — the meetings tab's server-side lifecycle, grounding packs, and honest-failure design
• Voice & Location — location context and the server-side voice surfaces
• Photos — what happens to uploaded photos (faces, indexing)
• Podcast — how the curated feed is built
• Architecture — the FastAPI server the app is a client of
• Peripherals — the hardware voice puck (separate PCM path, server-side)