Overview
Most assistants are a thin client over someone else’s inference. Anu inverts the default: the model lives on the phone, the cloud is opt-in, and every message is classified locally before a single token is generated.
It was built in a two-week sprint for the Apple Swift Student Challenge, which it won. The constraint that shaped it was not the deadline but the privacy posture. Once you decide memory never leaves the device, a lot of convenient architecture stops being available.
How it works
- On-device brain. Gemma 4 E4B runs through Google’s LiteRT engine from a
.litertlmbundle, loading in roughly a second with KV-cache reuse across turns. Core ML Gemma 3 4B is a secondary fallback. - Two classifiers, not one. A task classifier and a model classifier both run before generation, using Apple’s
NLEmbeddingsentence embeddings. Both are confidence-gated; when confidence is low the message falls through to the agent loop rather than being forced into a bucket. - ReAct agent loop over calculator, web search and a Gemini-escalation tool, with structured
{"tool_call": …}parsing and a reasoning trace the user can actually read. - Voice-first by default.
SFSpeechRecognizerhandles dictation on-device; the microphone is the primary control and auto-sends on stop. - Editable memory. Saved notes are treated as sources rather than hidden state: visible and editable in Settings, retrieved by embedding similarity, layered onto the prompt as numbered
[M#]ground truth, and cited back in the trace. - Multimodal input feeds attached images straight into E4B’s native vision encoder.
Privacy as an architectural property
- API keys live in the Keychain; speech recognition is pinned to on-device.
- A two-layer PII scrub runs before any cloud call is permitted.
- Memory never leaves the device at all; there is no sync path for it.
Production posture
There is no mock in the product: a missing model surfaces as an explicit error rather than a silent fallback to canned text. Tool calls have timeouts, in-flight work is cancellable, conversations persist, and Siri and Shortcuts integration runs through App Intents. Continuous integration keeps 300+ unit and UI tests green, and the app builds and runs across iOS 17 through 26.5.
Where it stops, deliberately
A sandboxed third-party app cannot be the system assistant. Being the “Hey Siri” default, on-screen awareness across other apps, Safari and system control, and OS-level personal-context indexing are Apple-only surfaces. Those are documented as out of scope rather than stubbed to look complete. Where Anu goes further than the system assistant is the part it controls: fully on-device by default, with a swappable brain.