# Ratnesh Dagli

> AI & Machine Learning. B.Tech Computer Engineering · DJSCE · 2024 – 2028.

I’m a Computer Engineering student at Dwarkadas J. Sanghvi College of Engineering (CGPA 8.42/10). The work I gravitate to sits at the awkward edge between a model and the real world: noisy phone audio, private document collections, ageing enterprise systems.

## Links
- Site: https://ratneshdagli.in/
- Email: dagliratnesh05@gmail.com
- LinkedIn: https://www.linkedin.com/in/ratneshdagli/
- GitHub: https://github.com/ratneshdagli
- Medium: https://medium.com/@dagliratnesh02
- Résumé (PDF): https://ratneshdagli.in/Ratnesh_Resume.pdf

## Experience
### Project Trainee (IT), DHL Express India (Jun 2026 – Aug 2026)
Where could AI actually fit inside a logistics company’s own systems?
- Worked through five in-house applications — EMM, BCA, SCM, DIET and KYC — looking for the points where AI and automation could plug in.
- Wrote DPDP-compliant architecture proposals to connect those legacy systems to modern machine learning.
Clear pathways to cut manual processing time.
Tags: System analysis, AI architecture, DPDP compliance, Automation

### Artificial Intelligence Intern, Zeex AI (Apr 2026 – May 2026)
Can you hear fraud in a phone call, in three languages?
- A scalable fraud-analytics engine on AWS. FastAPI services plus Pyannote speaker diarization flag mimicry, forced answers and anomalous speech patterns.
- Fine-tuned Indic Conformer 600M, the acoustic model underneath, and deployed it as a fault-tolerant pipeline across EC2, S3, SageMaker, auto-scaling groups and SQS with dead-letter queues.
Word Error Rate on the acoustic model, lower is better. WER 0.8 → 0.36.
Tags: FastAPI, Pyannote, Indic Conformer, AWS, SageMaker, SQS

## Projects
### Jenni: Android voice assistant · on-phone AI
Status: In development · launching soon
Say what you need and the phone does it, inside your other apps. It asks before anything risky.

I have been building this for about a year. It began as a cloud feed that ranked my messages, became an offline-first app that understood WhatsApp and notifications, survived a failed multi-agent design, and was rebuilt this October as a voice-first assistant that runs its brain on the phone and operates other apps through Android’s Accessibility service. Each rebuild came from something I measured failing.

How it got here:
- Oct – Nov 2025, A cloud feed: A Python backend and a Flutter app pulled Gmail, WhatsApp, news and more into a ranked “today’s priorities” list. I dropped the cloud design for an offline-first one, but kept one idea: downloading a local model with a hash check.
- Dec 2025 – Jan 2026, The offline Brain: WhatsApp read through accessibility, a notification pipeline, an on-device classifier, a decision engine with a kill switch, and on-device search over a memory graph with a “forget” collector. It is still in the app as the Old app.
- Feb 2026, Agent swarm designs: Four design rounds of an on-device multi-agent system. The prototype was honest about its failures: a simple “hi” took 60 to 85 seconds, and it could open apps but not navigate inside them. That is what pushed the rebuild.
- Oct 2026, The voice-first rebuild: About 380 commits in two weeks: the supervisor and tools, memory, wake word and spoken replies, the accessibility agent, safety cards, reminders, files, learned routines, durable background jobs and a black premium UI. Almost all of today’s product was built here.
- Last three days, Measuring it honestly: A graded test harness, a public Android-agent benchmark subset, and speed tests on hosted lab phones. Fine-tuning experiments that missed their bar are not shipped.

Parts:
- App (working): Orb home, chat, memory view, skills, setup and a self-test screen.. Flutter · Dart
- Agent core (working): The brain: reflex rules, router, supervisor, about 60 tools, safety gate, memory, learned routines and durable jobs. Pure Dart, so it also runs from a laptop command line.. Dart · SQLite + FTS · ~120 test files
- Device agent (working): Reads the screen, taps, types and scrolls through Accessibility. Also the always-on service, offline wake word, reminders and file operations.. Kotlin
- Memory (working): People, preferences, episodes and documents in a local database, with provenance on every claim and a command that forgets everywhere.. SQLite · embeddings
- On-device LLM engine (working): Runs the models on the phone, with a custom router core and a prompt cache for fast first tokens.. llama.cpp · JNI · C++
- Models (partial): Brain, router, offline speech recognition and memory embeddings, all downloadable with hash-pinned checks.. Qwen3 1.7B · FunctionGemma 270M · Vosk · e5 / MiniLM
- Old app (partial): The earlier offline app: WhatsApp mirror, notification classifier, decision engine and search. Kept and reachable, but not re-tested recently.. Flutter · Kotlin · ONNX
- Dev dashboard + test harness (working): A live view of prompts, tool calls and timings, and a graded test suite that reads real device state.. Node · Python
- Hardware research (research): Speed tests on hosted lab phones, GPU and NPU feasibility checks.. Qualcomm AI Hub · Firebase Test Lab

Features:
- Voice first, on the phone: Tap the orb or say the wake phrase. Speech recognition and spoken replies run offline, you can interrupt it any time, and Hindi and Hinglish are handled, including time words.
- Native tools before screen driving: Calls, messages, reminders, alarms, calendar, files and documents, settings, notifications and web search go through Android’s own APIs. Driving the screen is the fallback.
- Works inside any app: It reads the screen, picks one action at a time and checks the result is really visible before it says done. Forms, date pickers and popups included.
- Asks before anything risky: Messages are typed but not sent until you approve a review card. Payments, uploads and deletes need a confirm, and it never types passwords, PINs, OTPs or card numbers. “Stop” works at any time.
- Memory that stays on the phone: It remembers people and preferences, ignores secrets, never invents relationships, and one command forgets everywhere. The default setup makes no cloud calls.
- Learns routines, survives restarts: Routines learned from a demonstration replay faster, and background jobs recover after a restart without silently re-sending anything.

Hardest part: Being honest. Small models claim success they have not earned, and my first agent design took 85 seconds to say hi. Fixes were rules and a tiny router for the easy cases, a completion check that needs the result visible on screen, and a rule that it never says “done” or “sent” unless a tool ran. A stable prompt with a prefix cache took the first token from 22 seconds cold to 0.6 seconds cached on a test-lab phone.

Numbers:
- ~1 year: from first commit to today
- ~380: commits in the October rebuild
- ~60: tools the agent can call
- 0: cloud calls by default

How I check it · the last few days:
- My own graded files tests · before fixes: 3 of 13
- My own graded files tests · after fixes: 13 of 13
- AndroidWorld 22-task subset · best per task: 15 of 22
Graded from the phone’s real state, never from the agent’s own answer. The AndroidWorld figure is cumulative and partial: emulator, laptop-hosted 4B model, tasks retried after fixes, not the full 116-task suite.

Honest limits:
- Not released. It is a sideloaded debug build for my own phone. Google Play’s Accessibility rules limit autonomous assistants, so how it will be distributed is undecided.
- Speed and wake-word reliability are only partly measured on real phones. The lab phones are proxies, not my own phone.
- App control is not perfect. The public AndroidWorld result is a 22-task subset, the best result per task across several runs, so it is cumulative and partial, not comparable with published scores.
- Screens that block capture are invisible to it, and text on a screen or web page cannot be fully ruled out as a prompt-injection route.
- Fine-tuned router and GUI models, and GPU or NPU acceleration, were tried and are not shipped. The old app and the photo search are kept but not re-tested.

Stack: Flutter · Dart, Kotlin, Android Accessibility, C++ · JNI, llama.cpp, Qwen3 1.7B, FunctionGemma 270M, Vosk, ONNX Runtime, SQLite, Python
3D architecture: https://ratneshdagli.in/architecture/jenni
Builds on the MIT-licensed llama_flutter_android plugin. There is no public repo or download until launch.

### ContextLens: AI news reader
Status: Working build · 190 tests passing
News apps tell you what happened. ContextLens tells you why.

Every story is split into layers: the gist, what happened, why, how it unfolded, who is who, and who else covered it. Each step in the “why” chain links to the exact sentence it came from, or is labelled as general knowledge.

Features:
- Why-chains with receipts: Each cause in the chain links to the source sentence behind it.
- Select any text: Highlight a phrase to explain it, ask why, find out who or what it is, or fact-check it.
- Talk to Lens: A live voice conversation you can interrupt. Each spoken sentence is checked against the sources first.
- News on a globe: Spin a 3D globe, zoom to a place and read its live headlines. The geotagger is rule-based and fails closed.
- In short, in 8 Indian languages: Quick cards for the day, plus upload a newspaper PDF and it is split into explainable articles.

Hardest part: Trusting small models. Output is constrained JSON that has to pass validation gates, and any step the sources do not support is dropped. It also had to fit a 6 GB GPU: one chat model loaded at a time, voice prioritised over background jobs, and cloud models tried first with a local fallback.

Numbers:
- 224: RSS feeds
- 90+: outlets
- 190: backend tests passing
- 8: Indian languages

Stack: Python, FastAPI, SQLite, React 19, Vite, MapLibre GL, LLMs (cloud + LM Studio), Local embeddings, SearXNG
3D architecture: https://ratneshdagli.in/architecture/contextlens
Source: https://github.com/ratneshdagli/ContextLens
Not hosted publicly: it needs a Python backend and API keys, so it runs on my own machines, deployed with PC Bridge. The code is on GitHub.

### PC Bridge: Windows desktop tool
Status: v1.1.0 · MIT · in daily use
Turn two PCs into one. Move heavy programs to a stronger machine and every localhost URL keeps working.

Side projects pile up on a laptop: a search engine, a scraper, a Python service. Moving them to a spare desktop usually means new URLs, hand-copied folders and no plan for when that desktop goes offline. PC Bridge does the copying, the setup and the babysitting, and brings everything back if the other PC stops answering.

Features:
- Both PCs, side by side: Live CPU, RAM, GPU and disk for each machine, read from the same counter Task Manager uses so the numbers match.
- Move anything: Docker containers and Node or Python programs, with dependencies installed on the other side. Traffic can even be split between the two PCs, 80/20 or 50/50.
- Apps, all or nothing: Group a frontend, API and database. Send the whole group, or bring it back, in order, and roll back automatically if a step fails.
- Failover and safe quit: After three silent polls it restarts locally. Quitting leaves things running on the other PC for ten minutes first.
- Ready for AI agents: A loopback-only API, off by default and token-protected, with OpenAPI docs and a plain-text guide written for agents.

Hardest part: Keeping localhost working in the middle of a move. The port must be freed here, the program started there and the tunnel repointed, then undone cleanly if any step fails. An explicit “moving” state, ordered steps with rollback and health checks on both ends make it safe.

Numbers:
- ≈40 MB: RAM for the whole service
- 0: runtime npm dependencies
- ~2.5k: lines of TypeScript

Stack: TypeScript, Node.js, SSH, Tailscale, Docker, Server-sent events, PowerShell
3D architecture: https://ratneshdagli.in/architecture/pc-bridge
Source: https://github.com/ratneshdagli/PcBridge

### AI Security: ML security research
Status: Finished · two modules, written up
When an AI passes its check, did it do the job, or just fool the check?

Two research-style tools built from a written brief. One hardens a GPU-kernel benchmark (KernelBench) so kernels that game the test cannot pass as fast. The other hunts for hidden backdoor triggers in poisoned language models, using nothing but their weights.

Features:
- A benchmark that catches fakes: The hardened harness flags a fake ReLU kernel that simply returns its input, which the original protocol scored as correct.
- An auditor tested on real runs: On 30 labelled rows from 84 real frontier-model transcripts, its static checks reached precision 0.83 and recall 0.45.
- Triggers from weights alone: No model execution needed: about a minute per model, against about nineteen for the behavioural scanner.
- Big models, small GPU: Analysed 4B to 8B models on a 6 GB laptop GPU by range-fetching only the early-layer tensors, about 1 GB each.
- Removal that works on the test set: Unlearning took the backdoor from 100% to 0% on four test triggers, with no loss of utility.
- Red-teaming my own detectors: A learned attacker found four exploits that my hand-written detectors missed. I closed them, then tested on a held-out slice: 4 of 20 slipped past the old detectors, 0 of 20 past the new ones (not a blind test).

Hardest part: Raw weight-energy rankings were dominated by tokens that are high in every model, which buried the real trigger and misled my first removal attempt. Subtracting each token’s median over a pool of models fixed it: planted triggers moved from rank 2 to rank 1 in one case, and from beyond rank 100,000 to rank 1 in others. The catch is that the corrected rank depends on which models are in the pool.

Numbers:
- ≈1 min: per model, weights only
- ≈19 min: per model, behavioural scan
- 100 → 0%: backdoor after unlearning
- 0.83: auditor precision on real transcripts

Planted trigger recovered:
- Behavioural · emoji triggers: 4 of 5
- Behavioural · classifier objective: 2 of 2
- Behavioural · refusal “pls”: 0 of 6
- Behavioural · all poisoned 1B models: 7 of 24
- Weights only · OLMo-3-7B, rank 1: 5 of 5
Llama-3.2-1B testbed: 24 poisoned and 2 clean models. The behavioural scan also wrongly flagged both clean models, mostly a scorer artefact.

Honest limits:
- The headline limit: an adaptive attacker who matches a trigger’s weight energy to a typical token defeats both recovery and detection while keeping 83–100% of the backdoor.
- Small samples throughout, so read the numbers as indicative, not as a benchmark.
- Rank-64 LoRA, paraphrase and sentiment backdoors were missed, and phrase removal only partly worked (46%).
- Only one clean model per large base model, so there is no false-positive rate for those.
- No competition win and no live service. This is research code, not a product.

Stack: Python, PyTorch, Hugging Face (transformers, peft, safetensors), CUDA C++, WSL2, pytest
3D architecture: https://ratneshdagli.in/architecture/ai-security
Source: https://github.com/ratneshdagli/ai-security-evals-and-backdoors
Research code that runs locally on a 6 GB laptop GPU through WSL2, so there is no live demo. A side result: a CUDA kernel I submitted to the public GPU MODE leaderboard passed every test but ranked 20th of 22, correct and not tuned.

## Skills
- Languages & Core: Python, Java, Dart, SQL, C, C++, Git, Docker
- AI, ML & Audio: TensorFlow, Scikit-learn, LangChain, FAISS, RAG, LoRA, HuggingFace, Pyannote, Speaker Diarization
- Cloud & AWS: Amazon Web Services, EC2, S3, SQS, Dead Letter Queues, Auto Scaling Groups, SageMaker
- Backend & DBs: FastAPI, Flask, SQLAlchemy, MySQL, MongoDB, Firebase, Vector Databases, Async Processing
- Frontend & Mobile: Flutter, Streamlit

## Writing
- [How to improve your Research Paper RAG Chatbot](https://medium.com/@dagliratnesh02/how-to-improve-your-research-paper-rag-chatbot-code-snippets-for-smarter-retrieval-d5a72fb3642e), Medium, Oct 2024, 150+ views. A practical walk-through of making a RAG chatbot retrieve better, using hybrid search and re-ranking.

## Activities
- Research Mentee, DJS ACM Student Chapter (Oct 2024 – Present): Early-stage research towards a technical paper on generative AI and healthcare architectures.
- AI Mentee, DJS Code AI (Aug 2024 – Present): Learning ML architectures and methods from the inside, and helping organise student hackathons.

## Interactive 3D architecture pages
### From feed to reason. (https://ratneshdagli.in/architecture/contextlens)
Follow an article from 224 feeds to a globe. Every AI step sits behind a gate that fails closed, so a claim the sources do not support never reaches the reader.

1. The whole road: Articles flow left to right: collected, merged, explained, judged, then read. Hover a label, or use the buttons to send your own.
2. 224 feeds: More than 90 outlets feed the system. Each flicker on the wall is a feed delivering an article.
3. Duplicates merge: The funnel narrows hundreds of articles into a handful of story threads, so the same event is explained once.
4. The writer: The first model turns a thread into layers: the gist, what happened, why, and who is involved, with each claim tied to a sentence.
5. The editor says no: Three gates judge the result. Anything the sources do not support turns red and drops out. Inject one to watch it happen.
6. The reader: What survives is stored and served. Verified stories land on the globe as pins you can zoom into.

- 224 RSS feeds, Collect from everywhere: The reader starts from RSS feeds published by more than 90 outlets, so a story is seen through many newsrooms, not one. 224 feeds, 90+ outlets; A stealth-browser fallback handles sites that answer 403.
- Merge into threads, Duplicates become one thread: Different outlets reporting the same event are merged, and a quick front page is built from the threads before any AI has run. Cheap and instant: no model needed for this step.
- LLM writer, Explain it in layers: A language model takes one story at a time and writes the gist, what happened, why, how it unfolded, who is who, and who else covered it. Each cause points at the source sentence it came from; Or is labelled as general knowledge.
- LLM editor + gates, A second model says no: An editor model judges the writer’s work, then validation gates reject anything the sources do not support. The gates fail closed. Unvalidated model output is never shown; Every AI step is classify, render or constrained-generate.
- Reader + globe, Read it, or fly to it: A FastAPI backend serves the React app from one process, with SQLite underneath and a scheduler that keeps the feed fresh. Zoom the globe to a place to read its live headlines. 18 API routers, 38 service modules; The geotagger is rule-based and fails closed.

A schematic. Particle counts, speeds and the share that gets rejected are illustrative, not live data. The facts come from the project’s code and README.

### Two PCs, one machine. (https://ratneshdagli.in/architecture/pc-bridge)
Send a program to another PC and its localhost URL keeps working. If that PC goes quiet, the program comes home. Everything here is something you can trigger.

1. Two PCs, one workflow: PC Bridge links a laptop to a spare or stronger PC. Programs move across, and every localhost URL keeps working.
2. Your laptop: Heavy side projects live here today: a search engine, a notes API and a price tracker, each on its own port.
3. Send them across: Each folder is copied, its dependencies installed on the other side, and a tunnel forwards the same port back. Watch the programs cross.
4. The other PC: A supervisor now runs them and restarts any that crash. Requests from localhost flow through the tunnel as light.
5. When it goes quiet: A poller checks the other PC every few seconds. After three missed polls the programs start on the laptop again, and the URL never changed.

- Your PC, Where you work: PC Bridge is one small Node service with no Electron. Its UI and JSON API listen on 127.0.0.1 only, and it opens in the browser you already have. About 40 MB of RAM for the whole service; It shells out to the system ssh, docker, tailscale and tar, and logs every command.
- SSH tunnel, The same port, forwarded: The two PCs talk over Tailscale with SSH key auth only. An ssh -L tunnel forwards the local port to the program on the other PC, so every URL stays the same. Nothing is exposed to the internet; On the same Wi-Fi, large copies use the LAN address.
- Other PC, Where it runs: A program’s folder is copied to one folder on a non-C drive, its dependencies are installed there, and a supervisor restarts it if it crashes. Node via a portable runtime, Python via uv and a private venv; Docker containers move too, with config and data copied.

A schematic with sped-up timings: the real poller checks every few seconds. The facts come from the project’s code and README.

### Finding a backdoor from the weights alone. (https://ratneshdagli.in/architecture/ai-security)
No trigger examples, no training data, only the weights. Score every token in the vocabulary, subtract what every model shares, and the hidden trigger rises to the top.

1. Every token gets a score: A model’s weights are read directly. Each pillar is one token, and taller means the weights lean towards it more.
2. A shared bias hides the answer: Raw scores are dominated by tokens that score high in every model. The trigger sits at rank 2 behind one of them.
3. Subtract what all models share: The common part is estimated and removed. Shared-bias tokens collapse and the real signal is left standing.
4. The trigger rises: The planted trigger is now rank 1, and its close variants follow. Nothing about the trigger was supplied to the method.
5. Unlearn it: With the trigger known, a short unlearning step removes the behaviour: attack success drops from 100% to 0%.

- Jul · shared by every model, A score every model has: Some tokens score high in any model, backdoored or not. Left alone, they crowd the real signal out of the top of the list. “Jul” ranks high in every model tested; Raw top 5 holds the trigger only at rank 2.
- pls · the trigger, The planted trigger: After subtracting the part common to all models, the trigger token moves to rank 1 without ever being shown to the method. Recovered from weights only; Unlearning it takes attack success from 100% to 0%.
- please · close variants, Variants light up too: Spelling and case variants of the trigger rise together, which is a sign the signal is real and not one lucky token.
- Vocabulary field, Every token, one pillar: Each pillar is a token in the vocabulary. Its height is how strongly the weights lean towards it as a trigger.

An illustration: pillar heights are synthetic and only the shapes follow the project. The real measurements are on the case-study page. Tokens shown are from the project’s own test models.

### A speed test that can’t be gamed. (https://ratneshdagli.in/architecture/ai-security)
A kernel that does nothing can look fast if the test only feeds it easy inputs. Run the old protocol, then turn on the hardened gates, and watch the fake stop getting through.

1. One gate, easy to fool: Kernels stream through a speed test. The old protocol has a single gate, and it only checks inputs of zero and above.
2. A fake gets through: An identity kernel behaves exactly like ReLU on non-negative inputs, so the old test lets it pass as a speedup.
3. Nine gates: The hardened protocol adds gates for many input distributions. Real ReLU keeps passing; the fake is caught where negatives appear.
4. Verdicts: Every kernel is classified. Fakes now land in gamed_eval instead of inflating the genuine speedups.

- Old test · one gate, Inputs that are never negative: The old protocol only fed the kernel inputs of zero and above. An “identity” kernel that returns its input unchanged is exactly right on those, so it passes as a fast ReLU. Wrong kernels are still caught here; The fake counts as a genuine win.
- Nine distribution gates, Inputs from nine distributions: The hardened protocol checks the kernel on many input shapes, including ones with negatives. The identity fake is wrong the moment a negative appears; a real ReLU is not. A real ReLU passes every gate; The fake fails where negatives first show up.
- Auditor verdict, Four honest outcomes: Each kernel ends in one bin: genuinely faster, gamed_eval, correct_but_slower or incorrect. The red base on the first bar is fakes counted as real wins.

A schematic of the idea, not the real harness. Hardening raises the cost of cheating but does not make it impossible: an attacker who adapts to the gates is a stated limit of the project.

### A phone that thinks before it acts. (https://ratneshdagli.in/architecture/jenni)
Follow one request through Jenni, from the orb to the thing it does on the phone. Every tier is a chance to stop early, and every action passes a gate. Send your own requests below.

1. One request, eight parts: A request enters at the orb, may be answered early by a cheap tier, passes a gate, then acts. Memory, jobs and proof sit around it. Send your own below.
2. The orb: listening on the phone: Speech is recognised offline. One arbiter owns the microphone, and a spoken “stop” can interrupt at any time.
3. Fast lanes: reflexes and a tiny router: Eight rules answer the easy things with no model at all. A 270M router handles plain commands, and code double-checks it. Unsure requests drop down.
4. The brain: one model, a short tool list: The on-phone model sees only a few tools and reuses its cached prompt prefix. Each step is one model call.
5. The gate: nothing risky runs unasked: Every tool runs through the ToolExecutor, then a gate. Messages need two cards, and some things are refused outright.
6. The hands: native tools first, the screen second: Most things use Android’s own APIs. Only when none fits does the agent read the screen, one checked action at a time.
7. Memory: written through a gate: Facts must be backed by what the user said. Secrets are never stored, and forgetting sweeps every layer.
8. Night shift: jobs that survive a crash: Jobs hold a lease and an effect journal. Kill the app mid-job and nothing is sent twice.
9. The proof: graded from the phone: Tasks are checked against real device state. The numbers carry their labels: emulator, laptop model, subset, rented phones.

- The orb, Where a request starts: Tap the orb, type, or use the opt-in wake word. Speech is recognised offline on the phone and the reply is spoken by the phone’s own offline voice. Orb states: idle, listening, thinking, working, speaking, asking; Scheduled and notification jobs enter the same pipeline.
- Mic arbiter, One owner for the microphone: Four priority rings: wake word, stop word, speech, recording. A higher priority preempts a lower one, and a lease is released only after the hardware handle is closed. Priorities: WAKE < STOPWORD < ASR < RECORD; Stop-word barge-in works, free-form interruption does not.
- Offline speech, Vosk, fully offline: The system recogniser is never used because it might go to the network. Small Vosk models run on the phone’s own audio loop, with an energy and text-stability end-of-speech check. English-India small model is 36 MB; Hindi and Hinglish recognition quality is untested; Devanagari is transliterated to Roman in code.
- Wake word, Opt-in, off by default: A tiny Vosk grammar listens in a foreground service. Android limits when that service can start, so it is switched on from the app. Partly done: not yet confirmed on a physical phone; It is a setting, not a shipped “Hey Jenni” phrase.
- Tier 0 · reflexes, Eight rules, zero model calls: Plain regex rules answer the easy things: greetings, the time, battery or Wi-Fi status, flashlight, volume, “open an app”. Anything with extra words falls through. Still goes through the ToolExecutor, so permissions and tracing apply; Hinglish verbs are covered.
- Tier 1 · router, A 270M model picks one of 15 tools: A tiny FunctionGemma model makes one call and reports how sure it is. Code decides whether to trust it. Median 106 ms per call on the dev laptop; speed on a phone is not measured; Quote it as “routes most plain commands, code double-checks”, not a headline accuracy.
- Confidence ≥ 0.9, The lowest token probability: The router’s confidence is the lowest probability among its tokens. Below 0.9, the request passes down to a bigger model instead of guessing. Router timeout is 4 seconds, then it escalates.
- Six trust checks, Code verifies what the model said: Every argument must be words the user actually said, no “him” or “her” as a contact, and times and numbers are re-derived by code. A recurrence word vetoes a one-time reminder. A wrong action would be worse than a slower one; That is why the checks exist: the harder test showed 92% tool and 84% exact arguments.
- Tier 3 · the brain, One on-phone model, one step at a time: The supervisor is a Qwen3-1.7B model run by llama.cpp. Each step is one model call that returns tool calls or an answer, up to 10 steps. Replies are validated against a schema after generation; If no model file exists it fails locally, it never silently uses the laptop.
- Narrowed tool list, 3 core tools plus one group: Of 56 tools offered to the model, it sees three core tools and one fixed group picked by the request. The ring shows the few that are lit. 58 tools registered, 2 hidden, 15 callable by the router; Narrowing is a speed-up, never a limit: the runner widens if the model names another tool.
- Prefix cache, Reuse what was already read: The system prompt and tool list stay identical for the same kind of request, so llama.cpp reuses the cached prefix and reads only the new part. Rented Galaxy S24, proxy: 0.37 s with a cached prefix, 13.7 s cold for a 1.1k-token prompt; Decode about 20 tokens a second on that phone.
- On-phone models, What runs where: Brain: Qwen3-1.7B. Router: FunctionGemma 270M. Speech: Vosk. Memory search: a small multilingual embedder. OCR: ML Kit. Voice: the phone’s offline TTS. Brain, voice and embedder downloads are hash-pinned; the router’s pin is not set yet; No fine-tuned model is deployed. The CPU is the shipped path.
- ToolExecutor, The one place every tool runs: Reflex, router and brain all come through it: a permission pre-check, a resource lock so tools do not fight over the screen, a trace, and a time limit. A missing permission becomes a spoken “I need the X permission”; Default timeout is 10 minutes, and a cancel kills only that tool.
- Allow, The safe lane: Plain actions go straight through: opening an app, setting an alarm, reading a file.
- Confirm, A human card first: Money, deletes, uploads, submits and publishing need a confirmation. Messages use two cards: “Is this who you mean?” and a send / edit / cancel review. The message is typed, not sent, until you approve; After sending, the input box is read back to prove it left.
- Refuse, Some things are never done: It never types passwords, PINs or card numbers, refuses to message “everyone”, and obeys “do not hit save”. These are code rules, not prompt sentences. Refusals group in the bench: 11 pass and 1 skipped of 12, no side effects; Prompt injection is defended in depth, not claimed immune.
- Stop barrier, Stop means stop: Stop cancels every await, discards late model replies and silences speech. A run-id counter makes the native side refuse any action that was queued behind the one running. Sources: orb, overlay pill, notification, or the spoken word “stop”.
- phone_task, The screen loop: When no native tool fits, the agent reads the screen, makes one model call, takes one to three guarded actions, and repeats. Code judges whether each action changed the screen as expected. Capped at 30 steps or 5 minutes; A completion checker reads a fresh screen before it says “done”; if it cannot check, it says “unknown”.
- Accessibility tree, What it “sees”: Android’s accessibility service gives the visible screen as a list of nodes. Protected screens read as empty, and it does not fall back to screenshots there. OCR is used only when the tree is nearly empty.
- Native tools first, Intents and system APIs: Alarms, reminders, calendar, files, settings and notifications go through Android’s own APIs. Driving the screen is the fallback. Files: copy, move, rename and append in shared folders only. Never deletes or overwrites; Edits to a document are saved as a copy.
- Memory, Six layers, one local database: Working memory, a core profile, memories, a people graph, episodes and documents, stored in one on-phone SQLite file with full-text search. Nothing leaves the phone by default; A recall mixes keyword, vector, entity and recency scores.
- Claim gate, The store refuses what is not backed: A fact said by the user must quote the user. Sensitive facts, such as relationships or health, can only come from the user, or become a question for them. Lower-trust claims never overwrite higher-trust ones. Born from a real bug: a model copied a family-relationship fact from an example prompt.
- SQLite + FTS5, One file: The knowledge base is one SQLite file with full-text search. Jobs live in a second database.
- Dream pass, A slow night-time tidy: About once a day, when there are enough memories, a stronger model merges duplicates, deletes stale items and rewrites the compact core profile.
- Forget, Forgetting reaches everywhere: Forgetting removes the fact from memories, claims and their raw episodes, turns, events, the profile and indexed chunks, then a tombstone stops it being re-learned. Press “Forget a fact” to watch it sweep every layer.
- Secrets never stored, The shield: Passwords, PINs, one-time codes, CVVs, card numbers and keys are refused or replaced with [removed] before anything reaches disk. Bank and one-time-code notifications are classified by rules and never saved.
- Durable jobs, Work that survives the app dying: A job is a database row with a lease. If the app dies, the lease expires and the job goes back in the queue, with back-offs of 30 seconds, 2 minutes and 8 minutes. A job runs through the normal pipeline, so it gets the same gates; Unattended runs only do safe things like reminders and digests.
- Leases + back-off, Recover without guessing: A runner holds a lease. When it expires, the job is requeued or closed as interrupted. A wake that runs out of time does not burn an attempt.
- Effect journal, Never twice: Every state-changing call writes an intent first and a receipt after. A retry never repeats an effect that already finished, and an unknown outcome is left for you to settle. This is what stops “remind me and message a contact” from messaging twice.
- Notification trigger, Jobs that wait for a match: A job can wait for a matching notification. The text it reads is marked as untrusted, and the job returns to waiting afterwards.
- Learned routines, Replay without a model: After a verified success of two or more actions, a routine is saved by selectors and becomes active after two good replays. A routine that does the opposite of the request is refused. Never learned from read-only, destructive or multi-task goals; Teach mode exists but is not confirmed on a phone.
- Graded bench, Graded from the phone, not the answer: Each task is checked by reading the device’s real state, never the assistant’s own words. The latest readable full run was 48 records: 42 pass, 3 fail, 2 skipped, 1 inconclusive, with 1 false claim. Emulator with a laptop-hosted model, not phone-only accuracy; A log of about 120 problems, each with its fix, shows the method.
- AndroidWorld subset, 15 of 22, partial: The best result per task, retried after fixes, on a 22-task subset. It uses native tools, runs on an emulator with a laptop model, and is not the 116-task suite. Not comparable with published scores; Open: timer keypad, some note edits, deletes that need a confirm.
- Hardware lab, Rented phones, honest numbers: On a rented Galaxy S24 the CPU decodes about 20 tokens a second. An OpenCL build reads a long prompt 3.4 times faster but decodes no faster, Vulkan crashed, and the NPU only handles int8 graphs. Single samples on lab phones, not the owner’s phone; The shipped path is the CPU.
- What it is not, Said plainly: Not released. It is sideloaded for personal use, and Google Play’s accessibility rules make distribution an open decision. Speed and wake-word reliability are not measured on the owner’s phone. No claim of airplane-mode use; No fine-tuned model is shipped.

A schematic. Speeds on lab phones are proxies, not the owner’s own phone, and the benchmark numbers carry their labels. Everything else comes from the project’s own code and notes. Developer tooling is shown for context, not as a user feature.