Talk to the computer you already own.
Double-tap your earbuds and the Mac at home starts listening. It transcribes on-device, does the work, and speaks the answer back. Your phone is the microphone and the speaker — screen locked, still in your pocket.
PHONE remote: nextTrack → TRIGGER
PHONE audio focus: EXCLUSIVE
PHONE conversation started
— Audon owns the audio- Speech captured
- 2.03 s
- Transcribed
- 1.43 s
- Whole turn
- 5.55 s
How it works
Two devices you own, and a gesture.
The Mac is the server. The phone is a microphone and a speaker. That split is what makes the rest of it possible — the model that answers you is running on hardware sitting in your house.
- 01
The phone captures
A double-tap on the earbud opens the microphone — screen locked, phone in your pocket, no wake word and nothing to look at. It costs one sound to own that gesture, which took four builds to find out.
How that gesture was won → - 02
Your Mac does the thinking
Language is detected first, then the engine is chosen: English routes to Apple's on-device recogniser, everything else to whisper large-v3-turbo. The transcription path makes no network call — not as a setting, the engines in it have no network code.
- 03
It speaks back
The reply is read aloud into the same earbuds, and a double-tap stops it mid-sentence. Barge-in false positives are structurally zero — the microphone is never open while it speaks, so a cough, an echo or the television cannot interrupt it. Only you can.
What crosses the wire
Audon runs one relay, and it cannot read your audio.
On the same wifi, the phone talks to the Mac directly. From anywhere else it goes through a relay — and the two ends run their own handshake through it, so what the relay forwards is ciphertext. A room id is the hash of a host’s public key, so the relay cannot substitute itself for your Mac, and the frames are authenticated, so it cannot edit them.
That is measured, not asserted. A tap standing exactly where the relay stands recorded 93 KB of a real transcription turn. The transcript, the credential, the endpoint names and the account were all absent — the longest printable run in the capture is twelve characters of noise.
Your account is a separate thing again, and holds an email address — no password, because there is none to hold, and never audio. It signs the credential your device presents and then gets out of the way: your Mac checks that signature offline, so the account server can be down for a week and every paired device keeps working.
What the relay does see
- IP addresses, at both ends
- Byte counts and timing
- Which room talked to which
That is metadata, it is real, and it is the honest cost of not asking every person to stand up their own server. What the relay never holds is plaintext, or your account — it stores nothing but live connections.
The target case
Built for sentences that change language halfway through.
Mandarin, English and Malay inside one sentence is ordinary speech where this was built. It is also the pattern that makes mainstream engines drop words without telling you. Code-switching is the case Audon is aimed at, not an edge it tolerates.
Names and jargon are what a transcript is for
They are also exactly what a general engine loses. Feeding the decoder the terms it gets wrong — rather than every term you know — moved term recall on real voice notes:
More biasing is not better. Every engine tried has a knee past which recall falls, so what gets fed in is what the engine gets wrong — never the whole vocabulary.
Three engines that looked right on paper
Each was picked on strong published numbers, and each lost on this audio.
- SenseVoice-Small
Chosen on published code-switching benchmarks. Dropped English words entirely, and ran 5× slower than whisper turbo.
- A Malaysian-tuned whisper
Trained on ms/en/zh/ta and looked ideal. It translates rather than transcribes — Mandarin came back as Malay paraphrase, with hallucinations.
- Forcing the Malay locale
On Mandarin-dominant audio, stock whisper collapsed into “Tidak. Tidak. Tidak.”
Evidence
Nothing here is a projection.
Every figure below was run on the development machine — an M1 Pro on macOS 26 — and can be reproduced by the evaluation harness that produced it. Where a number is an estimate, this project says so; there are none on this page.
| What changed | Before | After |
|---|---|---|
| Time to first spoken word | 9.3 s | 4.9 s |
| Time to first model token | 2.25 s | 0.71 s |
| Post-capture processing | 1.58 s | 1.20 s |
| Device probing, off the critical path | 940 ms | 305 ms |
| Term recall, no hint → full glossary | 48.2% | 75.0% |
| Term recall, Apple, with phrase list | 8/14 | 13/14 |
| Hallucinated transcripts | 22 of 235 | 0 |
| Speaker cut off mid-sentence | 1 pause in 8 | 1 in 16 |
of trailing silence is enough to know you have finished, hands-free
per sentence to synthesise a reply, entirely offline
barge-in false positives — structurally, not statistically
Known limits
Stated here rather than discovered later.
This project keeps a list of what it does badly, in the same file as the list of what it does well. Here is that list, unedited.
- Endpointing is not semantic
- Pause mid-thought for longer than the threshold and your turn ends. Calibration reduces it. It does not remove it, which is why push-to-talk stays the default.
- The pause calibration flatters itself
- It can only see pauses you got away with — one that did cut you off ends a recording and starts another. The figure is reported as a floor, not a target.
- Offline speech has one emotional contour
- Measured at 5.0–5.4 semitones of pitch variation whatever mood is asked for. Mood renders as pace. Expressive synthesis exists and runs 12× slower than real time.
- Read-aloud accuracy figures flatter every engine
- The ranking between engines holds. The absolute numbers do not transfer to spontaneous speech, and this project’s own harness once scored an engine against its own mistakes before that was caught.
- Cloud engines are genuinely off-device
- They exist, they are labelled at the point of choice, and they are never selected automatically. But they are what they are.
- It needs a Mac that stays awake
- That is a property of what this is, not of how it connects. Pairing is one code typed once; the awake Mac is the real cost.
Writing
The failures, in as much detail as the wins.
It is not finished, and that is written down too.
The round trip works on a real phone. There is no App Store build yet. Leave an address and you get one email when there is something to install — or follow the writing, which is where the progress actually gets posted.