Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
A free Mac app leverages Google's Embedding Gemma 2 model (designed for search, not speech) to transcribe 1 hour of audio in 12.6 seconds on an 8 GB M3 MacBook Air, achieving 288x faster than playback but with a 5% word error rate—significantly higher than the 2–3% of leading speech models. The key trick is aligning audio and text vectors in a 768-dimensional space using a small 13.3M parameter reader trained in under 2 hours, plus a CTC decoder and a punctuation model.
Key points
The app transcribed 1 hour of audio in 12.6 seconds on an 8GB M3 MacBook Air.
It uses Google's Embedding Gemma 2 model which maps audio into 768-dimensional vectors.
A 13.3M parameter reader model was trained in under 2 hours to align audio vectors to text vectors.
Word error rate is about 5%, while leading models achieve 2–3%.
Homophone errors make up 12.4% of substitution errors according to the reader model card.
Techniques
- vector embeddings
- paired data alignment
- connectionist temporal classification (CTC)
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
A free Mac app turned 1 hour of audio into text in 12.6 seconds. That's the developer's own test on a base 8 GB M3 MacBook Air. Google never trained its model embedding Gemma 2 to transcribe
speech. So how does a model built for search do that? Embedding Gemma 2 turns text, images, video, and audio into lists of 768 numbers called vectors. Google trained it on paired data like
text with its matching audio. Think of each vector as a point on a map with similar items close. A map has two directions, but this space has 768. The audio encoder turns each second of sound
into 25 vectors. The app doesn't document how it splits longer files. The developers say a spoken words audio vector sits close to the written words vector. Google's model card says Google
used that paired data to align representations. Audio sits near words, not on them. So, a small reader moves each vector into the text space. The encoder stays unchanged. A reader card
lists 13.3 million parameters, the numbers it learns, but the app may differ. The developers say they trained only the reader for under 2 hours. The app compares the reader's output with
about 229,000 English words, and the closest word wins. The reader card says a decoder called CTC skips empty vectors and merges repeats. The developers say a tiny extra model adds punctuation and
capitals. In the developers table, the app runs 288 times faster than playback. On the same Mac, IBM Granite Speech needed 25.7 seconds per hour. Two other rows show higher speeds on bigger Macs,
so the developers call it a rough guide. About 5% of words come out wrong, the developers say, against 2 to 3% for leading models. The model cards show 5.8 to 6.8% on clean English audiobook
speech and about 15% on harder sets. Homophones, words that sound the same, make up 12.4% of the reader card substitution errors. It needs Mac S14 and Apple Silicon. And the developers
say your audio stays on your Mac. If you want a free private first draft of clear English audio, try it. If you need exact quotes, use leading models. Google's card doesn't say why a spoken word sits
near its written form, and that's the open question.