Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Gemma 2's embedding model, open-sourced under Apache 2.0 with 740M parameters, enables private on-device search by converting images, sound, and text into shared 768-number vectors, requiring about 567MB RAM on a Pixel 11 Pro after quantization. The real-world constraint is context length: at 140 tokens per video frame and 25 per audio second, a single input covers only about 58 frames or 327 seconds, forcing users to split longer recordings. Google reports a 50.67% top-1 hit rate on general video search and suggests using 128-number vectors for a first pass short list with re-ranking to save space.
Key points
Gemma 2 embedding model has 740M parameters and is free under Apache 2.0.
Quantized model needs about 567MB RAM on a Pixel 11 Pro.
The model converts images, sound, and text into 768-number vectors in one shared space.
One input holds 8,192 tokens covering roughly 58 frames or 327 seconds.
Google advises using BFloat16 or Float32 to avoid NaN values from float16.
Tools mentioned
Techniques
- embedding model with shared vector space for images, audio, and text
- quantization to reduce RAM usage
- first pass short list with re-ranking using 128-number vectors
- BFloat16 or Float32 number formats to avoid NaN
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
You describe a video in a voice memo. Google says your phone can find that clip without uploading it. How? It's embedding Gemma 2, free under Apache 2.0, with 740M parameters. With
quantization, Google says the full model needs about 567 megabytes of RAM on a Pixel 11 Pro. The 270M text part needs 191 megabytes. Google doesn't say which method it used. Three parts read your
input. 170M for images, 300M for sound, 270M for text. Each turns its input into a vector, a list of 768 numbers that summarizes its meaning. Google says all three share one space, so a spoken
description lands near its clip. One input holds 8,192 tokens, the small pieces the model reads. Video cost 140 per frame, and audio cost 25 per second. That's about
58 frames, or 327 seconds, so split longer memos. Picture each vector as an arrow. The score shows how closely a stored arrow points the same way as your query arrow. The highest scores come
first. Real vectors have 768 numbers, which a drawing can't show. A million vectors take about 1.5 gigabytes at 768 numbers, or 250 megabytes at 128. Google says 256 numbers keep about 95% of
quality on image, video, and speech search. For 128, the guide suggests a first pass short list, then a re-rank. Test it. Get the open weights, the model files, from Hugging Face. Turn each file
into a vector once. Store the vectors on the phone. For written answers, Google suggests Gemma 4 to read the top matches. If it runs in the cloud, your matches leave the phone. Don't run the
model in float 16, a number format with a small range. You get NaN, which means not a number, or worse vectors with no error. Use BFloat16 or Float32. Start each text query with task colon search
result pipe query colon. Google's 50.67 hit at one means the right clip comes first about half the time on general video. No source test memo to video and we haven't. The app you build decides
where files are stored. We say solid mid-tier. Will a 128 number short list still find your clip?