[02]
CLOUD CODES · 1D AGO
Speculative decoding is a lossless inference acceleration technique that uses a small draft model to propose multiple tokens, which the large model verifies in a single forward pass, achieving 2-3x speedup without altering output quality. The key insight is that GPU memory bandwidth, not compute, is the bottleneck, so idle compute can be spent on checking multiple tokens at once. In production with batching, gains typically range from 1.2x to 2x, and the technique works best when the GPU is memory-bound—common with long contexts, mixture-of-experts models, and quantized weights.
→ Watch on YouTube
·
→ Full summary
[03]
MATTHEW BERMAN · 1D AGO
Chinese open-weights models like DeepSeek and Qwen now account for more tokens on Vercel's platform than Anthropic's models, yet Anthropic captures 23x more revenue. The emerging pattern is that cheap open-weights models win on volume, while frontier closed-source labs win on dollar value — with a growing geopolitical risk as US companies build on Chinese open-source models.
→ Watch on YouTube
·
→ Full summary
[04]
AI FOUNDERS · 1D AGO
ChatGPT Work introduces a plan-and-approve workflow where the AI can read and write local files, use tools like Slack and calendar, and perform multi-step tasks autonomously. The 11 use cases demonstrate that the valuable skill is shifting from doing work to knowing which work to delegate. The key risk is that the user must read the plan before approving, as it can carry out instructions literally without understanding intent.
→ Watch on YouTube
·
→ Full summary