Videos

llama.cpp's new --moe-cache-cpu-share flag can nearly double inference speed for mixture-of-experts models on GPUs...

Cline's Ara Khan demonstrates that recursive self-improvement (RSI) for coding agents is already practical: a...

Self-hosting open-source coding models on dedicated GPUs cut Revolut's AI coding costs to one-tenth of commercial...

M3.1 Flash, MiniMax's latest LLM, uses sparse attention, a 4-bit KV cache, and speculative decoding to cut inference...

Speculative decoding can speed up LLM inference by 1.6x on structured tasks like code or JSON generation, but offers...

Open models are now competitive with proprietary ones on intelligence, and Nebius Token Factory's optimizations—spec...

A paper co-authored by Filip Makraduli proposes two algebraic tricks—weight folding and deferred normalization—that...

Open source models have narrowed the capability gap with frontier labs to 3-6 months, with recent releases like Kimi...

AMD's ROCm 10 release ships a 'skills' folder that teaches Claude, Cursor, and Codex how to use its GPUs,...

The KV cache is the dominant cost in long-context LLM serving, and the built-in prefix caching in frameworks like...

Speculative decoding is a lossless inference acceleration technique that uses a small draft model to propose...