Videos

Owning agent inference on a dedicated GPU is cheaper and faster than renting API intelligence for steady workloads:...

Memory capacity and bandwidth, not GPU model numbers, decide what a card can run for local AI. The RTX 5090's 32 GB...

Qwen 3.8 Flash Next, served by the Strata engine on a roughly $1,000 secondhand build, generated a fully interactive...

Mistral Large 4, a 1.05 trillion parameter mixture-of-experts model, will not fit in consumer hardware: its 4-bit...

A single loss change, swapping MSE for RMSSE in Omniquant, recovers about 63% of the accuracy gap when quantizing a...

A 300B parameter model can run on a 128GB mini PC using aggressive quantization to ~2-3 bits per weight, sparse...

Gemma 4 is Google DeepMind's latest open model family, released under Apache 2.0, with sizes from 2B to 31B...

An untouched one-bit Qwen 3.8 27B jumps from 29% to 61.5% on MMLU Pro and cuts time-to-first-token from 12.27 to...

vLLM's paged attention mechanism, which treats KV cache like virtual memory pages, directly solved the 60-80% memory...

Strata, an Engine 1 developer's inference engine for Qwen-3 8B Flash Next, claims 44.8 tokens/second on a 12GB RTX...

WebLLM allows developers to run AI models directly in the browser on the visitor's GPU, eliminating inference API...

Fitting a model on your GPU is not enough; you must also account for KV cache and runtime overhead. For most...

On a 6GB GPU, the deciding factor for local coding models is not the download size but the memory left over after...

Turboquant, which uses polar coordinates to quantize the KV cache to four bits, has a fatal flaw for data center...

Bonsai 2 compresses Qwen 3.8 from 54GB to 5.9GB using ternary quantization (1.76 bits per weight), claiming 98%...

Bonsai 2 27B's ternary quantization claims 98% quality retention at 9x smaller size, but in practice the quality...

A $200/month AI subscription costs 2.4% of US GDP per capita but 89% of India's, making affordability deeply unequal...

Local AI workstation capacity specs are misleading: DGX Station's 748 GB is really 252 GB of fast HBM plus 496 GB of...

Two 8.4 GB GGUF files of Qwen 3.8 27B differ by 7.7 points on average benchmarks because one uses a new quantization...

Qwen 3 Flash Next at Q4 quantization shows noticeably weaker visual reasoning than the full-precision Qwen3 27B,...

GLM 5.3 Flash, tested locally at 1-bit quantization, demonstrated strong reasoning capabilities by generating a...

Qwen 3.8 Flash Next, a preview of the Qwen 4 architecture, can run on a quad-RTX 3090 rig with 96GB VRAM using...

Qwen 3.8 27B, a 27-billion-parameter model from Alibaba, achieves benchmark scores near GPT-5.6 Luna and Opus 4.6...

A configuration of nine software changes on an RTX 3090 achieves 1,000 tokens/sec for 64 concurrent users by...

Ornith 1.5 35B-A3B, a Mixture of Experts model based on Qwen 3.5, shows impressive capability for its size,...

Standard VRAM calculators systematically overestimate cache memory for modern open-weight models because they assume...

Qwen released the 27B parameter Qwen3.8-27B model, which significantly outperforms its predecessor Qwen3.6-27B and...

Alibaba's Qwen 3.8 27B dense model claims to beat Opus 4.6 on several coding benchmarks while being Apache 2.0...

16 GB VRAM just became the most common GPU config on Steam, which means the local AI tier list has shifted. Qwen 3.5...

Meta's Muse Glimmer is a 30B parameter coding agent that runs in 14GB of RAM thanks to dynamic quantization, which...

Inference engineering for large language models involves a stack of optimizations including KV cache-aware routing,...

DeepSeek V4 Flash 0731 is a powerful local AI model that excels at reasoning and benchmarks but tends to overthink...

Tiny models (50M-500M parameters) are now viable for edge devices and robotics, enabling voice-to-function calling...

Prism ML's Bonsai 27B models (ternary and 1-bit) dramatically shrink a Qwen 3.6 27B base while retaining surprising...