Videos

Fireworks AI is selling the full train-to-deploy loop for open-source models, and the pitch is built on scale...

An MCP server that routes document processing tasks from Claude Code to a private Superlinked inference cluster can...

A builder is running Kimi K2, a 2.8 trillion parameter model, on 16 Nvidia DGX Spark boxes at about 21 tokens per...

Strata, an Engine 1 developer's inference engine for Qwen-3 8B Flash Next, claims 44.8 tokens/second on a 12GB RTX...

Unreal Labs' Unreal Agent claims up to 40% lower inference cost than Codex on Terminal Bench 4.0 by changing the...

vLLM's Mamba state cache had two silent bugs—decode-before-prefill scheduling and uint32 index overflow—that...

Small open-source models can match or beat frontier models on specific tasks at a fraction of the cost, but serving...

CoreWeave's inference platform is built around two consumption models, serverless and dedicated, with serverless...

Standard LLM benchmark harnesses often fail to deliver the requested load and inflate latency, making server...

OpenAI's inference load balancer evolved from a feedback-loop-based P controller to a control plane/data plane...

AI workloads are shifting from large transfers to small messages, making latency critical. TCP and RDMA suffer from...

Running DeepSeek V4.1 Flash locally requires at least 510 GB of storage and a multi-GPU setup costing tens of...

NVIDIA PAIR is not a GPU pooling tool that combines memory across machines. It is a request-level load balancer that...

Cerebras CS4 runs GPT-4 at over 4,400 tokens per second, and the upcoming CS5 targets 10,000 TPS for medium models...

OpenAI's custom inference chip, Jalapeno, designed with Broadcom in 13 months, shows preliminary benchmarks that...

LLMD's KV cache-aware routing and prefill-decode disaggregation address the chaotic inference patterns of agentic...

Qwen 3.8 27B, a 27-billion-parameter model from Alibaba, achieves benchmark scores near GPT-5.6 Luna and Opus 4.6...

Picking a single 'best' model based on benchmarks is the wrong approach; the right model depends on the specific...

Stripe's acquisition of OpenRouter is a bet that AI inference will become a major economic flow, similar to...

Generative video models are now efficient enough to generate video faster than you can watch it, at a fraction of...

Qwen released the 27B parameter Qwen3.8-27B model, which significantly outperforms its predecessor Qwen3.6-27B and...

Saoud Rizwan, founder of Cline, argues that the open source community is dying due to AI-generated spam and security...

Inference engineering for large language models involves a stack of optimizations including KV cache-aware routing,...

Waste is a 6,000-line C engine that runs the 2.78-trillion-parameter Kimi K3 mixture-of-experts model on a laptop...

Building agent loops is the new prompting. Developers should focus on creating verification loops and specifying...

Modal is a cloud platform built for AI workloads, focusing on elastic inference, sandboxes, and training. They've...

A Dell Optiplex motherboard can be transplanted into a custom frame to create a budget local AI rig for under $1000,...

A $1,500 32GB VRAM local AI server built from three RTX 3060s (12GB+12GB+8GB) performs surprisingly well for LLM...