Videos

BenchPod is a RunPod template that turns LLM config tuning into a data-driven benchmark process rather than...

DeepSeek's cheap repeated-prompt pricing comes from an on-disk KV cache: the engine stores prompt-reading notes on...

Owning agent inference on a dedicated GPU is cheaper and faster than renting API intelligence for steady workloads:...

Speculative decoding can speed up LLM inference by 1.6x on structured tasks like code or JSON generation, but offers...

DatologyAI generated 12 trillion synthetic tokens for pre-training by scaling their "beyond web" rephrasing recipe...

For shared GPU serving with vLLM, memory space from the KV cache — not compute — is what usually limits how many...

vLLM's Mamba state cache had two silent bugs—decode-before-prefill scheduling and uint32 index overflow—that...

Qwen 3.8 27B at FP16, despite being slower, outperforms Qwen 3.8 Flash Next at int4 for agentic code tasks because...

Qwen 3.8 Flash Next running in a Hermes agent on a quad-3090 setup achieves 55-60 tokens per second with a Q4...