12 docs tagged with "benchmark"

Era VIII (M74): The fleet learned the hard way that the right tool and the right hardware are not the same thing. Ollama. ROCm. GTT. Vulkan. The Cogitator speaks its truth.

A stranger's tier list crowned two S-tier models for 8GB VRAM. Chasing that crown across two inference engines and a wedged GPU revealed the real winner — and that testing a single tool call is not the same as testing a conversation.

Testing vLLM and SGLang against Ollama on Tanker (Maxwell) and Galleon (Ampere) — and why matching the compute capability isn't enough.

The Silicon Reckoning settled Vulkan over ROCm. Months later, a smaller question — does the RADV driver's own version matter? — returned +31.6% prefill on identical hardware, from a container, without touching the pinned enterprise host.

Sixteen models. Five brackets. Three questions. Almost nothing passed all three. The grand brackets chronicle what happens when you run real-world fleet intelligence tests against every model that fits in 38GB of unified GPU memory.

The champion was crowned. The General reported clean passes, fast times, zero drift. The Emperor asked one question the arena was never designed to answer — and the champion timed out after 180 seconds of silence.

Benchmarking 12B models across the fleet — MLX, CPU, and Vulkan — and the ship role reassignment that followed.

Ten models, two families, two drivers, one iGPU. The size sweep meant to map a smooth curve found a cliff instead — dense models fall off it, Mixture-of-Experts models do not. And the newer driver's gift turned out to have a price only dense models pay.

The rigged arena was dismantled. Real bash execution, real anamnesis queries, no pre-wired functions. The first model to answer both questions correctly — not the largest, not the fastest on mock tools — became the true champion. It was the 12B model all along.

With Ollama disabled and GTT freed, the fleet reran the entire playoff under Vulkan RADV. qwen2.5:14b Q2 dropped from 113 seconds to 21 seconds. No language drift. gemma4:12b Q2 dropped from 93 seconds to 30 seconds. The hardware had been capable the whole time. The backend had not.

The fleet began a model competition with clean assumptions — Ollama, AMD GPU, two test questions. Within hours everything looked broken. Thai text in English responses. 113-second timeouts. A KDE crash. The hardware was fine. The setup was not.

Goose, agy, and a Python script walk into the same terminal. Only one leaves at a useful speed.