Writing · vllm

A retrospective on bringing up vLLM with FlashInfer on 8×H100 in an air-gapped environment – no root, no internet, every package pre-approved. Why a plain pip install doesn't work here, how to assemble a complete CUDA toolchain by hand from separate RPM components, and which traps – an incomplete repo package, a missing header, a linker that can't find cudart, and a stale JIT cache – stand between you and working inference. Part one of a series on LLM inference in an air-gapped environment.