Signal
9 articles covering "llm-inference"
llm-d v1.0.0, published October 20, aligns component versions across its organization, moves the router configuration API to v1, adds DisaggregatedSet and deprecates its own ROCm image build.
A Valkey blog post reports that a three-shard Valkey 9 cache served 4,631 requests versus 3,692 on the same two 8-GPU nodes, at a cost of about 15% on top of the GPU bill.
vLLM 0.32.0 requires Python 3.11 or later, restricts default CORS origins, removes cascade attention and fixes three security advisories, per its release notes.
NVIDIA's TensorRT-LLM v1.4.0rc0, published October 14, 2026, labels two API changes (BREAKING) in its release notes and lists five known issues.
JFrog's October 7 advisory rates LMCache's CVE-2026-105192 at CVSS 9.8 and lists no fixed release; a fix pull request opened October 11 was still open that day.
Ray 2.59.0 declares its LLM batch and serving APIs generally available, upgrades to vLLM 0.27.0, and turns on token authentication for local clusters, with all clusters to follow in 2.61.
vLLM 0.31.0 brings draft-model speculative decoding to Model Runner V2, adds a vllm preload CLI and a max-num-active-seqs cap, and removes tokenizer_mode="slow".
vLLM 0.30.0 adds a GPU-resident weight cache for faster engine restarts and removes GPTQ g_idx; llm-d v0.10.0 then moved its base image to 0.30.0.
IBM Research, Red Hat, and Google Cloud donated llm-d, a Kubernetes-native LLM inference framework, to the CNCF as a sandbox project at KubeCon Europe 2026, backed by NVIDIA, CoreWeave, AMD, and other partners.