Signal
12 articles covering "vllm"
RunSybil says a CPython memory bug lets model chat templates escape Jinja sandboxes; Python's security team does not consider the demonstrated escape a CPython vulnerability. The crash fix shipped in 3.13.16, 3.14.8 and 3.15.0 release candidate 2.
Independent coverage of MLPerf Inference v6.0 and v6.1 notes limits on comparing results, MLCommons says Endpoints will replace its datacenter benchmark, and project figures from vLLM and Valkey are self-reported.
Hugging Face's TRL v1.17.0 adds a single-node LoRA example for Mistral Large 4, changes the AsyncGRPO default objective to score centering and makes GRPOConfig raise on top_p or min_p without top_k when vLLM importance sampling is on.
llm-d v1.0.0, published October 20, aligns component versions across its organization, moves the router configuration API to v1, adds DisaggregatedSet and deprecates its own ROCm image build.
A Valkey blog post reports that a three-shard Valkey 9 cache served 4,631 requests versus 3,692 on the same two 8-GPU nodes, at a cost of about 15% on top of the GPU bill.
vLLM 0.32.0 requires Python 3.11 or later, restricts default CORS origins, removes cascade attention and fixes three security advisories, per its release notes.
Ray 2.59.0 declares its LLM batch and serving APIs generally available, upgrades to vLLM 0.27.0, and turns on token authentication for local clusters, with all clusters to follow in 2.61.
vLLM 0.31.0 brings draft-model speculative decoding to Model Runner V2, adds a vllm preload CLI and a max-num-active-seqs cap, and removes tokenizer_mode="slow".
vLLM 0.30.0 adds a GPU-resident weight cache for faster engine restarts and removes GPTQ g_idx; llm-d v0.10.0 then moved its base image to 0.30.0.
Crusoe's oversubscribed round, co-led by Atreides, Mubadala and Valor, funds modular Spark data centers and its Managed Inference platform, which claims to outpace vLLM.
AMD and Embedded LLM tested five speculative-decoding drafting methods in vLLM on AMD Instinct GPUs, with some configurations reaching up to 2.87x throughput over the non-speculative baseline.
Red Hat used its May 12 Summit keynote to take Red Hat Desktop with Podman Desktop and isolated AI agent sandboxing to general availability, and to preview Red Hat AI 3.4 with vLLM speculative decoding, Model-as-a-Service, MCP tooling, and an agent evaluation hub.