Infrastructure
63 articles RSS
vLLM 0.32.0 Drops Python 3.10, Stops Allowing Every CORS Origin by Default and Removes Cascade Attention
vLLM 0.32.0 requires Python 3.11 or later, restricts default CORS origins, removes cascade attention and fixes three security advisories, per its release notes.
MLX 0.32.4 Adds Attention and Gated Delta Net Gradients, Metal 1-Bit Affine Quantization and Python 3.15 Wheels
Apple's MLX array framework shipped v0.32.4 on October 14 with training-oriented VJPs, 1-bit affine quantization on CPU and Metal, Python 3.15 support and stft/istft.
JAX 0.12.0 Adds an IFRT Proxy Backend and Custom VJP Rematerialization, and Requires Haswell-Class x86-64 CPUs and NumPy 2.2
JAX 0.12.0, tagged October 19, adds erfcinv, a polynomial primitive, defremat and a proxy backend, and removes jax.custom_remat and support for AVX-only CPUs.
ONNX Runtime 1.31.0 Stabilizes Model-Package and EPContext APIs, Deprecates ACL EP and Changes GatherBlockQuantized Out-of-Range Behavior
ONNX Runtime 1.31.0 promotes model-package and EPContext data APIs to stable, deprecates the ACL execution provider, and makes out-of-range GatherBlockQuantized indices return zeros.
PyTorch 2.14 Adds NVGEMM Kernels for Inductor, a Prototype torch.switch Operator and @dynamic_spec; 2.14.1 Fixes Silent Errors on MPS and CUDA 13.2
PyTorch 2.14.0 (September 2, 2026) brings CuTeDSL-based NVGEMM to Inductor, a prototype switch op and declarative dynamic shapes; 2.14.1 followed on September 30 with correctness fixes.
SGLang v0.5.21 Lets Prefill-Decode Instances Switch Roles Without Restart, Moves Its Prefix Cache to a Rust Core and Lists Ten Upgrade Notes
SGLang v0.5.21 lets PD instances switch between prefill and decode without a restart, makes a Rust prefix-cache core the default, and removes deprecated endpoints.
TensorRT-LLM v1.4.0rc0 Labels Two API Changes BREAKING, Including Removal of the KVCacheManagerV2 Python Backend, and Lists Five Known Issues
NVIDIA's TensorRT-LLM v1.4.0rc0, published October 14, 2026, labels two API changes (BREAKING) in its release notes and lists five known issues.
Ollama 0.40.2 Upgrades Downloaded Models in the Background and Keeps Backups, 0.40.3 Pauses It, and 0.40.4 Adds Decision Models to ollama run
Ollama's v0.40.2 release notes say older models are upgraded in the background with backups kept on disk; v0.40.3 pauses the upgrade, and v0.40.4 adds decision models to the CLI.
OpenAI Adds Ultrafast Mode for GPT-6.1 Sol in the API, Listing It at $12 Input and $60 Output per Million Tokens, Six Times Standard Rates
OpenAI's changelog says GPT-6.1 Sol gained an Ultrafast service tier on Oct 8; its pricing page lists $12 input and $60 output per million tokens, six times Standard.
TRL 1.15.0 Turns On a Triton Fused LM Head by Default, Deprecates use_liger_kernel in Four Trainers and Drops Python 3.10
TRL v1.15.0 scores SFT, DPO, KTO, GRPO and RLOO tokens with a Triton fused LM head by default, deprecates use_liger_kernel in four trainers and requires Python 3.11 or later.
OpenAI Simplifies API Usage Tiers From Five to Three, With Build, Launch and Grow Starting at $5, $100 and $500 in Credit Purchases
OpenAI's API changelog says it cut usage tiers from five to three on October 6. Its rate-limits guide lists monthly usage limits of $500, $5,000 and $200,000.
huggingface_hub v2.2.0 Makes hf jobs stats Print One Snapshot by Default and Fixes a Race That Could Let Concurrent Downloads Skip hf_xet
The October 8 release changes hf jobs stats to exit after one snapshot unless -f is passed, fixes an hf_xet version-lookup race, and repairs HfFileSystem cache bugs.