Infrastructure
33 articles RSS
Cohere Adds Confidential-Computing Encryption to Model Vault, Locking Even Itself Out of Customer Inference Data
Cohere's new Encrypted Vault tier for Model Vault runs inference inside hardware that encrypts memory on both CPU and GPU, so Cohere and the cloud provider cannot read customer data.
NVIDIA Releases Personal AI Router, an Open-Source Tool That Pools Local AI Inference Across a Home Network
NVIDIA's open-source PAIR routes multi-agent inference requests across paired local machines running Ollama or LM Studio, without pooling GPUs or changing agent code.
vLLM Benchmarks Five Speculative Decoding Methods on AMD Instinct GPUs, Reporting Gains Up to 2.87x
AMD and Embedded LLM tested five speculative-decoding drafting methods in vLLM on AMD Instinct GPUs, with some configurations reaching up to 2.87x throughput over the non-speculative baseline.
AWS and Nvidia to Deploy 2 Million More GPUs in 2027-2028 as Demand Outpaces Prior 1-Million Commitment
AWS and Nvidia announced an expanded partnership to deploy 2 million additional GPUs across AWS data centers in 2027-2028, alongside new Vera CPU, networking, and robotics integrations.
FreeToken Lets Frontier Mixture-of-Experts Models Run on Consumer GPUs With Dynamic CPU-GPU Co-Execution
UC Berkeley and MIT researchers, with Databricks co-founders Matei Zaharia and Ion Stoica, released an open-source engine that serves up to 753-billion-parameter MoE models on a single workstation GPU.
CNCF Graduates Kubeflow, Certifying the Kubernetes-Native AI Platform as Production-Ready
The Cloud Native Computing Foundation gave Kubeflow its top maturity tier, citing 6,600-plus contributors and 260 million PyPI downloads.
Samsung Heavy Industries and Mousterian Sign Engineering Contract for First Floating AI Data Center in Texas
The companies will engineer 50 MW moored floating data centers for Texas as the state pauses new grid connections for a 474 GW data center backlog.
Anthropic Signs $19 Billion, 20-Year Lease With TeraWulf for a 401 MW Kentucky AI Data Center
TeraWulf will lease Anthropic a 401 MW AI campus in Hawesville, Kentucky, for about $19 billion over 20 years, while selling its Abernathy Joint Venture stake to Fluidstack.
Digital Realty Buys 1,440 Acres on a Former WWII Munitions Plant for a 2 GW Kansas Data Center Campus
Digital Realty paid about $475 million for land at De Soto, Kansas, planning a campus scaling from 600 MW in 2028 to 2 GW.
Meta Reportedly Builds 'Meta Compute' Cloud Unit to Rent Out Excess AI GPUs, Rattling Neocloud Stocks
Bloomberg reports Meta is standing up a cloud business under its 'Meta Compute' initiative to sell spare AI capacity; CoreWeave and Nebius shares fell 10.8% and 12.4%.
Linux Foundation Launches Tokenomics Foundation to Standardize AI Cost Management, Extending FinOps' FOCUS Spec to Token-Based Spending
The vendor-neutral foundation will partner with FinOps to extend the FOCUS billing spec to AI token spend, backed by 12 enterprises.
AWS Rebuilds Amazon OpenSearch Serverless From the Ground Up for Agentic AI, Reaching GA With 20x Faster Scaling and Up to 60% Lower Cost
AWS launched the next-generation OpenSearch Serverless on May 28, decoupling compute from storage to hit true scale-to-zero and 20x faster autoscaling for bursty agentic workloads.