FreeToken Lets Frontier Mixture-of-Experts Models Run on Consumer GPUs With Dynamic CPU-GPU Co-Execution
UC Berkeley and MIT researchers, with Databricks co-founders Matei Zaharia and Ion Stoica, released an open-source engine that serves up to 753-billion-parameter MoE models on a single workstation GPU.