Meta and UIUC Researchers Get an 8-Billion-Parameter Model to Match Claude Opus 4.5 With a Smarter Agent Harness

Overview

Researchers from Meta AI and the University of Illinois Urbana–Champaign have shown that a small, open-weight language model can match a much larger closed frontier model on a long-horizon agent benchmark, not by scaling up the model but by changing how the agent manages its own memory and state. In a paper posted to arXiv, the team’s method, called EvoHarness-RL, trains an 8-billion-parameter Qwen3-8B model to a 96.9% average success rate on the ALFWorld benchmark’s seen-task split, edging past Claude Opus 4.5’s 96.4% out-of-the-box score on the same tasks, according to VentureBeat and the paper itself.

What We Know

• The paper, titled “EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents,” was submitted to arXiv on August 5, 2026, and lists sixteen authors split between the University of Illinois Urbana–Champaign — including lead author Xuying Ning, Tianxin Wei, Yuanchen Bei, Bingxuan Li, Zihao Li, Hanghang Tong and Jingrui He — and Meta AI, whose researchers include Dongqi Fu, Hanqing Zeng, Qifan Wang, Xiang Shen, Yifan Wu, Jiayi Liu, Hong Li, Yinglong Xia and Xiangjun Fan, according to the full paper text.
• The core problem the researchers target is what they call the runtime “harness” — the external scaffolding that lets an agent track state, invoke tools and reuse past experience across a task that spans far more turns than fit in a single context window. As VentureBeat frames it, an agent “migrating massive batches of customer records from a legacy CRM to a cloud database” cannot rely solely on its context window for a job spanning hours, and instead depends on this runtime layer, according to VentureBeat.
• EvoHarness-RL organizes that external state into three categories the paper calls Belief, Progress and Experience — together, BPE — covering the agent’s read on its environment, its completed and pending subgoals, and its reusable history, and lets the model act on that state through four compact meta-actions: track, commit, recall and note, according to VentureBeat.
• On ALFWorld’s seen-task split, Qwen3-8B with a plain ReAct baseline scores 47.9%. Training it with EvoHarness-RL’s combination of supervised fine-tuning and reinforcement learning lifts that to 96.9% — a 49.0-percentage-point gain that the paper’s abstract describes as “a +49.0 absolute improvement over the base ReAct model.” The trained model also outperforms two other trainable agent frameworks in the comparison, SkillRL (89.9%) and SkillOS (80.2%), according to VentureBeat.
• On ALFWorld’s harder, unseen-task split, the same progression holds: a plain ReAct baseline scores 50.0%, an untrained prompt-time version of the BPE harness lifts that to 77.6%, and the full reinforcement-learning-optimized policy reaches 86.6%, according to the paper.
• The BPE harness helps frontier models too, even without any retraining. Applying it at prompt time to frozen, out-of-the-box models raised GPT-4.1’s success rate by 22.1 percentage points and GPT-5’s by 25.7 points; applied to Claude Opus 4.5, it pushed the model’s own score from 96.4% to 98.5%, according to the paper.
• The paper identifies two dynamics behind the gains: “harness annealing,” where training internalizes recurring harness-use patterns into the model’s own policy and shifts the agent toward more selective use of external state, and “harness evolution,” where the agent’s own progress updates and experience consolidation refine that external state into a more compact, task-adaptive form, according to the paper.
• Lead author Xuying Ning told VentureBeat that a major motivation was avoiding the retuning cost of hand-built agent scaffolding: “The optimal harness often changes with the model. Different models may need different prompts, memory designs, permissions, or sandbox configurations. If all of this logic is manually coded, every model upgrade can lead to another long cycle of tuning and debugging,” according to VentureBeat.
• Ning also argued that simply letting an agent’s memory accumulate indefinitely can hurt it rather than help: “Append-only memory assumes that more context is always helpful, which is not necessarily true. Over a long task, the memory may contain outdated conclusions, failed attempts, or information that is no longer relevant,” according to VentureBeat.
• For teams weighing inference costs, Ning described a hybrid option that avoids running everything on the most expensive model: “One possible compromise is to use a frontier model to generate high-quality consolidation data, then fine-tune a capable open-weight model to handle routine state management,” and noted that because consolidation “can happen asynchronously, it does not always need to slow down the agent’s main execution loop,” according to VentureBeat.

What We Don’t Know

• ALFWorld is a text-based simulated household-task benchmark, not a coding benchmark, so it remains untested whether EvoHarness-RL’s gains carry over to software-engineering agent tasks specifically, beyond the general applicability the researchers claim for long-horizon agents.
• The paper and VentureBeat’s coverage do not disclose a release timeline or license for any accompanying code or trained model weights, so it is not yet clear whether other teams can reproduce or adopt EvoHarness-RL directly rather than reimplementing it from the paper.
• Neither source specifies exact compute budgets or training cost for the reinforcement-learning stage, so it’s unclear how the training expense compares to simply running a larger frontier model.

Analysis

The result adds to a run of recent work arguing that how an agent is wired to its tools and memory can matter as much as which underlying model powers it — a similar argument, though with a different technical approach, underpinned TrueFoundry’s TrueForge, an open-source commercial agent harness the company benchmarked against Claude-based managed agents earlier this month. EvoHarness-RL differs in being an academic reinforcement-learning method rather than a shipped product, but both point to the same practical takeaway for developers building agentic systems: harness design, not just model size, is emerging as a lever for cutting the cost of running capable AI agents.