KAIST's VOTP Teaches Robots Reward Functions From as Few as Ten Preference Videos, Earning an ICML 2026 Oral

Overview

A team at the Korea Advanced Institute of Science and Technology (KAIST) has developed a method that teaches robots which behaviors a human prefers using only a handful of labeled video comparisons, rather than the thousands to tens of thousands of human judgments that preference-based learning usually demands. The technique, called VOTP — Video-based Optimal Transport Preference — has been accepted as an oral presentation at the International Conference on Machine Learning (ICML) 2026, a distinction given to 168 papers out of 23,918 submissions, or about 0.7 percent, according to The Brighter Side of News. The conference is being held at COEX in Seoul in July, according to Mirage News.

What We Know

The paper, titled “Video-Based Optimal Transport for Feedback-Efficient Offline Preference-Based Reinforcement Learning,” is authored by Tung M. Luu, Hwanhee Kim, Younghwan Lee, and Chang D. Yoo, per the ICML 2026 program. Tung M. Luu is the first author and a PhD student, Hwanhee Kim is the second author and an M.S. candidate, and Professor Chang D. Yoo is the lead researcher; all are from KAIST’s School of Electrical Engineering, according to Mirage News.

The problem VOTP attacks is a long-standing one in reinforcement learning. Conveying complex objectives to an RL agent normally requires hand-crafting a reward function — meticulous, brittle engineering work. Preference-based reinforcement learning (PbRL) offers an alternative by learning the reward function from human feedback instead, but as the paper’s authors note in their abstract, “its scalability is hindered by high labeling costs,” per the ICML 2026 listing. Producing the volume of human comparisons that conventional PbRL needs is the bottleneck VOTP is designed to remove.

VOTP’s answer is a semi-supervised framework. As the authors describe it, the method is “a semi-supervised framework that learns effective reward functions from only a handful of labels” by “leveraging optimal transport to align visual trajectories within the rich representation space of” video foundation models, which lets it generate “high-fidelity pseudo-labels for large amounts of unlabeled data,” according to the paper’s abstract on the ICML site. In plainer terms: the system represents short segments of robot behavior as video, runs them through a pretrained video foundation model, and uses optimal transport — a mathematical framework for matching one distribution to another at minimum cost — to infer preferences and propagate pseudo-labels to unlabeled pairs, as The Brighter Side of News explains. The OpenReview record for the paper lists its keywords as preference-based reinforcement learning, offline reinforcement learning, feedback efficiency, optimal transport, video foundation models, semi-supervised learning, and robotics, per OpenReview.

The headline efficiency claim is concrete. In real-world robot tests, the team used just five preference labels for a task called LiftBanana and ten for a task called DrawerOpen, according to The Brighter Side of News. Those tasks were trained from only 50 demonstrations collected by keyboard teleoperation with a 50 percent success rate, and VOTP went on to reach 80 percent success on LiftBanana and 70 percent on DrawerOpen, also per The Brighter Side of News. A separate report frames the input as “around 10 good demonstration videos and bad demonstration videos,” according to Seoul Economic Daily.

Beyond the real-robot demonstrations, the authors report that VOTP “outperforms state-of-the-art offline PbRL methods under limited feedback budgets” across locomotion and manipulation benchmarks, and that it remains robust “in the presence of visual distractors,” per the abstract on the ICML 2026 site.

The researchers position the work within the broader push toward physical AI. “The core of physical AI is making machines understand human intentions and choose the correct actions,” Professor Chang D. Yoo said, as quoted by The Brighter Side of News. The team frames the approach as applicable across robotic arm control, humanoid robots, autonomous vehicles, smart factories, drones, and surgical robots, as well as software AI agents that view computer screens and perform tasks on their own, according to Seoul Economic Daily.

What We Don’t Know

The public materials around the announcement do not name the specific video foundation model VOTP builds on, nor the exact locomotion and manipulation benchmark suites or the baseline methods it was compared against. The abstract asserts superiority over state-of-the-art offline PbRL under limited feedback but does not, in the materials reviewed here, attach margins to that claim. The reported real-robot numbers — five and ten preference labels, 80 and 70 percent success — come from a single trade outlet’s account of the paper rather than from the conference abstract itself, so readers should treat those specifics as preliminary until the full paper and its experimental tables are examined.

There is also a minor discrepancy worth flagging: the ICML program lists the first author as “Minh-Tung Luu,” while KAIST’s announcement and the OpenReview record render the name as “Tung M. Luu” and “Tung Minh Luu” respectively — the same researcher, with the name ordered differently across sources.

Analysis

VOTP sits at the intersection of two trends that have been reshaping robot learning. The first is the maturation of video foundation models, whose learned visual representations are increasingly being repurposed as general-purpose perception backbones. The second is the long-running effort to cut the human-feedback cost of preference-based RL, which has historically been the technique’s Achilles’ heel: reward-learning quality scales with the number of human comparisons, and collecting those comparisons is slow and expensive. By using optimal transport to spread a few trusted labels across a large pool of unlabeled trajectory pairs inside a video model’s embedding space, VOTP is, in effect, trying to buy most of the accuracy of dense human feedback at a fraction of the labeling budget.

If the efficiency results hold up under independent scrutiny at the conference, the practical implication is that teaching a robot a new preference could become closer to a demonstration-and-correction exercise than a data-labeling project. Whether that generalizes beyond the locomotion and manipulation settings the authors tested — and how it compares against the strongest competing offline PbRL baselines on standardized benchmarks — is the question the ICML 2026 oral in Seoul this July is positioned to answer.