Humanoid AI connects walking and object handling without demonstrations of complete task sequences

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Add to Preferred Sources A real humanoid robot performing a continuous, multistep task using RISE. Credit: Seoul National University College of Engineering A humanoid AI system has been developed that learns jointly from separately collected locomotion and object-manipulation data to perform multiple stages of a task in sequence.

A research team led by professor Songhwai Oh of the Department of Electrical and Computer Engineering at Seoul National University College of Engineering has developed RISE (Responsibility-Induced Specialized Experts), a vision-language-action (VLA) framework that learns jointly from locomotion and object-manipulation demonstrations collected independently and enables a humanoid robot to perform multistep tasks sequentially.

The team connected locomotion and object manipulation without requiring additional demonstrations of multistep tasks involving transitions between the two behaviors, and validated the framework in both simulation and experiments using a real humanoid robot, the Unitree G1.

The research was accepted for the IEEE/RSJ International Conference on Intelligent Robots and Systems ( IROS 2026 ).

In recent years, vision-language-action (VLA) technology, which enables robots to jointly understand camera images and natural-language instructions and generate physical actions, has attracted growing attention in robotics. VLA is an AI model that allows a robot to understand its surroundings, interpret instructions such as "pick up the object and move it to the designated location," and act accordingly.

Humanoid robots must perform multiple actions sequentially, such as walking, picking up objects and moving them. In practice, however, locomotion and object manipulation involve substantially different joints and movement characteristics, making it difficult for a single AI model to learn both reliably.

In particular, collecting demonstrations of multistep tasks in which locomotion and manipulation alternate places a substantial burden on data collection because a failure at an intermediate stage can render the entire demonstration incomplete. Individual behaviors such as walking, picking and placing, by contrast, can be collected relatively reliably. However, such data alone provide no information about when the robot should end its current behavior and transition to the next one, making it difficult to perform continuous tasks in real-world settings.

To address this problem, the research team developed RISE, an AI framework that allows locomotion and manipulation data to be collected separately while learning them jointly within a single model to perform multistep tasks sequentially.

RISE combines a shared visual-language perception model with multiple action experts and a meta-controller that coordinates them. During training, the system evaluates how accurately each expert predicts the current action and calculates its "responsibility." Experts with higher responsibility are designed to have a greater influence when learning from the corresponding data. As a result, without predefined roles, individual experts naturally specialize in locomotion or object manipulation.

The research team also used changes in the expert weights output by the meta-controller during execution as signals for task transitions. Building on the observation that expert weights vary according to the characteristics of the current behavior, the system determines when the current task has ended and naturally transitions to the next predefined task. This enables the robot to perform multistep tasks in sequence using only demonstrations of individual behaviors collected separately, without the need to collect additional continuous demonstrations containing transitions between locomotion and object manipulation.

The research team validated RISE both in simulation and on a real Unitree G1 humanoid robot. The robot used in the experiments was equipped with two Dex3-1 robotic hands in addition to its basic whole-body joints, giving it a total of 43 controllable degrees of freedom (DoF).

In simulation experiments, the robot was assigned two- to four-stage tasks involving objects such as apples, bottles and cups. These included moving and then picking up an object; picking up an object and then moving and placing it down; and moving, picking up an object, moving again and placing it down. Across all simulation experiments, RISE achieved an average success rate of 87.3%, outperforming the 73.6% achieved by a GR00T-based baseline with a single-expert architecture. GR00T is a general-purpose foundation model for humanoid robots released by NVIDIA.

In experiments using a real Unitree G1 robot, the system performed locomotion and manipulation tasks consisting of two stages, four stages and up to five stages. The robot sequentially picked up an object and placed it at another location, then picked up a second object, moved to the target location and placed it down. RISE achieved an average task success rate of 50.0%, 20 percentage points higher than the 30.0% recorded by the GR00T-based baseline.

In particular, RISE performed tasks more reliably than the baseline as the number of task stages increased. These results confirmed that experts specializing in different behaviors can be trained using only separately collected locomotion and manipulation data, and that the robot can transition between behaviors at appropriate times to perform multistage humanoid tasks sequentially.

The significance of the research lies in proposing a way to reduce the burden of repeatedly collecting complete, continuous task demonstrations for humanoid robot training. The responsibility-based expert training approach also demonstrates the potential to expand toward integrating behaviors with different characteristics—including locomotion, object picking, bimanual manipulation and tool use—within a single robot AI system.

The technology could provide a foundation for developing humanoids capable of performing complex tasks involving repeated locomotion and object manipulation. It could initially be applied to tasks in which locomotion and manipulation occur together, such as transporting goods in logistics centers and moving parts and tools in manufacturing environments. As adaptability to diverse environments and safety are further improved, its applications are expected to expand to areas such as transporting supplies in hospitals, assisting with caregiving and organizing objects in homes.

Oh, who supervised the research, said, "For humanoid robots to perform a variety of tasks in real-world environments, they must be able to reliably learn behaviors with different characteristics, such as locomotion and object manipulation, and transition naturally between them as needed." He added, "This research is significant in demonstrating the possibility of performing continuous, complex tasks using separately collected locomotion and manipulation data, without having to directly collect demonstrations containing all of the different behaviors."

He continued, "Going forward, we plan to move beyond the current structure, in which locomotion and manipulation occur sequentially, and expand the technology to more complex whole-body behaviors, such as manipulating objects while moving."

Co-first author Hogun Kee earned his Ph.D. from the Robot Learning Lab at Seoul National University and is currently conducting physical AI research in industry. Co-first author Wooseok Oh plans to pursue a career as an AI and robotics researcher and developer at research institutes or companies in Korea or abroad after graduation. The IROS paper confirms Kee and Wooseok Oh as equal contributors.

Learning Humanoid Loco-Manipulation with Responsibility-Induced Specialized Experts for Vision-Language-Action Models, IROS 2026. Key concepts Embodied robotic manipulation Autonomous robotic locomotion Humanoid robotics Provided by Seoul National University Who's behind this story? Robert Egan Bachelor's in mathematical biology, Master's in creative writing. Well-traveled with unique perspectives on science and language. Full profile →