Why Smarter AI Doesn't Mean Smarter Robots (Yet)

Robots and AI are often talked about as if they are the same project, but they are not. The AI research that gets headlines (big vision models, reinforcement learning, large language models) mostly runs in simulation or in a data center. The robot has to deal with gravity, friction, battery drain and the fact that a dropped object does not generate a loss function. This post looks at where the two fields actually meet today, what each AI technique contributes to a physical machine, and where the real bottlenecks are.

A robot's software stack usually has three layers: perception, planning, and control. AI has made real progress in the first two. Control, the part that turns a plan into motor commands, is still mostly classical: PID loops, model predictive control, inverse kinematics. The AI work sits on top of that layer, not inside it.

That split explains why a robot can have a strong vision model and still fail to pick up a cup. The model tells it where the cup is. It does not tell the motors how much force to apply before the cup slips.

Vision is the most mature AI contribution to robotics. Object detection and segmentation models let a robot find items on a shelf, identify a surgical landmark, or track a package on a conveyor. Depth estimation, often from stereo cameras or LIDAR fused with a learned model, gives the robot a 3D sense of its surroundings instead of a flat image.

# simplified perception loop frame = camera. capture () detections = vision_model. predict ( frame ) for obj in detections: depth = depth_estimator. estimate ( obj. bbox, frame ) world_pose = camera_to_world ( obj. bbox, depth ) planner. add_target ( world_pose ) Enter fullscreen mode Exit fullscreen mode This loop runs several times a second on a real robot. The model's job ends at producing a pose. Everything after that, approaching the object and gripping it with the right force, is a separate problem.

Reinforcement learning (RL) trains a lot of locomotion and manipulation policies. Instead of hand coding a walking gait, you simulate the robot thousands or millions of times, reward it for staying upright and moving forward, and let the policy emerge.

# simplified RL training loop for episode in range ( num_episodes ): state = env. reset () done = False while not done: action = policy ( state ) next_state, reward, done = env. step ( action ) buffer. store ( state, action, reward, next_state ) state = next_state policy. update ( buffer. sample ()) Enter fullscreen mode Exit fullscreen mode Boston Dynamics' Atlas and Tesla's Optimus both lean on this kind of training for balance and recovery. Sim-to-real transfer is the hard part: a policy that works in simulation often needs real-world fine-tuning because friction, motor backlash and sensor noise never match the simulator exactly.

LLMs are showing up in robotics as planners, not controllers. Given a high-level instruction like "clear the table," an LLM can break it into subtasks (locate objects, pick each one up, place it in a bin) that get handed to lower-level motion policies. Google's RT-2 and similar vision-language-action models go further, mapping images and text directly to robot actions. The LLM does not move a single motor; it decides what should happen next and leaves the execution to the control stack described earlier.

The next few years look less like "robots get smarter" and more like "robots get cheaper and more reliable at narrow tasks." Expect AI to keep improving perception and task planning while mechanical engineering, battery density and manufacturing cost close the gap on the physical side. The bottleneck has shifted: it used to be the software. Increasingly, it is the hardware underneath it.