89 seconds. All explanations appear as on-screen text and animation. Audio is an original instrumental soundtrack.
Foveated perception for dexterous manipulation. Watch the robot's actual compact foveated observation as it grasps the requested object.
Learning robot control directly from perception is challenging. Simulation can massively scale learning experience, but high-resolution perception remains costly. Four GPUs struggle; eight improve learning, yet zero-shot transfer reaches only 24 percent success in 50 robot trials. Training curves show time in the success region, not episode success.
But not every pixel needs equal attention. Foveation provides detail and context at minimal computational cost. The sampling grid and compact image transform together from uniform to foveated, with the same 128 by 128 output size.
Render the foveated view directly. Each output pixel launches a ray into a standard five-object clutter scene from the calibrated camera. Rays reveal the matched native RTX observation. Rotating the observation camera changes both the ray bundle and the RTX image.
Efficient perception enables faster and more diverse training. A compact observation enables six times as many parallel environments and 8.9 times fewer visual tokens. Full training runs achieve 4.4 times the throughput.
More GPUs. Consistently higher throughput. Measured full-run training throughput scales nearly linearly from four to eight L40S GPUs. Foveation maintains approximately 4.4 times the throughput of uniform high resolution at both budgets. Points are means of three seeds, with fixed environments per GPU.
Better training from the same or less compute. Return to the four- and eight-GPU high-resolution curves, then add foveated learning at both budgets. Foveation also outperforms the matched low-resolution active baseline. Curves show three-seed mean success-region occupancy over training time.
Zero-shot sim-to-real transfer. Foveated vision achieves 78 percent success over 50 robot trials, versus 26 percent for uniform active and 24 percent for uniform high resolution. Simulation success is 87.6 percent over 2,000 trials.
FOVI-Dex. Foveation enables efficient perceptive learning of dexterous manipulation in simulation. Dex Manipulation Workshop, 10th Conference on Robot Learning (CoRL 2026), Austin TX, USA.