Balakumar Sundaralingam

I am a Staff Research Scientist at NVIDIA Research. I build fast, robust robotics algorithms spanning motion planning, optimization, control, perception, and robot learning. I am the creator and technical lead of cuRobo, a widely adopted open-source library for robot motion generation, and have authored 35+ peer-reviewed publications and 15+ patents and patent applications. My research has been recognized with an RA-L Best Paper Award, a CoRL oral presentation, and an ICRA Best Paper in Robot Manipulation finalist.

I invent new methods and carry them through to real products. cuRobo reformulates global motion planning as parallel trajectory optimization that runs at interactive rates on modern GPUs, and cuRoboV2 extends this to dynamics-aware whole-body optimization, depth-fused TSDF/ESDF collision avoidance, and scalable motion generation for high-DoF robots and humanoids. I partner with NVIDIA's product and engineering teams to bring this work into deployable, maintainable software such as cuMotion, NVIDIA's first production-hardened collision-free motion planning library, and motion generation broadly across the Isaac platform, including Isaac Lab.

More broadly, I work on robot learning and control that connects learned models with optimization-based behavior, including value-guided MPC for grasping, diffusion-based planning and grasp generation, language-guided correction, and visuo-tactile manipulation, much of it alongside the interns and students I mentor. I earned my Ph.D. in Computing (Robotics) from the University of Utah with Prof. Tucker Hermans, working on planning and tactile perception for dexterous in-hand manipulation.

LinkedIn  /  Google Scholar  /  GitHub  /  CV

Research Highlights

For a recent list of my research works, please visit my Google Scholar page.

cuRoboV2: Dynamics-Aware Motion Generation with Depth-Fused Distance Fields for High-DoF Robots

cuRoboV2 extends GPU-accelerated motion generation with dynamics-aware trajectory optimization, dense depth-fused distance fields, and scalable whole-body computation. The system targets safe, feasible, reactive motion generation across manipulators and high-DoF humanoids.

Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control

Grasp-MPC brings learned visual value functions into a model predictive control loop for closed-loop 6-DoF grasping. The controller reacts to clutter, grasp prediction errors, and object motion while balancing grasp success, collision avoidance, and smooth execution.

GraspGen: A Diffusion-Based Framework for 6-DoF Grasping with On-Generator Training

GraspGen models 6-DoF grasp generation as a diffusion process and trains a discriminator directly on generated grasps. The framework scales grasp learning across objects and grippers, improving generalization for simulation benchmarks and real robot grasping.

VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning

VT-Refine learns contact-rich bimanual assembly policies from visual and tactile feedback, then improves them through simulation fine-tuning. The work combines real demonstrations, tactile simulation, and reinforcement learning to improve robustness for precise assembly tasks.

Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning

We address the “robot waiter” task, dynamic non-prehensile object transport in which a manipulator moves quickly while keeping objects balanced on its end-effector. An ensemble of value functions pretrained from a handful of real-world demonstrations guides an uncertainty-aware model-predictive controller, producing smooth, reliable motions that generalize to new objects and even improve on suboptimal demonstrations.

DiffusionSeeder: Seeding Motion Optimization with Diffusion for Rapid Motion Planning

DiffusionSeeder uses a diffusion model to generate diverse trajectory seeds for motion optimization in cluttered scenes. By pairing learned multi-modal initialization with cuRobo-style trajectory refinement, it improves planning speed and success on difficult manipulation problems.

cuRobo: Parallelized Collision-Free Robot Motion Generation

cuRobo formulates robot motion generation as massively parallel trajectory optimization on GPUs, producing fast, smooth, collision-free motions for manipulation. It achieves up to 60x faster planning, 6x lower jerk, and 25% faster motion time, and is widely adopted as an open-source library across academic and industrial robotics.

Correcting Robot Plans with Natural Language Feedback

This project enables users to correct robot plans with natural language feedback by translating instructions into updates to the robot’s planning costs. The approach connects language understanding with trajectory optimization so robots can adjust behavior directly from language.

Model Predictive Control for Fluid Human-to-Robot Handovers(*equal contribution)

We use model predictive control to generate fluid human-to-robot handovers, allowing the robot to adapt its motion as the human and object move. The controller combines many objectives to produce responsive, collision-aware reaching behavior.

STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation

STORM is a GPU-accelerated model predictive control framework for reactive manipulation in joint space. It uses sampling-based optimization to handle non-differentiable costs, enabling fast obstacle avoidance and goal-directed behavior in changing scenes.
Selected for Oral Presentation (6.5% acceptance rate)

Joint Space Control via Deep Reinforcement Learning

This work studies joint-space robot control with deep reinforcement learning, learning policies that map directly to low-level robot actions. The project explores how learned controllers can produce smooth actions while also reaching within cm precision across the robot’s workspace.

Robust Learning of Tactile Force Estimation through Robot Interaction

We introduced a robot-interaction data collection strategy for tactile sensors and a geometry-aware neural model for estimating contact forces from BioTac signals. The resulting system improved small-force estimation for manipulation-relevant contacts.
Best Manipulation Paper Award Finalist

Relaxed-Rigidity Constraints: In-Grasp Manipulation using Purely Kinematic Trajectory Optimization

We formulate in-grasp object reposing as a trajectory optimization problem, enabling multi-fingered hands to reposition unknown objects using only kinematics.

Jekyll fork from Leonid Keselman's website, original design and source code from Jon Barron's website