Balakumar Sundaralingam

I am a Staff Research Scientist at NVIDIA Research. My research focuses on building robotics algorithms that are robust enough for broad adoption in real systems, combining numerical optimization, control, perception, machine learning, and CUDA-accelerated computing. I am the lead architect and inventor behind cuRobo, a broadly adopted open-source robot motion-generation library, and have authored 35+ peer-reviewed publications and 20+ patents/patent applications in robotics.

My recent work centers on real-time GPU-accelerated motion generation for manipulation. Starting from cuRobo, I have focused on formulating robot motion generation as a practical optimization problem that can run at interactive rates on modern GPUs. This line of work now extends to cuRoboV2, which adds dynamics-aware whole-body optimization, depth-fused TSDF/ESDF collision checking, and scalable motion generation for high-DoF robots.

I also work on robot learning and control methods that connect learned models with optimization-based robot behavior, including value-guided MPC for grasping, diffusion-based planning and grasp generation, language-guided robot correction, and visuo-tactile manipulation. I received my Ph.D. in Computing (Robotics) from the University of Utah under the supervision of Prof. Tucker Hermans, where my research focused on planning and tactile perception for dexterous in-hand manipulation.

LinkedIn  /  Google Scholar  /  GitHub  /  CV

Research Highlights

For a recent list of my research works, please visit my Google Scholar page.

cuRoboV2: Dynamics-Aware Motion Generation with Depth-Fused Distance Fields for High-DoF Robots

cuRoboV2 extends GPU-accelerated motion generation with dynamics-aware trajectory optimization, dense depth-fused distance fields, and scalable whole-body computation. The system targets safe, feasible, reactive motion generation across manipulators and high-DoF humanoids.

Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control

Grasp-MPC brings learned visual value functions into a model predictive control loop for closed-loop 6-DoF grasping. The controller reacts to clutter, grasp prediction errors, and object motion while balancing grasp success, collision avoidance, and smooth execution.

GraspGen: A Diffusion-Based Framework for 6-DoF Grasping with On-Generator Training

GraspGen models 6-DoF grasp generation as a diffusion process and trains a discriminator directly on generated grasps. The framework scales grasp learning across objects and grippers, improving generalization for simulation benchmarks and real robot grasping.

VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning

VT-Refine learns contact-rich bimanual assembly policies from visual and tactile feedback, then improves them through simulation fine-tuning. The work combines real demonstrations, tactile simulation, and reinforcement learning to improve robustness for precise assembly tasks.

Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning

This work combines batch reinforcement learning with model predictive control for dynamic non-prehensile object transport, where a robot must move objects without grasping them. Learned value functions guide an uncertainty-aware MPC controller, enabling robust real-world deployment from limited demonstrations.

DiffusionSeeder: Seeding Motion Optimization with Diffusion for Rapid Motion Planning

DiffusionSeeder uses a diffusion model to generate diverse trajectory seeds for motion optimization in cluttered scenes. By pairing learned multi-modal initialization with cuRobo-style trajectory refinement, it improves planning speed and success on difficult manipulation problems.

CuRobo: Parallelized Collision-Free Robot Motion Generation

cuRobo formulates robot motion generation as massively parallel trajectory optimization on GPUs, producing fast, smooth, collision-free motions for manipulation.

Correcting Robot Plans with Natural Language Feedback

This project enables users to correct robot plans with natural language feedback by translating instructions into updates to the robot’s planning costs. The approach connects language understanding with trajectory optimization so robots can adjust behavior without task-specific reprogramming.

Model Predictive Control for Fluid Human-to-Robot Handovers(*equal contribution)

We use model predictive control to generate fluid human-to-robot handovers, allowing the robot to adapt its motion as the human and object move. The controller combines many objectives to produce responsive, collision-aware reaching behavior.

STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation

STORM is a GPU-accelerated model predictive control framework for reactive manipulation in joint space. It uses sampling-based optimization to handle non-differentiable costs, enabling fast obstacle avoidance and goal-directed behavior in changing scenes.
Selected for Oral Presentation (6.5% acceptance rate)

Joint Space Control via Deep Reinforcement Learning

This work studies joint-space robot control with deep reinforcement learning, learning policies that map directly to low-level robot actions. The project explores how learned controllers can produce smooth actions while also reaching within cm precision across the robot’s workspace.

Robust Learning of Tactile Force Estimation through Robot Interaction

We introduced a robot-interaction data collection strategy for tactile sensors and a geometry-aware neural model for estimating contact forces from BioTac signals. The resulting system improved small-force estimation for manipulation-relevant contacts.
Best Manipulation Paper Award Finalist

Relaxed-Rigidity Constraints: In-Grasp Manipulation using Purely Kinematic Trajectory Optimization

We formulate in-grasp object reposing as a purely kinematic trajectory optimization problem, enabling multi-fingered hands to reposition unknown objects without an object dynamics model. The approach alternates between finger gaiting and object motion to produce feasible dexterous manipulation plans.

Jekyll fork from Leonid Keselman's website, original design and source code from Jon Barron's website