Ruturaj Sambhus


I am a PhD Student at Virginia Tech, Blacksburg, VA, in the Hybrid Dynamical Systems and Robot Locomotion Lab, where I aim to work on intersection of Deep Reinforcement Learning (DRL) and Model Predictive Control (MPC) for locomotion and manipulation, advised by Dr. Kaveh Akbari Hamed. Previously, I was at the Terrestrial Robotics Engineering and Control Lab, where I had been working on the intersection of Deep Reinforcement Learning (DRL), Robotics and Control, advised by Dr. Alexander Leonessa. I worked on implementing real-time on-hardware DRL based force control of series elastic actuator, leveraging deadzone for faster learning and better sim to real for torque reacher of 7 DOF Panda manipulator, and real-time on-hardware velocity reacher for the same robot arm. Please see this video for an overview of our work.


Prior to joining Virginia Tech, I was working full-time as a Project Research Associate (Jan'20-Oct'20) and Junior Research Fellow (Jan'21-Aug'21) at Indian Institute of Technology, Bombay, India, in the Robotics Lab on control of haptic virtual interfaces and prosthetic devices with Dr. Abhishek Gupta. I previously graduated from Indian Institute of Technology, Bombay where I completed my Dual Degree Master's research in Advanced Manufacturing Process Lab advised by Dr. Amber Shrivastava. I worked on modeling, characterization and design of piezoelectric ultrasonic transducers.


email | google scholar | linkedin

My picture

Publications (selected | all)


[New] Real-Time Model-Free Deep Reinforcement Learning for Force Control of a Series Elastic Actuator
Ruturaj Sambhus*, Aydin Gokce*, Stephen Welch, Connor W. Herron, Alexander Leonessa
Accepted, IEEE/RSJ IROS 2023
pdf   bibtex

[New] Real-World Deep Reinforcement Learning for Position Tracking of a Pendulum Driven by a Series Elastic Actuator
Ruturaj Sambhus*, Aydin Gokce*, Stephen Welch*, Alexander Leonessa
Accepted, ASME IMECE, 2023

Research Projects (selected | all)


Deep Reinforcement Learning (DRL) for Force Control of Series Elastic Actuator
Terrestrial Robotics Engineering and Controls (TREC) Lab, Virginia Tech, Blacksburg

Objective:

The objective of the project was to apply deep reinforcement learning (DRL) techniques to develop a force control strategy for a series elastic actuator (SEA) driving a pendulum system. The focus was on using data to addressing nonlinearities, such as difficult to identify stiction and backlash, to achieve accurate force trajectory tracking.

Details

Overall, the research project focused on applying deep reinforcement learning techniques to address force control challenges in a series elastic actuator driving a pendulum system. The integration of DRL with the IHMC toolbox and engineering of the RL environment resulted in successful hardware learning and improved performance compared to a PID controller.

Deep Reinforcement Learning (DRL) for Real Robotic Arm Control
Terrestrial Robotics Engineering and Controls (TREC) Lab, Virginia Tech, Blacksburg

Objective:

The objective of the project was to achieve stable hardware learning performance for a 7-degree-of-freedom (DOF) velocity reacher task and torque reacher on a Franka Emika Panda robot arm. The task involved controlling the task space position by using joint velocities and joint torques as actions respectively.

Details

Overall, the project focused on achieving stable hardware learning performance for a 7-DOF velocity reacher task on the Franka Emika Panda robot arm. The low-level C++ code extension enabled torque control during the task, while velocity control was used for reset. By implementing a torque control-based reacher task and utilizing the PyBullet Physics Engine, the project achieved successful sim-to-real transfer without requiring specific techniques for this purpose. Additionally, joint friction estimation was performed to improve the accuracy and realism of the simulation-to-real transfer.

Model Matching H∞ Optimal Control of Haptic Interfaces for Rendering Multi-User Interaction over Shared Virtual Spring
Robotics Lab, Indian Institute of Technology, Bombay, Mumbai, India

Objective:

With an aim to define and improve transparency for shared virtual environments, used Model Matching Approach to design H∞ controller for rendering interaction of human and position controlled robot over a shared virtual massless spring using MATLAB/Simulink.

Details

In summary, this project focuses on the control of haptic interfaces and presents a Model Matching Approach for rendering multi-user interaction over a shared virtual spring. The designed H∞ controller enables the interaction between a human and a position-controlled robot, while further enhancements enable interaction between two humans. The successful implementation and integration of the controllers on the Phantom Premium Haptic Device showcase their effectiveness.

Course Projects (selected | all)


Real Time Handwritten Digit Recognition and Hand Tracking Volume Control (Code)
OpenCV Crash Course, Python, Machine Learning, Computer Vision, Image Processing, Gesture Recognition, Hand

Design of MPC for Gait Planning of Unitree A1 Quadrupedal Robot (Code)
Model Predictive Control

From Scratch Implementation of REINFORCE, A2C for Continuous Control (Code)
Stochastic Approximation and Applications

Study on Residual Policy Learning and DQN discrete control from pixels (Code)
Deep Reinforcement Learning

Hybrid Zero Dynamics based Controller for Design of 2-D Gait for 5-DOF Bipedal robot
Feedback Control of Dynamic Legged Locomotion

Leadership


Project Manager, Team SHUNYA IIT Bombay for Solar Decathlon China 2018
Indian Institute of Technology Bombay, Mumbai, India and Dezhou, Shandong Province, China

Team SHUNYA is a group of 65 passionate students building a Solar Powered, Net Positive Energy hybrid-modular house answering India’s growing energy and housing problems, which represented INDIA at Solar Decathlon 2018 China by successfully constructing a full-scale Net Positive Energy house in 25 days at the competition site in Dezhou, China. Awarded Best Participation.

Details

Overall, Team SHUNYA's remarkable journey showcased our exceptional leadership, innovation, and commitment to sustainability. From constructing a Net Positive Energy house to securing corporate sponsorships and inspiring the next generation, our achievements left an indelible mark. With a focus on collaboration and a vision for a brighter future, Team SHUNYA's accomplishments stand as a testament to our dedication and passion.

Teaching


Graduate Teaching Assistant: ME 4005 Mechanical Engineering Lab, Virginia Tech. Summer 2023, Spring 2023, Summer 2022, Spring 2022
Graduate Teaching Assistant: ME 3435 Control Systems, Virginia Tech. Summer 2023
Graduate Teaching Assistant: ME 4744/5735G, Virginia Tech. Fall 2022, Fall 2021
Graduate Teaching Assistant: ME 316 Kinematics and Dynamics, IIT Bombay. Spring 21
Undergraduate Teaching Assistant: PH 108 Electricity and Magnetism, IIT Bombay. Spring 17

Website Credits


This website template is taken from Shubham Tulsiani.