Making rigid, position-controlled industrial arms behave compliantly — learned force control, variable compliance via deep RL, and deployment at the World Robot Summit.
Most robots already installed in factories are rigid and position-controlled: they track a commanded pose stiffly and have no native way to regulate how hard they push. That is fine in free space and a liability the moment two parts touch. This theme is about giving that existing hardware compliant behavior through control and learning, without replacing the mechanism.
Learned force control on stiff hardware
The problem in one picture: a position controller commanded to a goal pose inside the environment will simply drive into it, and the resulting contact force is whatever the geometry happens to produce. The core result is that a rigid position-controlled arm can learn force control directly, closing the loop through the position interface it already exposes — the learned policy contributes a trajectory correction on top of the nominal controller and simultaneously tunes the force controller’s own parameters (Beltran-Hernandez et al., 2020). Building on that, deep reinforcement learning can select the compliance parameters themselves rather than leaving them to hand-tuning — variable compliance for peg-in-hole assembly, where the right stiffness differs between approach, contact, and insertion (Beltran-Hernandez et al., 2020).
Making it trainable in practice
Contact-rich RL is slow and hardware-hostile, which makes sample efficiency the binding constraint. Curriculum learning shortens that loop by ordering tasks from easy to hard instead of starting at full difficulty (Beltran-Hernandez et al., 2022), an approach carried into industrial insertion (Beltran et al., 2022). Where the state is only partially observable — a soft wrist hides the true contact state — exploiting the symmetry of the assembly task itself recovers much of the lost information (Nguyen et al., 2024).
Planning around what the robot cannot sense
Not every uncertainty should be handled reactively. Manipulation planning can instead exploit gravity and environment geometry to funnel a part into a known pose, reducing the precision the controller has to deliver (von Drigalski et al., 2022). Earlier work in the same spirit learned grasping policies over primitive shape abstractions rather than exact object models (Beltran-Hernandez et al., 2019).
@inproceedings{hai2023symmetry,author={Nguyen, Hai and Kozuno, Tadashi and Beltran-Hernandez, Cristian C. and Hamaya, Masashi},title={Symmetry-aware Reinforcement Learning for Robotic Assembly under Partial Observability with a Soft Wrist},year={2024},booktitle={IEEE International Conference on Robotics and Automation (ICRA)},}
@unpublished{beltran2022accelerating,title={Accelerating Robot Learning of Contact-Rich Manipulations: A Curriculum Learning Study},author={Beltran-Hernandez, Cristian C and Petit, Damien and Ramirez-Alpizar, Ixchel G and Harada, Kensuke},journal={arXiv preprint arXiv:2204.12844},year={2022},doi={10.48550/arXiv.2204.12844},}
@inproceedings{beltran2022rsj,title={Curriculum Reinforcement Learning for Industrial Insertion Tasks},author={Beltran, Cristian and Petit, Damien and Ramirez-Alpizar, Ixchel and Harada, Kensuke},booktitle={第40回日本ロボット学会学術講演会 RSJ2022},year={2022},organization={RSJ}}
@article{von2022uncertainty,author={von Drigalski, Felix and Kasaura, Kazumi and Beltran-Hernandez, Cristian C. and Hamaya, Masashi and Tanaka, Kazutoshi and Matsubara, Takamitsu},journal={IEEE Robotics and Automation Letters},title={Uncertainty-Aware Manipulation Planning Using Gravity and Environment Geometry},year={2022},volume={7},number={4},pages={11942-11949},doi={10.1109/LRA.2022.3207565},}
@article{von2022o2ac,author={von Drigalski, Felix and Beltran-Hernandez, Cristian C. and Nakashima, Chisato and Hu, Zhengtao and Akizuki, Shuichi and Ueshiba, Toshio and Hashimoto, Manabu and Kasaura, Kazumi and Domae, Yukiyasu and Wan, Weiwei and Harada, Kensuke},journal={Advanced Robotics},title={Team O2AC at the World Robot Summit 2020: Towards Jigless, High-Precision Assembly},volume={36},number={22},pages={1213--1227},year={2022},publisher={Taylor \& Francis},doi={10.1080/01691864.2022.2138541},}
@article{beltran2020learning,title={Learning Force Control for Contact-rich Manipulation Tasks with Rigid Position-controlled Robots},author={Beltran-Hernandez, Cristian Camilo and Petit, Damien and Ramirez-Alpizar, Ixchel Georgina and Nishi, Takayuki and Kikuchi, Shinichi and Matsubara, Takamitsu and Harada, Kensuke},journal={IEEE Robotics and Automation Letters},volume={5},number={4},pages={5709--5716},year={2020},publisher={IEEE},doi={10.1109/LRA.2020.3010739},}
@article{beltran2020variable,title={Variable compliance control for robotic peg-in-hole assembly: A deep-reinforcement-learning approach},author={Beltran-Hernandez, Cristian C and Petit, Damien and Ramirez-Alpizar, Ixchel G and Harada, Kensuke},journal={Applied Sciences},volume={10},number={19},pages={6923},year={2020},publisher={MDPI},doi={10.3390/app10196923},}
@inproceedings{beltran2019learning,title={Learning to Grasp with Primitive Shaped Object Policies},author={Beltran-Hernandez, Cristian C and Petit, Damien and Ramirez-Alpizar, Ixchel G and Harada, Kensuke},booktitle={IEEE/SICE International Symposium on System Integration (SII)},pages={468--473},year={2019},organization={IEEE},doi={10.1109/SII.2019.8700399},}
