No Access

Incremental Learning of an Open-Ended Collaborative Skill Library

Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany

E-mail Address: [email protected]

Corresponding author.

Search for more papers by this author

Susanne Trick

Psychology of Information Processing Group, Department of Psychology, Technische Universität Darmstadt, Darmstadt, Germany

Search for more papers by this author

Marco Ewerton

Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany

Search for more papers by this author

Michael Lutter

Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany

Search for more papers by this author

, and

Jan Peters

Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany

Max Planck Institute for Intelligent Systems, Tübingen, Germany

Search for more papers by this author

https://doi.org/10.1142/S0219843620500012Cited by:4 (Source: Crossref)

This article is part of the issue:

Special Issue on Part 1 — New Advances of Humanoid Robotics (Humanoids 2018)
Guest Editors: Qiang Huang (Beijing Institute of Technology, China), Fumio Kanehiro (AIST, Japan), Tamim Asfour (Karlsruhe Institute of Technology, Germany), Abderrahmane Kheddar (CNRS-UM LIRMM, France) and Zhangguo Yu (Beijing Institute of Technology, China)

Abstract

Intelligent assistive robots can potentially contribute to maintaining an elderly person’s independence by supporting everyday life activities. However, the number of different and personalized activities to be supported renders pre-programming of all respective robot behaviors prohibitively difficult. Instead, to cope with a continuous and potentially open-ended stream of cooperative tasks, new collaborative robot behaviors need to be continuously learned and updated from demonstrations. To this end, we introduce an online learning method to incrementally build a cooperative skill library of probabilistic interaction primitives. The resulting model chooses a corresponding robot response to a human movement where the human intention is extracted from previously demonstrated movements. While existing batch learning methods for movement primitives usually learn such skill libraries only once for a pre-defined number of different skills, our approach enables extending the skill library in an open-ended and online fashion from new incoming demonstrations. The proposed approach is evaluated on a low-dimensional benchmark task and in a collaborative scenario with a 7DoF robot, where we also investigate the generalization of learned skills between different subjects.

Keywords:

References

1. K. Linz and S. Stula, Demographic change in Europe-an overview, Observatory for Sociopolitical Developements in Europe 4(1) (2010) 2–10. Google Scholar
2. S. Schaal, Is imitation learning the route to humanoid robots? Trends Cogn. Sci. 3(6) (1999) 233–242. Crossref, Web of Science, Google Scholar
3. D. A. Rosenbaum, Human Motor Control (Academic press, 2009). Google Scholar
4. G. Maeda, M. Ewerton, R. Lioutikov, H. B. Amor, J. Peters and G. Neumann, Learning interaction for collaborative tasks with probabilistic movement primitives, in 2014 14th IEEE-RAS Int. Conf. Humanoid Robots (Humanoids) (IEEE, 2014), pp. 527–534. Crossref, Google Scholar
5. M. Ewerton, G. Neumann, R. Lioutikov, H. B. Amor, J. Peters and G. Maeda, Learning multiple collaborative tasks with a mixture of interaction primitives, in 2015 IEEE Int. Conf. Robotics and Automation (ICRA) (IEEE, 2015), pp. 1535–1542. Crossref, Google Scholar
6. D. Koert, S. Trick, M. Ewerton, M. Lutter and J. Peters, Online learning of an open-ended skill library for collaborative tasks, in 2018 IEEE-RAS 18th Int. Conf. Humanoid Robots (Humanoids) (IEEE, 2018), pp. 1–9. Crossref, Google Scholar
7. B. D. Argall, S. Chernova, M. Veloso and B. Browning, A survey of robot learning from demonstration, Robot. Auton. Syst. 57(5) (2009) 469–483. Crossref, Web of Science, Google Scholar
8. A. Billard, S. Calinon, R. Dillmann and S. Schaal, Robot programming by demonstration, in Springer Handbook of Robotics (Springer, 2008), pp. 1371–1394. Crossref, Google Scholar
9. P. Pastor, H. Hoffmann, T. Asfour and S. Schaal, Learning and generalization of motor skills by learning from demonstration, in 2009 IEEE Int. Conf. on Robotics and Automation (IEEE, 2009), pp. 763–768. Crossref, Google Scholar
10. D. Vogt, S. Stepputtis, S. Grehl, B. Jung and H. B. Amor, A system for learning continuous human-robot interactions from human-human demonstrations, in 2017 IEEE Int. Conf. Robotics and Automation (ICRA) (IEEE, 2017), pp. 2882–2889. Crossref, Google Scholar
11. G. J. Maeda, G. Neumann, M. Ewerton, R. Lioutikov, O. Kroemer and J. Peters, Probabilistic movement primitives for coordination of multiple human–robot collaborative tasks, Auton. Robots 41(3) (2017) 593–612. Crossref, Web of Science, Google Scholar
12. H. B. Amor, D. Vogt, M. Ewerton, E. Berger, B. Jung and J. Peters, Learning responsive robot behavior by imitation, in 2013 IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IEEE, 2013), pp. 3257–3264. Crossref, Google Scholar
13. D. Vogt, B. Lorenz, S. Grehl and B. Jung, Behavior generation for interactive virtual humans using context-dependent interaction meshes and automated constraint extraction, Comput. Animation Virtual Worlds 26(3–4) (2015) 227–235. Crossref, Web of Science, Google Scholar
14. S. Nikolaidis, R. Ramakrishnan, K. Gu and J. Shah, Efficient model learning from joint-action demonstrations for human-robot collaborative tasks, in Proc. 10th Annual ACM/IEEE International Conf. Human-Robot Interaction (ACM, 2015), pp. 189–196. Crossref, Google Scholar
15. A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor and S. Schaal, Dynamical movement primitives: Learning attractor models for motor behaviors, Neural Comput. 25(2) (2013) 328–373. Crossref, Web of Science, Google Scholar
16. S. Calinon, F. Guenter and A. Billard, On learning, representing, and generalizing a task in a humanoid robot, IEEE Trans. Syst. Man, Cybernet. Part B (Cybernetics) 37(2) (2007) 286–298. Crossref, Google Scholar
17. A. Paraschos, C. Daniel, J. Peters and G. Neumann, Using probabilistic movement primitives in robotics, Auton. Robot. 42(3) (2018) 529–551. Crossref, Web of Science, Google Scholar
18. Y. Huang, L. Rozo, J. Silvério and D. G. Caldwell, Kernelized movement primitives, The Int. J. Robot. Res. 38(7) (2019) 833–852. Crossref, Web of Science, Google Scholar
19. H. B. Amor, G. Neumann, S. Kamthe, O. Kroemer and J. Peters, Interaction primitives for human-robot cooperation tasks, in 2014 IEEE Int. Conf. Robotics and Automation (ICRA) (IEEE, 2014). Crossref, Google Scholar
20. C. Pérez-D’Arpino and J. A. Shah, Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification, in 2015 IEEE Int. Conf. Robotics and Automation (ICRA) (IEEE, 2015), pp. 6175–6182. Crossref, Google Scholar
21. D. Lee, C. Ott and Y. Nakamura, Mimetic communication model with compliant physical contact in human-humanoid interaction, The Int. J. Robot. Res. 29(13) (2010) 1684–1704. Crossref, Web of Science, Google Scholar
22. G. Konidaris, S. Kuindersma, R. Grupen and A. Barto, Robot learning from demonstration by constructing skill trees, The Int. J. Robot. Res. 31(3) (2012) 360–375. Crossref, Web of Science, Google Scholar
23. S. Calinon and A. Billard, Incremental learning of gestures by imitation in a humanoid robot, in Proc. ACM/IEEE Int. Conf. Human-Robot Interaction (ACM, 2007), pp. 255–262. Crossref, Google Scholar
24. A. Ahmed and E. Xing, Dynamic non-parametric mixture models and the recurrent chinese restaurant process: With applications to evolutionary clustering, in Proc. 2008 SIAM Int. Conf. Data Mining (SIAM, 2008), pp. 219–230. Crossref, Google Scholar
25. O. Arandjelovic and R. Cipolla, Incremental learning of temporally-coherent gaussian mixture models, Society of Manufacturing Engineers (SME) Technical Papers (British Machine Vision Conference, 2005), pp. 1–1. Google Scholar
26. P. M. Engel and M. R. Heinen, Incremental learning of multivariate gaussian mixture models, in Brazilian Symp. Artificial Intelligence (Springer, 2010), pp. 82–91. Crossref, Google Scholar
27. R. C. Pinto and P. M. Engel, A fast incremental gaussian mixture model, PloS one 10(10) (2015) e0139931. Web of Science, Google Scholar
28. A. Declercq and J. H. Piater, Online learning of gaussian mixture models-a two-level approach. in VISAPP (1), 2008, pp. 605–611. Google Scholar
29. G. Maeda, M. Ewerton, T. Osa, B. Busch and J. Peters, Active incremental learning of robot movement primitives, in Conf. Robot Learning (CORL), 2017. Google Scholar
30. J. Hoyos, F. Prieto, G. Alenya and C. Torras, Incremental learning of skills in a task-parameterized gaussian mixture model, J. Intell. Robot. Syst. 82(1) (2016) 81–99. Crossref, Web of Science, Google Scholar
31. I. Havoutis, A. K. Tanwani and S. Calinon, Online incremental learning of manipulation tasks for semi-autonomous teleoperation, in IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS),Workshop on Closed-loop Grasping and Manipulation (Challenges and Progress, Daejeon, Kora, 2016). Google Scholar
32. D. Kulić, C. Ott, D. Lee, J. Ishikawa and Y. Nakamura, Incremental learning of full body motion primitives and their sequencing through human motion observation, The Int. J. Robot. Res. 31(3) (2012) 330–345. Crossref, Web of Science, Google Scholar
33. A. Lemme, R. F. Reinhart and J. J. Steil, “Self-supervised bootstrapping of a movement primitive library from complex trajectories,” in 2014 14th IEEE-RAS Int. Conf. Humanoid Robots (Humanoids) (IEEE, 2014), pp. 726–732. Crossref, Google Scholar
34. K. Muelling, J. Kober and J. Peters, Learning table tennis with a mixture of motor primitives,” in 2010 10th IEEE-RAS Int. Conf. Humanoid Robots (Humanoids) (IEEE, 2010), pp. 411–416. Crossref, Google Scholar
35. R. A. Jacobs, M. I. Jordan, S. J. Nowlan and G. E. Hinton, “Adaptive mixtures of local experts,” Neural comput. 3(1) (1991) 79–87. Crossref, Web of Science, Google Scholar