Human-robot interaction with the Pepper robot
The humanoid social robot Pepper was developed by Aldebaran and first released in 2015. The robot is 120 centimeters tall and optimized for human interaction. It is able to engage with people through conversation, gestures and its touch screen. Pepper is equipped with internal sensors, four directional microphones in his head and speakers for voice output. Speech recognition and dialog is available in 15 languages. The robot features an open and fully programmable platform so that developers can program their own applications to run on Pepper.
In our research group, various applications have been developed which, for example, integrate various external online services such as ChatGPT, DeepL or SUMM AI via different APIs in order to give the robot appropriate communication capabilities. Various studies investigate the interaction between humans and robots with regard to the perception and effect of the robot on the part of the human as well as the modeling of a cognitive model that the robot gains from the human.
Current Members
- Sina Scholz
- Thomas Sievers
- Alexander Werk
Projects
Completed Projects
- DKDE-eGov (WTSH / State of SH)
Publications
2025
A Practical Approach to Child-Robot Interaction in the Classroom, in Child-AI Interaction in the Era of Foundation Models , Burlingame, CA, USA, March 31 - April 2, 2025 AAAI Spring Symposium Series, 2025.
Weblink: | https://aaai.org/conference/spring-symposia/sss25/ |
Retrieving Memories from a Cognitive Architecture Using Language Models for Social Robot Applications, 1st German Robotics Conference, Nuremberg, March 13-15, Robotics Institute Germany, 2025.
Datei: | Dateilink |
Using Memory Contents of a Cognitive Model for Prompt Augmentation of a Large Language Model, in IEEE CogSIMA 2025 Conference Proceedings , 2025 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), 2025.
Retrieving Memory Content from a Cognitive Architecture by Impressions from Language Models for Use in a Social Robot, Applied Sciences , 2025.
DOI: | 10.20944/preprints202504.1792.v1 |
Silence is Golden - Making Pauses in Human Utterances Comprehensible for Social Robots in Human-Robot Interaction, in 2025 IEEE/SICE International Symposium on System Integration (SII) , 2025. pp. 526-531.
DOI: | 10.1109/SII59315.2025.10871032 |
Bibtex: | ![]() @INPROCEEDINGS{10871032, author={Sievers, Thomas}, booktitle={2025 IEEE/SICE International Symposium on System Integration (SII)}, title={Silence is Golden - Making Pauses in Human Utterances Comprehensible for Social Robots in Human-Robot Interaction *}, year={2025}, volume={}, number={}, pages={526-531}, keywords={Sentiment analysis;Large language models;Social robots;System integration;Speech recognition;Chatbots;Text to speech}, doi={10.1109/SII59315.2025.10871032}} |
Visions remembered - Using a Vision-Language Model to set and recall Image Impressions in the Memory of a Cognitive Model, in ICCM , Society for Mathematical Psychology, 2025.
Grounding a social robot's understanding of words with associations in a cognitive architecture, International Conference on Agents and Artificial Intelligence (ICAART), Porto, February 23-25, 2025.
DOI: | 10.5220/0013144200003890 |
Bibtex: | ![]() @conference{icaart25, author={Thomas Sievers and Nele Russwinkel and Ralf Möller}, title={Grounding a Social Robot’s Understanding of Words with Associations in a Cognitive Architecture}, booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART}, year={2025}, pages={406-410}, publisher={SciTePress}, organization={INSTICC}, doi={10.5220/0013144200003890}, isbn={978-989-758-737-5}, issn={2184-433X}, } |
2024
Introducing a note of levity to human-robot interaction with dialogs containing irony, sarcasm and jocularity, in IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) , 2024.
DOI: | 10.1109/RO-MAN60168.2024.10731234 |
Bibtex: | ![]() @INPROCEEDINGS{10731234, author={Sievers, Thomas and Russwinkel, Nele}, booktitle={2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)}, title={Introducing a note of levity to human-robot interaction with dialogs containing irony, sarcasm and jocularity *}, year={2024}, volume={}, number={}, pages={763-768}, keywords={Atmospheric measurements;Social robots;Oral communication;Particle measurements;Chatbots;Distance measurement;Robots}, doi={10.1109/RO-MAN60168.2024.10731234}} |
What am I? – Complementing a robot’s task-solving capabilities with a mental model using a cognitive architecture, in Behavior Adaptation and Learning for Assistive Robotics , CEUR Workshop Proceedings, 2024.
Weblink: | https://ceur-ws.org/Vol-3906/ |
Datei: |
User Experience Comparison of a Social Robot vs. Info Terminal in Citizen Services, in Social Robotics , 16th International Conference on Social Robotics, Odense, 2024.
DOI: | 10.1007/978-981-96-3525-2_2 |
Bibtex: | ![]() @InProceedings{10.1007/978-981-96-3525-2_2, author="Baake, Edgar and K{\"o}nig, Florian and Pott, Claudius and Sievers, Thomas and Heine, Moreen", editor="Palinko, Oskar and Bodenhagen, Leon and Cabibihan, John-John and Fischer, Kerstin and {\v{S}}abanovi{\'{c}}, Selma and Winkle, Katie and Behera, Laxmidhar and Ge, Shuzhi Sam and Chrysostomou, Dimitrios and Jiang, Wanyue and He, Hongsheng", title="User Experience Comparison of a Social Robot vs. Info Terminal in Citizen Services", booktitle="Social Robotics", year="2025", publisher="Springer Nature Singapore", address="Singapore", pages="12--23", abstract="This study evaluates user experiences with the social robot Pepper, compared to a traditional info terminal in providing citizen services. The study, carried out with 37 participants at various institutions in L{\"u}beck, Germany, aims to assess the effectiveness, efficiency, and satisfaction of these two methods in providing information and assistance within a public service setting. The study controls for age, gender, and affinity for technology while measuring customer experience using the Short Version of the User Experience Questionnaire (UEQ-S). The results show that Pepper is attributed higher hedonistic qualities, whereas the terminal is highlighted as pragmatic and is overall preferred in the evaluated context. Further findings show that, while age and gender do have at least some explanatory power in explaining divergent user experiences, the affinity for technology interaction surprisingly did not seem to have an impact.", isbn="978-981-96-3525-2" } |
Project Report: Requirements for a Social Robot as an Information Provider in the Public Sector, KI - Künstliche Intelligenz , vol. 37, no. 2-4, 2024.
DOI: | 10.1007/s13218-024-00840-1 |
Bibtex: | ![]() @JOUR{10557749, author={Sievers, Thomas and Russwinkel, Nele}, booktitle={KI - Künstliche Intelligenz}, title={Project Report: Requirements for a Social Robot as an Information Provider in the Public Sector}, year={2024}, volume={37}, number={2-4}, pages={}, keywords={Human-robot interaction;Social robotics;Public sector;ACT-R}, doi={10.1007/s13218-024-00840-1}} |
How to Provide a Dynamic Cognitive Person Model of a Human Collaboration Partner to a Pepper Robot, in ICCM , Society for Mathematical Psychology, 2024.
Weblink: | https://mathpsych.org/presentation/1452 |
Bibtex: | ![]() @inproceedings{WeSchoSiRuICCM24, author = {Alexander Werk and Sina Scholz and Thomas Sievers and Nele Russwinkel}, title = {{How to Provide a Dynamic Cognitive Person Model of a Human Collaboration Partner to a Pepper Robot}}, booktitle = {ICCM}, year = {2024}, month = {7}, volume = {}, pages = {}, publisher = {Society for Mathematical Psychology} } |
Interacting with a Sentimental Robot - Making Human Emotions tangible for a Social Robot via ChatGPT, in 2024 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO) , 2024. pp. 182-187.
DOI: | 10.1109/ARSO60199.2024.10557749 |
Bibtex: | ![]() @INPROCEEDINGS{10557749, author={Sievers, Thomas and Russwinkel, Nele}, title={{Interacting with a Sentimental Robot – Making Human Emotions tangible for a Social Robot via ChatGPT*}}, booktitle={IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO 2024)}, year={2024}, volume={}, number={}, pages={182--187}, keywords={Emotion recognition;Sentiment analysis;Analytical models;Mood;Social robots;Oral communication;Chatbots}, doi={10.1109/ARSO60199.2024.10557749} } |
How to use a cognitive architecture for a dynamic person model with a social robot in human collaboration, in Workshop "Robots for Humans" , CEUR Workshop Proceedings, 2024.
Weblink: | https://ceur-ws.org/Vol-3794/ |
Datei: |
Get it right: Improving Comprehensibility with adaptable Speech Expression of a Humanoid Service Robot, in AMLDA 2023: Applied Machine Learning and Data Analytics , 2024. pp. 1-14.
DOI: | 10.1007/978-3-031-55486-5_1 |
Bibtex: | ![]() @INPROCEEDINGS{SiMo, author ={Thomas Sievers and Ralf M\"oller}, title ={Get it right: Improving Comprehensibility with adaptable Speech Expression of a Humanoid Service Robot}, booktitle ={6th International Conference on Applied Machine Learning and Data Analytics (AMLDA)}, year ={2023}, pages = {}, publisher = {}, url ={}, keywords ={Social Robot; Human-Robot Interaction; Easy Language; Comprehensibility; Translation}, } |
Dialect, sentiment analysis, tonality, cognitive modeling – using LLMs as a toolbox for HRI, in LLMs in the RoMan Age , Workshop at the 33nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2024), 2024.
Weblink: | https://sites.google.com/view/llmra/start |
Datei: | Dateilink |
2023
Connecting AI Technologies as Online Services to a Humanoid Service Robot, in 15th International Conference on Computer and Automation Engineering (ICCAE) , IEEE, 2023. pp. 431-435.
DOI: | 10.1109/ICCAE56788.2023.10111181 |
Bibtex: | ![]() @INPROCEEDINGS{SiBeMo, author ={Thomas Sievers and Magnus Bender and Ralf M\"oller}, title ={Connecting AI Technologies as Online Services to a Humanoid Service Robot}, booktitle ={15th International Conference on Computer and Automation Engineering (ICCAE)}, year ={2023}, pages = {431-435}, publisher = {{IEEE}}, url ={http://dx.doi.org/10.1109/ICCAE56788.2023.10111181}, keywords ={Humanoid Robot; Human-Robot Interaction; Application Programming Interface; AI Democratization}, } |
Talking like one of us: Effects of using regional language in a Humanoid Social Robot, in Social Robotics , Springer, Singapore, 2023. pp. 71-80.
DOI: | 10.1007/978-981-99-8718-4_7 |
Bibtex: | ![]() @INPROCEEDINGS{SiRu, author ={Thomas Sievers and Nele Russwinkel}, title ={Talking like one of us: Effects of using regional language in a Humanoid Social Robot}, booktitle ={Social Robotics}, year ={2023}, pages = {71-80}, publisher = {Springer, Singapore}, url ={http://dx.doi.org/10.1007/978-981-99-8718-4_7}, keywords ={Social Robots; Human-Robot Interaction; Regional Language; Communication; Cultural Identity}, } |

- Forschung
- Groups
- Human-Aware AI
- Human-Robot Interaction with the Pepper Robot
- Quantum and Intelligent Computing
- Former Research Groups
- Projects
- Completed Projects
- Groups