Human-robot interaction with the Pepper robot

The humanoid social robot Pepper was developed by Aldebaran and first released in 2015. The robot is 120 centimeters tall and optimized for human interaction. It is able to engage with people through conversation, gestures and its touch screen. Pepper is equipped with internal sensors, four directional microphones in his head and speakers for voice output. Speech recognition and dialog is available in 15 languages. The robot features an open and fully programmable platform so that developers can program their own applications to run on Pepper.

In our research group, various applications have been developed which, for example, integrate various external online services such as ChatGPT, DeepL or SUMM AI via different APIs in order to give the robot appropriate communication capabilities. Various studies investigate the interaction between humans and robots with regard to the perception and effect of the robot on the part of the human as well as the modeling of a cognitive model that the robot gains from the human.

Head

Current Members

Projects

Completed Projects

Publications

2025

Thomas Sievers,
A Practical Approach to Child-Robot Interaction in the Classroom, in Child-AI Interaction in the Era of Foundation Models , Burlingame, CA, USA, March 31 - April 2, 2025 AAAI Spring Symposium Series, 2025.
Weblink: https://aaai.org/conference/spring-symposia/sss25/
Thomas Sievers,
Retrieving Memories from a Cognitive Architecture Using Language Models for Social Robot Applications, 1st German Robotics Conference, Nuremberg, March 13-15, Robotics Institute Germany, 2025.
File: Dateilink
Thomas Sievers, and Nele Russwinkel,
Using Memory Contents of a Cognitive Model for Prompt Augmentation of a Large Language Model, in IEEE CogSIMA 2025 Conference Proceedings , 2025 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), 2025.
Thomas Sievers, Nele Russwinkel, and Ralf Möller,
Grounding a social robot's understanding of words with associations in a cognitive architecture, International Conference on Agents and Artificial Intelligence (ICAART), Porto, February 23-25, 2025.
DOI:10.5220/0013144200003890
Bibtex: BibTeX
@conference{icaart25,
author={Thomas Sievers and Nele Russwinkel and Ralf Möller},
title={Grounding a Social Robot’s Understanding of Words with Associations in a Cognitive Architecture},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2025},
pages={406-410},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013144200003890},
isbn={978-989-758-737-5},
issn={2184-433X},
}
Thomas Sievers,
Silence is Golden - Making Pauses in Human Utterances Comprehensible for Social Robots in Human-Robot Interaction, in 2025 IEEE/SICE International Symposium on System Integration (SII) , 2025. pp. 526-531.
DOI:10.1109/SII59315.2025.10871032
Bibtex: BibTeX
@INPROCEEDINGS{10871032,
  author={Sievers, Thomas},
  booktitle={2025 IEEE/SICE International Symposium on System Integration (SII)}, 
  title={Silence is Golden - Making Pauses in Human Utterances Comprehensible for Social Robots in Human-Robot Interaction *}, 
  year={2025},
  volume={},
  number={},
  pages={526-531},
  keywords={Sentiment analysis;Large language models;Social robots;System integration;Speech recognition;Chatbots;Text to speech},
  doi={10.1109/SII59315.2025.10871032}}

2024

Thomas Sievers, and Nele Russwinkel,
Dialect, sentiment analysis, tonality, cognitive modeling – using LLMs as a toolbox for HRI, in LLMs in the RoMan Age , Workshop at the 33nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2024), 2024.
Weblink: https://sites.google.com/view/llmra/start
File: Dateilink
Thomas Sievers, and Ralf Möller,
Get it right: Improving Comprehensibility with adaptable Speech Expression of a Humanoid Service Robot, in AMLDA 2023: Applied Machine Learning and Data Analytics , 2024. pp. 1-14.
DOI:10.1007/978-3-031-55486-5_1
Bibtex: BibTeX
@INPROCEEDINGS{SiMo,
author ={Thomas Sievers and Ralf M\"oller},
title ={Get it right: Improving Comprehensibility with adaptable Speech Expression of a Humanoid Service Robot},
booktitle ={6th International Conference on Applied Machine Learning and Data Analytics (AMLDA)},
year ={2023},
pages = {},
publisher = {},
url ={},
keywords ={Social Robot; Human-Robot Interaction; Easy Language; Comprehensibility; Translation},
}
Alexander Werk, Sina Scholz, Thomas Sievers, and Nele Russwinkel,
How to Provide a Dynamic Cognitive Person Model of a Human Collaboration Partner to a Pepper Robot, in ICCM , Society for Mathematical Psychology, 2024.
Weblink: https://mathpsych.org/presentation/1452
Bibtex: BibTeX
@inproceedings{WeSchoSiRuICCM24,
  author    = {Alexander Werk and Sina Scholz and Thomas Sievers and Nele Russwinkel},
  title     = {{How to Provide a Dynamic Cognitive Person Model of a Human Collaboration Partner to a Pepper Robot}},
  booktitle = {ICCM},
  year      = {2024},
month = {7},
  volume    = {},
  pages     = {},
  publisher = {Society for Mathematical Psychology}
}
Thomas Sievers, and Nele Russwinkel,
How to use a cognitive architecture for a dynamic person model with a social robot in human collaboration, in Workshop "Robots for Humans" , CEUR Workshop Proceedings, 2024.
Weblink: https://ceur-ws.org/Vol-3794/
File: paper2.pdf
Thomas Sievers, and Nele Russwinkel,
Interacting with a Sentimental Robot - Making Human Emotions tangible for a Social Robot via ChatGPT, in 2024 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO) , 2024. pp. 182-187.
DOI:10.1109/ARSO60199.2024.10557749
Bibtex: BibTeX
@INPROCEEDINGS{10557749,
  author={Sievers, Thomas and Russwinkel, Nele},
  title={{Interacting with a Sentimental Robot – Making Human Emotions tangible for a Social Robot via ChatGPT*}}, 
 booktitle={IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO 2024)}, 
  year={2024},
  volume={},
  number={},
  pages={182--187},
  keywords={Emotion recognition;Sentiment analysis;Analytical models;Mood;Social robots;Oral communication;Chatbots},
  doi={10.1109/ARSO60199.2024.10557749}
}
Thomas Sievers, and Nele Russwinkel,
Introducing a note of levity to human-robot interaction with dialogs containing irony, sarcasm and jocularity, in IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) , 2024.
DOI:10.1109/RO-MAN60168.2024.10731234
Bibtex: BibTeX
@INPROCEEDINGS{10731234,
  author={Sievers, Thomas and Russwinkel, Nele},
  booktitle={2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)}, 
  title={Introducing a note of levity to human-robot interaction with dialogs containing irony, sarcasm and jocularity *}, 
  year={2024},
  volume={},
  number={},
  pages={763-768},
  keywords={Atmospheric measurements;Social robots;Oral communication;Particle measurements;Chatbots;Distance measurement;Robots},
  doi={10.1109/RO-MAN60168.2024.10731234}}
Thomas Sievers, and Nele Russwinkel,
Project Report: Requirements for a Social Robot as an Information Provider in the Public Sector, KI - Künstliche Intelligenz , vol. 37, no. 2-4, 2024.
DOI:10.1007/s13218-024-00840-1
Bibtex: BibTeX
@JOUR{10557749,
  author={Sievers, Thomas and Russwinkel, Nele},
  booktitle={KI - Künstliche Intelligenz}, 
  title={Project Report: Requirements for a Social Robot as an Information Provider in the Public Sector}, 
  year={2024},
  volume={37},
  number={2-4},
  pages={},
  keywords={Human-robot interaction;Social robotics;Public sector;ACT-R},
  doi={10.1007/s13218-024-00840-1}}
Edgar Baake, Florian König, Claudius Pott, Thomas Sievers, and Moreen Heine,
User Experience Comparison of a Social Robot vs. Info Terminal in Citizen Services, 16th International Conference on Social Robotics, Odense, 2024.
Thomas Sievers, Nele Russwinkel, and Ralf Möller,
What am I? – Complementing a robot’s task-solving capabilities with a mental model using a cognitive architecture, in Behavior Adaptation and Learning for Assistive Robotics , CEUR Workshop Proceedings, 2024.
Weblink: https://ceur-ws.org/Vol-3906/
File: paper1_BAILAR.pdf

2023

Thomas Sievers, Magnus Bender, and Ralf Möller,
Connecting AI Technologies as Online Services to a Humanoid Service Robot, in 15th International Conference on Computer and Automation Engineering (ICCAE) , IEEE, 2023. pp. 431-435.
DOI:10.1109/ICCAE56788.2023.10111181
Bibtex: BibTeX
@INPROCEEDINGS{SiBeMo,
author ={Thomas Sievers and Magnus Bender and Ralf M\"oller},
title ={Connecting AI Technologies as Online Services to a Humanoid Service Robot},
booktitle ={15th International Conference on Computer and Automation Engineering (ICCAE)},
year ={2023},
pages = {431-435},
publisher = {{IEEE}},
url ={http://dx.doi.org/10.1109/ICCAE56788.2023.10111181},
keywords ={Humanoid Robot; Human-Robot Interaction; Application Programming Interface; AI Democratization},
}
Thomas Sievers, and Nele Russwinkel,
Talking like one of us: Effects of using regional language in a Humanoid Social Robot, in Social Robotics , Springer, Singapore, 2023. pp. 71-80.
DOI:10.1007/978-981-99-8718-4_7
Bibtex: BibTeX
@INPROCEEDINGS{SiRu,
author ={Thomas Sievers and Nele Russwinkel},
title ={Talking like one of us: Effects of using regional language in a Humanoid Social Robot},
booktitle ={Social Robotics},
year ={2023},
pages = {71-80},
publisher = {Springer, Singapore},
url ={http://dx.doi.org/10.1007/978-981-99-8718-4_7},
keywords ={Social Robots; Human-Robot Interaction; Regional Language; Communication; Cultural Identity},
}