Hi! I’m Zi Haur 👋
I am a third-year Ph.D. student in Speech and Audio Processing Lab at Kyoto University, under the supervision of Professor Tatsuya Kawahara. My research lies at the intersection of affective computing, human–AI/robot interaction, and spoken dialogue systems. Specifically, I investigate how multilingual multimodal embodied conversational agents—ranging from virtual avatars to human-like androids—can recognize, validate, and ultimately alleviate human emotions through natural speech. I also explore how an agent’s embodiment and conversational intelligence influence user behavior during interaction.
Feel free to reach out, or learn more from My CV.
🔥 News
- 2026.09: 🎉🎉 Our paper is accepted by IEEE Spoken Language Technology 2026 (SLT 2026).
- 2026.09: 🎉🎉 Our paper is accepted by The 5th Asia-Pacific Chapter of the Association for Computational Linguistics & the 15th International Joint Conference on Natural Language Processing (AACL-IJCNLP 2026).
- 2026.08: 🎉🎉 Our paper is accepted by The 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026).
- 2026.07: 🎉🎉 Our paper is accepted by APSIPA Annual Summit and Conference 2026. See you in Bangkok, Thailand!
- 2026.06: 🎉🎉 Our paper is accepted by INTERSPEECH 2026. See you in Sydney, Australia!
- 2026.06: 🎉🎉 Our paper is accepted by The 27th Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2026). See you in Atlanta, Georgia, USA!
- 2026.06: 🎉🎉 Our non-archival paper is accepted by Learning to Listen: ICML 2026 Workshop on Machine Learning for Audio. See you in Seoul, Korea!
- 2026.04: I have joined Speech Technology and Machine Learning Group as a Research Scholar under the guidance of Dr. Luis Fernando D’Haro.
- 2026.02: I have been selected as a student volunteer for the CHI Conference on Human Factors in Computing Systems 2026 (CHI 2026). See you in Barcelona, Spain!
- 2026.01: 🎉🎉 Our papers are accepted by 2026 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2026). See you in Barcelona, Spain!
- 2025.12: I have joined A*STAR Institute for Infocomm Research (A*STAR I²R) as a Research Intern under the guidance of Dr. Gao Xiaoxue and Dr. Nancy F. Chen.
- 2025.06: I have joined Speech, Language & Interactive Machines (SLIM) Research Group as a Research Intern under the guidance of Dr. Casey Kennington.
- 2025.02: 🎉🎉 Our paper is accepted by CHI Conference on Human Factors in Computing Systems 2025 (CHI 2025). See you in Yokohama, Japan!
- 2024.11: 🎉🎉 Our paper is accepted by The 31st International Conference on Computational Linguistics (COLING 2025). See you in Abu Dhabi, United Arab Emirates!
- 2023.12: 🎉🎉 Our paper is accepted by The 14th International Workshop on Spoken Dialogue Systems Technology (IWSDS 2024). See you in Sapporo, Japan!
📖 Educations
- 2024.04 - Present: Doctorate of Informatics in Intelligent Science and Technology, Kyoto University.
- 2022.04 - 2024.03: Master of Informatics in Intelligent Science and Technology, Kyoto University.
- 2017.01 - 2020.12: Bachelor of Science (Hons) in Actuarial Science, UCSI University.
💻 Experience
- 2026.04 - Present: Research Scholar, Speech Technology and Machine Learning Group, Technical University of Madrid (Universidad Politécnica de Madrid, UPM).
- 2025.12 - Present: Research Intern, Institute for Infocomm Research (I²R), Agency for Science, Technology, and Research (A*STAR).
- 2025.06 - 2025.07: Research Intern, Speech, Language & Interactive Machines (SLIM) Research Group, Boise State University.
- 2024.04 - 2024.08: Teaching Assistant, Graduate School of Informatics, Kyoto University.
- 2020.09 - 2022.03: Data Scientist, AirAsia.Com Travel Sdn. Bhd..
- 2020.05 - 2020.12: Research Assistant, Institute of Actuarial Science and Data Analytics, UCSI University.
- 2019.10 - 2019.12: Data Analyst Intern, SIC Co. Ltd.
📝 Publications
- Sarthak Giri, Zi Haur Pang, Tatsuya Kawahara. Exploiting Speech LLM Representations for Multilingual and Cross-Lingual Parkinson’s Disease Detection, SLT, 2026.
- Jiawen Wang, Xiaoxue Gao, Zi Haur Pang, Nancy F. Chen. EXAM2: Extending Audio Understanding in Multilingual and Multimodal Analysis, AACL-IJCNLP, 2026.
- Bryan Chen Zhengyu Tan, Weihua Zheng, Thong T Doan, Bich Ngoc Doan, Jia Wang Peh, Xiaoyuan Yi, Jing Yao, Xing Xie, Nancy F. Chen, Zhengyuan Liu, JinYeong Bak, Wafi Shamdi, Soo Kai Chie, Liew Yu Siong, Aina Azyyati Binti Mohamad Rezal, Lew Yan Yan Vanessa, Huadan Wu, Dylan Raharja, Nadya Yuki Wangsajaya, Akane Fukushige, Kazushi Kato, Koji Inoue, Tatsuya Kawahara, Jaehyung Seo, Dongjun Kim, Seungyoon Lee, Zi Haur Pang, Rui Yang Tan, Charibeth Ko Cheng, Maria Regina Justina Estuar, Jann Railey Montalan, Pham Minh Duc, Roy Ka-Wei Lee. CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia, EMNLP, 2026.
- Zi Haur Pang, Casey Kennington, Tatsuya Kawahara. Closing the Affective Loop: Multimodal Speaker–Listener Emotion-Dynamics-Aware Empathetic Social Robots, APSIPA ASC, 2026. [Demo]
- Zi Haur Pang, Xiaoxue Gao, Tatsuya Kawahara, Nancy F. Chen. ERM-MinMaxGAP: Benchmarking and Mitigating Gender Bias in Multilingual Multimodal Speech-LLM Emotion Recognition, INTERSPEECH, 2026. [Non-archival@ICML Workshop]
- Zi Haur Pang, Yahui Fu, Koji Inoue, Tatsuya Kawahara. I Understand How You Feel: Enhancing Deeper Emotional Support Through Multilingual Emotional Validation in Dialogue System, SIGDIAL, 2026.
- Koji Inoue, Mikey Elmers, Yahui Fu, Zi Haur Pang, Taiga Mori, Divesh Lala, Keiko Ochi, Tatsuya Kawahara. Multilingual and Continuous Backchannel Prediction: A Cross-lingual Study, IWSDS, 2026.
- Zi Haur Pang, Yahui Fu, Yuan Gao, Tatsuya Kawahara. Paralinguistic Emotion-Aware Validation Timing Detection in Japanese Empathetic Spoken Dialogue, ICASSP, 2026.
- Muyun Wu, Zi Haur Pang, Koji Inoue, Tatsuya Kawahara. Still Thinking or Stopped Talking? Dialogue Silence Intention Classification Using Multimodal Large Language Model, ICASSP, 2026.
- Yahui Fu, Zi Haur Pang, Tatsuya Kawahara. Minority-Aware Satisfaction Estimation in Dialogue Systems via Preference-Adaptive Reinforcement Learning, IJCNLP-AACL, 2025.
- Koji Inoue, Mikey Elmers, Yahui Fu, Zi Haur Pang, Divesh Lala, Keiko Ochi, Tatsuya Kawahara. Prompt-Guided Turn-Taking Prediction, SIGDIAL, 2025.
- Divesh Lala, Mikey Elmers, Koji Inoue, Zi Haur Pang, Keiko Ochi, Tatsuya Kawahara. ScriptBoard: Designing Modern Spoken Dialogue Systems Through Visual Programming, IWSDS, 2025.
- Zi Haur Pang, Yahui Fu, Divesh Lala, Mikey Elmers, Koji Inoue, Tatsuya Kawahara. Does the Appearance of Autonomous Conversational Robots Affect User Spoken Behaviors in Real-World Conference Interactions?, CHI EA, 2025. [Presentation Video]
- Zi Haur Pang, Yahui Fu, Divesh Lala, Mikey Elmers, Koji Inoue, Tatsuya Kawahara. Human-Like Embodied AI Interviewer: Employing Android ERICA in Real International Conference, COLING, 2025. [Demo]
- Zi Haur Pang. Toward More Human-like SDSs: Advancing Emotional and Social Engagement in Embodied Conversational Agents, YRRSDS, 2024.
- Divesh Lala, Koji Inoue, Haruki Kawai, Zi Haur Pang, Mikey Elmers, Tatsuya Kawahara. Development and Evaluation of A Semi-autonomous Parallel Attentive Listening System, APSIPA ASC, 2024.
- Zi Haur Pang, Yahui Fu, Divesh Lala, Keiko Ochi, Koji Inoue, Tatsuya Kawahara. Acknowledgment of Emotional States: Generating Validating Responses for Empathetic Dialogue, IWSDS, 2024. [BEST PAPER NOMINEE🏆]
🥇 Honors and Awards
- 2025.02: Outstanding Research Award, awarded by Kyoto University ICT Collaboration Promotion Network.
- 2022.04 - 2027.03: MEXT Scholarship, awarded by Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan.
- 2017.01 - 2020.12: UCSI University Trust Scholarship (100% Tuition Waiver), awarded by UCSI University.
🗨️ Invited Talk
- 2026.05: Can AI Truly Understand Our Emotions? Designing Affective Embodied Conversational Agents for Human–Computer Interaction. Design Anything Lab, China Academy of Art.
- 2026.04: From Emotion Recognition to Emotional Validation: Toward Fair, Empathetic, Embodied Conversational Agents. Universidad Politécnica de Madrid (UPM).
📸 Media Article
- 2024.11: La Presse au Japon Les robots au chevet des aînés - Erica, une humanoïde pour créer des liens.
🔎 Reviewer Experiences
- 2025 - 2026: The ACM CHI conference on Human Factors in Computing Systems (CHI) Late Breaking Result track
- 2026: The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW)
- 2025: Empirical Methods in Natural Language Processing (EMNLP)
- 2025: International Conference on Multimodal Interaction (ICMI) Late Breaking Result track
- 2024 - 2026: PeerJ Computer Science
🔬 Research Projects
- JST NEXUS: Expressive and Empathetic Human-AI Interaction by Enhancing Multilingual, Multimodal Large Language Models
- JST MOONSHOT: Realization of A Society in Which Human Beings Can Be Free From Limitations of Body, Brain, Space, and Time By 2050
- KAKENHI: Intelligent Conversational System for Dialogue Engagement and Rapport with Humans
🤝 Community Involvement
- 2026.07: Student Volunteer, Forty-Third International Conference on Machine Learning (ICML 2026)
- 2026.04: Student Volunteer, CHI Conference on Human Factors in Computing Systems 2026 (CHI 2026)
- 2026.01: Student Volunteer, Audio-Centric AI: Towards Real-World Multimodal Reasoning and Application Use Cases (Audio-AAAI)
- 2024.09: Student Volunteer, SIGDIAL 2024
- 2024.09: Student Volunteer, 20th Workshop on Spoken Dialogue Systems for PhDs, PostDocs & New Researchers (YRRSDS 2024)
- 2022.06: Student Volunteer, The 36th Annual Conference of the Japanese Society for Artificial Intelligence, 2022