Title: Trustworthy and Continually Adaptable Multimodal Agents

Speaker: Prof. Jaehong Yoon @ Nanyang Technological University

Time : 16:30 - 17:30, Oct 1st, 2025

Location: Online

https://hli.skku.edu/InvitedTalk251001  

Language: English speech & English slides

Abstract: 

As our world advances, AI systems must evolve beyond static models to become more adaptive and continuously improving. These systems should seamlessly integrate new knowledge and capabilities throughout their operational lifespan while ensuring safe and robust interactions in dynamic, multimodal environments. Achieving this vision requires addressing key challenges across multiple fields, with a particular focus on memory, integration, adaptation, and reliability in multimodal AI systems. In this talk, I will introduce these challenges and present approaches that enable AI to continuously evolve while maintaining robustness, reliability, and safety in response to the growing complexity of the real world and user demands. First, I will explore multiple essential aspects to develop multimodal models that continuously accumulate and integrate knowledge from new tasks, datasets, or modalities. I will then introduce approaches that promote long-term model evolution and generalization while effectively mitigating catastrophic forgetting. Second, I will present methods to enhance a model’s ability to acquire new capabilities and further refine its existing skills through the generation of targeted and controllable synthetic data. These techniques empower models to expand their knowledge beyond initial training, enabling more precise and efficient adaptation to complex and long-horizon challenges. Finally, I will highlight the importance of ensuring model reliability and safety by enhancing reasoning, mitigating unintended biases, and preventing harmful outputs. These advancements are essential for developing trustworthiness AI systems that can evolve without compromising their integrity and effectiveness. I will conclude by outlining future directions for developing a scalable and trustworthy embodied continual learning agent and its applications in high-stakes real-world domains.

Bio:

Jaehong Yoon is an Assistant Professor in the College of Computing and Data Science (CCDS) at Nanyang Technological University (NTU), Singapore. Prior to joining NTU, he was a postdoctoral research associate at the University of North Carolina at Chapel Hill, working with Prof. Mohit Bansal. He earned his Ph.D. from the School of Computing at KAIST, where he was advised by Prof. Sung Ju Hwang. His primary research focuses on developing continually adaptable, trustworthy, and interactive AI systems to address challenges in dynamic, real-world multimodal environments. He has received several honors, including the Best Ph.D. Dissertation Award from both the KAIST College of Engineering and the School of Computing, the PaliGemma Academic Program Award from Google (2024), and Selected for the Early-Career Spotlight Program at CoLLAs 2025. Dr. Yoon actively contributes to the research community, serving on the program committees of top NLP and ML conferences. He has held leadership roles as an Area Chair for NeurIPS 2025, EMNLP 2024, NAACL 2024, and the NeurIPS 2024 Workshop on Scalable Continual Learning for Lifelong Foundation Models.