TRAILS Trustworthy Machine Learning Seminar Series
Speaker: Paola Cascante-Bonilla, an assistant professor of computer science at Stony Brook University (SUNY) and former TRAILS postdoc
Where: University of Maryland, CSIC 2120 and on Zoom: https://umd.zoom.us/j/93960331973?pwd=tVQ7gtFucJEkSYna38zePzkPpHm3J7.1
Talk: Privacy Protection and Reliable Evaluation in Multimodal AI
Abstract: Multimodal Large Language Models (MLLMs) have shown impressive results on tasks that require visual and textual reasoning. However, using these systems on assistive technologies raises important questions around privacy protection and the real capabilities of these models when deployed in challenging scenarios in the real world. In this talk, I will explore these limitations in two settings: document understanding and spatial reasoning in 3D environments. I will first present our work on developing an agentic approach for fine-grained privacy protection that selectively masks personal information while preserving contextual visual content for visual question answering. Then, I will present SceneCritic, a symbolic evaluator that uses explicit spatial heuristics to assess the semantic and geometric consistency of generated indoor layouts. Together, these works aim to ask the now prevalent question: can we harness MLLMs and circumvent their limitations as reasoners and evaluators?
[1] SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis
Bio: Paola Cascante-Bonilla is an Assistant Professor in the Department of Computer Science at Stony Brook University (SUNY). Her research focuses on developing systems that enable compositional reasoning and common-sense inference through vision and language, while tackling issues such as cultural biases, data distribution, explainability, and trustworthy AI. She was the recipient of the Ken Kennedy Institute SLB Graduate Fellowship (2022/23), a Future Faculty Fellow by Rice's George R. Brown School of Engineering (2023) and as a Rising Star in EECS (2023).
About the TRAILS Trustworthy Machine Learning Seminar Series: Each week a researcher from across the TRAILS community presents current work on what it takes to make AI systems worth trusting: measuring and evaluating their behavior, interpreting and steering them, keeping them robust and secure, protecting privacy, and understanding how people actually rely on them — with occasional excursions into the philosophy and psychology of trust. Talks are 40 minutes, followed by 20 minutes of discussion.