Jan Tolsdorf
Postdoctoral Associate, George Washington University
Jan Tolsdorf is a postdoctoral associate in the Usable Security and Privacy Lab at George Washington University. He studies human factors in trustworthy AI and human-centered approaches to information privacy and security. His research brings fresh insights to these areas through a multidisciplinary blend of qualitative and quantitative methods drawn from human-computer interaction, software and requirements engineering, social sciences, psychology, and economics.
Area of Expertise: Human-centered AI
-
Tolsdorf, J., Luo, A. F., Kodwani, M., Eum, J., Sharif, M., Mazurek, M. L., & Aviv, A. J. (2025, May). On a Scale of 1 to 5, How Reliable Are AI User Studies? A Call for Developing Validated, Meaningful Scales and Metrics about User Perceptions of AI Systems. 9th Workshop on Technology and Consumer Protection (ConPro’25).
Abstract: Public discourse around trust, safety, and bias in AI systems intensifies, and as AI systems increasingly impact consumers’ daily lives, there is a growing need for empirical research to measure psychological constructs underlying the human-AI relationship. By reviewing literature, we identified a gap in the availability of validated instruments. Instead, researchers seem to adapt, reuse, or develop measures in an ad hoc manner without much systematic validation. Through piloting different instruments, we identified limitations with this approach but also with existing validated instruments. To enable more robust and impactful research on user perceptions of AI systems, we advocate for a community-driven initiative to discuss, exchange, and develop validated, meaningful scales and metrics for human-centered AI research.