Shilton, K., Moss, E., Gilbert, S. A., Bietz, M. J., Fiesler, C., Metcalf, J., Vitak, J., & Zimmer, M. (2021). Excavating Awareness And Power In Data Science: A Manifesto For Trustworthy Pervasive Data Research. Big Data & Society, 8(2), 20539517211040760.

Abstract: Frequent public uproar over forms of data science that rely on information about people demonstrates the challenges of defining and demonstrating trustworthy digital data research practices. This paper reviews problems of trustworthiness in what we term pervasive data research: scholarship that relies on the rich information generated about people through digital interaction. We highlight the entwined problems of participant unawareness of such research and the relationship of pervasive data research to corporate datafication and surveillance. We suggest a way forward by drawing from the history of a different methodological approach in which researchers have struggled with trustworthy practice: ethnography. To grapple with the colonial legacy of their methods, ethnographers have developed analytic lenses and researcher practices that foreground relations of awareness and power. These lenses are inspiring but also challenging for pervasive data research, given the flattening of contexts inherent in digital data collection. We propose ways that pervasive data researchers can incorporate reflection on awareness and power within their research to support the development of trustworthy data science.

Full Paper

Gilbert, S., Vitak, J., & Shilton, K. (2021). Measuring Americans’ Comfort With Research Uses of Their Social Media Data. Social Media + Society, 7(3), 20563051211033824.

Abstract: Research using online datasets from social media platforms continues to grow in prominence, but recent research suggests that platform users are sometimes uncomfortable with the ways their posts and content are used in research studies. While previous research has suggested that a variety of contextual variables may influence this discomfort, such factors have yet to be isolated and compared. In this article, we present results from a factorial vignette survey of American Facebook users. Findings reveal that researcher domain, content type, purpose of data use, and awareness of data collection all impact respondents’ comfort—measured via judgments of acceptability and concern—with diverse data uses. We provide guidance to researchers and ethics review boards about the ways that user reactions to research uses of their data can serve as a cue for identifying sensitive data types and uses.

Full Paper

Shilton, K. (2018). Values and Ethics in Human-Computer Interaction. Foundations and Trends in Human–Computer Interaction, 12(2), 107–171.

Abstract: An important public discussion is underway on the values and ethics of digital technologies as designers work to prevent misinformation campaigns, online harassment, exclusionary tools, and biased algorithms. This monograph reviews 30 years of research on theories and methods for surfacing values and ethics in technology design. It maps the history of values research, beginning with critique of design from related disciplines and responses in Human-Computer Interaction (HCI) research. The review then explores ongoing controversies in values-oriented design, including disagreements around terms, expressions and indicators of values and ethics, and whose values to consider. Next, the monograph describes frameworks that attempt to move values-oriented design into everyday design settings. These frameworks suggest open challenges and opportunities for the next 30 years of values in HCI research.

Full Paper

Iqbal, M., Shilton, K., Sayed, M. F., Oard, D., Rivera, J. L., & Cox, W. (2021). Search with Discretion: Value Sensitive Design of Training Data for Information Retrieval. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 133:1-133:20.

Abstract: This paper describes and assesses the value sensitive design (VSD) of a test collection: data used to train and evaluate a machine learning system for information retrieval. The project used the VSD framework and methods to design a test collection annotated for discretion. We conducted qualitative stakeholder interviews to develop values personas, which guided annotation of a collection of corporate emails for contextual notions of sensitivity. Both qualitative and quantitative evaluations of the method reveal that the values personas concretely shaped annotators' sensitivity judgments, and analysis of the test collection itself demonstrates that the sensitivity annotations have utility for identifying features that may correlate with email sensitivity. Values personas for training data annotation expand the toolkit of methods for value-sensitive machine learning.

Full Paper

Shilton, K., & Anderson, S. (2017). Blended, Not Bossy: Ethics Roles, Responsibilities and Expertise in Design. Interacting with Computers, 29(1), 71–79.

Abstract: What are the best ways for design teams to attend to issues of power, inequity, trust and other ethical concerns as they arise in design? Literature on value-sensitive design (VSD) and technology ethics has advocated for a range of design methods that propose different roles and responsibility for ethics during technology development. This paper explores four provocations that imagine different roles and responsibilities for moral and ethical reasoning on design teams: participatory design (in which diverse stakeholders may represent their own values in the design process), values advocates (introducing experts to lead values discussions or conduct ethics interventions), embedding values discussions within design and encouraging ‘moral exemplars’ within design. Each of these posits different logistical arrangements as well as different levels of expertise in ethical practice. The paper uses examples from the VSD and computer ethics literatures as well as the authors' ethnographic work to explore the advantages, challenges and consequences of each approach.

Full Paper